Why My ESM-C Flow Model Lost the AMP Challenge

I placed 29th in the AMP Challenge with an Aggregation Score of 16.45 out of 100.

I fine-tuned EvolutionaryScale’s ESM-C 300M with a discrete masked flow objective, sampled 396,000 candidate peptides on an M5 Pro, and selected a 50,000-sequence library evaluated against held-out splits. Across 12 internal metrics, 11 beat the my DiffTabPFN baseline,¹ but when the organizers released the scoring post-mortem, I saw the divergence between my local validation and the leaderboard. The selection pipeline optimized for a narrow surrogate distribution, treated physical synthesizability as a soft penalty rather than a hard constraint, and incurred heavy perplexity penalties from non-causal flow sampling.

What I built

The goal of the competition is de novo generation of linear antimicrobial peptides between 8 and 50 amino acids. Participants submitted a 50,000-sequence library for computational scoring and a prioritized top-100 list. For the top 20 teams, 25 peptides from each team’s top-100 list are synthesized and tested in wet-lab assays against 20 clinical pathogen isolates, including multidrug-resistant ESKAPE strains.²

My generative model used ESM-C 300M with a discrete masked flow objective. The training sequence conditioned generation on explicit control tokens:

[BOS] [charge] [length] [hydro] [residues ...] [EOS]

Net charge was discretized into seven buckets, sequence length into seven bins matching natural length distributions, and mean hydrophobicity into seven ranges. During training, control tokens were dropped to a learned NULL token 15% of the time to support classifier-free guidance.

Sampling ran through 12-step confident discrete unmasking: all residue positions started masked, and the model iteratively unmasked its highest-confidence predictions.

To select 50,000 sequences from the 396,334 sampled candidates, I set up a held-out split on DBAASP.³ I fit density ratios and kernel density estimates on the first half and scored candidates only against the second half:

$\text{Score}=2z(\text{Conformity})+1.8\cdot \text{Active}+0.5\cdot \text{F1}+4z(\text{density ratio})+1z(\text{Diversity proxy})$

I also applied a nearest-neighbor gate: candidate sequences had to land within the neighborhood of known DBAASP peptides in at least 97% of random draws (prec20 >= 0.97) to anchor generated candidates close to verified antimicrobial space.

The comparison against the organizers’ baseline looked favorable across biophysical and latent metrics:

Metric	DiffTabPFN Baseline	My Run 1 Submission	Difference
AMPredictor Median MIC	13.80 µM	9.07 µM	-4.73 µM
Active $\le 16\mu \text{M}$	54.9%	79.8%	+24.9%
ESM-2 FBD / MMD	2.38 / 8.75	1.55 / 2.81	-0.83 / -5.94
ESM-C FBD / MMD	0.11 / 0.58	0.05 / 0.08	-0.06 / -0.50
Latent Precision	0.995	0.996	+0.001
Latent Recall	0.845	0.865	+0.020
Biophysical Conformity	0.752	0.808	+0.056
Synthesizability (Torres Filter)	9.2%	20.2%	+11.0%
Novelty Compliance ($\le 80\%$)	73.1%	99.4%	+26.3%
Sequence Diversity	0.828	0.826	-0.002

My top-100 candidates were predicted active at $\le 16\mu \text{M}$ under TabPFN, had an AMPredictor median of 5.34 µM, and were gated against erythrocyte hemolysis ($p<0.35$, with 0% above 0.55). Because the internal metrics improved consistently on the baseline, I assumed the 50,000-sequence library would clear the top-20 cutoff, leaving the primary test to wet-lab screening.

What the top approaches did

Autoregressive language models took the top positions on the leaderboard.

1. Sandra Clemens / IMI Münster (Rank 1: 38.86 ± 0.22)

Sandra Clemens (Institute of Medical Informatics, Münster) submitted predictions from the base checkpoint of AmpGPT2, published earlier in 2026. The submission required no task-specific retraining.

• Generative backbone: A 738M-parameter causal transformer with ProtGPT2 architecture trained on the COMPASS database⁴ (over 75,000 non-redundant antimicrobial peptides).
• Library generation: Sampled sequences until 50,000 candidates met an APEX⁵ mean MIC threshold of $\le 256\mu \text{M}$, removing exact duplicates and training references.
• Top-100 selection: Sorted candidates by APEX mean MIC and applied distance filtering: Levenshtein distance ratio $\le 0.80$ against known antibacterials, local alignment similarity $\le 0.60$ against references, and pairwise local similarity $\le 0.40$ among selected candidates.

2. Evgeniy Nam (Rank 2: 36.61 ± 0.34)

Evgeniy Nam adapted ProtGPT-1.3B using LoRA and continuous soft prompts.

• Distribution steering: Analyzed natural antimicrobial peptides along net charge and Wimley-White interfacial hydrophobicity. Identified five density peaks in natural sequences and steered soft prompts toward those regions.
• Candidate scoring: Combined potency predictions from a LightGBM model trained on GRAMPA⁶ using ESM-2 embeddings with safety scores from an ESM-2 regression model trained on DBAASP ${\text{HC}}_{50}$ hemolysis assays.

3. Rishyanth Reddy (Rank 3: 32.19 ± 0.32)

Reddy used ProGen2 paired with directed evolutionary search. Starting from natural sequence likelihoods, the pipeline generated local sequence variants guided by surrogate fitness functions evaluating predicted MIC and edit diversity.

Why my model scored 16.45

1. Language model perplexity penalized non-causal flow sampling (Estimated loss: ~10 pts)

The official metric evaluated sequence plausibility under hyformer_peptides_34M. In discrete flow matching, unmasking several residue positions simultaneously without causal left-to-right attention produces local transition anomalies. Under high charge conditioning, my sampler generated repetitive cationic motifs such as RFRWFRVF and RWKIWYRWKF.

2. Synthesizability rules were soft-weighted rather than filtered (Estimated loss: ~9 pts)

The competition’s synthesizability metric evaluated rules defined by Torres et al.:⁷ hydrophilic fraction between 30% and 70%, at most 2 consecutive glycines, at most 1 bulky hydrophobic run, no aggregation motifs, proline fraction $\le 20\%$, at most 1 cysteine, and net charge strictly within $[+1.9,+10.0]$.

I included synthesizability as a soft term in my multi-objective selection score ($0.5\cdot \text{F1}$). In the submitted 50,000-sequence library, only 20.2% of peptides cleared all filters, dropping an 11.1-point metric to roughly 2.2 points.

3. Active Coverage penalized library concentration (Estimated loss: ~14 pts)

The diversity pillar allocated 16.7 points to Active Coverage. The calculation works in three steps:

1. Filter the 10,000 sequences in an evaluation block down to only those predicted to be active against the bacterial panel.
2. Group those active sequences into clusters using CD-HIT at 60% sequence identity (sequences sharing more than 60% of their amino acid positions merge into the same cluster).
3. Count the number of distinct active clusters, and divide that count by the total block size of 10,000 sequences, rather than by the number of active peptides:

$\text{Active Coverage Ratio}=\frac{\text{Number of Active Clusters}}{10,000}$

My selection pipeline suffered in two ways:

• Over-concentrating on nearest neighbors: To keep generation close to validated antimicrobial peptides, my selection algorithm applied a nearest-neighbor precision filter (prec20 >= 0.97). This filter heavily penalized any candidate peptide that did not land within the immediate neighborhood of known DBAASP reference sequences in at least 97% of random evaluation draws. While this improved local conformity scores, it packed thousands of candidate sequences into dense clouds around the same known reference peptides.
• Lower multi-strain hit rate: Because my selection relied on AMPredictor (E. coli only), fewer sequences cleared the competition’s conservative eleven-strain activity threshold.

With fewer active peptides overall, and with those active peptides concentrated into a small number of redundant clusters, dividing the active cluster count by the fixed block size of 10,000 reduced the coverage score to near zero, losing roughly 14 of the 16.7 available points.

4. Surrogate mismatch across bacterial strains

I selected candidates using AMPredictor, which regresses MIC against E. coli. The official evaluation used an eleven-strain AMPFormer ensemble (combining pepCATS, ESM-2, Hyformer, and APEX models) across both Gram-negative and Gram-positive pathogens, applying an uncertainty penalty ($\mu -0.5\sigma$). Sequences optimized for AMPredictor’s specific charge preferences lost points when the other ensemble models differed on Gram-positive targets. Clemens filtered using APEX directly, which was part of the evaluation ensemble.

---

Why my data pipeline missed COMPASS

My training pipeline used three stages:

• Stage A (5.6M rows): UniProt protein fragments, protein tiling, ToxProt.
• Stage B (889k rows): Curated databases combined with AMPSphere.⁸
• Stage C (4,460 rows): Broad-spectrum potent peptides ($\le 16\mu \text{M}$).

The composition of Stage B explains the data quality issue:

• DBAASP provided 286 rows, and DRAMP⁹ provided 250 rows.
• Over 840,000 rows (more than 94% of Stage B) came from AMPSphere.
• AMPSphere contains computational predictions from metagenomic contigs generated by a random forest classifier (Macrel), without experimental validation.

The model spent its training budget on unvalidated metagenomic predictions. Meanwhile, AmpGPT2 trained on COMPASS (compass.imi.uni-muenster.de), an experimentally curated database of over 75,000 non-redundant antimicrobial peptides. I assumed pretraining on 5.6M UniProt protein fragments would teach general peptide grammar. In practice, slicing globular cytoplasmic proteins introduced hydrophobic core fragments that do not reflect linear antimicrobial sequences.

Technical adjustments for a competitive flow model

1. Clean training data
2. Allowing the sampler to revise unmasked tokens resolves local transition errors and reduces Hyformer perplexity.
3. Hard synthesizability pre-filtering

Footnotes

1. TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second (Hollmann et al., Nature 2025). ↩

2. ESKAPE is an acronym for the six nosocomial bacterial pathogens that frequently exhibit multidrug resistance and virulence: Enterococcus faecium, Staphylococcus aureus, Klebsiella pneumoniae, Acinetobacter baumannii, Pseudomonas aeruginosa, and Enterobacter species. See Rice (2008), Federal funding for the study of antimicrobial resistance in nosocomial pathogens: no ESKAPE. ↩

3. DBAASP (Database of Antimicrobial Activity and Structure of Peptides) is a curated repository of experimentally validated antimicrobial peptides, logging minimal inhibitory concentrations (MIC) across microbial targets and erythrocyte hemolytic toxicity (${\text{HC}}_{50}$). ↩

4. COMPASS Database is a curated repository compiled by the Institute of Medical Informatics at the University of Münster, aggregating over 75,000 non-redundant, experimentally validated antimicrobial peptides. ↩

5. APEX (Antibacterial Peptide EXplorer) is a deep-learning ensemble model that predicts antibacterial minimum inhibitory concentrations (MIC) across diverse Gram-positive and Gram-negative bacterial isolates. ↩

6. GRAMPA (Genome-wide Recorded Antimicrobial Peptide Activity) is an open database of antimicrobial peptides with curated experimental MIC measurements, assembled by Witten & Witten (2019). ↩

7. Torres et al. (2019), Structure-function relation of antimicrobial peptides. The filter requires 30–70% hydrophilic residues, at most 2 consecutive glycines, at most 1 bulky hydrophobic run, no aggregation motifs, proline $\le 20\%$, at most 1 cysteine, and net charge strictly in $[+1.9,+10.0]$. ↩

8. AMPSphere is a large-scale database of computationally predicted antimicrobial peptides identified by the Macrel random-forest classifier from global microbial genomes, lacking direct experimental wet-lab validation. ↩

9. DRAMP (Data Repository of Antimicrobial Peptides) is an annotated repository of natural, synthetic, and clinical antimicrobial peptides detailing experimental bioactivity and physicochemical attributes. ↩