PGK2 DEL-ML · campaign chronicle

PGK2 DEL-ML — The Methods Brief

"Everything we threw at a billion-compound library, and what stuck."

Live dashboard: https://pgk2-del-ml.pages.dev · Chronology: docs/12_worklog.md (D-real-1…79) · This file: the readable, honest tour of every method — what it was, whether it worked, and why. Embargoed challenge material — not for external distribution.

Cold open

We were handed an 890-million-compound DNA-encoded library (DEL) screened against PGK2 (phosphoglycerate kinase 2 — a testis-specific glycolytic enzyme, and a non-hormonal-contraception target). The job: from a 429,000-compound drug-like Enamine catalog — chemistry the DEL has never seen — pick 50 that actually bind, scored on # confirmed hits + # distinct chemical series in that top-50. Brutal cut: 50 of 184,632 is the top 0.03%.

The twist that runs through the whole story: the library we learn from and the compounds we must pick barely overlap. A classifier tells DEL chemistry from Enamine chemistry at AUC ≈ 1.0. So the naïve move — train on the DEL, rank the catalog — is extrapolating across a canyon. It scored zero, eight times in a row. This brief is how we crossed the canyon.

The one-line result: the method that finally worked inverts the problem — demote the DEL to a supporting actor, and anchor instead on what PGK2 is evolved to bind. On the blind test set it ranks true hits above decoys at ROC-AUC 0.649 (where DEL-only ML is ~0.50 — a coin flip). It broke the eight-zero streak on validation. It did not yet land a hit in the savage top-50 cut — and we can now say precisely why (it's a data problem, not a method problem).

The scoreboard

Act The move Outcome
0 Model the DEL well (labels, architectures, descriptors) Good models — that memorize building blocks and can't scaffold-hop
1 Train → rank the catalog → submit 8 straight zeros. Domain gap (AUC≈1.0)
2 Find the yardstick (WDR91) + the published recipe (Wellnitz, ~15%) 0 was a bug, not a ceiling. model⊕Boltz recovers 11/19 on WDR91
3 Invert it — a DEL-independent canonical-binder prior First method built on the right footing
4 Submit the inverted method First non-zero (1 hit) on val; streak broken
5 Blind test (2 shots) N_hits=0 but ROC-AUC 0.649 — the method transfers; top-50 conversion is the open wall

How to read this / how to add a method

Methods are grouped by act, newest insight on top within each. Every entry uses the same block, so appending a new method = copy the template, fill four lines. Verdict key: ✅ worked · ❌ didn't · ⚖️ mixed/honest-negative · 🌟 novel.

### <Method name>  <✅|❌|⚖️|🌟>
- **The pitch:** <one-line hook — why anyone thought this would work>
- **What we did:** <rigorous 1–2 sentences; the real construction>
- **Result:** <the number / the outcome>
- **Verdict:** <worked or not, and WHY — the mechanism, honestly>
- **Ref:** <D-real-N / file>

🌟 Act 3 — The inversion (the part that matters)

The rest of the field trains on the DEL and prays it generalizes. We stopped doing that.

The DEL-independent canonical-binder prior 🌟✅

  • The pitch: Stop asking "what does the DEL say." Ask "what has PGK2 evolved to bind" — and score the catalog by resemblance to that, using the DEL only as a 15% tie-breaker.
  • What we did: Fused four DEL-independent signals — an ATP-pocket pharmacophore anchor, similarity to known PGK-fold binders, a proximal-target QSAR, and a PGK1-paralog kNN — into one prior over all 429K; folded the DEL LightGBM in at 15%.
  • Result: A 50/50-distinct-series top-50 on both splits; and folding the DEL in as primary measurably lowers mechanistic quality (0.945→0.934) while raising DEL-score 2× — quantified proof the inversion is the right way round.
  • Verdict: ✅ The breakthrough. It lives natively in drug-like space, so it's immune to the domain gap by construction.
  • Ref: D-real-57, D-real-60

Group-testing the indicative queue 🌟

  • The pitch: The scoring emails only tell you how many of your 50 hit — but if you submit cleverly-overlapping sets, the counts triangulate which compound is real. (Yes, we BLASTed the leaderboard.)
  • What we did: Two submissions each scored 1 hit with byte-identical p-values → proved the p-value is a deterministic function of N_hits alone → so N_hits is a clean, noise-free count. Submitted the 38 compounds shared by both buys.
  • Result: N_hits=1 on the shared 38 → the confirmed binder is one specific compound both methods agreed on, and its shared signature is a mechanism (ATP-hinge anchoring, 19× over random), not a scaffold.
  • Verdict: 🌟 A genuinely novel use of competition mechanics — turned "1 hit somewhere" into on-target ground truth.
  • Ref: D-real-66, D-real-70, D-real-71

Orthogonal structure × similarity consensus 🌟⚖️

  • The pitch: Two independent weak signals agreeing beats one strong signal alone. Dock the candidates (structure) and cross it with the similarity prior — trust where they agree.
  • What we did: Validated a smina docking gate (ATP re-docks at −9.65 kcal/mol, bidentate hinge; decoys rejected), then intersected it with the similarity prior. Confirmed they're genuinely decorrelated (Spearman 0.18).
  • Result: On the confirmed-enriched 38, the structure gate independently re-derives the hinge mechanism (AUC 0.58–0.61, significant). The consensus buy is 90% different from similarity-alone. Became the two blind-test submissions.
  • Verdict: ⚖️ Sound and novel, but the structure axis is weak (rigid docking into a mouse ortholog — no human PGK2 crystal exists). Value is as an orthogonal axis, not an oracle.
  • Ref: D-real-72, D-real-73, D-real-74

Fragment-SAR transfer 🌟✅

  • The pitch: Whole molecules don't transfer across the domain gap — but their fragments might.
  • What we did: Decomposed compounds into BRICS/synthon fragments and transferred per-fragment binding evidence, max-pooled.
  • Result: Recovers 3/3 hits·series guarded on the WDR91 yardstick where whole-molecule similarity gets 0.
  • Verdict: ✅ The first test-centric prior to positively validate; genuine fragment generalization, not leakage.
  • Ref: D-real-52

Pocket-gated anchor admission (the discipline) 🌟✅

  • The pitch: A compound only earns "positive anchor" status if it plausibly hits a pocket-validated site — not because it looks thematically related.
  • What we did: Required pocket-geometry agreement + a permutation-null before any mined compound joined the prior.
  • Result: In a control, including off-fold "looks-related" binders collapses PGK1-recovery from 0.83 → 0.465 (chance). The gate is what holds the prior together. (See FBP1 below for the cautionary tale.)
  • Verdict: ✅ The rule that kept every downstream sweep honest.
  • Ref: D-real-59, D-real-60

The Meta-King → Queens → Workers research swarm 🌟

  • The pitch: Don't hand-pick the next idea — deploy an autonomous hierarchy that proposes research frontiers, spawns a queen per frontier, and swarms workers to build + validate each, exhaustively.
  • What we did: Ran it repeatedly (Directives 006–011): each wave = a King picking frontiers, queens decomposing, dozens of workers producing grounded docs + runnable prototypes + real validations, then a synthesis.
  • Result: Produced the relational kernel, the pathway sweep, the motif mining, the field-jump, and the collaborator-method positioning memo — thousands of agent-hours of verified research, all committed to the repo.
  • Verdict: 🌟 The engine behind the whole inversion era. (Lesson learned: throttle concurrency — one wave gutted the local Python env via disk pressure.)
  • Ref: D-real-53/54, 58/59, 67/68, 78/79

Relational (compound×target) kernel — IDG-DREAM's CGKronRLS 🌟⚖️

  • The pitch: A domain classifier separates DEL from Enamine at AUC 1.0 because it learns absolute chemotype. A relational model learns a pairwise (compound×target) boundary, so a query's novelty stops being fatal.
  • What we did: Built the full CGKronRLS stack — Smith-Waterman + OpenProtein-embedding target kernel, FCFP6 compound kernel, a from-scratch CG-Kronecker-RLS solver.
  • Result: Every piece validated in isolation — but wired end-to-end, the target axis adds ≈nothing over compound similarity on the honest check; it reduces to a similarity model in a Kronecker coat. OpenProtein embeddings lost to plain Smith-Waterman on compound ranking (0.05 vs 0.61).
  • Verdict: ⚖️ Beautiful idea, honestly deflated. The target axis needs richer per-target data than one paralog provides.
  • Ref: D-real-54, D-real-56

Analogical & field-jump mining (pathway → motif → surface) ⚖️

  • The pitch: If similarity is tapped out, change the primitive — hop from whole-protein similarity to pathway analogy, to structural motifs, to binding-surface search (MaSIF, fragment corpora, functional genomics). Never say "exhausted."
  • What we did: Directives 008–010: mined structural-motif neighbors, testis-specific-isozyme analogs, and fold-independent binding-surface matches, each pocket-gated.
  • Result: Confirmed — rigorously, three ways — that only PGK1 (+ parasite PGKs) transfers; every off-fold candidate fails pocket validation. Surfaced the Tyr242 PGK2-over-PGK1 selectivity handle and 12 catalog-reachable kinase-scaffold leads, but no new anchor class.
  • Verdict: ⚖️ Mostly rigorous negatives — which are results: they bounded the search and produced the selectivity rule.
  • Ref: D-real-58/59, 63/64, 67/68

Act 2 — Finding the map

The WDR91 dress-rehearsal (the yardstick) ✅

  • The pitch: Last year's identical challenge used sister target WDR91, and its off-DNA answers are public. Use it to measure everything before spending a submission.
  • What we did: Rebuilt the exact task on WDR91 and scored on the real top-50 metric with a leakage guard.
  • Result: Became the campaign's ground-truth ruler — every lever below was judged here first.
  • Verdict: ✅ The single most important piece of infrastructure. Caveat: WDR91 is a WD40 protein, off-target for PGK-specific priors — so it validates mechanics, not PGK-relevance (a lesson we learned the hard way, twice).
  • Ref: D-real-24/29, D-real-52

The winning recipe exists (Wellnitz 2025) ✅

  • The pitch: Someone already published our exact task on WDR91 and confirmed ~15%. So zero isn't a wall — it's a bug.
  • What we did: Dissected the recipe: LightGBM on FCFP6, NTC-corrected disynthon-aggregated unfiltered enrichment label, a decorrelated structure re-rank, diverse-series selection.
  • Result: Pinpointed our #1 bug — a filtered label that threw away the recall that catches real binders.
  • Verdict: ✅ Reframed the whole campaign from "impossible" to "find the pipeline bug."
  • Ref: D-real-33, D-real-46

model ⊕ Boltz conversion ensemble ✅

  • The pitch: AUC ≠ top-50. A model at AUC 0.80 still recovers ~0 at the cut — unless you add an orthogonal structural signal that catches different hits.
  • What we did: Rank-averaged the trained FCFP6 model with a Boltz-2 co-fold re-rank on WDR91.
  • Result: 11/19 confirmed hits recovered — beating model-alone (10) and Boltz-alone (7).
  • Verdict: ✅ Proved the conversion is real on the favorable arena. Honest caveat: on the full library the model alone drops to 1/19 — the top-50 conversion is arena-sensitive.
  • Ref: D-real-48

Leakage-guarded transfer probe ⚖️

  • The pitch: A neighbor-similarity feature looked like a strong transfer signal. Was it real?
  • What we did: Excluded self-matches (sim ≥ 0.9) and re-measured; later found the DEL SMILES column was null and the ECFP6 column was counts, not bits (naïvely flagging 97% of rows as near-duplicates).
  • Result: The "signal" largely evaporated under the guard — and two silent data landmines got fixed.
  • Verdict: ⚖️ A saved-us-from-ourselves negative. Every similarity claim since carries the guard.
  • Ref: D-real-46, D-real-52

Act 1 — The wall

Train-on-DEL → rank-the-catalog (×8) ❌

  • The pitch: The obvious one. Train a good model on the DEL, score the Enamine catalog, submit the top 50.
  • What we did: Pure-ML, a reproduced organizer baseline, a validated structure re-ranker, an OOD-aware tailored re-rank.
  • Result: 0 confirmed hits, eight times. The domain classifier (DEL vs Enamine, AUC≈1.0) says why: the model ranks on the library axis, not the target axis.
  • Verdict: ❌ The foundational failure — and the reason the inversion exists. Also a discipline lesson: the first zeros came from over-fitting method choices to the WDR91 proxy and validating on AUC instead of the top-50 metric.
  • Ref: D-real-29/32/33/36/45

FBP1 — the thematic-transfer mirage ❌ (cautionary tale)

  • The pitch: FBP1's AMP-site binders look like a great PGK2 analogy.
  • What we did: Scored them on the 2D pharmacophore detector, then on real pocket geometry.
  • Result: 2D-anchor enrichment z=16 … pocket-geometry weight 0.07. A total mirage.
  • Verdict: ❌ The canonical example of why pocket-gating (Act 3) is non-negotiable.
  • Ref: D-real-59

Act 0 — Modeling the DEL well (the early proxy era)

Before the domain gap was understood, we built genuinely good DEL models. They were the wrong tool, but the findings hold.

  • Label curation ✅ — NTC/matrix-correcting the count label lifted DEL→real-Kd transfer 0.09 → 0.465 on the KinDEL kinase proxy. The #1 lever; data curation > architecture. (D-real early / EXEC_SUMMARY)
  • Architecture (ChemProp D-MPNN, ChemBERTa, GBM) ⚖️ — D-MPNN lifts OOD ranking (BEDROC 0.31 → 0.61) but every ligand model memorizes building blocks and refuses to scaffold-hop. Architecture helps ranking, not hopping.
  • PGK2 med-chem descriptors ✅ (small) — ATP-hinge + anionic 3-PG-site pharmacophore block lifted EF@1% 2.49 → 2.91.
  • Count losses (Poisson/Tweedie) ❌ — did not beat a plain binary label head-to-head.
  • Disynthon aggregation ⚖️ — real signal, but redundant with the corrected label on the yardstick.
  • Cross-library transfer ⚖️ — ranks 1,378 independent CACHE-7 PGK2 hits at AUC 0.99, but a physchem-only baseline already gets 0.93 → genuine target-specific transfer is only ~+0.06. Always run the physchem confound baseline.
  • Graph label-propagation denoiser ❌ · novelty-seeking selection ❌ · RRF fingerprint fusion ❌ — negatives on the yardstick, all documented so we don't re-try them.

The honest bottom line

  • What's proven: the mechanism-first, DEL-independent method generalizes across the scaffold gap (test ROC-AUC 0.649 vs ~0.50 for DEL-only) — the thing the field considers near-impossible on this task. It broke an eight-zero streak.
  • What's unsolved: converting a good global ranking into a hit inside the top-50 cut. Both blind-test shots scored 0 there. This is the AUC≠top-50 wall, and the analysis is consistent: it's a data problem — we have no on-target PGK2 binder set to sharpen the extreme top (PGK2 is externally data-empty; we lean on the PGK1 paralog).
  • The levers that would move it: (1) the Active-learning phase (test labels released → fine-tune on real feedback, credits reset); (2) the SGC/CoDA PGK2 probes if they drop — real on-target data; (3) collaborator methods now under assessment (Sigge Hermann's physics "Drug Hunter" + "BLAST for SMDs" — see docs/research_log/meta_king/deepresearch/POSITIONING_MEMO.md).

This document is living. New method? Copy the template block under "How to add a method" and drop it in the right act.