Audit of all twelve ACMG/AMP evaluators against BRCA1's real specification, showing which needed changes and what kind
01 — THE AUDIT
Twelve evaluators, three outcomes
Not a binary pass/fail. Two evaluators needed attention — but "needed attention" split into two categories that look identical from the outside and are not the same thing.
Unchanged — needed nothing
Code fix — evaluator logic changed
Data-curation fix — evaluator untouched, fixture data corrected
PVS1
Unchanged
PS1
Unchanged
PM3
Unchanged
PM2
Code fix
PM4
Unchanged
PM5
Unchanged
PS3
Unchanged
BS3
Unchanged
PP3
Unchanged
BP4
Unchanged
BA1
Data-curation fix
BS1
Data-curation fix
9 unchanged
1 code fix (PM2)
2 data-curation fixes (BA1, BS1)
11 / 12 needed zero evaluator code changes
How a guard test caught PM2 incorrectly firing on BRCA1 indel variants, and the gene-gated code fix that followed
02 — PM2: CODE FIX
The safety net catching something, not just the patch
BRCA1's real spec says PM2 doesn't apply to insertion/deletion variants. The original plan assumed that could be handled by fixture choice alone. It couldn't — and a guard test is what proved it.
1
Guard test written
test_brca1_fixtures_do_not_exercise_pm4_pm5_or_unenforced_pm2_indel_gaps — asserting no curated BRCA1 fixture accidentally exercises PM4, PM5, or an unenforced PM2-indel call.✕
Test fails on PM2
PM4/PM5 stayed inert through fixture-shape avoidance alone, as planned. PM2 didn't: two real, cited Ashkenazi-founder frameshift fixtures needed
population_evidence anyway, for their own BA1/BS1 handling — so PM2 always ran a real frequency comparison, contradicting the spec's "does not apply" rule.✓
Gene-gated fix, test passes
One small, additive, opt-in config key —
pm2_excludes_indel_delins: true, set only for BRCA1. Checked before any retrieval-status branch. CAPN3/DMD provably unaffected.| Fixture | PM2 result before | PM2 result after | |
|---|---|---|---|
| BRCA1_c.68_69delAG | NOT_MET | → | NOT_APPLICABLE |
| BRCA1_c.5266dup | NOT_MET | → | NOT_APPLICABLE |
How BRCA1's founder-population BA1 and BS1 evaluators were kept correct through fixture data curation rather than a code change
03 — BA1 / BS1: DATA-CURATION FIX
A different kind of "not for free"
ba1.py and bs1.py have zero BRCA1- or batch-31-specific code — no new branch, no new config key. What needed fixing was which number got curated into the fixture, not the evaluator that reads it.BS1 threshold — 1 × 10⁻⁴
WHY IT MATTERED
Both evaluators compare
overall_af against the gene's threshold before ever consulting ancestry_specific_max_af — so curating only the ancestry field as founder-excluded, the original plan, would not have worked. The raw total alone already crossed BS1's threshold.
"BA1/BS1's founder-frequency handling needed the right value curated into
overall_af for BRCA1's two Ashkenazi-founder fixtures, not a code or config change — the evaluators themselves were already correct and untouched."