The pipeline was frozen before 18 held-out liquid samples (influent, effluent, pond; Days 2 and 3) were opened, then scored once against the counts published by Taylor et al. (2025). Each dot is one sample: ours divided by published. Choose a measure and a sample type, and hover a dot for the numbers.
The total ratio is our summed read count at the paper’s published ARG subtypes (called at ≥ 50 reads) divided by the paper’s summed count for the same sample; it is not the sample’s total read count. The shaded band is the registered pass range for the chosen measure; the dashed line is a ratio of 1.00. Dots outside the band are ringed.
“Subtypes” are the ResFinder gene subtypes the paper published a positive count for (called at ≥ 50 reads). Red values fall outside the registered range.
Criteria were fixed in the registration (version 2) before the evaluation reads were opened.
| Criterion | Registered requirement | Result | Status |
|---|---|---|---|
| P1 total ratio | 0.93–1.07 in ≥ 16 of 18 samples and ≥ 5 of 6 per type | 17 of 18 (influent 6/6, effluent 5/6, pond 6/6); range 0.911–0.991 | met |
| P2 direction | ratio below 1.00 in ≥ 15 of 18 | 18 of 18 | met |
| P3 gene level | ≥ 90% of gene–sample pairs within 0.90–1.10 | 68 of 70 (97.1%); outside: ermB in E021 (0.22) and E036 (0.71) | met |
| P3b extra genes | ≤ 2 detections the paper does not report | 0 | met |
| P4 subtype ratio | median subtype ratio 0.90–1.10 in ≥ 16 of 18 | 17 of 18 (P036 outside) | met |
| P3c subtype recall | reported, outside the verdict | recall 0.934; 9 extra subtype calls | reported |
70 gene–sample pairs rather than 90: a ratio can only be formed where the paper reports a positive count, and 20 of the 90 pairs have none. Additional detections go to P3b.