Wastewater ARG Metagenomics · Preregistered Reproduction

Registered Criteria Scorecard

The pipeline was frozen before 18 held-out liquid samples (influent, effluent, pond; Days 2 and 3) were opened, then scored once against the counts published by Taylor et al. (2025). Each dot is one sample: ours divided by published. Choose a measure and a sample type, and hover a dot for the numbers.

17/18P1: total ratio within 0.93–1.07 (registered ≥ 16/18, and ≥ 5/6 per type)
18/18P2: our ARG-read sum below the published sum (registered ≥ 15/18)
68/70P3: gene–sample pairs within 0.90–1.10 (registered ≥ 90%)
0P3b: additional gene detections (registered ≤ 2)
17/18P4: median subtype ratio within 0.90–1.10 (registered ≥ 16/18)

Ours versus published, per sample

The total ratio is our summed read count at the paper’s published ARG subtypes (called at ≥ 50 reads) divided by the paper’s summed count for the same sample; it is not the sample’s total read count. The shaded band is the registered pass range for the chosen measure; the dashed line is a ratio of 1.00. Dots outside the band are ringed.

Measure
Sample type
inside the registered band outside the band registered band

Sample table

“Subtypes” are the ResFinder gene subtypes the paper published a positive count for (called at ≥ 50 reads). Red values fall outside the registered range.

Registered criteria and outcome

Criteria were fixed in the registration (version 2) before the evaluation reads were opened.

CriterionRegistered requirementResultStatus
P1 total ratio0.93–1.07 in ≥ 16 of 18 samples and ≥ 5 of 6 per type17 of 18 (influent 6/6, effluent 5/6, pond 6/6); range 0.911–0.991met
P2 directionratio below 1.00 in ≥ 15 of 1818 of 18met
P3 gene level≥ 90% of gene–sample pairs within 0.90–1.1068 of 70 (97.1%); outside: ermB in E021 (0.22) and E036 (0.71)met
P3b extra genes≤ 2 detections the paper does not report0met
P4 subtype ratiomedian subtype ratio 0.90–1.10 in ≥ 16 of 1817 of 18 (P036 outside)met
P3c subtype recallreported, outside the verdictrecall 0.934; 9 extra subtype callsreported

70 gene–sample pairs rather than 90: a ratio can only be formed where the paper reports a positive count, and 20 of the 90 pairs have none. Additional detections go to P3b.