Alignment-free QC accuracy across the full k-mer configuration space
How closely SNIPE's sketch-based QC estimates match alignment ground truth, scored per sequencing workload and per k-mer configuration. Genomic assays and RNA are scored independently.
How the score is built ▸
Every value below is in [0,1], higher is better, and is the best achievable score for the
current filters — shown with the k-mer configuration that achieves it.
Four QC metrics
Sequence-change rate, mean depth, mapping rate, coverage breadth. (The manuscript states why these metrics were selected; here they are all shown.) SNIPE's estimate for each is compared to the alignment ground truth (Qualimap / bamqc; read-level mapping rate; event-based sequence-change rate).
Per-metric accuracy — hybrid
Closeness to truth, 1 − min(MAPE, 1), blended with rank tracking max(0, r)
wherever the ground truth actually varies (so a correlation is meaningful); closeness only where it doesn't.
Aggregation
Per assay: the four metric accuracies are averaged (equal) or seq-change-weighted (×2). Per workload: the geometric mean across the assays in the bundle — a weak assay pulls the bundle down.
Two k-modes
Fixed restricts to k1 ∈ {21, 31, 51} (the manuscript's operating points).
Free searches the entire sweep — k1 ∈ {21…71}, all edge extensions, all scales — for the best score.
Why RNA is separate
RNA is scored against the transcriptome and is k-sensitive (short reference, long edge-mers saturate); the genomic assays are k-robust. Mixing them would hide both behaviors, so they are scored independently.
Findings & conclusions ▸
What the benchmark shows. On the Canis familiaris CanFam3.1 reference, across simulated sequencing workloads, SNIPE's alignment-free QC estimates reproduce alignment-derived ground truth with high fidelity — composite accuracy above 0.96 for every genomic workload and ≥ 0.88 in the worst case.
Genomic QC is recovered without alignment
For whole-genome (150 bp), exome-capture, indel-enriched and metagenomically-contaminated reads, composite scores span 0.96–0.99 (WGS 0.98, exome 0.97, indels 0.98, metagenomic 0.98). Sequence-change rate is the strongest signal: SNIPE's estimate matches the event-based alignment truth at r ≈ 1.0 for whole-genome, indel and metagenomic reads and r ≈ 0.91 for exome. Where a metric carries genuine dynamic range — sequence-change, and depth in coverage-varying designs — SNIPE tracks it closely; where the truth is saturated by construction (mapping and breadth for fully on-reference reads) the estimates agree in value even though a correlation there is uninformative.
Mapping rate reflects on-reference content in mixed samples
The metagenomic workload is dog whole-genome reads spiked with a fixed block of off-reference reads; mapping such a sample to the dog reference recovers only its on-reference fraction. Measured against the appropriate read-level ground truth (mapped reads / total), SNIPE's alignment-free mapping estimate agrees closely — 0.46 versus a true 0.458 — that is, the sketch-based rate reflects the same on-reference fraction the aligner does. The base-level Qualimap rate is computed only over reads that already mapped, so it saturates near 1.0 and is blind to the unmapped fraction; using it as ground truth is what previously made this metric look like a failure. This is agreement on a mixed sample once the unmapped noise is isolated — not an active contamination-detection claim.
RNA is the systematic exception
Scored independently against the transcriptome, RNA is the one workload where accuracy is materially lower (0.88–0.90) and where k-mer choice matters. Its sequence-change accuracy rises from r ≈ 0.70 at k1 = 51 to r ≈ 0.92 at k1 = 21, and the composite peaks near k1 = 21–25 before declining monotonically. This is consistent with edge-mer saturation on a short, isoform-redundant reference: a single change removes roughly k2 consecutive k-mers, so long extensions leave too few surviving k-mers and the inferred per-base rate runs to its ceiling. A residual ~1% floor in the alignment truth itself — cross-isoform mismatches under a splice-unaware aligner — further limits agreement at low divergence. Small core k-mers are therefore the appropriate operating point for transcriptomic input.
Accuracy is robust to k, with one caveat
For genomic workloads the composite is nearly invariant to core k-mer size, edge extension and FracMinHash scale: a single compromise configuration (k1 = 31, k2-extension = 6, scale = 1,000) lands within 0.04 of every workload's own optimum. Per-workload tuning yields a measurable benefit only for RNA (+0.04 by moving to k1 = 21). Searching the full sweep instead of the fixed k1 ∈ {21, 31, 51} grid recovers only marginal accuracy (for example k1 = 25 lifts the full genomic bundle from 0.969 to 0.975), indicating the method is not sensitive to precise parameter choice within a sensible range.
These results are simulation-derived and specific to CanFam3.1; absolute values will shift with reference completeness, read chemistry and divergence. The metrics and configurations emphasized in the manuscript are selected for the reasons stated there — this page previews the entire configuration space so the sensitivity of each conclusion to the k-mer parameters can be inspected directly below.
RNA — scored independently
transcriptome · k-sensitiveRNA is not mixed with the genomic assays. Its accuracy peaks at small k1 and degrades as k1 grows (edge-mer saturation on the short transcriptome reference).