AFQuery: A High-Speed, Capture-Aware Engine for Accurate Clinical Allele Frequency Estimation
In clinical genomics, determining the Allele Frequency (AF)—the proportion of a specific genetic variant in a population—is essential for deciding if a mutation is harmless or causes disease. Clinicians rely on standardized guidelines to classify variants. If a variant is extremely rare, it is more likely to be pathogenic (disease-causing). If it is common, it is likely benign.
However, institutional databases are rarely uniform. Over years of operation, a hospital might transition through several generations of Whole Exome Sequencing (WES) kits. Even successive versions of the same manufacturer's product often target slightly different genomic regions. The status quo for calculating AF involves a "naive" approach. This approach assumes every sample in the cohort was tested at every position. This ignores the reality that a sample sequenced with an older kit might not have been "captured" (physically targeted and enriched for sequencing) at a specific location.
As a result, the Allele Number (AN)—the denominator in the AF equation—becomes artificially inflated. When you divide the number of observed mutations by an incorrectly large denominator, the resulting AF is deflated. This creates a dangerous systematic bias. Variants appear rarer than they actually are. This causes them to be erroneously flagged as pathogenic [Figure 1C].
The Problem
Accurate AF estimation is vital because it underpins several clinical classification criteria. One major issue arises when cohorts combine different technologies, such as WGS (Whole Genome Sequencing) and various WES kits. Standard methods compute AN as twice the number of eligible samples at every position. They ignore that WES and panel samples lack coverage outside their specific capture regions.
This kit-driven batch effect is an invisible source of error. Many labs label all samples simply as "WES," regardless of the kit version used. Consequently, the denominator is inflated by samples that never actually looked at the position in question. This leads to a downward bias in frequency estimates. Such errors can push variants into categories that suggest they cause disease, even when they are common in the local population.
How It Works
AFQuery addresses this by shifting from a cohort-wide calculation to a per-position, technology-aware calculation. The architecture relies on three core mechanisms:
- Bitmap Indexing via Roaring Bitmaps: Instead of scanning massive text files, the engine indexes genotype data into Roaring Bitmaps. These are highly compressed bitsets (collections of bits representing presence or absence) stored in Apache Parquet files. This allows the engine to perform lightning-fast intersections between sample metadata (like sex or phenotype) and genotype data.
- Capture-Aware Denominator Construction: For every genomic position being queried, AFQuery consults technology-specific BED files (files containing genomic coordinates). If a sample was sequenced using a kit that does not cover the queried position, that sample is excluded from the AN calculation. This ensures the denominator only counts individuals who were actually capable of providing data for that specific spot.
- Ploidy-Awareness: The engine adjusts the AN based on the biological reality of sex chromosomes. It uses the specified genome build to identify pseudo-autosomal regions (areas of the X and Y chromosomes that behave like autosomes). This ensures the math correctly accounts for the number of alleles present in different sexes.
By combining these steps, AFQuery treats a heterogeneous cohort as a collection of overlapping, technology-specific subsets.
Numbers
The authors demonstrate that AFQuery provides significant advantages in both speed and clinical reliability. Regarding computational efficiency, the paper reports that point-query latency remains effectively constant at approximately 14 ms. This speed is maintained whether the cohort contains 1,000 or 50,000 samples [Figure 1A]. This constant-time scaling is a direct consequence of the bitmap indexing. Furthermore, in bulk export tasks, AFQuery achieved a 33-fold speedup over the industry-standard BCFtools when processing a 2,504-sample cohort [Figure 1D].
The most critical improvements are found in accuracy. In simulated mixed-technology cohorts, the authors find that capture-aware AN reduced the Mean Absolute Error (MAE) of allele frequencies by 8 to 13-fold. Most importantly, in "extreme" scenarios involving a transition between kit versions, the naive method produced a staggering 6.87% of variants with shifts toward pathogenicity. AFQuery reduced these spurious pathogenic-evidence calls by 45-fold. This brought the error rate down to just 0.15% [Figure 1C].
What's Missing
While the results are compelling, the paper leaves several questions unanswered. First, the accuracy gains were demonstrated using simulated technology assignments on Chromosome 22. While this proves the mathematical logic holds, the authors note that the robustness of the engine regarding actual technology-specific "calling" differences requires validation on real-world, mixed-kit datasets.
Second, the scaling benchmarks were produced by resampling from a fixed set of 2,504 samples. While this simulates larger cohorts, it may not fully capture the compression dynamics of true biobank-scale data. Real biobanks contain much higher levels of biological and technical noise.
Finally, the current ploidy model is limited to binary sex. It does not currently account for X-aneuploidies (abnormal numbers of X chromosomes). For clinical applications involving certain chromosomal abnormalities, the current implementation would require further refinement.
Should You Prototype This
Yes, if you manage an institutional genomic database that has grown organically over several years. If your cohort spans multiple generations of sequencing kits or combines WES and WGS data, the risk of systematic misclassification is real. AFQuery offers a serverless, low-overhead way to fix this without requiring heavy Spark or HPC infrastructure. The code is available under the MIT license at https://github.com/babelomics/afquery.