Machine learning models to improve targeting of blood culture testing

Doctors often order blood tests that end up being negative. This wastes valuable hospital resources. Bloodstream infections are a major cause of death. Yet, the primary testing method—blood cultures—often has low positivity rates. Typically, fewer than 10% of cultures yield a pathogen. Turnaround times are also slow, taking 24 to 48 hours. Many cultures are taken from low-risk patients. Meanwhile, some truly infected patients are diagnosed too late.

A new study from the University of Oxford and University College London proposes a solution. The researchers developed a machine learning model to predict blood culture positivity. They suggest hospitals could perform "resource-neutral reallocation." This means swapping low-risk tests for high-risk ones. This increases the number of pathogens found without increasing total testing volume.

The inefficiency of the gold standard

Blood culture remains the diagnostic gold standard. It directly identifies the bacteria causing an infection. However, the current process is inefficient. Clinicians often collect samples without a clear indication. This leads to many "negative" results where no pathogen grows. This creates a data bottleneck in hospitals.

Existing clinical prediction tools try to help. However, they often use simple scoring systems. These systems struggle to capture complex, non-linear relationships. They miss nuances in vital signs and lab results. Many existing models also use small sample sizes. Some are restricted to specific clinical settings. This makes them hard to deploy across different hospitals.

Training an XGBoost classifier

The authors developed a model using XGBoost. This is a gradient-boosted decision tree algorithm. Think of XGBoost like a committee of experts. Instead of one person making a decision, a sequence of specialized models is trained. Each new model focuses on correcting errors made by the previous ones. This helps find subtle patterns in structured medical data.

The researchers used the Infections in Oxfordshire Research Database (IORD). This dataset contains 294,064 blood cultures from 110,210 patients. The modeling process involved several stages:

1. Feature Engineering: The team selected 32 variables across ten domains. These include demographics, vital signs, and blood test results. They used a fine-tuned BERT model to handle unstructured data. BERT is a transformer-based natural language processor. It classifies antibiotic indications from free-text clinical notes.
2. Handling Sparsity: Medical records are often incomplete. The authors used XGBoost's native ability to handle missing values. It uses "sparsity-aware split finding" to route missing data points. This maximizes predictive gain during decision-making.
3. Addressing Class Imbalance: Only 5.6% of the cultures were positive. The authors used a weight parameter to upweight positive cases. This prevents the model from simply guessing "negative" every time.

The model's logic aligns with biology. As shown in, the most influential features include elevated C-reactive protein (CRP) and neutrophil counts.

It also looks for decreased lymphocyte and eosinophil counts. These patterns characterize an immune response to a pathogen.

High yield through strategic reallocation

The authors report high predictive accuracy. The model achieved an AUROC (Area Under the Receiver Operating Characteristic curve) of 0.853 in the temporal test set. The AUROC measures a model's ability to distinguish between classes. In an external validation cohort, the AUROC was 0.847 .

The most striking result involves a simulation of hospital workflow. The researchers modeled a "resource-neutral" scenario. They replaced the 10,000 lowest-risk blood cultures with the 10,000 highest-risk untested emergency admissions. The paper finds this swap yields 627 additional positive cultures. This is a 28.3% relative increase in diagnostic yield [Table 3].

The XGBoost model outperformed existing clinical tools. It showed a higher "net benefit" across various probability thresholds [Figure 3A]. It beat common benchmarks like the National Early Warning Score (NEWS). It also outperformed the modified Shapiro rule [Figure 3B].

Constraints on real-world deployment

The authors highlight several critical trade-offs. First, performance drops when using only immediate data. If restricted to data available at the moment of collection, the AUROC falls to 0.769. This happens because the model relies on lab results. These results might not be ready when a doctor decides to take a sample.

Second, the model is not a standalone diagnostic tool. Its positive predictive value (PPV) is only 15.9%. PPV is the probability that a positive test is actually a true positive. Therefore, a "high risk" prediction is a screening tool. It should guide further investigation rather than replace a diagnosis.

Finally, the study shows performance disparities. The model performed significantly worse in pediatric patients. It also performed lower for patients of Black ethnicity. Performance was also lower in the most deprived socioeconomic areas. The authors suggest this may stem from lower positivity rates in these groups. It may also be caused by lower data coverage.

The verdict: A powerful screening layer

This model is not a replacement for clinical judgment. It is best viewed as a screening layer. It could be integrated into an Electronic Health Record (EHR) system. Such a system manages patient data digitally.

The model could act as a digital safety net. It could flag high-risk patients overlooked during emergency admissions. Because the model is well-calibrated, it is highly useful. Calibration means predicted probabilities match actual observed risks . This allows for reliable individual risk assessment. However, the "front-door" latency issue remains. Until lab results are available instantly, the model's main value is optimizing testing for already admitted patients.

Figures from the paper