AI segmentation requires accounting for brain size to maintain performance on developmental MRI cohorts

AI Brain Segmentation Improved for Infants via Rescaling and Cropping Pipelines

Artificial intelligence tools used to map brain structures often fail on infants. This happens because their brains are much smaller than the adult brains the AI was trained on. Researchers found that by virtually "stretching" infant brain scans to adult size and cropping them to a standard view, they could make the AI much more accurate. This allows scientists to study brain development from birth through adulthood using a single, consistent method.

The mismatch between adult training and infant biology

In computational neuroanatomy, segmentation is a foundational task. This is the process of partitioning an MRI scan into distinct anatomical regions like grey matter, white matter, or the thalamus. Modern deep learning tools, most notably SynthSeg, aim to provide "contrast-agnostic" segmentation. This means the AI identifies structures regardless of variations in signal intensity. Such variations are caused by different MRI scanner settings or biological changes in tissue composition.

However, a fundamental problem persists. Most of these models are trained on adult datasets. During the first two years of life, the human brain undergoes a massive transformation. Total cerebral volume (TCV, the total volume of the cerebrum) increases by a factor of four to five. The very appearance of tissues shifts as white matter myelinates (the process of coating nerve fibers in insulating fatty tissue).

Because SynthSeg was trained on adult cohorts, it expects a specific scale and field of view (the spatial extent of the image captured by the scanner). As the authors demonstrate, when presented with the much smaller brains of infants, the model fails. In the infant period (under 3 months), the original SynthSeg pipeline achieved a 0% pass rate for automated quality control (QC) [Figure 1C]. This means none of the scans met the minimum quality threshold. Even as children grow, the performance remains inconsistent. Accuracy trails off as brain size decreases [Figure 1E].

Bridging the scale gap with geometry

The researchers hypothesized that the failure was not due to the AI's inability to recognize brain shapes. Instead, it was due to the extreme scale discrepancy. To fix this, they developed a preprocessing pipeline. This pipeline moves infant scans closer to the "adult" distribution the model expects. The approach consists of two distinct mechanical stages:

1. Age-specific rescaling: The authors used established normative trajectories of brain growth to calculate a scaling factor for each infant. They adjusted the voxel-size (the 3D equivalent of a pixel) in the MRI header. This makes the infant's brain volume mathematically approximate that of an adult.
2. Field-of-view cropping: Infant MRI scans often capture much more of the surrounding anatomy, such as the neck and shoulders. The authors used SynthStrip (a tool for removing non-brain tissue) to create a brain mask. They then cropped the image to a 25mm box around that mask. This ensures the AI looks at a window of space comparable to its adult-centric training .

By applying these geometric transformations, the researchers essentially "tricked" the model. The AI sees a brain that looks biologically plausible according to its training history.

Quantifying the recovery of accuracy

The effectiveness of this pipeline was validated using the Baby Open Brains (BOBs) repository. This dataset provides "ground truth" segmentations. These are labels meticulously drawn by human experts. The authors measured success using the Dice coefficient. This is a statistical metric ranging from 0 to 1 that quantifies the spatial overlap between the AI's prediction and the expert's map. A score of 1 represents perfect overlap.

The results show a dramatic recovery in both precision and reliability. The "rescale + crop" pipeline increased the Dice score significantly compared to the original method [Figure 3A]. More importantly, this improvement translated into the automated QC scores. Researchers rely on these scores to filter large datasets. While the original pipeline saw only 36% of scans aged 0–2 years pass the quality threshold, the new pipeline pushed that number to 91% [Figure 4C]. This jump from roughly one-third to nearly all scans is a massive gain for large-scale studies.

Crucially, the authors found that rescaling alone improved the Dice score (accuracy). However, it actually depressed the automated QC scores. This happened because the QC network expects an adult-sized brain. It perceives the extra "empty" space in an uncropped infant scan as a failure. Only by combining rescaling with cropping did the automated QC scores align with the actual biological accuracy of the segmentation [Figure 3B].

Limitations of geometric scaling

While the "rescale + crop" method is a powerful workaround, it is not a perfect biological simulation. The authors note that a neonatal brain is not simply a shrunken adult brain. Biological development is non-linear and multi-dimensional. For instance, cortical thickness and the depth of the sulci (the grooves on the brain's surface) follow unique developmental trajectories. Simple volumetric scaling cannot capture these complexities.

Furthermore, the efficacy of the rescaling step depends on the accuracy of the normative growth models. If these models are inaccurate for a specific sub-population, the rescaling will introduce systematic errors. Finally, while the pipeline handles the geometry of the brain, it does not change the underlying training data. It remains a patch for a model that lacks native understanding of pediatric neuroanatomy.

The verdict: A vital bridge for longitudinal studies

The "rescale + crop" pipeline offers a computationally efficient way to bypass the need for expensive model retraining. It provides a single, consistent pipeline that functions from birth to adulthood. This removes "methodological discontinuities." These are the jarring jumps in data quality that occur when switching between infant-specific and adult-specific software. Such jumps have historically plagued longitudinal studies.

For those interested in implementing this, the authors have made the code and the pipeline openly available at https://github.com/imaginelab/SynthSeg-resize-and-crop. This work serves as a critical reminder. In the era of AI, the "data distribution" is not just about the type of signal. It is also about the physical scale at which that signal exists.

Figures from the paper