Beyond Gene Reconstruction: Learning Cell Representations through Complementary Transcriptomic Views

Beyond Predicting Missing Values: Teaching AI to Understand Whole Cells

Current AI models for single cells focus on predicting missing gene values. This doesn't always help them understand the whole cell. CoCoS improves this by splitting a cell's genes into two complementary views. It teaches the model to recognize they belong to the same cell. This makes the model better at identifying cell types and gene networks.

Bridging the Gap Between Genes and Cells

The central challenge in single-cell biology is translating raw data into a coherent picture of cellular identity. Most modern foundation models—large-scale AI systems trained on vast biological datasets—use masked expression-value prediction. In this setup, the model is given a partial list of genes and their activity levels. It must then "fill in the blanks" for the missing ones.

This approach helps the model learn how individual genes depend on one another. However, the authors argue it creates an objective mismatch. Predicting a single missing value is a local task. It does not force the model to build a robust, global representation of the entire cell. This is problematic for downstream applications. Identifying a cell's specific type or inferring gene regulation requires a holistic "embedding" (a compressed mathematical summary of the cell's state).

The Limits of Reconstruction

Existing models like scGPT and Geneformer primarily use masked language modeling (MLM). This is a process where parts of the input are hidden and the model must reconstruct them. As shown in [Figure 1a], this focuses the model's attention on gene-level features.

The authors identify a fundamental flaw here. A model can become excellent at reconstructing a specific gene by looking at its neighbors. It may do this without grasping the "cellular program" (the collective biological activity) driving that expression. Attempting to fix this with standard "contrastive learning" introduces new errors. Contrastive learning is a method where a model learns by distinguishing between similar and dissimilar examples.

In computer vision, you might create a "positive pair" by rotating an image. However, in transcriptomics, perturbing gene values can change the cell's identity [Figure 1b]. Additionally, single-cell data is often sparse (containing many zeros). A model might take a "shortcut." It could identify a cell simply by which specific genes are present, rather than their actual expression levels.

The CoCoS Framework

The authors propose CoCoS (Complementary Transcriptomic Views). This framework optimizes whole-cell representations without corrupting biological truth. The method relies on three pillars, summarized in .

First, the authors use Co-expression-Guided Gene Partitioning (GPC). They build a graph showing how frequently genes appear together. They then split each cell into two "views." One view sees one set of genes. The other view sees the remaining complementary set. Both views use original, unperturbed values. Therefore, they represent the same biological state.

Second, to stop the "gene-identity shortcut," the authors implement Expression-Aware Contrast-Set Construction. They create "hard negatives" by taking a cell's actual genes but shuffling (permuting) their expression values. As shown in, this forces the model to pay attention to how much genes are expressed.

Without this, the model might ignore the actual biological signal and only recognize gene IDs.

Finally, the authors introduce Competence-Gated Contrastive Onset. A model cannot relate two different views until it understands how genes work individually. They use a "sentinel" group of cells to monitor progress. The system only activates the contrastive objective once the model shows stable "competence" (accurate reconstruction and diverse representations) .

Superiority in Biological Tasks

The impact of this approach is evident in real-world biological questions. The authors report that CoCoS-GPC achieves a mean k-nearest neighbors (k-NN) accuracy of 74.82% for cell-type annotation across ten datasets [Table 1]. This represents a notable improvement over scGPT (70.99%) and Geneformer (60.64%). Higher k-NN accuracy means the model is better at grouping similar cells together in a mathematical space.

The model also excels in Gene Regulatory Network (GRN) inference. This task determines which genes act as "switches" for others. The study finds that CoCoS-GPC records the highest mean AUROC (0.8733) and AUPRC (0.7120) point estimates among the compared variants [Table 5]. These metrics measure how accurately the model identifies true regulatory connections. This suggests the model learns the underlying logic of cellular regulation.

Where the Edges Are

The framework is not a universal solution. The authors note that the "winner" of the GRN evaluation varies across different biological networks. This means the method's superiority is not uniform in every context.

The complexity of the training process also adds significant overhead. Managing a gated onset and hard negatives requires more careful coordination. While the authors demonstrate success with frozen embeddings, they do not explore how these representations behave in multi-modal models. Such models combine transcriptomics with other data types like proteomics. Finally, the effectiveness of the "competence gate" depends on specific sentinel metrics. This leaves open questions regarding how to tune these thresholds for new biological data.

Figures from the paper