Feed 0% source
AI/ML AI-generated

LaRA: Layer-wise Representation Analysis for Detecting Data Contamination in RL Post-Training

Generated by a local model from a scientific paper, claim-checked against the full text. Provenance is open by design.

When training large language models (LLMs) using reinforcement learning (RL), there is a growing risk of data contamination. This happens when the model memorizes test questions instead of learning to reason. Traditional detection methods often fail here. They typically check if the model assigns high probability to a specific answer. However, RL optimizes for rewards over entire reasoning trajectories. This makes token-level likelihood (the probability assigned to a single word) an unreliable signal.

The authors propose LaRA. This method moves the search from the output to the model's internal "brain waves." Instead of looking at final text, LaRA examines hidden layers (the intermediate mathematical representations inside a transformer). It observes how these representations react to small changes. A model that has truly learned a concept should be robust to minor wording shifts. A memorized sample, however, will show telltale signs of rigidity and geometric collapse.

The Problem

Current detection relies on output-level signals. These include log-likelihood and entropy (a measure of a model's uncertainty). In standard pre-training or supervised fine-tuning (SFT), these signals work well. This is because the training objective is to maximize token likelihood.

RL post-training breaks this paradigm. As shown in, RL shapes behavior through trajectory-level rewards.

Figure 1
Figure 1. Output vs. Representation-level detection. Output-level signals are sensitive to output miscalibration, such as overconfidence of superficially plausible tokens. In contrast, representation geometry provides more robust and interpretable contamination signals.

This can cause severe miscalibration. The model might produce highly confident but superficial outputs. These outputs mask the fact that the model is just reciting a memorized path. Because RL does not optimize for token-wise likelihood, old tools like perplexity (PPL) become unreliable. We need to probe the model's internal geometry to bypass these output distortions.

How It Works

The LaRA framework treats contamination detection as a Membership Inference Attack (MIA). This is a method to determine if a specific sample was part of the training set. The core idea is to perturb a question and observe how the hidden states ($h_\ell$) shift across transformer layers.

The authors use three geometric metrics:

  1. Representation Shift Magnitude (RSM): This measures perturbation sensitivity. The system removes key information from a question. It then calculates the Euclidean norm (the straight-line distance) of the change in hidden representations. A high RSM means the model is overly dependent on specific, fragile information.
  2. Directional Collapse (DC): This measures organizational diversity. It calculates the cosine alignment (a measure of how much two vectors point in the same direction) between a specific perturbation and the average direction of similar questions. High DC indicates that the model's responses are collapsing into a narrow, low-dimensional direction.
  3. Representation Stability Index (RSI): This measures local rigidity. The system generates multiple paraphrased versions of a question. It then quantifies how much the representations vary. A high RSI means the model's internal state is unexpectedly invariant to meaning-preserving changes.

As shown in, these metrics are extracted across all transformer layers.

Figure 2
Figure 2. Overview of LaRA, the proposed layer-wise representation geometry analysis framework. Given an original question q0, we generate semantically similar questions and remove shared key information to construct perturbed variants.

The SLaRA protocol then standardizes these values. It uses Median Absolute Deviation (MAD) to ensure outliers do not skew the detection threshold. Finally, it aggregates these into a single anomaly score.

Numbers

Moving to the representation layer provides a cleaner signal than output-level heuristics. The authors report that LaRA achieves up to a +9.6% improvement in AUC (Area Under the ROC Curve, a measure of classification accuracy) compared to the strongest prior output-level methods.

More importantly for high-precision needs, the method shows a 3.5× higher True Positive Rate at a 5% False Positive Rate (TPR@FPR=5%) than previous baselines. This means the detector catches significantly more contaminated samples while keeping false alarms very low.

Looking at the layer-wise dynamics in, contamination is not localized to one part of the model.

Figure 3
Figure 3. Layer-wise representation geometry patterns of RL post-trained model. We compare layer-wise representation geometry between contaminated and clean samples under input perturbations.

It produces progressive geometric deviations. For example, contaminated samples show a sharp spike in RSM around layers 7–9. Ablation studies in Table 2 show that while DC provides a strong standalone signal, the full combination of all three metrics is necessary for peak performance.

What's Missing

There are several gaps to consider:

  • Computational Overhead: LaRA is more expensive than output-level methods. It requires generating multiple semantic neighbors and paraphrased variants. It also requires multiple forward passes to extract hidden states.
  • Unclear Causality: The paper shows that these geometric shifts correlate with contamination. However, it does not fully explain why RL specifically drives these exact geometric shapes.
  • Edge Case Overlap: Detection is not perfect. As seen in [Table 3], some member samples have representation geometries that closely overlap with the "clean" manifold. These samples may go undetected.

Should You Prototype This

Yes, but as a secondary audit layer.

If you are shipping reasoning models, do not rely solely on output-level metrics. The authors show that SLaRA and Self-Critique (SC) are complementary. Combining them (SC + SLaRA) achieved the strongest overall performance in several tests.

Do not use this for real-time monitoring. The cost of generating perturbations and extracting layer-wise states is too high for production inference. Instead, integrate it into your offline evaluation pipeline. Use it to vet your RL training sets and verify that your reasoning scores are genuine.

Figures from the paper

Figure 4
Figure 4. Beta sweep over score mix. Default β = 0.65. only the SLaRA, with performance degrading as β increases. For OLMO, AUC is highest at β = 0, while TPR@FPR=5% peaks at β = 0.65.
Figure 5
Figure 5. (a) Correlation plot of output-level signals and SLaRA. (b) Analysis on the # of perturbations
Figure 6
Figure 6. Perturbation type analysis. Results show that SLaRA is robust to perturbation types. Analysis on Perturbation Types. The perturbation analysis in Figure 6 shows that LaRA remains relatively robust across different perturbation types, with all variants achieving comparable AUC values between 0.56
Novelty
0.0/10
Impact
0.0/10
Overall
0.0/10
#ai#nlp#reinforcement_learning#data_contamination#representation_learning