MAAT: Multi-phase Adapter-Aware Targeted Unlearning

MAAT: Solving the Causal Knowledge Gap in LLM Unlearning

Current methods for making AI forget information often fail at explaining "why" things happen. They also accidentally break the AI's general knowledge. Researchers have developed a new tool called MAAT. It surgically targets specific parts of the AI's memory. This erases facts without making the model act confused or lose other useful skills.

In machine unlearning, the goal is to remove specific knowledge from a Large Language Model (LLM). We must keep the rest of its capabilities intact. This is critical for regulatory compliance, like GDPR. Users may demand the removal of personal data. Until now, the industry has relied on benchmarks that primarily test "What" or "Who" questions. These are simple entity-attribute lookups. However, a massive gap exists in evaluating "Why" questions. These probe the causal and relational chains that underpin intelligence. Previous methods have struggled here. They often force a bad choice: the model forgets the fact but loses its reasoning, or it retains reasoning but fails to forget the target information.

The Problem

The status quo in unlearning evaluation is structurally broken. Most existing benchmarks, such as CounterFact or TOFU, are heavily skewed toward entity-centric knowledge. As the authors demonstrate in [Table 1], "Why"-type questions—which test causal reasoning—comprise less than 0.06% of CounterFact and only 1.2% of TOFU. This creates a blind spot. An unlearning method can fail catastrophically on causal logic. Yet, it can still appear highly successful in aggregate metrics.

The authors argue that causal knowledge is qualitatively harder to unlearn. This is due to multi-hop reasoning chains and gradient dilution (where the learning signal becomes too weak to be effective). While a "What" question might have a four-token answer, "Why" answers average 40.1 tokens [Table 7]. They involve complex relational dependencies. When applying standard gradient ascent (maximizing the loss on the forget set), the signal spreads too thin. This leads to a failure in the forget-retain Pareto frontier (the optimal balance between erasing data and keeping utility). Aggressive methods like Adapter Negation (AN) achieve high forgetting but trigger catastrophic collapse of all other knowledge. Conversely, conservative methods like Gradient Ascent with KL Divergence (GA+KL) preserve retention by simply failing to forget the causal links.

How It Works

MAAT (Multi-phase Adapter-Aware Targeted Unlearning) moves away from uniform gradient pressure. Instead, it performs "structured adapter surgery" directly on LoRA (Low-Rank Adaptation) weights. The base model remains frozen .

The framework operates in three distinct phases:

1. Gradient-Projected Unlearning: To prevent the unlearning process from eroding retained knowledge, MAAT applies a conditional orthogonal projection. If the forget gradient ($g_f$) conflicts with the retain gradient ($g_r$), the algorithm projects $g_f$ to be orthogonal to $g_r$. This ensures the update only moves in directions that do not actively harm the retained manifold (the space of useful knowledge).
2. Structural Compression and Task Negation: This phase targets the internal geometry of the adapter. First, the authors use Singular Value Decomposition (SVD) to score rank dimensions in the MLP (Multi-Layer Perceptron) modules. They score them based on their activation by the forget set. They then prune the top-$\rho$ fraction of these dimensions. Second, they perform "task vector negation." They construct a forget task vector ($\tau^F_l$) from the highest-scoring dimensions. They then subtract this vector from the LoRA $B$ matrix. Unlike previous methods that negate the entire adapter, MAAT confines this negation to the specific rank dimensions responsible for the forget-set signal.
3. Multi-Objective Utility Repair: Finally, a hybrid repair engine restores model utility. It uses a complex loss function. This combines KL divergence (staying near the reference model) and hidden-state distance (preserving internal representations). It also uses a negative entropy term. This term is crucial. It maximizes entropy on the forget-set predictions. This effectively "locks" the door to prevent the model from re-learning the forgotten content during the repair phase.

Numbers

The authors report that MAAT reaches a new operating point on the forget-retain Pareto frontier. On the Llama 3.2-3B model using the 5WBENCH benchmark, MAAT achieves an average Forget Success Rate (FSR) of 77.4% across all 5W categories. It also achieves an average Retain Success Rate (RSR) of 71.6% [Table 3].

The most significant result is found in the "Why" category. While baselines struggle to pass the 60% threshold, MAAT is the only method to simultaneously exceed 60% FSR and 60% RSR on all five 5W categories on Llama 3.2-3B. Specifically, it achieves 63% FSR and 65% RSR for "Why" questions [Table 3]. This means it successfully erases the causal reason while keeping the model's general reasoning intact. Compared to Retain-Only Fine-Tuning (RO-FT), MAAT delivers a +36.4 point gain in retention at zero cost to forgetting on Llama 3.2-3B.

The ablation studies clarify the necessity of the multi-phase approach .

The authors show that replacing task vector negation with simple attention pruning (Condition C) causes a massive drop in retention. Retention falls from 76% down to 54%. This happens because pruning attention modules destroys the model's fundamental instruction-following pathways.

What's Missing

The paper is technically dense, but there are gaps for practitioners:

• Architecture Sensitivity: Performance is architecture-dependent. MAAT performs exceptionally on Llama 3.2-3B. However, it shows lower separability and performance on Gemma 3-4B [Table 3]. This suggests that SVD-based rank pruning depends on how the base model encodes knowledge. You cannot treat these hyperparameters as "set and forget" across different model families.
• Judge Reliability: Evaluation relies on an "LLM-as-a-Judge" (Qwen 2.5-7B). This captures semantic nuances better than exact-match metrics. However, it introduces potential calibration errors. A practitioner might want to verify these results with manual audits. This ensures the model hasn't just learned to avoid certain keywords.
• Scope of Operations: The framework is strictly designed for unlearning. It does not address knowledge insertion (adding new facts) or modification (updating old facts).

Should You Prototype This

Yes, if you are dealing with causal or relational data.

If your unlearning needs are limited to simple "What is X?" queries, standard gradient ascent is likely sufficient. It is much easier to implement. However, if you must erase complex reasoning chains or "Why" relationships, MAAT is worth the effort. This is common in legal, medical, or sensitive social contexts. The ability to target specific LoRA rank dimensions via SVD avoids the "catastrophic collapse" seen in simpler negation methods. Code is reportedly available; see the paper for the canonical link. Since the method fits on a single consumer GPU (≤24 GB VRAM), the barrier to entry is low.

Figures from the paper