Beyond Local Inspection: Global, Guideline-Grounded Evaluation of Post-hoc XAI Methods for ECG Classification

Can We Trust the Heatmaps?

Researchers have found that many AI explanation tools for heart monitoring (ECG) are misleading. Instead of looking at the actual medical problem areas, these tools often just highlight the largest electrical spikes in the signal. By using official medical guidelines as a benchmark, the authors of this paper developed a better way to ensure AI explanations are actually clinically useful.

Does the explanation follow the pathology or the physics?

The central question investigated by Gumpfer et al. is whether post-hoc Explainable AI (XAI) methods—tools designed to show which parts of an input drove a model's decision—actually highlight the clinically relevant features of an electrocardiogram (ECG). In cardiology, diagnosing conditions like myocardial ischemia (a lack of blood flow to the heart muscle) depends on subtle, low-amplitude changes in specific temporal segments, such as the ST segment.

The authors wanted to know if the "relevance" assigned by these methods aligns with established medical guidelines. Or, are the methods simply being hijacked by the sheer physical magnitude of the signal? If an explanation method highlights a massive R-peak (the sharp upward spike in a heartbeat) while ignoring the tiny, diagnostically vital ST segment, it fails to audit the model's actual reasoning.

The trap of local visual plausibility

Until now, the field has largely relied on "local inspection." This is the practice of looking at a single prediction and its corresponding heatmap to see if it "looks right" to a human. This approach assumes that if a heatmap overlaps with a known feature, the method is working. However, the authors argue that this creates a dangerous feedback loop. Visual plausibility often reflects agreement with human expectations rather than faithfulness (how accurately the explanation tracks the model's internal logic) to the model’s decision process.

As illustrated in, a local explanation might appear to highlight a relevant region in a single heartbeat.

But this view is deceptive. Without an aggregated perspective, one cannot tell if the method is systematically identifying the pathology. One cannot tell if it is merely reacting to the signal's amplitude (the height of the wave). The "cracks" in the previous paradigm are most evident when methods transferred from computer vision are applied to 1D physiological signals. In these signals, the relationship between amplitude and diagnostic meaning is fundamentally different.

Testing against clinical gold standards

To move beyond subjective visual checks, the authors developed a global, guideline-grounded framework. Instead of judging single samples, they aggregate explanations across many heartbeats. They compare these against "reference masks"—temporal regions defined by international cardiology guidelines [Figure 2a].

The investigation involved 13 different gradient-based attribution methods. These were applied to four binary CNN classifiers (convolutional neural networks) trained on the PTB-XL dataset. The researchers categorized the pathologies into two distinct groups to stress-test the methods. They looked at high-amplitude morphology patterns (like bundle branch blocks). They also looked at low-amplitude segments (like the PR interval or ST segment). This setup was designed to see if methods would "collapse" onto high-amplitude peaks when asked to find low-amplitude features. They quantified this using Spearman’s Rank Correlation Coefficient (SCC) to measure how much the explanation's relevance tracked the raw input amplitude. They also used "Normalized Coverage" (NCov) to measure how well the explanations hit the guideline-defined targets.

Amplitude-driven bias and the SIGN solution

The results reveal a systematic failure. Many popular XAI methods are heavily biased by signal amplitude. The paper reports that methods like Integrated Gradients and LRP-$\epsilon$ showed high mean SCC values (up to 0.69). This means their explanations were essentially tracking the shape of the raw ECG signal rather than the model's decision logic.

This bias is catastrophic for low-amplitude diagnostics. For myocardial ischemia, the authors find that LRP-$\epsilon$ assigns only 4.6% of its relevance to the ST segment. In contrast, the SIGN-adjusted variant (LRP-$\epsilon$/SIGN) assigns 63.8% [Table 1]. The study shows that nine of the thirteen methods actually fell below chance for at least one condition. This means they were systematically avoiding the correct diagnostic regions.

However, the paper identifies a clear path forward. The SIGN-adjusted methods replace the input factor in the gradient calculation with its sign. This discards magnitude while keeping direction. These methods consistently outperformed the others. Specifically, LRP-$\epsilon$/SIGN achieved the highest "worst-case" normalized coverage (NCovmin) of 41.5%. This proves that these lightweight adaptations can restore clinical relevance even in the most challenging, low-amplitude scenarios.

Moving from heatmaps to clinical audits

The implications of this work are significant for the deployment of medical AI. If this finding generalizes to other physiological signals, it implies that many "interpretable" medical AI tools may provide a false sense of security. We are seeing "clever Hans" explanations. These are heatmaps that look convincing because they follow the signal's energy. However, they can hide the fact that the model might be relying on irrelevant artifacts.

I read this as implying three distinct shifts in practice: 1. Metric Shift: Practitioners should stop relying on mean coverage or local visual inspections. A method can have high average performance by succeeding on easy, high-amplitude tasks. However, it can still fail catastrophically on critical, low-amplitude ones. "Worst-case" reliability (NCovmin) is a more critical metric for clinical reliability. 2. Methodological Shift: When working with time-series data, standard gradient-times-input methods should be viewed with skepticism. Using SIGN-based adaptations is a way to mitigate amplitude-driven biases in time-series explanations. 3. Audit Shift: XAI should be treated as a tool for auditing the model, not just verifying the explanation. The paper notes that while they measured clinical plausibility, they did not prove faithfulness. A high coverage score does not guarantee the model is using that region. It only proves the method is capable of finding it.

The paper does not show that these biases are universal to all possible architectures. However, the systematic nature of the failure across 13 methods suggests it is a fundamental property of the math behind gradient-based attribution. An obvious follow-up experiment would be to test whether these amplitude-driven biases persist in transformer-based architectures. These models handle temporal dependencies differently than the CNNs used here.

Figures from the paper