Interpretable MEG Decoding of Perceived Speech: Cortical Sources and the Stimulus Features That Drive Retrieval

Interpretable Deep Learning Decodes Perceived Speech from MEG via Cortical Sources

Researchers have developed AI models capable of looking at brain activity recorded via magnetoencephalography (MEG) and identifying exactly which spoken audio segment a person was hearing. While previous "black box" models achieved impressive accuracy, they functioned as opaque mathematical transformations. Their internal weights matched nothing an electrophysiologist could recognize. This left it unclear whether the AI was truly "listening" to the brain or simply exploiting peripheral artifacts. This paper introduces a redesigned decoder that bridges this gap. It maps neural signals to specific cortical locations (the outer layer of the brain) and temporal dynamics (how signals change over time). By forcing the architecture to respect the physics of the measurement and the physiology of the brain, the authors transform a high-performing retriever into a legitimate tool for neuroscience.

The opacity of the black box

The ability to reconstruct speech from non-invasive brain recordings is a holy grail for neuroprosthetics (devices that restore lost functions) and communication interfaces. Current state-of-the-art approaches use deep networks to map MEG sensor signals—weak magnetic fields measured by a helmet of sensors—into high-dimensional audio embeddings (mathematical summaries of sound). However, these models suffer from a fundamental disconnect between machine learning success and biological meaning.

As noted by the authors, existing decoders turn sensor signals into speech embeddings through a chain of transformations. These transformations do not correspond to cortical locations or neural rhythms. This creates a significant scientific risk. The model might achieve high accuracy by performing "shortcut learning." For instance, eye movements (ocular activity) or heartbeats (cardiac components) can carry stimulus-locked information that mimics linguistic structure. If a model relies on these peripheral signals to succeed, its "accuracy" is a mirage. It tells us nothing about how the auditory cortex processes language. Until a decoder's weights can be read as actual neural populations, the move from a high accuracy score to a neuroscientific claim remains speculative.

Designing a physiologically constrained front-end

To solve this, the authors implement a decoder whose architecture is explicitly constrained by the biophysical reality of MEG. Rather than treating the sensor array as a flat grid, they employ a 3D spatial attention layer parameterized by real spherical harmonics .

Because MEG sensors sit on a roughly spherical helmet, spherical harmonics provide the natural mathematical basis for the magnetic fields they measure.

The architecture follows a structured, factorized approach to decompose the signal into interpretable components:

1. Spatial Attention: The 3D spherical-harmonic layer maps the 208 sensor channels to 270 virtual channels. This respects the geometry of the head.
2. Unmixing and Projection: A $1 \times 1$ convolution mixes these channels. They are then projected into a compact set of $K=25$ branches via a subject-specific layer.
3. Temporal Filtering: Crucially, each branch is equipped with a trainable depthwise temporal filter (a 150 ms kernel). This ensures that each branch is matched to a specific neuronal source not just in space, but in time.
4. Non-linear Decoding: These filtered signals are fed into a shallow convolutional module. This module produces the final MEG-derived embedding.

By reducing the representation to just 25 branches, the authors impose a physiological bottleneck. This forces the network to prioritize only the most informative neural sources. To prevent "shortcuts," they also remove ocular and cardiac components from the data before training begins.

Evidence from cortical sources and stimulus features

The effectiveness of this constrained design is demonstrated through its ability to recover meaningful biological signals. On the MEG-MASC dataset, the model achieves 39.75% $\pm$ 0.34% Top-1 accuracy among 1005 candidates. This means the model correctly picks the exact audio segment about 40% of the time. Remarkably, the authors report that this is accomplished using approximately 20 times fewer trainable parameters than previous large-scale decoders.

The true value of the model lies in its interpretability. By mapping the learned weights back to cortical space using Minimum Norm Estimation (MNE), the authors recover generators consistent with known speech-perception networks. As shown in, the dominant spatial patterns involve bilateral auditory cortices, the frontal lobe, and the medial temporal lobe.

Further clustering of these patterns reveals three distinct functional groups: * Superior temporal clusters that track the speech envelope (the overall shape of the sound).

• Dorsal-stream sites (supramarginal/central sulcus) related to phonological and sensorimotor functions.
• Frontal clusters (left frontal operculum) involved in predictive and conceptual processing.

Beyond localization, the authors use "paired MEG occlusion"—replacing specific segments of the MEG signal with matched "donor" intervals—to see what the model actually uses. They find that 15 of 19 tested features contribute to retrieval. The largest effects are seen for silence, high loudness, vowels, and acoustic onsets . Intriguingly, the model's performance drops significantly when presented with randomly ordered word lists compared to coherent narrative speech. This suggests the network relies on the predictive, structural information inherent in natural language to track the audio.

Limits of the interpretable approach

While the model succeeds in bridging the gap between AI and electrophysiology, several caveats remain. First, the interpretation of the "front-end" relies on the assumption of linear space-time factorized processing. This mathematical simplification means the model might miss more complex, inseparable cortical waves. In these cases, space and time are inextricably linked.

Second, the "paired occlusion" method does not provide a perfectly isolated causal measure. Features like vowels, loudness, and acoustic onsets often overlap or co-vary in natural speech. The authors admit that the observed effects cannot be viewed as entirely independent contributions. The results identify which features the decoder uses. However, they do not allow for a perfectly calibrated ranking of how much each individual feature contributes to the total signal.

Finally, the anatomical mapping is conducted via a common-template approach. While this allows for cross-subject aggregation, it introduces spatial uncertainty. A more precise mapping would require applying the pipeline to individual subject anatomies. This would ensure that the localized sources are not blurred by the registration process.

The verdict: A new instrument for discovery

The research presents a definitive shift in how we should approach neural decoding. The authors prove that you do not need massive, opaque architectures to achieve high-performance speech retrieval. In fact, a compact, physically grounded model is more efficient and more useful. By aligning the decoder's architecture with the physics of MEG and the biology of the cortex, they have turned a "black box" into a microscope.

This is a "yes" for the utility of constrained deep learning in neuroscience. The model's ability to compress the 768-dimensional wav2vec target into just 12 learned dimensions without losing accuracy proves that the relevant information for the brain is highly compact. For practitioners developing speech neuroprostheses, this work highlights that successful systems must account for both rapid acoustic transients and broader linguistic context. Code and data are available at https://github.com/ivsemenkov/LISA/.

Figures from the paper