Hierarchical learning creates invariant schema within plastic neural networks.

Most artificial intelligence learns by constantly changing all its internal connections to minimize errors. While efficient for specific tasks, this process is fundamentally unstable. Learning something new often overwrites what was previously known. This is called "catastrophic forgetting." In contrast, the human brain maintains a delicate balance. It uses plasticity (the ability of synapses to strengthen or weaken) to learn new information. At the same time, it preserves a stable underlying reasoning framework. This structure is known as a "schema."

How neural circuits form and protect these schemata from being overwritten remains a major open question. Current state-of-the-art models, such as large language models, lack these protected structures. This may explain why they struggle with consistent reasoning despite their impressive capabilities. A new study in the Journal of Computational Neuroscience proposes that the secret lies in the hierarchy of the learning rule itself.

The instability of end-to-end learning

The prevailing paradigm in deep learning is end-to-end backpropagation. In this framework, an error signal is calculated at the output. It is then propagated backward through every layer to adjust every weight. Both the ENN and the backpropagation models in this study used two hidden layers with 3 interneurons (intermediate processing units) each. They also utilized the tanh (a hyperbolic tangent) activation function.

Because every parameter is treated as a variable to be optimized, the network has no mechanism to protect specific logical structures. As the authors demonstrate, this approach leads to continuous weight drift. When a backpropagation-trained network sees more data, its weights continue to fluctuate. They never settle into a permanent configuration. This lack of stability means the network's "knowledge" is always in flux. This malleability prevents the emergence of the reliable cognitive frameworks seen in human intelligence.

Topological protection through hierarchical training

The researchers propose an alternative: Essence Neural Networks (ENNs). These use a hierarchical, layer-wise learning framework. Instead of adjusting everything at once, the ENN breaks learning into discrete stages. The mechanism functions through two primary components:

1. Differentia Neurons: The first layer of neurons acts as separators. Using a linear support vector machine (SVM)—a method that finds the optimal hyperplane (a decision boundary) to divide data—these neurons distinguish between pairs of data clusters.
2. Subconcept Neurons: The second layer takes the outputs of the differentia neurons as inputs. These neurons approximate specific regions of the input space defined by the first layer.

This structure creates "topological protection." As shown in, more training data may cause differentia neurons to rotate their decision hyperplanes. However, as long as the fundamental data clusters do not merge or split, the subconcept neurons do not change their weights. They only need to know which side of the existing hyperplanes the data falls on. This effectively "digitizes" the downstream logic. It turns a shifting landscape of weights into a stable, invariant schema.

Evidence of stability and biological alignment

The authors trained both ENN and backpropagation models on a visual boundary-detection task. The networks had to identify "soft" or "hard" transitions in video clips. The results highlight a stark divergence in how these models organize information.

The authors report that ENNs achieve a higher average test accuracy of 62.3% (±3.8%). This is a significant improvement over the 56.1% (±4.7%) achieved by backpropagation models. More importantly, the ENN's hidden layers converge to a stable schema even with sparse initial training. Once the core logic is established, additional data only refines the upstream input representations. The "cognitive" layers remain untouched.

Beyond accuracy, the ENN demonstrates superior robustness. When subjected to parameter noise or the removal of important input features, the ENN retains its performance much better than backpropagation models .

Crucially, the study finds that the ENN's internal organization mirrors human biology. In human medial temporal lobe (MTL) recordings, researchers observe two distinct cell types. "Boundary" cells fire at the onset of a transition. "Event" cells fire later for harder transitions. The hierarchical ENN reproduces this functional segregation. The first layer acts as the boundary detector, while the second layer distinguishes the boundary type .

Limitations of the hierarchical model

While the results are compelling, the study is constrained by several factors. First, the investigation is limited to feed-forward neural network architectures. While these are common in AI, they lack the recurrent (looping) connections found in biological brains.

Second, the input representation used in the simulation involves four discrete frames. This lacks the fluid temporal continuity of actual cinematic motion. This simplification may mask how a continuous stream of sensory data would interact with a schema. Finally, the task relies on unimodal visual inputs. Human boundary detection is a multimodal process involving sound and context. It remains unknown if this hierarchical protection holds when integrating disparate sensory streams.

A blueprint for robust cognition

The verdict is clear: hierarchical learning is a powerful mechanism for generating stable, interpretable, and biologically plausible cognitive structures. By decoupling the refinement of sensory features from the preservation of logical schemas, ENNs avoid the trap of continuous weight drift.

For engineers, this suggests that moving away from monolithic gradient descent could be key. Structured, layer-wise learning might help build AI that learns continually without forgetting. The ability to "distill" these stable ENNs into concise, discrete circuits [Figure 4e] also offers a path toward highly interpretable models. The challenge moving forward will be integrating these hierarchical principles into the massive, multi-modal architectures of modern AI.