Architectural Invariance of Internal Language Representations

24 concepts24 papers1 proposal

When we peek inside a large language model, we often find remarkably clean, human-interpretable structures. Probing studies have revealed that models organize syntax hierarchically, isolate specific concepts in distinct neural subspaces, and even map different languages into a shared semantic space.

But these discoveries raise a fundamental question: Are these internal representations universal laws of neural language processing, or are they fragile artifacts of specific network architectures? If a structural pattern occurs in an encoder-only model like BERT, will it persist in an autoregressive decoder like Llama, a sparse Mixture-of-Experts (MoE) system, or a state-space model? Understanding architectural generalizability is crucial. If internal representations do not generalize across architectures, our interpretability insights are merely narrow anecdotes about specific models rather than universal principles of artificial intelligence.


The Objective Dilemma: Encoders vs. Decoders

Early mechanistic interpretability focused heavily on masked language models (encoders). In a seminal paper, Tenney et al. demonstrated that BERT internally reconstructs the classical NLP pipeline—processing parts of speech in early layers, syntax in middle layers, and complex semantics in higher layers. Subsequent work by Starace et al. showed that these models jointly encode related syntactic categories across languages, while Mohebbi et al. found that encoders anchor specific linguistic functions in distinct "playmaker" tokens.

Masked Language Model (Encoder)        Autoregressive Model (Decoder)
[Layer 1: POS] -> [Layer 12: Syntax]   [Layer 1: ???] -> [Layer 32: ???]
       Is this hierarchy invariant across training objectives?

However, we cannot assume these insights automatically apply to generative, autoregressive decoders. The fundamental training objective shapes how information flows through a network. For example, Swamy et al. noted that knowledge graph extraction pipelines optimized for masked language models face significant consistency challenges when applied to autoregressive architectures. Similarly, while encoder-based multilingual models maintain robust internal representations of mixed-language syntax, as shown in Code-Mixed Probes Show How Pre-Trained Models Generalise on Code-Switched Text, it remains an open question whether generative decoders organize code-switching through the same internal mechanisms.


Topology and Routing: Dense vs. Sparse Models

Even within the same model family, changes in structural topology alter how knowledge is localized. In standard dense transformers, complex cognitive tasks map to highly localized circuits. Zhao et al. identified that dense models process multilingual queries via a three-stage internal translation workflow governed by a tiny fraction (~0.1%) of specialized neurons. As these dense models scale, Zeng et al. observed that language-specific regions shrink, and the model increasingly relies on a shared, language-agnostic "Lingua Franca" semantic space.

Dense Transformer: Fixed Path          Sparse MoE: Dynamic Routing
   [Token] -> [Layer] -> [Output]         [Token] -> [Router] -> [Expert 3] -> [Output]
                                                              -> [Expert 7] 
       Does the language-agnostic "Lingua Franca" space survive dynamic routing?

But what happens when we introduce sparsity? In a Mixture-of-Experts (MoE) architecture, tokens are dynamically routed to different sub-networks. As Ju et al. point out, current findings on how models migrate context knowledge from entity tokens to output tokens are restricted to dense families; we do not yet know if these patterns generalize to sparse architectures. Dynamic routing might shatter the clean, localized semantic spaces observed in dense models, distributing concepts across experts in ways that defy current localized probing techniques.

Beyond transformers entirely, alternative architectures like state-space models introduce completely different computational boundaries. Amir Hossein Kargaran discovered that decoder-only code models use English as an internal pivot language, separating general syntax in lower layers from language-specific outputs in upper neurons. However, the author emphasizes that verifying whether this pivot mechanism persists in non-transformer topologies like state-space models remains a critical frontier.


The Measurement Problem: Do Our Probes Generalize?

Often, what appears to be a failure of architectural generalizability is actually a failure of our diagnostic tools. If a probe fails to find a syntactic hierarchy in a new model, is the hierarchy missing, or is the probe simply blind to it?

The tools we use to measure internal states are highly sensitive to the assumptions built into them:

  1. Linguistic Framing Bias: Kuznetsov et al. demonstrated that the choice of formal linguistic theory used to label data can completely flip probing results, making a layer look highly "syntactic" or highly "semantic" based purely on task framing.
  2. Information Redundancy: Standard linear probes often underestimate a model's internal structure because networks distribute information redundantly. By introducing dropout probes, Tucker et al. proved that models rely on syntax far more than traditional probes suggest.
  3. Methodological Mismatch: Probing methodologies do not translate seamlessly across architectures. Li et al. showed that while classifier-based probes work well for encoders, autoregressive models require "Probing via Prompting" to accurately reveal their internal knowledge.

This diagnostic friction makes it difficult to benchmark structural comprehension globally. As Wijesiriwardene et al. argue, to truly evaluate complex reasoning and syntax across the field, we must deliberately extend our diagnostic tools to accommodate diverse architectures like encoder-decoder models and hybrid variants.


Why Invariance Matters

The quest for architectural invariance echoes historical transitions in traditional natural language processing. In early parsing research, algorithmic shifts frequently invalidated engineering breakthroughs. For instance, Ficler et al. and Habash et al. both demonstrated that highly effective feature engineering—such as conjunction symmetry or functional morphology—yielded volatile results when moved between transition-based and graph-based parsing paradigms.

Today, the stakes are much higher. We rely on internal representations to align models with human values, extract factual knowledge graphs, and steer complex behaviors like Theory of Mind, as explored by Jafari et al.. If these internal states are highly architecture-dependent, an alignment or safety mechanism developed for a dense transformer might fail catastrophically if deployed on a sparse MoE or a state-space model.

Similarly, unsupervised structure extraction techniques, such as the self-attention parsing methods developed by Xiong et al., or cross-lingual transfer predictors, like those analyzed by Muller et al., risk becoming obsolete if they cannot adapt to changing underlying topologies. By systematically testing for architectural invariance, machine learning researchers can move past model-specific quirks and uncover the universal mathematical principles governing how neural networks represent human language.

Key Foundational Work

Morphological Features of the Irish Universal Dependencies Treebank (2017) — contributed 1 concept elements to this direction.
Improving Arabic Dependency Parsing with Form-based and Functional Morphological Features (2011) — contributed 1 concept elements to this direction.
Improving a Strong Neural Parser with Conjunction-Specific Features (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cross_architecture_generalizability_327f1724,
  title = {cross-architecture generalizability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-architecture-generalizability/rdr_327f17242d53457293621a6074410ad1},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_cross_architecture_generalizability_327f1724_v1,
  title = {cross-architecture generalizability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-architecture-generalizability/rdr_327f17242d53457293621a6074410ad1/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake