Can LLMs Faithfully Solve Ontology Engineering Tasks? A Mechanistic Interpretability Study

Bohui Zhang, Albert Meroño-Peñuela, Elena SimperlarXiv 2026 · arXiv

TL;DR

LLMs are increasingly used for ontology engineering, but do they actually "understand" the underlying taxonomic relationships, or are they just reciting memorized patterns? This paper introduces MIOE, a new benchmark that uses mechanistic interpretability to show that while ontological knowledge is causally accessible within models like Qwen3, it remains messy and deeply entangled with other subject-specific attributes.

What Problem Does This Paper Address?

While Large Language Models (LLMs) show promise in ontology tasks—like identifying that "Amsterdam" is a "City" (entity typing) or that "City" is a "Geographical Feature" (subsumption)—most evaluations only look at the final output. This "behavioral" approach doesn't tell us if the model is using a dedicated internal mechanism for these relationships or just relying on surface-level associations.

Furthermore, existing mechanistic interpretability research has focused heavily on simple factual recall (e.g., "The capital of France is..."). Ontological knowledge is different; it’s schema-level information that defines the very nature and constraints of an entity. The authors set out to determine if LLMs have a "faithful" internal representation of these taxonomic structures that can be isolated and manipulated without breaking other facts the model knows about an entity.

Methods

The researchers frame the problem as one of causal abstraction. They use interchange interventions: they identify a specific internal representation (a "target") and swap it between two different inputs to see if the model's behavior changes in a predictable way.

For example, if they swap the "type" representation of Amsterdam (a city) with that of Australia (a country), a faithful mechanism should cause the model to now say Amsterdam is a country, while still correctly remembering that Amsterdam is in the Netherlands (preserving auxiliary information).

They evaluated several state-of-the-art interpretability methods:

  • Sparse Autoencoders (SAEs): Decomposing activations into sparse, interpretable features.
  • Distributed Alignment Search (DAS): Finding low-dimensional linear subspaces that correspond to high-level concepts.
  • Multi-task variants (MDAS/MDBM): Training the intervention to not only change the target answer but explicitly preserve unrelated auxiliary facts.
  • Multi-layer interventions: Extending these searches across contiguous windows of transformer layers rather than just a single site.

Datasets

The authors introduced the MIOE (Mechanistic Interpretability for Ontology Engineering) benchmark. It is built from Wikidata, WordNet, and schema.org and includes:

  • Two core tasks: Entity typing (instance of) and hierarchy prediction (subclass of).
  • 54 classes for entity typing and 39 classes for subsumption.
  • Auxiliary attributes: For every subject, the dataset includes non-target facts (e.g., for a city: population, official language) to test if interventions are truly disentangled.
  • Prompt variants: Both controlled templates and naturalistic Wikipedia-derived contexts.

Key Insights and Ideas

  • Ontology is "Middle-Management": The internal mechanisms for ontological relations aren't found at the very beginning or end of the model. As shown in Figure 1, the "Cause" effect (the ability to change the model's answer) peaks in the middle-to-late layers.
Figure 1: Representative layer profiles for Qwen3-8B on instance of. The top panel shows single-layer MDAS with rank 2048, and the bottom panel shows four-layer MDAS with 256 × 4 contiguous windows. Both panels report Cause, Iso, and Disentangle.
  • The Disentanglement Struggle: It is difficult to change an entity's type without "leaking" into other attributes. There is a clear tension between changing the target answer and preserving auxiliary facts.
  • Class-Level Personality: Not all ontology classes are created equal. Concrete categories with regular attribute profiles (like airport or currency) are much easier to isolate than broad, heterogeneous ones (like body of water or biological process).
  • Role Asymmetry: A class might be a great "source" for an intervention (easy to graft onto others) but a poor "base" (hard to overwrite), or vice versa. For instance, protein and video game are strong sources, while national park is a stronger base.

Key Results

  • MDAS Wins: The Multi-task Distributed Alignment Search (MDAS) achieved the best balance of changing the target while preserving auxiliary info, reaching a disentanglement score of 0.592 for instance of on Qwen3-8B.
  • Multi-layer is Better: Intervening on a window of 4 layers outperformed single-layer interventions, suggesting that ontological computation is distributed across a short interval of the residual stream rather than a single point.
  • Supervision Matters: Methods that didn't use explicit preservation supervision (like standard SAEs or DAS) often "broke" the model's auxiliary knowledge while trying to change the taxonomic type, as illustrated by the trade-offs in Figure 2.
Figure 2: Cause–Iso scores for selected Qwen3-8B instance of configurations with different intervention dimensions. Percent labels indicate the dimensionality of Fτ under each method’s parameterization; for multi-layer interventions, total rank is reported relative to a single-layer residual-stream width.

Limitations and Future Work

The authors conclude that while ontological variables are "causally accessible," they are not yet "cleanly disentangled." Current methods can't perfectly isolate the "city-ness" of an entity from its other properties.

Future directions include: * Testing richer ontological constraints beyond simple typing and subsumption. * Evaluating a broader range of model families. * Investigating whether these internal mechanisms remain stable across even more complex reasoning contexts or different domains.

Summary updated

Abstract

Ontological knowledge underpins large language model (LLM) performance on knowledge-intensive tasks. However, existing evaluations of LLMs' ontological understanding largely rely on behavioral correctness, leaving the underlying mechanisms unexplained. Existing mechanistic interpretability work has largely focused on instance-level factual knowledge, leaving ontological knowledge under-explored. To bridge this gap, we introduce MIOE, a causal-abstraction benchmark that evaluates interchange interventions for two ontology engineering tasks: entity typing and subsumption prediction. MIOE tests whether learned internal intervention targets can change an ontology-level target answer while preserving class-specific auxiliary information. Across Qwen3-8B and Qwen3-4B, we evaluate SAE-, DBM-, and DAS-based methods, including multi-task and multi-layer variants. Distributed subspace methods give the strongest results and generalize across prompt settings, but the induced changes remain only partially separated from auxiliary class-specific information. Multi-task objectives and contiguous layer-window interventions improve this balance, while class-level analyses reveal substantial variation across ontology classes. These results suggest that ontological predictions are mechanistically accessible in LLMs, although current methods do not yet isolate a cleanly disentangled causal mechanism for ontological knowledge.
BibTeX
@misc{zhang2026can,
  author = {Bohui Zhang and Albert Meroño-Peñuela and Elena Simperl},
  title = {{Can LLMs Faithfully Solve Ontology Engineering Tasks? A Mechanistic Interpretability Study}},
  year = {2026},
  eprint = {2610.38326},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.38326},
}
Figure