When designing an educational curriculum or training an artificial intelligence, it is tempting to treat knowledge as a flat checklist of independent skills. In this simplified view, a student or a language model possesses a collection of isolated abilities—such as multi-digit addition, fraction multiplication, and algebraic equation solving—each acquired independently.
However, human learning and machine intelligence do not operate in a vacuum. Knowledge is inherently structured as a network of prerequisites, where mastering advanced concepts depends strictly on the stability of foundational building blocks. Failing to model these hierarchical dependencies creates a significant blind spot: systems treat learning events in isolation, making it difficult to diagnose why a learner struggles or how an agent can efficiently acquire complex behaviors.
The Fallacy of Flat Knowledge and the Conjunctive Reality
Traditional knowledge tracing frameworks often treat educational concepts as independent variables or flat sequences. As Congjie Liu points out, this omission limits a model's ability to represent how the mastery of foundational skills directly influences the acquisition of more complex knowledge. When a system assumes independence, it expects that a student's performance on a complex task can be predicted by simply averaging their disparate skill levels.
In reality, skill composition behaves like a logical AND gate (a conjunctive mechanism) rather than an OR gate (a compensatory mechanism). Zachary A. Pardos et al. analyzed student performance on multi-skill math questions and demonstrated that strength in one area cannot compensate for a weakness in another. If a task requires three distinct skills and the learner is missing even one foundational piece, their probability of success drops to near baseline.
Compensatory Model (Flat/OR):
[Skill A: 90%] + [Skill B: 10%] + [Skill C: 80%] --> High Chance of Success (Averages out)
Conjunctive Model (Hierarchical/AND):
[Skill A: 90%] × [Skill B: 10%] × [Skill C: 80%] --> Near-Zero Chance of Success
This structural dependency is not unique to human cognition; it is a fundamental property of complex systems. Even in early symbolic AI, Jonathan Schaeffer 0001 and T. Anthony Marsland observed that introducing sophisticated heuristics into expert systems could actually degrade performance unless the foundational "building block" routines were already firmly established to provide the necessary context.
Propagating Knowledge Through Structural Priors
If knowledge is hierarchical, how can computational models leverage this structure to better track and predict a learner's state? The answer lies in treating prerequisite relationships as structural priors that allow information to flow across a network of concepts.
Consider the challenge of data sparsity. In a typical classroom or online tutoring platform, a student answers only a fraction of all available questions. A flat model cannot infer how a student will perform on an unseen topic. However, by organizing knowledge components into a tree structure, models can perform cross-concept inference.
Xinyi Gao et al. implemented this using a Hidden Markov Tree framework ($KT^2$). When a student answers a question correctly, the performance data does not just update the estimate for that isolated concept; the success propagates upward to update parent concepts and laterally to update sibling concepts.
[Algebraic Equations] (Parent)
▲
│ (Upward Propagation)
[Multi-Step Linear] (Target Concept)
│
▼ (Lateral Inference)
[Single-Step Linear] (Sibling)
This mechanism mitigates the cold-start problem. If a student demonstrates mastery of an advanced topic, the model can algorithmically deduce that they have likely mastered its prerequisites, even if they have never encountered direct questions for those foundational skills. Conversely, as S. Park et al. and Chuan Qin et al. demonstrate, ignoring these inter-concept relationships leads to overconfident and inaccurate mastery estimates, particularly when evaluating complex concepts that rely heavily on unmeasured foundational knowledge.
Discovering the Hidden Scaffolding
While leveraging a known hierarchy is powerful, mapping out these prerequisite structures manually is a bottleneck. Human experts may disagree on the exact "natural order" of a curriculum, and manual tagging fails to scale to massive domains like Wikipedia or broad web datasets.
To solve this, researchers have turned to data-driven causal discovery. Instead of asking teachers to draw the connections, algorithms can infer the underlying hierarchy directly from patterns of student success and failure. Nischal Ashok Kumar et al. developed a "Causal GRU" module that utilizes a learnable dependency structure to discover prerequisite relationships end-to-end from sequential response data. Similarly, Shenbao Yu et al. framed skills as latent causes of exercise success, allowing their framework to map out global, multi-layered hierarchical structures rather than simple pairwise correlations.
When these structures are uncovered, they reveal the hidden scaffolding of a domain. For instance, Partha P. Talukdar et al. demonstrated that machine learning could successfully predict the prerequisite structure of technical concepts across Wikipedia, moving automated systems away from shallow fact extraction and toward deep comprehension of how ideas build upon one another.
Scaffolding in Machine Intelligence
The principles of hierarchical skill dependency are rapidly reshaping how we understand and train Large Language Models (LLMs). Just as a human student struggles with calculus without algebra, an LLM can experience "developmental delays" or training inefficiencies when fed a heuristically-mixed pile of data without structural progression.
Mayee F Chen et al. introduced a formal framework called Skill-it!, which demonstrates that language models exhibit clear hierarchical skill dependencies. By identifying which simpler tasks act as prerequisites for complex behaviors, their online sampling algorithm navigates a skill graph to prioritize foundational data early in training. This structured curriculum maximizes training efficiency, allowing models to achieve superior performance while using up to two-thirds fewer tokens than standard baseline training.
Standard Fine-Tuning:
[Random Mixture of Simple & Complex Data] ──> Inefficient Learning / High Data Cost
Skill-Scaffolded Training (Skill-it!):
[Foundational Skills] ──> [Intermediate Tasks] ──> [Complex Reasoning] ──> Faster Convergence
This dependency also explains why specialized models often fail when trained from scratch on narrow tasks. Eric J Michaud et al. discovered that complex cognitive capabilities depend on a foundational hierarchy of simpler sub-skills that are typically acquired during broad, general-purpose training. Without this underlying scaffolding—which is often nonlocally distributed across the network—a model trained strictly on a narrow dataset lacks the prerequisite environment required to sustain higher-order reasoning.
By shifting our framing from flat skill representation to hierarchical dependency modeling, we gain a more accurate reflection of both human learning and machine intelligence. Whether predicting a student's next test score or optimizing the sample efficiency of a neural network, respecting the natural order of prerequisites is essential for building deep, reliable comprehension.
Key Foundational Work
The Utility of Expert Knowledge (1985) — contributed 1 concept elements to this direction.
The Composition Effect: Conjuntive or Compensatory? An Analysis of Multi-Skill Math Questions in ITS (2008) — contributed 1 concept elements to this direction.