Learning Latent Steps for Weakly Supervised Reasoning

39 concepts34 papers1 proposal

Imagine trying to learn how to navigate a massive maze, but you are only told whether you successfully exited or not at the very end. You receive no feedback on which turns were correct and which were dead ends.

This is the core challenge of training machine learning models on complex, multi-step tasks—such as algebraic word problems, multi-hop question answering, and knowledge graph traversal—using only final-answer supervision. As highlighted by Bordes et al., models frequently fail at basic multi-step logic when they lack explicit intermediate guidance. This disconnect between the available training signal and the required execution steps is known as the intermediate supervision gap.

The Vulnerability of Weak Supervision

When a model is optimized solely on the final answer, it must navigate an exponentially large search space of potential reasoning paths (the authors of the Complex KBQA survey). Without step-by-step constraints, the model often falls victim to spurious reasoning (He et al.). Instead of learning a valid logical proof, the model exploits dataset shortcuts, surface-form correlations, and statistical biases to guess the correct answer for the wrong reasons (Dua et al.; Feng et al.).

To bridge this gap without the prohibitive cost of manual step-by-step annotations (Dua et al.), a prominent research direction treats intermediate reasoning steps as latent variables. By forcing models to generate an explicit, structured bridge (a "rationale," "program," or "retrieval path") between the question and the answer, researchers can optimize the underlying logic using the final answer as an indirect reward or filter.

Optimizing Latent Rationales and Sub-Questions

One powerful way to bridge the gap is to make the model generate its own text-based or symbolic reasoning steps before arriving at an answer. Wang Ling et al. demonstrated this by treating step-by-step algebraic rationales as a latent bridge to solve math word problems, showing that generating explanations directly improves final accuracy.

[Question] ──> [Latent Sub-questions / Rationales] ──> [Final Answer]
                     │                          ▲
                     └────── Optimized via ─────┘
                         EM / Reinforcement Learning

To handle multi-step question answering robustly when intermediate labels are missing, Zhu et al. introduced the Chain-of-Questions (CoQ) framework. CoQ decomposes a complex question into sequential sub-questions. Because the answers to these sub-questions are unannotated, they are treated as latent variables and optimized via a combination of Expectation-Maximization (EM) and Reinforcement Learning (RL). The final answer acts as the ultimate verification signal, pruning out incorrect sub-question chains and reinforcing the paths that yield valid logical deductions.

Differentiable Graph and Retrieval Paths

When reasoning over structured data like Knowledge Graphs (KGs) or large document corpora, the intermediate steps correspond to specific entities, relations, or text chunks (Khot et al.; Qiu et al.). Instead of relying on hand-crafted rules to order these steps (Khattab et al.), models can learn to navigate the graph continuously.

For instance, Shi et al. developed TransferNet, which models multi-hop reasoning as a differentiable transfer of entity scores across a relation graph. Rather than making hard, non-differentiable choices at each hop, the model maintains a probability distribution over the entire graph, allowing the entire path to be optimized end-to-end based purely on the final answer.

To prevent the model from getting lost in incomplete graphs or massive search spaces, Choi et al. introduced a look-ahead tree search mechanism (NuTrea). NuTrea probes future subtrees to evaluate the viability of an intermediate step before taking it, ensuring that the model avoids dead ends even under weak supervision.

Synthesizing Intermediate Supervision Signals

Another strategy is to use the final answer to work backward and synthesize the missing intermediate labels.

Forward Search:  [Question] ───> [Entity A] ───> [Entity B] ───?
                                                    ▲
Backward Search: [Answer]   <─── [Entity C] ────────┘ (Intersection = Gold Path)

He et al. proposed a bidirectional reasoning framework where a teacher network searches forward from the question and backward from the answer simultaneously. The intersection of these two paths isolates the true logical sequence, creating a reliable intermediate supervision signal to train a student model.

Similarly, Jo et al. showed that modular dialogue systems can bootstrap their own intermediate training data (such as search queries). By using "guided prompts" that contrast past mistakes with the correct final answer, the system self-corrects and generates high-quality intermediate steps without human intervention.

By treating intermediate steps as latent variables, continuous distributions, or bidirectional targets, these techniques successfully close the supervision gap. They transform weak final-answer feedback into a structured, step-by-step optimization signal, allowing models to learn verifiable logic rather than superficial shortcuts.

Key Foundational Work

Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks [data] Jason Weston (2016) — contributed 1 concept elements to this direction.
Coupling Distributed and Symbolic Execution for Natural Language Queries (2017) — contributed 1 concept elements to this direction.
Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_intermediate_step_supervision_gap_08f74d87,
  title = {intermediate step supervision gap},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/intermediate-step-supervision-gap/rdr_08f74d87c2674eaeabf9072c4a9df9a0},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_intermediate_step_supervision_gap_08f74d87_v1,
  title = {intermediate step supervision gap},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/intermediate-step-supervision-gap/rdr_08f74d87c2674eaeabf9072c4a9df9a0/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake