In the world of K-12 educational platforms, teachers and content creators constantly search for existing questions to build new assignments. You might think a simple visual search would suffice—if two questions look the same, they are the same, right? Not quite. In education, a single changed digit, a shuffled set of multiple-choice options, or a slightly expanded sub-question can completely change whether a candidate question is a direct replacement, a useful template, or a useless distraction.
The paper "ExRel: Learning Directed Reuse Relevance between Rich-Text K–12 Question Images with Executable Relation Contracts" introduces a novel framework to tackle this "fine-grained" retrieval challenge.
TL;DR
ExRel is an agentic framework that reconstructs K-12 question images into editable HTML/CSS "seeds" to synthesize high-quality training data for question reuse. By using "executable relation contracts" to govern precise semantic edits, it creates a balanced dataset that teaches models to distinguish subtle differences (like numeric changes or scope expansion) that generic similarity metrics miss.
What Problem Does This Paper Address?
Standard document retrieval often relies on topical or visual similarity. However, educational reuse hinges on directed reuse relevance. The authors categorize the relationship between a query question ($q$) and a candidate ($k$) into three grades:
- Strong: Direct reuse (e.g., same question, different layout).
- Medium: Requires adaptation (e.g., same core problem but different numbers or a rephrased stem).
- Weak: Solve independently (e.g., different key information or only partial overlap).
Crucially, this relevance is directional. If candidate $k$ contains query $q$ plus an extra sub-question, it might be "Medium" relevance for $q$. But if you reverse them, $q$ does not cover the full scope of $k$, making it "Weak."
Existing multimodal models struggle here. As shown in Figure 1, pixel-based metrics and standard rankers often suffer from "ranking inversions," where they mistakenly rank a "Weak" pair (with high visual overlap) higher than a "Strong" pair. Furthermore, naturally collected data is heavily skewed toward "Strong" matches, leaving models under-trained on the subtle "Medium" and "Weak" boundaries.
Methods
The authors propose ExRel, a framework for controllable and verifiable visual data synthesis. Instead of generating images from scratch (which often leads to "hallucinated" or unreadable math), ExRel follows a three-stage pipeline:
- Executable Seed Preparation: Real question images are reconstructed into editable HTML/CSS. This allows the system to manipulate the underlying structure (like swapping options or changing numbers) rather than just pushing pixels.
- Structure-Aware Relation Planning: The system profiles the question (identifying slots for numbers, options, etc.) and selects one of 15 fine-grained relation subtypes (e.g., "Numeric difference," "Option-order variant").
- Contract-Guided Execution and Admission: This is the "secret sauce." Each subtype is governed by an executable relation contract. This contract specifies what must be edited, what must remain invariant, and how the final pair should be oriented. An agentic controller executes the plan and then validates the result using both deterministic DOM checks and a "target-hidden" review by a Vision-Language Model (VLM) to ensure the intended relation holds.
Finally, the system applies mild photographic degradation (noise, blur, lighting shifts) to the rendered images to ensure the downstream model is robust to real-world capture conditions.
Datasets
- Training: ~6.5K natural pairs (Real) and 6.4K synthesized pairs (Synthetic).
- Release-200 (Benchmark): A new, human-adjudicated benchmark consisting of 1,000 real-image pairs across 200 queries.
- Uniform-300 (Diagnostic): A balanced set of 300 pairs (20 per subtype) used to evaluate subtype mastery and robustness to degradation.
Key Insights and Ideas
- Code-Grounded Synthesis > Pixel Manipulation: Editing the DOM of an HTML/CSS reconstruction is far more reliable for maintaining mathematical typesetting and diagram integrity than direct image editing.
- Contracts Prevent Semantic Drift: Without "contracts" (invariants), a VLM might accidentally change a critical condition while trying to rephrase a question, inadvertently flipping the relevance grade.
- The "Medium" Gap: Natural data lacks examples of questions that are "almost" the same. By specifically targeting these "Medium" subtypes, ExRel helps models learn the actual boundaries of utility rather than just visual distance.
- Cost-Efficiency: Synthesizing a valid pair with ExRel costs roughly 0.70 RMB, compared to 1.10 RMB for high-quality outsourced human annotation, while achieving higher post-admission validity (98%).
Key Results
The researchers fine-tuned a Qwen3.5-4B model on ExRel-synthesized data and found significant improvements:
- Superior "Medium" Classification: On the Release-200 benchmark, ExRel supervision improved the F1 score for the scarce "Medium" class by +2.80 points compared to natural data replay.
- Robustness: Models trained with ExRel showed much higher resilience to visual degradation. On the Uniform-300 set, the accuracy drop from "clean" to "degraded" views was only 2.0%, compared to 9.3% for zero-shot GPT-5.5.
- Downstream Utility: In an "answer routing" experiment, using ExRel's predicted grades to decide whether to reuse an archived answer or trigger a new RAG (Retrieval-Augmented Generation) process improved overall accuracy by over 11% compared to a query-only baseline.
Limitations and Future Work
While ExRel is highly effective for K-12 questions, the authors note a few areas for growth:
- Domain Expansion: The current framework is optimized for the structured layouts of math and science questions. Extending this to broader document domains (like legal or financial forms) would require new seed reconstruction techniques.
- Complex Pedagogical Relations: Future work could explore multi-step relations, such as identifying if one question is a "prerequisite" for another, rather than just a "reuse" candidate.
In summary, ExRel proves that for specialized AI tasks, "more data" isn't always the answer—smarter, contract-governed synthetic data is. By teaching models why two things are different, we can build tools that truly understand the needs of educators.
Summary updated