Schemex: Interactive Structural Abstraction from Examples with Contrastive Refinement

Sitong Wang, Samia Menon, Dingzeyu Li, Xiaojuan Ma, Richard S. Zemel, Lydia B. Chilton

Abstract

Each type of creative or communicative work is underpinned by an implicit structure. People learn these structures from examples - a process known in cognitive science as schema induction. However, inducing schemas is challenging, as structural patterns are often obscured by surface-level variation. We present Schemex, an interactive visual workflow that scaffolds schema induction through clustering, abstraction, and contrastive refinement. Schemex supports users through visual representations and interactive exploration that connect abstract structures to concrete examples, promoting transparency, adaptability, and effective human-AI collaboration. In our user study, participants reported significantly greater insight and confidence in the schemas developed with Schemex compared to those created using a baseline of an AI reasoning model. We conclude by discussing the broader implications of structural abstraction and contrastive refinement across domains.

Introduction

When we try to master a new form of communication—like writing a grant proposal, designing a pitch deck, or even making a viral TikTok—we usually look at examples. In cognitive science, the process of turning those examples into a mental template is called schema induction.

However, this is surprisingly difficult for novices. Surface-level details (like the specific topic of a paper) often hide the underlying structural patterns (like how a "Motivation" section transitions into a "Gap Statement"). While Large Language Models (LLMs) like GPT-4 can summarize patterns, they often produce vague, generic advice that lacks the precision needed for high-stakes creative work. Schemex addresses this by grounding abstract patterns in concrete evidence while allowing users to steer the abstraction process.

The Schemex Workflow

Schemex is an interactive system that helps users extract structured templates (schemas) from a messy set of real-world examples using a "human-in-the-loop" AI workflow. The system breaks schema induction into three main stages:

  1. Clustering: Instead of treating all examples as one block, the system uses an LLM to group examples by structural similarity. It presents a feature-example matrix so users can verify that the groups actually make sense.
  2. Abstraction: For a chosen cluster, Schemex infers "Dimensions" (high-level categories) and "Attributes" (specific, observable features). Crucially, every attribute is linked back to the original text with citations, ensuring the AI isn't just "hallucinating" a structure.
  3. Contrastive Refinement: This is the system's core innovation. To test if a schema is actually good, Schemex uses it to generate a new piece of content (e.g., a mock abstract). It then compares this generation side-by-side with a "gold" (real) example. The system highlights "gaps"—things the real example did that the schema-guided generation missed—and suggests specific edits to the schema to fix them.
The Schemex workflow: from clustering and abstraction to the iterative contrastive refinement loop.

The system uses a "tree-like" visual interface rather than a simple chat box, allowing users to "drill down" from broad clusters to specific textual evidence. This hierarchical, node-based layout matches the mental model of a schema as a nested set of rules rather than a flat list of instructions.

Key Insights and Results

The researchers evaluated Schemex using a technical evaluation across six diverse domains (e.g., faculty job talk abstracts, LinkedIn summaries, TikToks) and a user study where participants brought their own datasets.

A central finding was the importance of the "Apply-and-Test" cycle. Users often only realized their schemas were too vague once they saw the AI try (and fail) to use that schema to recreate a high-quality example. Furthermore, grounding builds trust: users reported much higher confidence in Schemex than in raw ChatGPT because they could click any "claim" the AI made about a pattern and see the exact sentence in the source material that proved it.

Quantitative comparison showing Schemex significantly outperforming the o1-pro baseline across multiple dimensions.

When compared against a state-of-the-art baseline (OpenAI’s o1-pro), Schemex demonstrated superior performance:

  • Higher Quality: Expert evaluations rated Schemex's schemas significantly higher in cluster quality, attribute precision, and overall usefulness.
  • Deeper Learning: Participants reported a 147% increase in structural insight and a 148% increase in "novel reflection" (noticing things they hadn't seen before) compared to the baseline.
  • Actionability: Users felt they could immediately use the Schemex output to perform the task, whereas baseline outputs felt too high-level.

Conclusion and Future Work

Schemex demonstrates that interactive structural abstraction can significantly improve how humans learn from examples. While currently text-centric, future work could expand these refinement steps to direct visual or interactive comparisons. Additionally, the researchers noted the risk of "overfitting" during contrastive refinement, where a schema might become too specific to a single example. Finding the "Goldilocks zone" of abstraction remains a key challenge for future iterations of human-AI collaborative design tools.

Summary updated

Abstract

Each type of creative or communicative work is underpinned by an implicit structure. People learn these structures from examples - a process known in cognitive science as schema induction. However, inducing schemas is challenging, as structural patterns are often obscured by surface-level variation. We present Schemex, an interactive visual workflow that scaffolds schema induction through clustering, abstraction, and contrastive refinement. Schemex supports users through visual representations and interactive exploration that connect abstract structures to concrete examples, promoting transparency, adaptability, and effective human-AI collaboration. In our user study, participants reported significantly greater insight and confidence in the schemas developed with Schemex compared to those created using a baseline of an AI reasoning model. We conclude by discussing the broader implications of structural abstraction and contrastive refinement across domains.
BibTeX
@misc{wang2025schemex,
  author = {Sitong Wang and Samia Menon and Dingzeyu Li and Xiaojuan Ma and Richard S. Zemel and Lydia B. Chilton},
  title = {{Schemex: Interactive Structural Abstraction from Examples with Contrastive Refinement}},
  year = {2025},
  eprint = {2504.11795},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.11795},
}
Figure