Large Language Models (LLMs) excel at summarizing text, but they often struggle to organize information into formats that humans can scan quickly. This paper introduces STRUCTSUM, a framework that uses specialized prompting and a "critic" pipeline to generate high-quality tables and mind maps, helping users comprehend complex information up to 43% faster than reading raw text.
The Information Processing Bottleneck
We are currently drowning in information. While LLMs like Gemini or GPT-4 can summarize long articles, their output is usually just more text. For a human trying to quickly grasp a topic, a well-organized table or a visual mind map is often much more efficient than a wall of prose. However, generating these structures is surprisingly difficult for LLMs. They often produce "hallucinated" facts, misalign data (e.g., putting a date in a "Location" column), or create structures that are either too shallow or overly complex. STRUCTSUM tackles the challenge of making LLMs "think" structurally to produce faithful, useful, and scannable summaries.
Methodology
The authors propose two distinct prompting strategies and a multi-stage verification pipeline to ensure structural integrity:
- Divide-and-Generate (Tables): Instead of asking an LLM to make one giant table for a long article, the system first segments the text into sub-topics. It then generates a specific table for each sub-topic. This prevents the model from omitting details or creating confusingly large schemas.
- Iterative Expansion (Mind Maps): Mind maps require "radial thinking." The system starts with a central root concept and then iteratively prompts the model to decide if more branches are needed and what they should be. This step-by-step growth ensures the map remains logical and manageable.
- The Critic Pipeline: To ensure quality, the authors developed three "critics" (LLM-based self-correction steps):
- Factuality Critic: Checks if every row or path can be attributed back to a specific sentence in the source text.
- Local Structure Critic: Ensures data types match their headers (e.g., verifying that a "Birth Date" column actually contains dates).
- Global Structure Critic: For mind maps, it converts the map into a "Table of Contents" format to check if the hierarchy makes logical sense.
Key Insights and Results
A primary finding of the study is that even powerful models like PaLM-2 struggle with "structural factuality"—they might get the facts right but place them in the wrong structural context. The "Divide-and-Generate" approach proves that LLMs perform significantly better when the structural task is narrowed down to a single sub-topic at a time.
Quantitative Performance
- Accuracy Boost: The critic pipeline led to an absolute improvement in accuracy of +37 percentage points for mind maps and +15 percentage points for tables.
- Faster Comprehension: In a timed user study, participants were 42.9% faster at answering questions when using tables and 31.9% faster when using mind maps, compared to reading the original text.
- Semantic Coverage: Using a novel AUTO-QA metric, the authors showed that their structured summaries retained a high percentage of the original information, performing significantly better than standard "single-table" generation.
Limitations and Future Directions
The research focused exclusively on tables and mind maps in English. Future work could explore other structures like flowcharts or timelines. Additionally, as Vision-Language Models (VLMs) improve, evaluating the visual layout of these structures—rather than just the text/JSON—will become increasingly important to ensure maximum human readability.
Summary updated