TL;DR
This paper introduces the "Structured Knowledge Hunter," a task-agnostic framework designed to make knowledge-enhanced text generation highly interpretable. By explicitly extracting a human-readable plan of entities and facts before generating text, it pairs a hierarchical retriever with large language models (LLMs) to set new performance standards on table-to-text and dialogue benchmarks while giving users a clear audit trail of the model's generation process.
What Problem Does This Paper Address?
While modern language models excel at generating fluent text, they remain notorious "black boxes." In knowledge-intensive tasks—such as summarizing a sports statistics table or pulling facts from a knowledge graph for a chatbot—this lack of transparency is a major hurdle. If a model hallucinates or makes an error, it is incredibly difficult to trace why it made that decision.
Existing solutions often rely on domain-specific retrievers tailored to a single type of data, which limits their ability to generalize across different tasks. Furthermore, they frequently struggle with factual consistency when dealing with large-scale structured inputs. This paper tackles these limitations by introducing a generalized, task-agnostic architecture capable of handling diverse structured data types (like tables and graphs) while ensuring high interpretability and factual grounding.
Methods
The authors propose a two-stage pipeline that decouples knowledge selection from text generation. Instead of feeding raw structured data directly into an LLM, they route it through a Structured Knowledge Hunter to extract a precise factual plan.
The architecture relies on a few key innovations:
- Entity-Centric Local-Global Encoder: It processes structured inputs from two distinct perspectives: a local intra-entity view (focusing on attributes within a single entity or table row) and a global inter-entity view (capturing relationships across different entities).
- Local-Global Attention Fusion: A gated mechanism using a Sigmoid function dynamically weights and merges these local and global representations.
- Hierarchical Knowledge Planner: Modeled as a transformer-based pointer network, it mimics human "macro-to-micro" planning by first selecting high-level "macro" entities and then diving down to select specific low-level "micro" knowledge triples.
- Multi-task Learning Optimization: The retriever is jointly optimized using three losses: triple selection loss, entity selection loss, and a novel matching loss that ensures selected triples strictly align with their parent entities.
Once the human-readable knowledge plan is extracted, it is converted into simple text templates and fed into a fine-tuned LLM (such as BART or T5) to generate the final output.
Datasets
The framework was evaluated on two distinct tasks using standard benchmarks:
- RotoWire-FG: A dataset for internal knowledge-enhanced table-to-text generation. Models must generate long, multi-paragraph NBA game summaries (averaging ~206 words) based on complex box-score and line-score tables containing hundreds of numerical records.
- KdConv: A multi-domain Chinese dataset for external knowledge-graph-driven multi-turn dialogue. It spans film, music, and travel domains, featuring complex topic transitions and rich external knowledge graphs.
Key Insights and Ideas
- Human-Readable and Human-Writable Memory: Instead of passing abstract vector embeddings to the text generator, the framework extracts explicit, raw-text knowledge triples. This makes the model's "thought process" fully auditable by humans and allows developers to dynamically update the model's knowledge base simply by editing the underlying text.
- The Power of "Macro-to-Micro" Planning: Mirroring human cognitive behavior—where we first identify the main subjects of interest before analyzing specific details—significantly boosts retrieval accuracy. The introduction of the matching loss ensures that the low-level facts selected actually belong to the high-level entities being discussed.
- Task-Agnostic Flexibility: By focusing on the universal two-tier hierarchy of structured data (Entities $\rightarrow$ Triples), the exact same architecture seamlessly adapts to both static numerical tables and dynamic relational knowledge graphs.
Key Results
The framework was evaluated on both knowledge selection accuracy and text generation quality, outperforming state-of-the-art baselines.
- Table-to-Text (RotoWire-FG):
HunterAug-BARTachieved a Content Selection F1 score of 55.78%, a substantial improvement over the previous SOTA baseline AuxEncoder (49.04%) and vanilla BART (50.12%). It also achieved the highest text quality with a BLEU score of 28.69 (compared to vanilla BART's 24.51). - Multi-turn Dialogue (KdConv):
HunterAug-BARTshowed massive improvements over the competitive KHRED baseline, raising the BLEU-4 score by 10.49 points in the travel domain, 5.00 in music, and 3.10 in film.
Ablation studies confirmed that the entity-aware design, local-global fusion layers, and multi-task loss combinations all progressively contributed to these performance gains.
Limitations and Future Work
The authors acknowledge a few key limitations:
- Handling Implicit Contexts: The Structured Knowledge Hunter can struggle when a dialogue context doesn't explicitly mention specific entities or relationships, occasionally leading to an empty extracted knowledge set and less informative responses.
- Trade-off Between Factualness and Diversity: Because the model heavily prioritizes factual grounding and strictly adheres to the extracted knowledge plan, it can suffer from reduced text diversity, particularly when generating longer narratives.
For future work, the authors suggest exploring better ways to handle incomplete or noisy structured data and investigating methods to boost stylistic diversity without sacrificing factual accuracy.
Summary updated