Abstract
Answering complex questions that span structured tables and unstructured text remains a significant challenge for Large Language Models (LLMs). We present ODYSSEY, a zero-shot framework that constructs unified "Hybrid Graphs" to navigate these data sources. By employing graph pruning and an iterative "hop-wise" retrieval strategy, ODYSSEY improves accuracy by up to 10% while reducing token costs by over 50% compared to standard long-context approaches.
Introduction
Real-world information retrieval often requires reasoning across a mix of structured data (e.g., Wikipedia tables) and unstructured text (e.g., hyperlinked biographies). This "Hybrid Question Answering" (Hybrid QA) task is difficult for AI because it involves multi-hop reasoning—identifying a table row, jumping to a linked document, and extracting a specific fact.
Current approaches typically rely on expensive fine-tuning or providing the entire context to long-context LLMs. The latter often suffers from "lost in the middle" syndrome, where the model misses critical information amidst irrelevant noise, and incurs high token costs. ODYSSEY addresses this by providing LLMs with precisely the necessary context through structured graph representation.
The ODYSSEY Framework
ODYSSEY operates in four stages to bridge the gap between structured and unstructured data without requiring model retraining.
- Question Analysis: Key entities are extracted from the query and mapped to table headers (e.g., mapping "position 4" to a "Pos" column).
- Hybrid Graph Construction: A unified graph is built where nodes represent table cells and entities from linked documents, with edges representing their relationships.
- Graph Traversal & Pruning: Using Breadth-First Search (BFS), the system explores the graph starting from query entities. Information not reachable within three hops is pruned to eliminate noise.
- Iterative Reader LLM: A "hop-wise" retrieval strategy is used. The LLM is first presented with 1-hop data; if the answer is not found, the context is expanded to the 2nd hop, and so on. This minimizes prompt size and focuses the model's attention.
Experimental Evaluation and Results
The framework was evaluated on the Hybrid-QA and OTT-QA benchmarks using models including GPT-3.5, GPT-4, and LLaMA-3.
Key Findings
- Accuracy and Efficiency: On Hybrid-QA, ODYSSEY improved Exact Match (EM) scores by 10% over the baseline. Simultaneously, it reduced token usage by 45.5% on Hybrid-QA and 53% on OTT-QA.
- Pruning vs. Summarization: Unlike standard summarization, which can erase specific facts needed for reasoning, graph-based pruning preserves connectivity and critical data points.
- Zero-Shot Performance: ODYSSEY with GPT-4 outperformed several fine-tuned models, demonstrating that structured context can be more effective than specialized training for multi-hop reasoning.
Conclusion
ODYSSEY demonstrates that hybrid graphs are a powerful tool for enhancing LLM performance in complex QA tasks. By intelligently pruning irrelevant data and using iterative retrieval, the framework achieves superior accuracy and significant cost savings. Future work may extend this approach to multi-modal data, including images and video.
Reference: Agarwal, A., Ganesh, S., & Devaguptapu, C. (2025). Hybrid Graphs for Table-and-Text based Question Answering using LLMs. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL).
Summary updated