FinGEAR: Financial Mapping-Guided Enhanced Answer Retrieval

Ying Li, Mengyu Wang, Miguel de Carvalho, Sotirios Sabanis, Tiejun MaEMNLP 2025

TL;DR

Financial disclosures like 10-K filings are dense and complex, making it hard for current AI models to find specific information. This paper introduces FinGEAR, a novel retrieval system that uses the structure and specialized vocabulary of financial documents to dramatically improve information retrieval accuracy, paving the way for more reliable financial analysis.

What Problem Does This Paper Address?

Financial disclosures, particularly the lengthy 10-K filings required by the SEC, are crucial for investment analysis, regulatory oversight, and risk assessment. However, these documents are challenging for AI due to their sheer length (often over 100 pages), intricate hierarchical structure (organized by SEC-mandated "Items"), and domain-specific language filled with synonyms and cross-references. Standard retrieval-augmented generation (RAG) systems often struggle with these documents because they treat them as flat text, ignoring the inherent structure and specialized terminology. This leads to inefficient retrieval, where relevant information might be missed or buried within irrelevant passages, ultimately hindering downstream tasks like question answering and analysis. The paper aims to bridge this gap by developing a retrieval system specifically tailored to the unique characteristics of financial disclosures.

Methods

FinGEAR introduces a multi-stage approach to tackle the challenges of financial document retrieval:

  • Structure Extraction: The system first parses 10-K filings and builds hierarchical representations within each SEC Item. This is achieved by segmenting items into chunks, applying dimensionality reduction (UMAP), and then clustering these segments using Gaussian Mixture Models (GMM) to create "Summary Trees" and "Question Trees." The Summary Tree uses node summaries for navigation, while the Question Tree uses LLM-generated sub-questions to guide retrieval more precisely.
  • Financial Lexicon-Aware Mapping (FLAM): This module leverages a financial lexicon (FinRAD) to identify domain-specific terms. These terms are clustered and then mapped to SEC Items, assigning weights based on their frequency. This weighting guides the retrieval process, allocating a budget to the most relevant Items for a given query.
  • Hybrid Retrieval and Reranking: FinGEAR employs a two-stage retrieval process. First, "Global Navigation" uses FLAM to select relevant SEC Items. Then, "Within-Item Search" traverses the Summary and Question Trees within those selected Items. This traversal combines sparse (BM25) and dense (embedding similarity) signals for a more robust search. Finally, a two-stage cross-encoder reranker refines the retrieved passages, first within an Item and then across all Items, to prioritize globally informative answers.

Datasets

  • FinQA: This dataset, containing 8,281 question-answer pairs derived from 10-K filings, was used for evaluating retrieval performance and for fine-tuning embeddings. The authors recovered the original 720 full-length 10-K filings corresponding to the FinQA instances to enable full-document retrieval.
  • FinRAD: A financial lexicon used for fine-tuning embeddings and for the FLAM module.

Key Insights and Ideas

  • Structure is King (for Financial Docs): The paper strongly suggests that ignoring the hierarchical structure of financial documents (like SEC Items) is a major limitation of current RAG systems. FinGEAR's success hinges on its ability to leverage this structure.
  • Domain Lexicons are Crucial: Simply using generic embeddings isn't enough. FinGEAR's use of a financial lexicon (FLAM) to guide retrieval towards relevant sections and terms significantly boosts performance.
  • Hybrid Retrieval Wins: Combining sparse (keyword-based) and dense (embedding-based) retrieval, along with a sophisticated reranking mechanism, provides a more balanced and accurate approach than either method alone.
  • Retrieval Quality Directly Impacts Downstream Tasks: The paper demonstrates that improving retrieval accuracy directly translates to better performance in downstream tasks like question answering, highlighting the foundational importance of effective retrieval.

Key Results

FinGEAR significantly outperforms several state-of-the-art baselines on the FinQA dataset:

  • Retrieval Performance: FinGEAR achieved up to 56.7% higher F1 score compared to flat RAG, 12.5% over graph-based RAGs, and 217.6% over prior tree-based systems. It also showed substantial improvements in precision, recall, and relevancy across various retrieval depths (k=5, 10, 15).
  • Answer Accuracy: Using a fixed reader model (GPT-4o-mini), FinGEAR achieved the strongest final answer accuracy, demonstrating that its improved retrieval leads to more reliable downstream task performance.
  • Ablation Studies: All core components of FinGEAR (Summary Tree, Question Tree, and FLAM) were found to be critical for its performance, with disabling any single component leading to significant degradation.

Limitations and Future Work

  • Domain Specificity: FinGEAR is primarily developed and evaluated on U.S. 10-K filings. Its generalizability to other financial documents (like earnings calls) or different regulatory structures remains to be tested.
  • Lexicon Dependence: The system's performance relies on stable financial terminology. Emerging terms or sector-specific language might require adaptation.
  • Parsing Sensitivity: Accurate document parsing is crucial; severe inconsistencies could impact retrieval quality.
  • Limited Reasoning: FinGEAR focuses on retrieval and does not perform explicit financial reasoning or computation.
  • Evaluation Coverage: The evaluation used a dataset with limited gold spans, potentially underestimating retrieval performance.

Future work suggested includes exploring generalization to more diverse corpora, integrating lightweight reasoning modules, and improving robustness to document parsing noise.

Summary updated

BibTeX
@inproceedings{li2025fingear,
  author = {Ying Li and Mengyu Wang and Miguel de Carvalho and Sotirios Sabanis and Tiejun Ma},
  title = {{FinGEAR: Financial Mapping-Guided Enhanced Answer Retrieval}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure