Abstract: Max Upravitelev, Premtim Sahitaj, Arthur Hilbert, Veronika Solopova, Jing Yang, Nils Feldhus, Tatiana Anikina, Simon Ostermann, Vera Schmitt. Proceedings of the Eighth Fact Extraction and VERification Workshop (FEVER).
2025.
Introduction: The Latency Bottleneck in Fact-Checking
Automated fact-checking is a critical tool in the fight against misinformation, yet its practical utility is often hampered by high latency. For news organizations and social media platforms, a verification system that takes minutes to process a single claim is insufficient for real-time needs. Most state-of-the-art systems utilize Retrieval-Augmented Generation (RAG) pipelines, which rely on multiple, computationally expensive calls to Large Language Models (LLMs) for tasks such as question generation, evidence re-ranking, and final verdict prediction.
The primary bottleneck in these pipelines is the LLM inference step. This paper introduces SFEFC (Semantic Filtering for Efficient Fact Checking), an "efficiency-aware" pipeline designed to minimize LLM usage. By implementing semantic heuristics that "short-circuit" the process for certain types of claims, the authors achieve a significant reduction in average runtime—slashing it from approximately 34 seconds to just 7 seconds per claim—while maintaining competitive accuracy.
Methodology: The SFEFC Pipeline
The SFEFC architecture is built around the Phi-4 (14B) model, optimized via 6-bit quantization and llama.cpp. The pipeline introduces three key innovations to improve both speed and retrieval quality:
- Contextual Sentence Chunking: Rather than retrieving isolated sentences, the system concatenates sentences into chunks of four. This preserves semantic context and reduces the search space for the retrieval model by 75%, leading to faster processing.
- Hybrid Retrieval: The system employs a dual-retrieval strategy, combining BM25 (sparse) with
gte-baseembeddings (dense) to identify the top 10 most relevant evidence sets. - Semantic Filtering (The "Short-Circuit"): This is the core contribution of the research. Instead of passing every retrieved set to the LLM, the system analyzes the cosine similarity scores from the dense retrieval step to make immediate heuristic judgments.
The filtering logic operates on two primary heuristics:
- NEI (Not Enough Information) Filter: If the maximum similarity score between the claim and the evidence is below 0.82, the system immediately labels the claim as NEI, bypassing the LLM.
- CoC (Conflicting Evidence) Filter: The system calculates the variance of the top 10 similarity scores. A high variance (above 0.0007) suggests a mix of highly relevant but potentially contradictory information, triggering a "Conflicting Evidence" label.
The LLM is only invoked for a final "Supported" or "Refuted" verdict if these semantic filters are not triggered.
Experimental Results and Efficiency
The system was evaluated using the AVeriTeC dataset (4,568 real-world claims) as part of the FEVER-8 shared task. Performance was measured using Hungarian METEOR (lexical) and Ev2R (semantic recall) metrics.
The results demonstrate that SFEFC is the fastest system on the FEVER-8 leaderboard, achieving an average runtime of 7.01 seconds per claim. In comparison, the parallelized baseline required 33.88 seconds. Despite this 80% reduction in runtime, the system remained highly competitive in accuracy. On the development set, SFEFC-phi4 achieved a Hungarian METEOR score of 0.572, outperforming the baseline's 0.554.
Key insights from the ablation studies indicate that the 14B Phi-4 model provides an optimal balance of speed and performance, even outperforming larger models like Gemma 3 (27B) in this specific RAG context. Furthermore, the "chunking" strategy proved to be a "free win," improving retrieval scores while reducing embedding overhead.
Discussion and Limitations
While the semantic filtering approach offers dramatic speedups, the authors acknowledge that static heuristics are a "blunt" instrument. Error analysis reveals that these filters can introduce false positives; for instance, some "Refuted" claims were mislabeled as NEI because their similarity scores fell slightly below the 0.82 threshold.
Future work will focus on replacing these static thresholds with learnable classification heads—small, trainable models that can analyze embedding vectors more nuancedly. Additionally, the authors suggest that the current 7-second runtime could be further reduced through standard parallelization techniques, as the reported times were achieved on a non-parallelized system.
Conclusion
SFEFC demonstrates that the "confidence" of retrieval models, expressed through similarity scores and variance, can serve as a powerful proxy for veracity verdicts. By strategically bypassing the LLM bottleneck, the researchers have paved the way for more scalable and responsive automated fact-checking systems.
Summary updated