NeuroSymbolic AI for Legal AI-TRISM: Trustworthy, Reliable, Interpretable, Safe Models

Deepa Tilwani, Yash Saxena, Ankur Padia, Srinivasan Parthasarathy, Manas GaurarXiv 2026 · DOI · arXiv

TL;DR

This paper introduces the TRISM framework and its practical pipeline, RASOR, which combine the natural language fluency of Large Language Models (LLMs) with the logical rigor of symbolic reasoning (Knowledge Graphs). By generating explicit rationales, dynamically selecting document chunks, and verifying outputs against legal constraints, RASOR slashes LLM hallucination rates in contract analysis from 75% to under 40%.

What Problem Does This Paper Address?

Large Language Models (LLMs) show immense promise in automating legal document analysis, but they suffer from a fatal flaw: hallucinations. In a high-stakes field like law, a single fabricated citation or missed precedent can jeopardize an entire case and lead to severe court sanctions.

Standard Retrieval-Augmented Generation (RAG) systems attempt to ground LLMs in authoritative texts, but commercial legal RAG tools still maintain error rates between 17% and 34%. This is because traditional RAG relies on opaque chunking and ranking heuristics that often strip away vital context. Furthermore, standard LLMs struggle with the complex, hierarchical nature of legal reasoning—such as understanding when a local municipal regulation overrides a general state principle, or identifying nuanced contextual exceptions within contract clauses.

Methods

The authors propose TRISM (Trustworthy, Reliable, Interpretable, Safe Models), a framework rooted in NeuroSymbolic AI. This approach fuses neural networks (excellent at pattern recognition and language generation) with symbolic systems (excellent at rule-based logic and structured knowledge representation).

To operationalize TRISM, the authors develop RASOR (Rationalize, Select, and Refine), a modular RAG pipeline that breaks down legal text generation into five distinct steps:

  1. Rationalize: A legal LLM (SaulLM) is fine-tuned to generate explicit, interpretable legal rationales based on a user's query.
  2. Select: A hyperparameter-free Neural Rationale-based Chunk Selector (NRCS) uses a Dense Passage Retriever to match these generated rationales with the most relevant document chunks.
  3. Refine: A Refiner module applies preference learning inspired by Direct Preference Optimization (DPO) to optimize the rationales, bypassing the need for complex reinforcement learning (RLHF).
  4. Verify: An unsupervised Verifier filters out irrelevant or contradictory document chunks to mitigate hallucination risks.
  5. Inference: The final LLM generates a response structured with explicit citations and clear logical pathways.
Figure 2. RASOR Architecture

Additionally, the paper outlines a formal framework for constructing and automatically updating Legal Knowledge Graphs (KGs). These KGs explicitly map legal entities (statutes, cases, principles) as nodes and their hierarchical relationships as edges, ensuring the AI respects jurisdictional and temporal boundaries.

Figure 4. NeuroSymbolic RAG Pipeline integrating TRISM principles. The framework combines an LLM with a Legal Knowledge Graph (KG), leveraging query enrichment and retrieval techniques (e.g., FAISS (Douze et al. 2024)) for scalable and secure document retrieval. It incorporates trust, responsibility, and security through enriched rule-based reasoning in the KG, ensuring robust query-answering in legal contexts.

Datasets

The framework was evaluated using the Contract Understanding Atticus Dataset (CUAD), a prominent legal benchmark containing over 13,000 expert annotations across 510 contracts. The evaluation was performed on the CUAD test set, which consists of 400 complex legal queries.

Key Insights and Ideas

  • "Show Your Work" Wins: Forcing the system to generate explicit, structured rationales before retrieving documents significantly improves both transparency and raw generation accuracy.
  • Dynamic Context Filtering: Instead of using a fixed number of retrieved document chunks ($K$), RASOR dynamically adjusts $K$ based on query complexity. This minimizes background noise for straightforward queries while capturing necessary context for intricate legal issues.
  • An Evolutionary Pathway: RASOR bridges pure neural approaches and full neurosymbolic systems. Because its pipeline is modular, each neural component has a clear symbolic counterpart that can be progressively swapped out for formal legal ontologies and strict rule-based verifiers.

Key Results

  • Massive Hallucination Drop: RASOR successfully reduced hallucination rates from roughly 75% (with SaulLM-52B) to under 40%.
  • Superior Accuracy: Evaluated using strict exact string matching (which heavily penalizes hallucinations), RASOR outperformed the state-of-the-art Contriever re-ranker by up to 17%. It achieved 52% accuracy at 128-token chunks and extended to 64% accuracy at 512-token chunks.
  • Robust Retrieval: In information retrieval tests, RASOR achieved a precision-recall balance with a robust recall rate of approximately 93% across varying document chunk sizes.

Limitations and Future Work

While RASOR establishes a strong foundation, its current implementation still relies heavily on neural pathways for rationale generation and verification.

The authors outline several immediate directions for future work to complete the neurosymbolic transition:

  • Replacing the neural chunk selector with a Knowledge Graph-enhanced selector using legal ontologies (e.g., GraphRAG).
  • Upgrading the unsupervised verifier to a Rule-based Logical Verifier that enforces formal legal constraints and constitutional AI principles.
  • Implementing multi-agent legal reasoning systems where specialized agents handle distinct legal domains (such as statutory interpretation vs. case precedents) and coordinate through structured legal protocols.

Summary updated

Abstract

Large Language Models (LLMs) have transformed natural language processing, but their lack of interpretable reasoning and tendency to hallucinate pose significant challenges for legal applications. While LLMs show promise for legal text analysis and generation, they struggle with accurate citation attribution and precedent verification. For example, in legal contexts, a single incorrect precedent can jeopardize a case. Current approaches to improve LLM reliability in legal domains suffer from two key limitations: inadequate integration of structured legal knowledge during training or fine-tuning, and insufficient verification mechanisms for generated legal content. To address these challenges, we propose the TRISM (Trustworthy, Reliable, Interpretable, Safe Models) framework, which integrates NeuroSymbolic AI principles with LLMs to leverage both neural learning capabilities and symbolic reasoning over structured legal knowledge. The TRISM approach addresses the above limitations while maintaining interpretable decision pathways. Our framework formalizes the extraction of symbolic knowledge from legal textual documents and incorporates Retrieval-Augmented Generation (RAG) as a core component for grounding LLM outputs in verified legal sources. In this position paper, we make the following contributions: (1) An analysis of the limitations of AI in law; (2) Introduce RASOR RAG which creates foundations for neurosymbolic RAG by generating explicit interpretable rationales that could be formalized into symbolic representations; (3) A formalized methodology for creating symbolic legal knowledge bases that support both interpretable reasoning and output verification in LLMs; and (4) The TRISM framework for integrating symbolic legal knowledge with LLMs.
BibTeX
@misc{tilwani2026neurosymbolic,
  author = {Deepa Tilwani and Yash Saxena and Ankur Padia and Srinivasan Parthasarathy and Manas Gaur},
  title = {{NeuroSymbolic AI for Legal AI-TRISM: Trustworthy, Reliable, Interpretable, Safe Models}},
  year = {2026},
  eprint = {2606.15646},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.15646},
}
Figure