TL;DR
AI-Voyager is a public-interest platform that empowers citizens to navigate, question, and co-design AI governance rules using a provenance-preserving agentic RAG (Retrieval-Augmented Generation) system. By binding AI assistance to the exact, versioned text of over 3,600 governance documents, it enables "civic reasoning"—allowing users to see how legislation affects them and draft their own revisions with verified evidence.
What Problem Does This Paper Address?
As governments worldwide scramble to regulate AI, the resulting "rules of the road" are often buried in thousands of fragmented, complex, and frequently updated documents. For the average citizen, finding the relevant bill, understanding its practical implications (e.g., "How does this affect my privacy in a hiring process?"), and contesting its wording is nearly impossible.
Existing AI tools often fail in this domain because they hallucinate legal details or provide citations that don't actually support their claims. This paper addresses the need for a system that makes the governance record legible, supports discovery without legal expertise, and keeps AI interpretations strictly grounded in verifiable source text.
Methods
The authors introduce a "grammar of civic reasoning"—a set of operations like Find, Reframe, Verify, and Revise—implemented through a novel technical architecture:
- ROAR (Reading Order-Aware Reconstruction): A pipeline that processes PDFs (scanned or digital) into structured Markdown blocks while preserving the original reading order, headings, and tables. This ensures that a provision's meaning isn't lost by being detached from its context.
- Voyager-RAG: An agentic retrieval system. Unlike standard RAG, it uses a controller to plan iterative searches. Most importantly, it includes a statement verifier that cross-checks every generated claim against its cited passage. If a claim isn't supported, the system triggers a targeted search to find the missing evidence or drops the claim.
- Analytical Lenses: Instead of biased "persona prompting," the system offers lenses (e.g., AI Developer, Civilian, Policymaker) that act as analytical viewpoints. These lenses steer the AI's focus toward different aspects of the same shared evidence base.
Datasets
The study introduces and utilizes several key resources:
- AI-Voyager Corpus: A newly collected dataset of 3,653 document versions (278,021 pages) spanning U.S. federal/state bills, city policies, and international AI strategies.
- Governance Benchmark: A new evaluation set of 360 source-reviewed questions designed to test document discovery, temporal reasoning (comparing versions), and multi-hop evidence assembly.
- LegalBench-RAG-mini: An existing benchmark used to validate the system's performance when documents are already identified.
Key Insights and Ideas
- Provenance is a First-Class Citizen: In legal reasoning, a citation isn't just a link; it's a specific character span in a specific edition of a bill. AI-Voyager maintains this "provenance" throughout the entire pipeline.
- The "Grounding Cue" Trap: A user study revealed that participants trusted AI responses more simply because citations were present, but they rarely actually clicked them to verify. This highlights why internal automated verification (like Voyager-RAG's verifier) is critical to prevent overreliance on potentially wrong AI.
- Practicality Over Ideology: When given the chance to "revise" laws, users didn't focus on high-level ethics. Instead, they asked highly practical questions about implementation: "Who enforces this?" or "What happens to my data after the 60-day retention period ends?"
- Lenses vs. Independent Thinking: While "lenses" helped users explore bills faster, some participants preferred "General Analysis" because it left more room for their own independent judgment.
Key Results
- Retrieval Accuracy: On the new Governance Benchmark (the "open" setting where bills must be found first), Voyager-RAG achieved 83.0% macro recall, outperforming the strongest hybrid baseline (80.5%).
- Ablation Success: Removing the iterative search and verification steps dropped recall significantly (down to ~73-76%), proving that the agentic "search-and-verify" loop is essential for complex legal queries.
- Fixed Document Performance: When the documents were pre-selected (LegalBench-RAG-mini), the system reached a near-perfect 99.1% recall.
- User Engagement: In a 24-person study, participants used the system to identify "underspecification" in real AI bills, proposing concrete clarifications that the authors argue are a vital form of public contribution to AI alignment.
Limitations and Future Work
The authors acknowledge that the current corpus is heavily weighted toward U.S. legislation. Additionally, the user study focused on a specific age demographic (18-24), and future work is needed to see how more diverse populations interact with the system over longer periods.
A major next step is closing the loop: moving from "drafting a revision" in the interface to actually getting those proposals into the hands of the institutions that draft the laws. The authors also suggest that future evaluations should involve legal experts to assess the "correctness" of AI-generated legal interpretations beyond just retrieval accuracy.
Summary updated