This paper, authored by Alex Chohlas-Wood from New York University, explores how state agencies can leverage artificial intelligence to navigate a looming fiscal crisis in the U.S. food assistance landscape.
TL;DR
New federal legislation threatens states with billion-dollar penalties if their SNAP payment error rates exceed 6%. This simulation study demonstrates that combining traditional machine learning risk scores with generative AI "flags" can reduce these errors seven times more cost-effectively than using risk models alone.
What Problem Does This Paper Address?
The Supplemental Nutrition Assistance Program (SNAP) is a lifeline for millions, but determining eligibility is complex and prone to error. Historically, the federal government covered the cost of these errors. However, the 2025 federal budget law (H.R. 1) shifted the stakes: states with a Payment Error Rate (PER) above 6% must now pay a significant share of those benefit costs.
States are under immense pressure to improve accuracy through "preauthorization reviews"—having experienced staff double-check cases before benefits are issued. The challenge is capacity. Staff cannot review every case, and they cannot check every detail within a case. Agencies need a way to prioritize the right cases and the right details to minimize errors without ballooning administrative costs.
Methods
The author simulates and compares three distinct targeting strategies:
- Risk-based Review: Uses a machine learning model (XGBoost) to rank cases from highest to lowest probability of containing an error. Reviewers then perform a broad check of specific categories (e.g., earned income) in these high-risk cases.
- AI-based Review: Envisions a Large Language Model (LLM) that scans case documents to identify specific discrepancies (e.g., a mismatch between a pay stub and the amount entered by a caseworker). Reviewers only check the specific "flags" raised by the AI.
- Combined Review: A hybrid approach where risk scores prioritize the queue, but AI flags guide the reviewer to specific issues within those cases.
The simulation accounts for human-in-the-loop factors, such as the time taken to perform a check (5–10 minutes) and the probability that a human successfully corrects a flagged error (50–80%).
Datasets
The study relies on the USDA’s public-use SNAP Quality Control (QC) Data from fiscal years 2022–2024.
- This dataset includes detailed records of household demographics, income, and expenses, alongside "elements of variance" that describe exactly where and why errors occurred in sampled cases.
- The analysis focuses on the 48 contiguous states and the District of Columbia.
- The author adapted the BGL SNAP risk model, an open-source tool developed by the Better Government Lab, to generate the risk-based rankings.
Key Insights and Ideas
- The "Search Cost" Problem: Traditional risk models tell you which case is risky but not where the error is. This forces reviewers to spend time searching for errors that might not exist. AI flags solve this by pointing directly to the discrepancy, significantly reducing "search time."
- Complementary Strengths: AI is excellent at catching "errors of commission" (data entry mistakes or calculation errors visible in the documents). However, it is blind to "errors of omission" (e.g., a job the applicant never reported). Traditional risk models are better at identifying the types of households likely to have these hidden omissions.
- The Power of Hybridization: The combined approach is the most robust. It uses the risk model to find the "needle-heavy haystacks" and the AI flags to find the "needles" within them.
Key Results
- Massive Cost Savings: In a median across 25 jurisdictions, the risk-based approach costs 7 times as much as the combined approach to reach the same 6% error rate threshold.
- High Success Rate: The combined approach brought 26 out of 27 high-error jurisdictions below the 6% threshold within plausible budget constraints.
- State-Specific Variance: In California, AI-only review was highly effective. In Massachusetts, AI-only review hit a ceiling because many of its errors were "undetectable" omissions, requiring the risk-based component to reach the target.
- ROI: The cost of implementing these reviews (labor at ~$30/hr after federal reimbursement) is a tiny fraction of the potential billion-dollar penalties states face.
Limitations and Future Work
- Simulated AI Performance: The study assumes AI detection and precision rates (80% each) based on plausible LLM capabilities, but these haven't been benchmarked on actual SNAP case files yet.
- Human Factors: The simulation does not account for "alert fatigue" or the possibility that AI flags might lead to burdensome requests for more documentation from applicants, potentially delaying benefits.
- Data Gaps: The public QC data excludes households found to be entirely ineligible, which are included in official federal error rates.
- Next Steps: The author calls for real-world pilots to measure how quickly and accurately human reviewers can actually resolve AI-generated flags in a live workflow.
Summary updated