TL;DR
OpenAI's GPT-4 demonstration used a simplified, hand-edited tax code dataset created by the authors, leading to an incorrect tax calculation. While GPT-4 shows promise in statutory reasoning, it struggles with accuracy, often misinterpreting statutes rather than making mathematical errors, highlighting the need for careful validation of AI models in specialized domains.
What Problem Does This Paper Address?
The paper addresses the critical question of whether large language models (LLMs) like GPT-4 can reliably perform complex legal and financial reasoning, specifically in the domain of tax law. OpenAI's public demonstration of GPT-4 using a tax scenario suggested impressive capabilities. However, the authors, who created the data used in that demonstration, found that GPT-4's calculation was incorrect. This raises concerns about the accuracy and trustworthiness of LLMs when applied to specialized, high-stakes fields where precise interpretation of complex rules is paramount. The paper aims to dissect why GPT-4 failed in this instance and to provide a more rigorous evaluation of its performance on statutory reasoning tasks.
Methods
The authors employed a two-pronged approach:
- Dataset Creation and Modification: They developed the "StAtutory Reasoning Assessment" (SARA) dataset, which includes handcrafted tax cases and heavily edited sections of the Internal Revenue Code (IRC). These edits were made to simplify the statutes, remove ambiguity, and create a manageable dataset for training and evaluating AI models. For the GPT-4 livestream demonstration, OpenAI used a subset of these SARA statutes and a specific tax case (
tax_case_9). - Empirical Testing of GPT-4: The authors systematically tested GPT-4 using their SARA dataset and the edited IRC statutes. They recreated the livestream scenario by feeding GPT-4 the SARA statutes and the
tax_case_9facts, confirming it produced the same incorrect answer. They then modified the test by reintroducing a crucial section of the actual IRC (section 1(j) related to TCJA rates) that had been removed in their edited version. This allowed them to observe GPT-4's ability to adapt to the correct statutory context. They also tested GPT-4 on a larger set of SARA cases, including both numerical tax liability calculations and true/false statutory interpretation questions.
Datasets
- SARA (StAtutory Reasoning Assessment): A newly introduced dataset developed at Johns Hopkins University. It comprises 376 handcrafted tax cases and nine sections of the IRC, which have been significantly edited by hand to be shorter and less ambiguous. This dataset was created for training and measuring AI models' ability to reason over statutory language.
- OpenAI's Livestream Data: The specific tax case (
tax_case_9) and edited IRC sections used in OpenAI's GPT-4 demonstration were derived directly from the SARA dataset.
Key Insights and Ideas
- "Garbage In, Garbage Out" (with a twist): OpenAI's GPT-4 demonstration was flawed not because of a fundamental inability to process text, but because it was fed a simplified and edited version of the tax code (from the SARA dataset). The model's incorrect answer matched the expected answer based on that edited code, leading OpenAI to believe it was correct.
- Editing Matters: The authors' hand-editing of the IRC statutes, while intended to simplify for AI training, fundamentally altered the legal landscape. Reintroducing a key edited-out section (TCJA rates) allowed GPT-4 to correctly calculate the tax liability, demonstrating its capacity when provided with more complete information.
- Reasoning vs. Calculation: GPT-4's errors were consistently due to misinterpreting the meaning of statutes, not due to mathematical errors in calculation. This suggests a deeper challenge in LLMs' ability to grasp nuanced legal language.
- "Shelter Check" Motivation: The authors' ultimate goal is to develop an AI tool called "Shelter Check" to proactively identify tax minimization strategies, underscoring the potential of AI in tax law if accuracy issues can be overcome.
Key Results
- Livestream Scenario: GPT-4 produced an incorrect tax liability of $10,597.68 for Alice and Bob using the SARA-edited IRC, which matched the expected answer for that specific edited dataset. When the TCJA rates (section 1(j)) were reintroduced, GPT-4 correctly calculated the liability at $6,216.72.
- True/False Questions: On 276 true/false statutory interpretation questions using the SARA IRC, GPT-4 achieved 67% accuracy, significantly better than random chance (50%) but far from perfect.
- Tax Liability Calculations: On 100 tax liability calculation cases from SARA, GPT-4 performed moderately well:
- 76 cases were within 10% of the correct liability.
- 46 cases were within 1% of the correct liability.
- 31 cases were within $1 of the correct liability.
- Error Analysis: For cases where GPT-4 was off by over 1%, the errors were almost exclusively due to misreading statutes, not mathematical mistakes. Common errors involved misinterpreting exemption rules and standard deduction provisions.
Limitations and Future Work
- Limitations:
- The SARA dataset, while designed to reduce ambiguity, is a simplified version of the actual IRC. The authors acknowledge that real-world tax law is far more complex.
- GPT-4's performance on the SARA dataset is not perfectly representative of its overall capabilities, as the authors note that less than a third of cases were within $1 of the correct liability.
- GPT-4 is not deterministic, requiring multiple runs for consistent results, especially with higher temperature settings.
- Future Work:
- The authors are optimistic that future models will improve and be able to assist in proactively finding tax minimization strategies, as envisioned by their "Shelter Check" project.
- They emphasize the need for continued research into AI models' ability to reason over complex statutory language and the importance of rigorous evaluation before deploying them in critical applications.
- The authors strongly advise users to "always check with your tax adviser," highlighting that current AI models are not yet reliable replacements for human expertise in tax matters.
Summary updated