Tool for Automated Tax Coding of Invoices

Tarun Tater, Sampath Dechu, Neelamadhav Gantayat, Meena Guptha, Sivakumar NarayananAAAI 2021 · DOI

Abstract

In corporate finance, assigning correct tax codes to invoices is a high-stakes task prone to human error and regulatory complexity. This paper introduces a hybrid AI system developed by IBM researchers that automates tax code determination. By integrating semantic similarity and rule-based engines with a human-in-the-loop feedback loop, the system achieves over 94% accuracy. Deployed at a multinational consumer goods company, it successfully automates 73% of invoices, significantly reducing manual effort and audit risks.

Introduction

Accounts Payable (AP) is a critical environment where every invoice must be assigned a specific tax code based on purchase details, vendor location, and destination. Incorrect coding leads to audits, interest, and heavy penalties. Researchers from IBM presented a tool at AAAI-21 designed to transform this daunting manual task into an automated pipeline that learns from its mistakes and adapts to regulatory changes.

Problem Statement: The Complexity of Indirect Tax

Tax code determination is a bottleneck in the invoice-to-pay process. Indirect taxes, such as Value Added Tax (VAT) or Goods and Services Tax (GST), are governed by complex, country-specific regulations that change frequently. Currently, most AP teams perform this manually, requiring processors to cross-reference item descriptions with vendor data and "cheat sheets." This process is slow, error-prone, and struggles with "concept drift"—where legal changes render previous automation rules obsolete.

Methodology: A Hybrid AI Approach

The proposed system utilizes a hybrid classifier that mimics human expert decision-making. The architecture consists of several integrated stages:

  1. Semantic Similarity Engine: To handle messy or abbreviated invoice descriptions, the system uses an Information Retrieval (IR) approach to find the most similar historical item descriptions.
  2. Rule Engine: Results are filtered through business rules checking geographical data (origin vs. destination), company codes, and vendor details.
  3. Confidence Scoring ($C_p$): A confidence score is calculated based on description similarity, vendor matching, and a majority vote from top historical results.
  4. Human-in-the-Loop Feedback: The system employs two thresholds ($C_{min}$ and $C_{max}$). High-confidence items ($C_p > C_{max}$) are auto-posted. Medium-confidence items are sent to human agents for validation, which fine-tunes the model. Low-confidence items are marked "Unknown" for manual entry.
  5. Drift Detection: To manage changing laws, the system prioritizes recent data and uses a sliding window precision algorithm to flag performance drops.
Hybrid architecture of the tax prediction engine.

Datasets and Evaluation

The system was trained and evaluated using real-world historical Enterprise Resource Planning (ERP) data from a large multinational consumer goods company. The training set included approximately 53,000 items from Italian invoices over a 14-month period. A test set of 10,745 data points was used for initial validation, followed by a production evaluation of 22,375 invoices to measure real-world impact.

Key Insights

  • Prioritizing Precision: In financial compliance, a wrong guess is costlier than no guess. The system is designed to return "Unknown" for low-confidence items, ensuring high precision for automated actions.
  • Recency as Adaptation: By weighting recent data more heavily, the system inherently adapts to shifts in tax law without requiring manual rule updates for every regulatory change.
  • Dynamic Thresholding: Agent feedback is used to dynamically adjust confidence thresholds. If agents begin downvoting a specific category, the system automatically raises the requirement for auto-posting in that category.

Results and Generalizability

The system was deployed in production for over six months in Italy, achieving an overall accuracy of 94.46%. Notably, for the 73% of invoices that were auto-posted (high-confidence), accuracy reached 99.54%. This automation saves approximately 400 work-hours per country per year. Furthermore, the system demonstrated high generalizability, maintaining accuracy above 90% across 19 other European countries with minimal configuration.

System performance across various European countries.

Limitations and Future Work

While successful, the system faces a "cold start" problem when entering new countries or facing complete tax regime overhauls, requiring an initial "hyper-care" period of human-led data collection. Future work focuses on more sophisticated data purging methods for irrelevant historical data and scaling the tool to more volatile regulatory environments, such as Latin America.

Summary updated

BibTeX
@inproceedings{tater2021tool,
  author = {Tarun Tater and Sampath Dechu and Neelamadhav Gantayat and Meena Guptha and Sivakumar Narayanan},
  title = {{Tool for Automated Tax Coding of Invoices}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i17.17782},
}
Figure