Collusion in the U.S. crop insurance program: applied data mining

Bertis B. Little, Walter L. Johnston, Ashley C. Lovell, Roderick M. Rejesus, Steve A. SteedKDD 2002 · DOI

TL;DR

This paper applies data mining techniques to identify potential collusion within the U.S. crop insurance program. By analyzing indemnity behavior of agents, adjusters, and producers, the study found that non-recursive triplet and agent-producer doublet relationships are the most probable indicators of waste, fraud, and abuse, offering a scalable method for detecting such patterns.

What Problem Does This Paper Address?

The U.S. crop insurance program, administered by the USDA Risk Management Agency (RMA), is susceptible to waste, fraud, and abuse. The Agriculture Risk Protection Act (ARPA) of 2000 mandated the use of data warehousing and data mining to improve program integrity. Specifically, the law requires identifying entities whose indemnity behavior exceeds 150% of the county mean. This paper addresses the challenge of systematically detecting collusion among the various actors (agents, adjusters, producers) involved in this complex system, moving beyond anecdotal evidence and time-consuming manual investigations. The "cart wheel" metaphor, where agents are the hub, adjusters are spokes, and producers are the rim, is a conceptual model used by investigators to understand potential conspiracies.

Methods

The core methodology employed is log linear analysis, a statistical technique well-suited for analyzing Poisson-distributed count data, such as the number of flagged entities. This approach allows for the flexible modeling of interactions between discrete variables. The researchers defined "anomalous" entities as those exceeding 150% of the county mean for six derived measures of indemnity behavior, using a generalized jackknife procedure to compute county means and reduce bias. They then tested several hypotheses: a null hypothesis of no relationship, hypotheses of linked doublets (e.g., Agent-Producer), and hypotheses of linked triplets (e.g., Agent-Adjuster-Producer). The "best fit" model was identified using a chi-square distributed deviance statistic (-2 log likelihood).

Datasets

The study utilized data from the USDA RMA's national book of insurance business for the year 2000. This dataset comprised approximately 157 million acres insured under 1 million producer policies, sold by 13,434 crop insurance agents, with $2.49 billion in indemnities adjusted by 3,842 adjusters. Catastrophic (CAT) policy indemnities were excluded to avoid biasing the analysis.

Key Insights and Ideas

  • Data mining as a scalable solution: The paper demonstrates that data mining can effectively analyze vast datasets (24 million records in this case) to detect patterns suggestive of collusion, a task previously reliant on manual investigation and luck.
  • "Cart Wheel" metaphor validated: The findings support the "cart wheel" conspiracy model, suggesting that linked anomalous behavior among agents, adjusters, and producers is a significant indicator of potential fraud.
  • Non-recursive triplets are key: The most significant finding is that a non-recursive triplet relationship (where all three entities – agent, adjuster, producer – are linked without a strict hierarchy) and Agent-Producer doublets probabilistically account for the largest proportion of waste, fraud, and abuse. This implies a more interconnected conspiracy than a simple hierarchical structure.
  • Beyond metric data: The analysis highlights that data mining, particularly log linear analysis, can identify collusion even when the data is not strictly metric and doesn't adhere to specific statistical distributions, relying instead on frequency counts of events.

Key Results

  • The non-recursive triplet model (Adjuster-Agent-Producer) provided the best statistical fit (Deviance = 102.9, d.f. = 1), indicating significant sharing of information among all three entities.
  • The Agent-Producer doublet was the second best fit (Deviance = 3,994.1, d.f. = 3).
  • The analysis identified 1,135 potential "triplet" entities for investigation, significantly reducing the scope from over 200,000 entities.
  • Presenting these findings to the USDA RMA led to entities being placed on a "watch" list, resulting in a decrease of over $100 million in indemnity claims from these entities in subsequent years, demonstrating the practical impact of the data mining approach.

Limitations and Future Work

  • Geographic clustering: While the study analyzed geographic distributions, no apparent visual geographic clusters were found for flagged triplets, suggesting the jackknife procedure adequately normalized county-level means. However, flagged Agent-Producer pairs showed some clustering in the Midwest and Great Plains.
  • Defining outliers: The authors suggest that further constraining the outlier definition (e.g., using 95th or 99th percentile thresholds instead of 150% of the county mean) could enhance the yield of investigations.
  • Prioritizing investigation: The paper recommends prioritizing investigation of linked triplets and Agent-Producer doublets with Indemnity-Liability (I-L) ratios approaching 1.0.
  • Broader applicability: The authors propose that this data mining technique can be applied to detect patterns of collusion or conspiracy in various domains, including criminal justice and intelligence agencies, by analyzing different types of event counts (e.g., emails, business transactions).

Summary updated

BibTeX
@inproceedings{little2002collusion,
  author = {Bertis B. Little and Walter L. Johnston and Ashley C. Lovell and Roderick M. Rejesus and Steve A. Steed},
  title = {{Collusion in the U.S. crop insurance program: applied data mining}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2002},
  doi = {10.1145/775047.775135},
}

Loading related content...

Figure