Measuring Data Collection Diligence for Community Healthcare

Ramesha Karunasena, Mohammad Sarparajul Ambiya, Arunesh Sinha, Ruchit Nagar, Saachi Dalal, Hamid Abdullah, Divy Thakkar, Dhyanesh Narayanan, Milind Tambe

Abstract

In developing countries, community health workers (CHWs) are essential for population health data collection, yet this data is often compromised by "non-diligence"—intentional or unintentional inaccuracies in reporting. This paper introduces a framework to quantify and predict CHW diligence in low-resource settings. By converting heuristic rules into relative probability scores via Kernel Density Estimation (KDE) and using k-means clustering for interpretability, the system achieved an 84% effective accuracy in identifying non-diligent data collection during field trials in Rajasthan, India.

Introduction

Community Health Workers, such as Auxiliary Nurse Midwives (ANMs) in India, form the backbone of rural healthcare systems. They collect vital data used for resource allocation and policy-making. However, "non-diligence"—such as rounding off blood pressure readings, skipping tests while recording results, or falsely reporting equipment failure—frequently plagues these datasets.

Traditional technical solutions face three major hurdles:

  1. Lack of Ground Truth: There are no labels for which records are fraudulent or lazy.
  2. Prevalence of Poor Data: In many regions, poor data collection is the norm rather than the exception, causing standard unsupervised anomaly detection to fail.
  3. Contextual Variability: Data quality may be affected by systemic issues (e.g., broken equipment) rather than individual negligence.

Methodology

The proposed system moves from raw logs to actionable insights through a multi-step pipeline involving rule processing, relative scoring, and clustering.

Overview of the Diligence Explanation and Prediction System architecture.

Relative Scoring via KDE

The core innovation is the use of Kernel Density Estimation (KDE) to calculate relative non-diligence. Instead of absolute thresholds, the system calculates rule violation percentages for each worker and maps them to a probability distribution across the entire cohort. This ensures that if a specific task is difficult for everyone due to systemic factors, an individual worker is not unfairly penalized.

Kernel Density Estimation (KDE) for blood pressure rule violations, showing the distribution of non-diligence across the cohort.

Scoring and Prediction

To rank workers, 11 different rule probabilities are condensed into a single scalar "Diligence Score" using the Euclidean norm of the probability vector. The system then employs a linear regressor to predict future diligence scores based on a six-month history. Simple linear models were found to outperform complex neural networks like LSTMs due to the limited dataset size.

Interpretable Clustering

For field managers, the system uses k-means clustering ($k=3$) to group workers into behavioral profiles, such as "Generally Diligent" or "Poor BP Recording." This provides context for interventions.

Datasets and Validation

The system was evaluated using data from 74 ANMs in Rajasthan, India, collected between January 2020 and January 2021. Field validation was conducted by monitors from the NGO Khushi Baby, who performed 37 physical audits of health camps to verify recorded data against actual performance.

Results and Discussion

The system demonstrated significant predictive and diagnostic power:

  • Field Accuracy: The AI's predictions matched field observations in 67% of cases. An additional 17% of cases identified by the AI as non-diligent were later verified through post-hoc audits, resulting in an 84% effective accuracy.
  • Predictive Power: A correlation of 0.54 between predicted and true diligence scores provides a viable "early warning" system for managers.
  • Superiority over Baselines: The relative scoring approach significantly outperformed VAE-based anomaly detection, which could not distinguish diligent groups when non-diligence was prevalent.

Conclusion

The transition from "black box" models to interpretable, relative scoring was key to making the tool production-ready. By focusing on relative performance, the system accounts for systemic challenges while identifying individual outliers. Future work will explore incorporating GPS and timestamp data to further refine clustering and develop ethical intervention strategies focused on training and encouragement rather than punishment.

The system is currently being deployed on government servers in Rajasthan to support health tracking for millions.

Summary updated

BibTeX
@inproceedings{karunasena2021measuring,
  author = {Ramesha Karunasena and Mohammad Sarparajul Ambiya and Arunesh Sinha and Ruchit Nagar and Saachi Dalal and Hamid Abdullah and Divy Thakkar and Dhyanesh Narayanan and Milind Tambe},
  title = {{Measuring Data Collection Diligence for Community Healthcare}},
  booktitle = {EAAMO},
  year = {2021},
  doi = {10.1145/3465416.3483292},
}
Figure