AIDRIN 2.0: A Framework to Assess Data Readiness for AI

Kaveen Hiniduma, Dylan Ryan, Suren Byna, Jean Luca Bez, Ravi K. MadduriarXiv 2025 · DOI · arXiv

Introduction

The efficacy of artificial intelligence (AI) is fundamentally constrained by the quality of its training data. In the domain of machine learning, the "Garbage In, Garbage Out" (GIGO) principle remains a persistent hurdle. Data professionals often struggle with manual, inconsistent processes for evaluating data readiness, a challenge that is significantly amplified in Federated Learning (FL) environments. In FL, data is decentralized across multiple organizations, and privacy regulations often prevent researchers from directly inspecting raw data. AIDRIN 2.0 addresses this by providing a framework to audit data readiness at the "edge" before it is utilized by AI models.

The AIDRIN 2.0 Framework

AIDRIN 2.0 categorizes data readiness into six core pillars:

  1. Data Quality: Assessing completeness, identifying duplicates, and detecting outliers.
  2. Understandability & Usability: Ensuring adherence to FAIR (Findable, Accessible, Interoperable, Reusable) principles.
  3. Structure & Organization: Validating data types and consistent formatting.
  4. Governance: Evaluating privacy risks, such as re-identification potential.
  5. Impact on AI: Analyzing feature relevance and statistical correlations.
  6. Fairness: Detecting class imbalances and representational biases.

The framework integrates with APPFLx (Advanced Privacy-Preserving Federated Learning as a Service). This allows AIDRIN 2.0 to run locally on edge devices, computing summary statistics and visualizations without exposing sensitive raw information. Only these high-level evaluation results are transmitted to a central server to generate a comprehensive readiness report.

The AIDRIN 2.0 interface provides a centralized view of data quality metrics, including summary statistics, distribution estimates, and completeness scores.

Experimental Validation

The framework was validated using two distinct datasets:

  • MIDRIC (Medical Imaging and Data Resource Center): A public collection of de-identified COVID-19 chest images used to demonstrate the revamped user interface and reporting capabilities.
  • FLamby Heart Disease Dataset: A real-world federated dataset from four hospitals, used to test AIDRIN’s ability to detect "poisonous" or unready data in decentralized settings.

Results and Impact

A key finding from the FLamby case study was the identification of a "bad actor" client. AIDRIN’s local analysis revealed that one client's dataset had a 100% class imbalance and highly sparse features.

Visualization of a problematic dataset showing extreme class imbalance (100%) and feature correlation patterns that can degrade global model performance.

By identifying and excluding this problematic client, the global model accuracy improved from 70.6% to 74.7%. This demonstrates that in federated learning, more data is not always better; uninformative or biased data from a single source can degrade the entire system. Early detection through AIDRIN 2.0 not only improves model performance but also prevents the waste of computational resources on flawed training runs.

Conclusion and Future Work

AIDRIN 2.0 provides a critical "pre-flight" check for AI, particularly in privacy-sensitive federated environments. While the framework significantly enhances data transparency, future work will focus on improving scalability for high-dimensional datasets at the extreme edge and developing automated remediation tools to suggest data cleaning steps based on the generated readiness reports.

Summary updated

BibTeX
@misc{hiniduma2025aidrin,
  author = {Kaveen Hiniduma and Dylan Ryan and Suren Byna and Jean Luca Bez and Ravi K. Madduri},
  title = {{AIDRIN 2.0: A Framework to Assess Data Readiness for AI}},
  year = {2025},
  eprint = {2505.18213},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.18213},
}
Figure