Abstract
We are developing an artificial intelligence (AI)-based health management system that suggests efficient and effective interventions for keeping people healthy based on the potential risk of disease predicted by using AI. We proposed a new feature extraction model for unequally spaced medical data that improves the disease risk prediction. We introduce the model in detail in this article and also describe an application of the model that demonstrates the improved prediction accuracy.
1. Introduction
In clinical practice, medical data is rarely a perfectly regular time series. Patients visit clinics at irregular intervals, and self-monitoring records often contain gaps due to missed entries. This "unequally spaced" nature of real-world data poses a significant challenge for standard machine learning models, which typically require uniform inputs.
The traditional approach to this problem involves a "completion" phase, where missing values are estimated using techniques like linear interpolation. However, these "guesses" introduce noise and errors into the dataset. If the initial interpolation is inaccurate, the subsequent feature extraction and risk prediction are compromised. Researchers at NTT have developed a novel AI-based health management system that bypasses this bottleneck by extracting meaningful features directly from sparse, irregular data.
2. The Corevo Health Management Ecosystem
The proposed system is part of NTT’s "corevo" AI brand, designed to create a closed-loop health management cycle. This cycle involves gathering data from electronic health records (EHRs) and self-monitoring devices, predicting disease risks using advanced AI models, and then delivering targeted interventions through clinicians or social robots.
3. Methods: Feature Extraction without Interpolation
The core innovation of this research is a feature extraction model based on a modified autoencoder. An autoencoder is a neural network that compresses input data into a lower-dimensional "feature" vector (encoding) and then attempts to reconstruct the original data (decoding).
Unlike standard autoencoders that penalize differences across all potential data points, the proposed model utilizes a masking technique. By introducing a Boolean vector that identifies which data points are actually present, the loss function is modified to ignore missing or null values during the training phase. This allows the middle layer of the network to learn the underlying health trends based solely on observed data, effectively skipping the error-prone completion step.
4. Experimental Evaluation and Results
The model's efficacy was tested using a large-scale dataset of EHRs from the University of Tokyo Hospital, comprising over 7,000 diabetic patients. The primary focus was predicting HbA1c (hemoglobin A1c) levels, a critical indicator of long-term blood sugar control. The dataset was highly irregular, with a mean testing interval of 5.9 weeks and a standard deviation of 2.6 weeks.
The results demonstrated a significant improvement over traditional methods:
- Baseline (with interpolation): Achieved an Area Under the Curve (AUC) of 0.72.
- Proposed Model (direct feature extraction): Achieved an AUC of 0.80.
This 0.08 increase in AUC represents a substantial improvement in the reliability of identifying patients at risk of poor glycemic control.
5. Conclusion and Future Directions
By treating missing data as a first-class citizen within the neural network's architecture, this system preserves the integrity of original medical records and improves diagnostic accuracy. Future work will focus on standardizing medical data formats for broader portability, integrating real-time data from wearable devices, and exploring the use of networked robotics to deliver personalized health interventions.
Summary updated