Abstract
Parkinson’s Disease (PD) is the world’s fastest-growing neurological disorder, yet it remains significantly under-diagnosed, particularly in underserved regions. We present UFNet (Uncertainty-calibrated Fusion Network), a multimodal AI system that detects PD using standard webcam videos of simple at-home tasks. By dynamically weighting inputs from finger-tapping, facial expressions, and speech based on their individual prediction "certainty," the model achieves a high diagnostic accuracy of 88% and an AUROC of 93%.
Introduction
Early intervention in Parkinson’s Disease is critical, but clinical diagnosis currently requires expensive, specialized care. While AI-based screening tools exist, they typically focus on a single "modality"—either just speech or just motor function. Because PD symptoms are highly variable, unimodal models often miss the broader clinical picture. The challenge lies in creating a system that can analyze multiple symptoms simultaneously and, crucially, determine which "signal" to trust when data is noisy or ambiguous.
Methods: The UFNet Architecture
The researchers proposed a two-stage classification architecture designed to handle the inherent variability of at-home data collection.
- Task-Specific Modeling: Three independent neural networks analyze distinct tasks: finger-tapping (motor), smiling (facial expression), and speaking a pangram (speech). These models utilize Monte Carlo (MC) Dropout during both training and inference. By running multiple passes on the same data, the model generates a prediction and a standard deviation ($\sigma$) representing its uncertainty.
- Uncertainty-calibrated Fusion Network (UFNet): This core component merges the three signals. Unlike standard fusion techniques that simply concatenate data, UFNet uses a Calibrated Self-Attention mechanism. It adjusts the attention weights of each task based on the uncertainty scores from Stage 1. If a specific model (e.g., speech) is "confused" by background noise, UFNet automatically down-weights its contribution and relies more on clearer signals from other tasks.
- Safety via Conformal Prediction: To prioritize patient safety, the model can "withhold" a prediction. If the system cannot distinguish between PD and Non-PD with high confidence, it marks the case as "uncertain" rather than risking a false result.
Datasets and Validation
The study introduced a significant new benchmark, the Multi-Task Video Dataset, comprising 1,102 sessions from 845 unique participants. This includes over 3,300 videos covering finger-tapping, smiling, and speech. Features were extracted using clinical hand-crafted methods (MediaPipe, OpenFace) for motor/facial tasks and high-dimensional embeddings (WavLM) for speech. To ensure generalizability, the model was also validated on the external YouTubePD dataset.
Results and Key Insights
UFNet demonstrated superior performance compared to traditional "Early" or "Late" fusion baselines. A key finding was that treating model uncertainty as a weighting factor significantly improves robustness.
- Performance Metrics: UFNet achieved 88.0% accuracy and 93.0% AUROC on a subject-separated test set. It maintained high sensitivity (79.3%) and specificity (92.6%).
- Modality Strength: In isolation, speech was the strongest signal (89.4% AUROC), while finger-tapping was the weakest. However, the multi-task combination consistently outperformed any single task.
- Generalization: On the external YouTubePD dataset, UFNet showed an 18% improvement in accuracy over previous state-of-the-art methods without specific fine-tuning.
- Demographic Robustness: The model showed no detectable bias across sex or ethnic subgroups, performing most effectively for the 50–80 age bracket.
Discussion and Limitations
While promising, the system has limitations. Accuracy drops for younger (30–49) and much older (80+) individuals, likely due to their under-representation in the training data. Additionally, the speech task is currently limited to English-language pangrams. Future work will focus on expanding language support and integrating real-time quality assessment to help users record higher-quality videos at home.
Summary updated