Abstract
The need for automated facial expression analysis arises in various clinical settings involving mental and physical health assessment of older adults. However, the effect of age and ability on the performance of available methods has not yet been investigated. In this paper, we demonstrate a bias affecting the performance of common facial landmark detection and expression recognition algorithms on the faces of older adults with dementia. We also investigate the ways of mitigating this bias via the addition of representative training examples. Results show that landmark placement is less accurate when tested on the faces of individuals with dementia as compared to older adults who are cognitively healthy. Retraining or fine-tuning the methods with images of older adults’ faces improves the performance significantly, but the gap between older adults with versus without dementia persists.
Introduction
Facial analysis technology—specifically landmark detection (identifying the eyes, nose, and mouth) and action unit (AU) recognition (detecting specific muscle movements)—is increasingly proposed for clinical use. These tools could revolutionize patient monitoring, such as tracking pain in non-verbal patients or monitoring the progression of neurological diseases. However, most state-of-the-art models are trained on datasets like 300-W or Multi-PIE, which consist primarily of young, healthy individuals. This study investigates whether these models carry an "ability bias," specifically evaluating if they perform as effectively on older adults with dementia as they do on their cognitively healthy peers.
Experimental Design and Methodology
The researchers evaluated five prominent facial landmark detection algorithms: Active Appearance Models (AAM), Face Alignment Network (FAN), Coarse-to-Fine Shape Searching (CFSS), Mnemonic Descent Method (MDM), and OpenFace. Additionally, two facial action unit detection tools, OpenFace and the Affdex SDK, were tested.
The core evaluation utilized a clinical dataset of 688 frontal images from 86 older adults (average age ~80). The cohort was split between 42 participants with dementia and 44 cognitively healthy controls. To address observed biases, the authors performed retraining and fine-tuning experiments on the AAM and FAN models using representative images of older adults.
The "Dementia Gap" in Landmark Detection
A primary finding of the research is the existence of a significant "Dementia Gap." Even when comparing individuals of the same age, algorithms performed significantly worse on those with dementia. This suggests that dementia-related changes—potentially in skin texture, muscle tone, or facial expressions—create unique challenges for AI that are not present in healthy aging.
For the FAN model, landmark accuracy (defined as a Normalized Root-Mean-Square Error under 5%) was 84.9% for healthy controls but dropped to 72.0% for the dementia group. Notably, the FAN model was trained on over 750,000 images, yet still exhibited this bias, highlighting that massive datasets are not a substitute for representative data.
Action Unit Recognition Challenges
The models struggled immensely with detecting specific facial actions, such as AU4 (brow lowering). Precision was often below 50%, rendering these tools currently unsuitable for automated pain assessment in clinical populations. The models are particularly prone to "false alarms" on older faces, likely because wrinkles or age-related skin sagging are misinterpreted as active muscle movements.
As shown in the confusion matrix, OpenFace achieved a precision of only 30.0% for AU4, while the Affdex SDK reached 41.2%. These low numbers indicate a fundamental failure of off-the-shelf models to generalize to the morphology of older faces.
Mitigation and Fine-Tuning
Retraining and fine-tuning the models with images of older adults significantly boosted performance for both groups. For example, after fine-tuning the FAN model, accuracy for the dementia group jumped from 72.0% to 90.2%. However, the healthy group simultaneously improved to 99.4%, meaning the statistical gap between the two groups persisted. The AI became more accurate overall, but it remained "less fair" to the clinical population.
Conclusion
This study serves as a critical call to action for the computer vision community. As interest in using facial analysis for clinical applications grows, researchers must address the limitations of existing models. Clinical AI must be developed with representative data from the start. Future work should expand to other conditions affecting facial movement, such as Parkinson’s disease or stroke, to ensure that automated health assessments are equitable and effective for all patient populations.
Summary updated