Problems of video-based pain detection in patients with dementia: a road map to an interdisciplinary solution

Miriam Kunz, Dominik Seuß, Teena Hassan, Jens-Uwe Garbas, Michael Siebers, Ute Schmid, Michael Schöberl, Stefan LautenbacherBMC 2017 · DOI · OpenAlex

Abstract

We hope to encourage colleagues to join forces and to inform end-users about an imminent solution of a pressing pain-care problem. For the near future, implementation of such systems can be foreseen to monitor immobile patients in intensive and postoperative care situations.

Introduction: The Challenge of Pain Assessment in Dementia

Pain assessment in patients with dementia represents a significant clinical hurdle. Statistics indicate that up to 80% of individuals in care facilities suffer from regular pain, yet a substantial portion remains undertreated. This discrepancy arises because cognitive decline often strips patients of their ability to communicate their distress effectively.

While the current gold standard relies on observation by trained professionals, this method is limited by time constraints, inherent observation biases, and the subtle, fleeting nature of pain-indicative facial expressions. Automated, video-based systems are proposed not as a replacement for human caregivers, but as a continuous "vital sign" monitor to alert staff to potential pain events in real-time.

The Proposed Technical Pipeline

The authors outline a multi-stage framework that bridges psychological theory with advanced machine learning techniques to detect pain objectively.

  1. Robust Face Capturing: Using the Sophisticated High Speed Object Recognition Engine (SHORE), the system localizes the face and key landmarks (eyes, nose, mouth). This is critical for bedridden patients who may have varying head orientations.
  2. Facial Action Coding System (FACS): Rather than identifying a holistic "pain face," the system decomposes expressions into individual Action Units (AUs)—specific muscle movements like brow lowering or lip raising.
  3. Temporal State Modeling: By connecting successive frames, the system accounts for the physical properties of facial muscles, filtering out noise and smoothing the output for better accuracy.
  4. Online Calibration: The system automatically determines a patient's baseline expression over time, avoiding the errors associated with static, one-time calibration.
  5. Knowledge-Level Diagnosis: Using Inductive Logic Programming (ILP), the system applies symbolic rules to detected AUs. This ensures that the AI's "decisions" are interpretable by clinicians.
Technical steps for the automated pain detection system

Data and Validation

The framework's development and validation draw from several key sources:

  • Public Benchmarks: The SHORE face detection engine achieved a 91.5% detection rate on the CMU + MIT dataset.
  • Proprietary Clinical Data: Video recordings of both dementia patients and cognitively healthy elderly individuals, manually annotated by FACS experts, were used to refine the AU detection.
  • Standardized Coding: Adherence to the international FACS standard ensures that the system's findings are compatible with broader psychological research.

Clinical Insights and Interpretability

A primary challenge in this field is distinguishing pain from other forms of distress, such as agitation. Research suggests that pain is not represented by a single expression but by four distinct facial activity patterns. For example, narrowed eyes often signal sensory pain, while furrowed brows relate to the affective dimension.

Furthermore, the system must account for pathological alterations; for instance, Alzheimer’s may increase facial expressiveness to pain, while Parkinson’s often reduces it. The use of ILP is vital here, as it allows the system to explain its reasoning (e.g., "detected AU4 and AU7 at high intensity"), fostering trust among medical staff.

Future Directions and Limitations

While foundational results are promising—identifying eye muscle contraction and upper lip raising as stable pain indicators—several hurdles remain. Current technology is best suited for immobile patients; future work must address mobile patients through multi-camera setups. Additionally, environmental factors like low-light conditions may require the integration of ultraviolet (UV) or infrared sensors. The ultimate goal is to transition from offline analysis to real-time monitoring in intensive and postoperative care environments.

Summary updated

Abstract

We hope to encourage colleagues to join forces and to inform end-users about an imminent solution of a pressing pain-care problem. For the near future, implementation of such systems can be foreseen to monitor immobile patients in intensive and postoperative care situations.
BibTeX
@inproceedings{kunz2017problems,
  author = {Miriam Kunz and Dominik Seuß and Teena Hassan and Jens-Uwe Garbas and Michael Siebers and Ute Schmid and Michael Schöberl and Stefan Lautenbacher},
  title = {{Problems of video-based pain detection in patients with dementia: a road map to an interdisciplinary solution}},
  booktitle = {BMC},
  year = {2017},
  doi = {10.1186/s12877-017-0427-2},
}
Figure