Person Identification and Relapse Detection From Continuous Recordings of Biosignals Challenge: Overview and Results

Athanasia Zlatintsi, Panagiotis P. Filntisis, Niki Efthymiou, Christos Garoufis, George Retsinas, Thomas Sounapoglou, Ilias Maglogiannis, Panayiotis Tsanakas, Nikolaos Smyrnis, Petros MaragosIEEE Open Journal of Signal Processing 2024 · DOI

Abstract

This paper presents an overview of the e-Prevention: Person Identification and Relapse Detection Challenge, which was an open call for researchers at ICASSP-2023. The challenge aimed at the analysis and processing of long-term continuous recordings of biosignals recorded from wearable sensors, namely accelerometers, gyroscopes and heart rate monitors embedded in smartwatches, as well as sleep information and daily step counts, in order to extract high-level representations of the wearer's activity and behavior, termed as digital phenotypes. Specifically, with the goal of analyzing the ability of these digital phenotypes in quantifying behavioral patterns, two tasks were evaluated in two distinct tracks: 1) Identification of the wearer of the smartwatch, and 2) Detection of psychotic relapses in patients in the psychotic spectrum. The long-term data used in this challenge were acquired during the e-Prevention project, an innovative integrated system for medical support that facilitates effective monitoring and relapse prevention in patients with mental disorders. Herein, we present an overview of the approaches and methods as well as the performance analysis and the results of the top-ranked participating teams.

Introduction

Clinical psychiatry has traditionally relied on reactive, in-person assessments, which often miss the early warning signs of a mental health crisis. Digital phenotyping—the moment-by-moment quantification of the individual-level human phenotype in situ using data from personal digital devices—offers a proactive alternative.

However, the field faces a major bottleneck: the lack of large-scale, long-term, and publicly available datasets. Most studies last only a few weeks. This paper summarizes a challenge designed to push the boundaries of signal processing by providing a massive dataset (20,000 human-days) to solve two problems:

  1. Person Identification: Can we uniquely identify a wearer just by how they move and their physiological response? This serves as a proxy for the distinctness of behavioral patterns.
  2. Relapse Detection: Can we detect a psychotic relapse as an "anomaly" in a patient's typical behavior using only wearable data?

Methodology

The challenge was divided into two tracks, with participants employing a mix of standard and cutting-edge techniques:

  • Track 1 (Identification): The baseline used a 1D Convolutional Neural Network (CNN). Top-performing teams improved on this using ensembles of 1D CNNs, LSTMs for temporal modeling, and Transformers. A key technical hurdle was handling "missingness"—data gaps caused by the watch being charged or taken off. Successful teams used Nearest Neighbor Interpolation (NNI) or developed "missingness-aware" frameworks.
  • Track 2 (Relapse Detection): This was treated as an unsupervised anomaly detection task. Since relapse data is scarce, models were trained on "stable" periods. Baselines used linear Autoencoders, where a high reconstruction error on new data signaled a potential relapse. Top teams experimented with CNN-based autoencoders, U-Nets, and Isolation Forests.
Model architectures and data modalities used by top teams for the Person Identification task.

Dataset Overview

The challenge utilized the e-Prevention dataset, which is now publicly available.

  • Scope: 60 participants (37 patients in the psychotic spectrum, 23 healthy controls).
  • Duration: Up to 2.5 years of continuous monitoring.
  • Sensors: Tri-axial accelerometer, gyroscope, heart rate (PPG), RR-intervals, sleep schedules, and daily step counts.
  • Uniqueness: Unlike many datasets that provide high-level summaries, this provided aggregated biosignals at a 5-second resolution, allowing for much finer-grained behavioral analysis.

Key Insights and Ideas

  • Rawer is Better: Top-performing teams moved away from traditional "hand-crafted" features (like mean/variance over long windows) and instead fed high-resolution (5-second) raw sensor data directly into deep networks.
  • The "Sleep" Signal: Features extracted during sleep were often more predictive of a patient's mental state than daytime activity. This aligns with clinical knowledge that sleep disruption is a primary indicator of psychotic relapses.
  • Personalization is Key: For relapse detection, "personalized" models (trained or normalized specifically for one patient) significantly outperformed global models. What constitutes an "anomaly" for one person might be "normal" for another.
  • Kinetic Dominance: In the identification task, kinetic data (accelerometer) proved more discriminative than physiological data (heart rate), suggesting our movement patterns are highly unique "biometric signatures."

Key Results

  • Track 1 (Identification): The winning team (SRCB-LUL) achieved 95% accuracy using a mid-depth 1D CNN. The top three teams all stayed above 91%, proving that digital phenotypes are remarkably stable and unique.
  • Track 2 (Relapse Detection): This proved much harder. The top team (PeRCeiVe) achieved a mean PR-AUC and ROC-AUC of approximately 0.6489. While this is a significant improvement over random chance, it highlights the inherent difficulty of detecting complex medical events from noisy wearable data.
Performance results for Person Identification and Relapse Detection tasks across top teams.

Limitations and Future Work

The authors and participants noted several areas for improvement:

  • Temporal Continuity: Future research should utilize the full temporal sequence to model how behavior evolves over weeks leading up to a relapse, rather than shuffling days.
  • Data Resolution: Plans are in place to release completely raw sensor data in future iterations to allow for even more complex feature extraction.
  • Broader Phenotypes: Future challenges aim to include depressive relapses and incorporate demographic data (age, gender) to see if it aids in predictive accuracy.
  • Real-world Deployment: Moving from a challenge setting to a real-time clinical tool requires addressing the "black box" nature of deep learning to provide interpretable results for clinicians.

Summary updated

Abstract

This paper presents an overview of the e-Prevention: Person Identification and Relapse Detection Challenge, which was an open call for researchers at ICASSP-2023. The challenge aimed at the analysis and processing of long-term continuous recordings of biosignals recorded from wearable sensors, namely accelerometers, gyroscopes and heart rate monitors embedded in smartwatches, as well as sleep information and daily step counts, in order to extract high-level representations of the wearer's activity and behavior, termed as digital phenotypes. Specifically, with the goal of analyzing the ability of these digital phenotypes in quantifying behavioral patterns, two tasks were evaluated in two distinct tracks: 1) Identification of the wearer of the smartwatch, and 2) Detection of psychotic relapses in patients in the psychotic spectrum. The longterm data that have been used in this challenge have been acquired during the course of the e-Prevention project [1] (https://eprevention.gr/), an innovative integrated system for medical support that facilitates effective monitoring and relapse prevention in patients with mental disorders. Two baseline systems, one for each task, were described and the validation scores for both tasks were provided to the participants. Herein, we present an overview of the approaches and methods as well as the performance analysis and the results of the 5-top ranked participating teams, which in track 1 achieved accuracy results between 91%–95%, while in track 2 mean PR- and ROC-AUC scores between 0.6051 and 0.6489 were obtained. Finally, we also make the datasets publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://robotics.ntua.gr/eprevention-sp-challenge/</uri> .
BibTeX
@article{zlatintsi2024person,
  author = {Athanasia Zlatintsi and Panagiotis P. Filntisis and Niki Efthymiou and Christos Garoufis and George Retsinas and Thomas Sounapoglou and Ilias Maglogiannis and Panayiotis Tsanakas and Nikolaos Smyrnis and Petros Maragos},
  title = {{Person Identification and Relapse Detection From Continuous Recordings of Biosignals Challenge: Overview and Results}},
  journal = {IEEE Open Journal of Signal Processing},
  year = {2024},
  doi = {10.1109/ojsp.2024.3376300},
}
Figure