Person Identification from Contextual Motion

Igor Kviatkovsky2019

TL;DR

This paper introduces a novel interactive framework for person identification using body motion, where a system actively probes users with visual cues to infer their identity. They propose a unifying probabilistic model for three scenarios (smart home, login, and interactive) and demonstrate its effectiveness on existing and a new dataset, achieving high accuracy even with noisy depth sensor data.

What Problem Does This Paper Address?

Identifying individuals based on their motion patterns is a valuable biometric modality, especially when traditional methods like face recognition are hindered by poor lighting, occlusions, or privacy concerns. While previous work focused on analyzing locomotion (like gait) or assumed fully cooperative subjects, this paper tackles the challenge of identifying people from more general, everyday actions in both passive (smart home) and active (interactive) scenarios. A key challenge is inferring identity from motion that may not have a clear semantic label, and doing so efficiently, especially in an interactive setting where the system can actively guide the identification process.

Methods

The core of the paper is a probabilistic generative framework that models how action instances are created. This framework is adapted for three scenarios:

  • Smart Home: The system observes motion but doesn't know the action type or identity. It infers both.
  • Login: The action type is known (e.g., a specific gesture for authentication), and the system infers the identity.
  • Interactive: This is the novel contribution. The system presents visual cues (like moving objects) to the user, who responds with motion. The system uses this interaction to infer identity. The key here is an active cue selection strategy based on mutual information. The system chooses cues that are expected to maximally reduce uncertainty about the user's identity. This is framed as a sequential decision-making problem.

For motion representation, they use normalized joint positions from skeletal data, temporally normalized using cubic spline interpolation or DTW+FTP. To improve discriminative power, they employ action-specific PCA and LDA for dimensionality reduction, creating a Mahalanobis distance metric.

Datasets

The authors evaluate their approach on several publicly available datasets and introduce their own:

  • Public Datasets: NTU RGB+D, MSRC-12, UCFKinect, MSR-Action3D, UTKinect, BodyLogin. These are primarily action recognition datasets, adapted for person identification.
  • CuedId Dataset (New): This dataset was collected using a custom-built interactive system (Kinect sensor, large screen) where subjects played a game involving intercepting virtual objects. It contains 4,476 recordings from 22 subjects responding to 15 types of cues. This dataset is specifically designed for evaluating the interactive person identification scenario.

Key Insights and Ideas

  • Interactive Identification is Powerful: Actively probing users with cues can significantly speed up and improve the accuracy of person identification compared to passive observation.
  • Mutual Information for Active Learning: Using mutual information as a criterion for selecting cues is an effective way to guide the identification process and reduce uncertainty efficiently.
  • Context Matters: The semantic richness of actions (e.g., "throwing" vs. "walking") strongly influences identification accuracy. More semantically rich actions lead to better performance.
  • Action-Specific Metrics Improve Performance: Learning distance metrics tailored to specific actions (using PCA-LDA) significantly outperforms generic Euclidean distance.
  • Heterogeneous Action Sets are Better: For identification, using a diverse set of action instances (heterogeneous) is generally more effective than relying on multiple instances of the same action (homogeneous), especially as the number of users increases.

Key Results

  • High Accuracy in Known Scenarios: For the Login and Smart Home scenarios, they achieve high True Positive Rates (TPRs), often exceeding 95% on datasets with semantically rich actions (e.g., MSRC-12, BodyLogin).
  • Interactive Scenario Performance: In the interactive CuedId dataset, their IG-Max cue selection strategy significantly outperforms a random strategy, requiring fewer cues on average to reach a desired confidence level. For 22 users, 90% confidence is reached in about 7 iterations with IG-Max versus 9 with random selection. This gap widens with more users.
  • Dataset Dependency: Performance is lower on datasets with more primitive actions (e.g., UTKinect, CuedId), highlighting the importance of semantic context.
  • Open-Set Recognition: The system shows promise in open-set scenarios, where it can distinguish between enrolled and unknown individuals.

Limitations and Future Work

  • Deep Learning Integration: The authors hypothesize that combining their generative models with powerful action representations learned by deep learning models could further improve performance across all scenarios.
  • Continuous Cues: Currently, cues are discrete. Future work could explore defining cues as continuous variables, allowing for more generalized interaction.
  • Broader Applications: The framework is suggested for other classification tasks beyond person identification, such as age/gender estimation or medical condition diagnosis.
  • Real-World Complexity: While the interactive system is functional, real-world deployment might require handling more complex user behaviors and environmental factors.

Summary updated

BibTeX
@misc{kviatkovsky2019person,
  author = {Igor Kviatkovsky},
  title = {{Person Identification from Contextual Motion}},
  year = {2019},
}
Figure