Recurrent Neural Network for Inertial Gait User Recognition in Smartphones

Pablo Fernández López, Judith Liu-Jimenez, Kiyoshi Kiyokawa, Yang Wu, Raul Sánchez-Reillo2019 · DOI

TL;DR

This paper introduces a novel gait recognition system for smartphones using Recurrent Neural Networks (RNNs) to analyze inertial sensor data. By processing gait cycles extracted from accelerometer and gyroscope readings, the system achieves competitive accuracy, demonstrating the potential of smartphone-based behavioral biometrics for enhanced security.

What Problem Does This Paper Address?

In an era where smartphones hold vast amounts of sensitive personal information, their security is paramount. While traditional methods like PINs and patterns are vulnerable, and physical biometrics like fingerprints and facial recognition have their own limitations, behavioral biometrics offer a promising alternative. Gait recognition, the identification of individuals based on their walking patterns, is particularly attractive for smartphones because the necessary sensors are already embedded, and the user doesn't need to perform any extra action. However, accurately recognizing gait from noisy, variable smartphone sensor data, especially with limited training data per user, remains a significant challenge. This paper aims to develop and optimize an RNN-based gait recognition system for smartphones that can achieve high accuracy and scalability.

Methods

The core of the proposed system is a Recurrent Neural Network (RNN) architecture designed to process gait cycles. The process begins with Gait Cycle Extraction, where raw inertial data (accelerometer and gyroscope) is first processed to create a magnitude signal. This signal is then analyzed to identify distinctive minima corresponding to foot strikes, which serve as starting points for gait cycles. These cycles, which are segments of the original signal, are then fed into the RNN for Feature Vector Generation. The RNN, specifically using Long Short-Term Memory (LSTM) cells, transforms each gait cycle into a fixed-size feature vector. To handle variations in gait cycle length, dynamic RNNs are employed, and shorter cycles are padded. For an entire walking session, the feature vectors from individual gait cycles are averaged to create a single template. Finally, Comparison is performed by calculating the Euclidean distance between the generated feature vector and stored user templates.

A key aspect of this research is the meticulous approach to finding the optimal RNN configuration. The authors employed a two-stage process: first, a random grid search to explore a wide range of hyperparameters (number of RNN layers, fully connected layers, filters, and feature vector size), and then hand-tuning to fine-tune the best configurations. This systematic approach allowed them to identify promising architectures and understand the influence of different parameters on performance.

Datasets

The primary dataset used for training and evaluation is the Osaka University Database (OUDB) [27]. This database is notable for its size, containing data from 744 users (389 male, 355 female), with an even gender distribution and a wide age range. It includes inertial data from both accelerometers and gyroscopes, collected from sensors placed on the user's waist. The OUDB consists of two datasets: one on a leveled surface and another on a sloped surface. For this study, the authors focused on the leveled surface dataset, which is a common practice for benchmarking. The paper also mentions that the OUDB is open access, facilitating benchmarking and comparison with other algorithms.

Key Insights and Ideas

  • Smartphone Inertial Sensors are Sufficient for Gait Recognition: The study demonstrates that the accelerometer and gyroscope embedded in standard smartphones are capable of capturing enough information to distinguish individuals based on their gait.
  • RNNs Excel at Sequential Gait Data: Recurrent Neural Networks, particularly LSTMs, are well-suited for processing the sequential nature of gait cycle data, effectively extracting discriminative features.
  • Gait Cycle Magnitude Signal Simplifies Extraction: Using the magnitude of the accelerometer signal simplifies the complex task of gait cycle extraction by providing a more robust and periodic signal, even when the phone's orientation changes.
  • User-Centric Database Splitting is Crucial for Scalability: The authors' decision to split the database by users rather than by visits is a critical insight. This approach provides a more realistic evaluation of the system's scalability and its ability to generalize to new users, which is essential for practical biometric applications.
  • Hyperparameter Tuning is Key to Performance: The extensive hyperparameter optimization process, combining random search and hand-tuning, highlights the significant impact of network architecture and configuration on the final accuracy of gait recognition systems.

Key Results

The proposed RNN-based gait recognition system achieved competitive results on the OUDB.

  • The best-performing configuration, RNN1_F64_FC2_V128, achieved an Equal Error Rate (EER) of 7.55% when trained with 70% of the users.
  • Even with a smaller training set (20% of users), the system achieved an EER of 11.48%.
  • The study demonstrated a clear trend: increasing the amount of training data generally led to lower error rates, suggesting that the system's performance could be further improved with larger databases.
  • The system's performance was benchmarked against several state-of-the-art algorithms, showing competitive results, particularly when considering the user-centric approach to database splitting.

Limitations and Future Work

  • Database Size and Visit Variability: While the OUDB is one of the largest available, the authors acknowledge that the number of visits per user is limited. This restricts the study of intravariability (how a single user's gait changes over time) and suggests that larger databases with more samples per user are needed for further advancements.
  • Comparison with Physical Biometrics: The paper notes that the achieved EER of 7.55% is not yet competitive with established physical biometrics like fingerprint or facial recognition. However, it emphasizes that gait recognition is a relatively young modality and can be used in conjunction with other biometrics.
  • Future Work: The authors suggest that future research should focus on acquiring larger databases with more visits per user. They also propose exploring the combination of gait recognition with other biometric modalities to enhance overall security. The potential for gait recognition in various applications beyond smartphone access, such as building access or healthcare systems, is also highlighted.

Summary updated

BibTeX
@misc{lopez2019recurrent,
  author = {Pablo Fernández López and Judith Liu-Jimenez and Kiyoshi Kiyokawa and Yang Wu and Raul Sánchez-Reillo},
  title = {{Recurrent Neural Network for Inertial Gait User Recognition in Smartphones}},
  year = {2019},
  doi = {10.3390/s19184054},
}
Figure