TL;DR
This paper introduces AirSign, a novel smartphone authentication system that uses a combination of acoustic and motion sensors to recognize users by the unique way they sign their name in the air. This approach offers a more convenient and secure alternative to traditional methods, achieving a high F-score of 97.1% without requiring any additional hardware.
What Problem Does This Paper Address?
As smartphones become repositories for increasingly sensitive personal information, robust and user-friendly authentication methods are crucial. Existing methods like PINs are vulnerable to shoulder surfing, fingerprint scanners can be affected by finger condition or spoofed, and advanced facial recognition systems often require specialized hardware that adds cost and bulk. The authors sought to develop an authentication system that is:
- Highly accurate.
- Requires no extra hardware.
- Resistant to shoulder surfing.
- Convenient and flexible to use.
They identified a gap in current authentication technologies that could be addressed by leveraging the ubiquitous sensors already present in smartphones.
Methods
AirSign employs a two-pronged approach using the smartphone's built-in acoustic and motion sensors:
- Acoustic Sensing:
- Hand Geometry: During an initial enrollment phase, the earpiece speaker emits inaudible chirp sound signals (20-23 kHz). These signals reflect off the user's hand held near the screen and are captured by the top and bottom microphones. Features like spectrograms are extracted to represent unique hand geometry.
- Signature Sensing: While the user signs their name in the air, the earpiece emits continuous cosine sound waves with varying frequencies. The echoes captured by the microphones are analyzed to extract features related to displacement, velocity, and acceleration during the signing motion. This utilizes the Doppler effect.
- Motion Sensing: The accelerometer and gyroscope sensors continuously capture the smartphone's movement (acceleration and angular velocity) as the user signs. This is based on the insight that the hand holding the phone will also move in a pattern influenced by the signing hand and the body's overall movement.
- Feature Extraction and Classification: Features from both acoustic and motion sensing are combined to form an "Air Signature." This signature is then passed through three classifiers: a hand geometry classifier (using KNN), a signature classifier, and a motion classifier. The system uses Multidimensional Dynamic Time Warping (MD-DTW) to calculate similarity between registered and new signatures. A cascade of these three classifiers determines if the signature is genuine or forged.
Datasets
The authors collected their own dataset due to the lack of public smartphone-based datasets for this specific task.
- Data Collection: Experiments were conducted with 30 participants over three sessions.
- Session 1: 30 participants provided registered, genuine, and forged signatures.
- Sessions 2 & 3: 10 participants were tested under different poses (standing, sitting, lying) and environments (quiet, talk, music) to evaluate robustness.
- Data Split: The collected data was used for training and testing the KNN model for hand geometry.
- Total Signatures: 3300 signatures were collected in total.
Key Insights and Ideas
- Human Body as an Entity: The core assumption that the entire body moves as a unit when signing in the air is crucial. This implies that the motion of the holding hand can be a reliable indicator of the signing user.
- 3D Signing for Enhanced Security: Signing in the air offers a 3D space, making it inherently harder to imitate compared to 2D signatures on a screen, which are more susceptible to shoulder surfing.
- Leveraging Existing Sensors: The innovation lies in effectively utilizing the readily available acoustic and motion sensors on smartphones, eliminating the need for specialized hardware.
- Complementary Features: Combining acoustic features (hand geometry, signing motion) with motion features (phone movement) provides a richer set of discriminative information, making the system more robust and secure.
Key Results
- Overall Performance: AirSign achieved an F-score of 97.1% when using both acoustic and motion classifiers.
- Individual Classifiers:
- Acoustic classifier only: 92.4% F-score.
- Motion classifier only: 92.0% F-score.
- Robustness: The system showed good performance across different poses and environments, with F-scores generally above 90%.
- Response Delay: The average response delay was 14.8 ms, primarily due to the MD-DTW calculation.
- Registration Requirements: The system achieved a stable F-score of 97.1% with seven registered signatures, balancing performance and time consumption.
- User Experience: Participants found AirSign convenient, flexible, and attractive, with 27 out of 30 finding the process easy to use.
Limitations and Future Work
- Active Motion: The current system is primarily tested with users in static poses. Authenticating users who are actively moving (walking, running) is challenging due to the increased dissimilarity in captured motion data. Future work could explore learning moving patterns or adapting the system for dynamic scenarios.
- Multiple Users: Interference between signals from multiple smartphones in close proximity could affect authentication accuracy. Future work aims to explore dynamic frequency band selection to mitigate this.
- Large-Scale Experiments: The current experiments involved 30 users. Larger-scale studies with over 100 users are needed for a more mature solution.
- Privacy and Memory: While current data storage is minimal, future growth in signature data per user might require memory management strategies. Data is currently anonymous, but encryption technologies could be adopted for international business purposes.
Summary updated