ArtiLock: Smartphone User Identification Based on Physiological and Behavioral Features of Monosyllable Articulation

Aslan B. Wong, Ziqi Huang, Xia Chen, Kaishun Wu

TL;DR

This paper introduces ArtiLock, a novel smartphone authentication system that uses a single spoken syllable to identify users. By analyzing both speech and ultrasonic signals, ArtiLock captures unique physiological and behavioral articulation patterns, achieving high accuracy (99%) and resistance to mimicry attacks with minimal training data.

What Problem Does This Paper Address?

Traditional smartphone authentication methods like PINs, fingerprints, and even voiceprints have significant drawbacks. PINs are easily snooped, fingerprints can be faked or rendered unusable by environmental factors, and voiceprints are susceptible to noise, require long passphrases, and need extensive training data. Existing behavioral authentication methods often rely on multiple syllables or complex gestures, making them less user-friendly. The paper aims to bridge this gap by developing a secure, user-friendly, and robust authentication system that leverages the subtle, unique ways individuals articulate speech, even with a single utterance.

Methods

ArtiLock employs a dual-signal approach, simultaneously capturing audible speech signals (below 5 kHz) and inaudible ultrasonic signals (above 20 kHz) using a smartphone's built-in speaker and microphone. The core idea is to analyze the articulation of speech organs, specifically the tongue and lips, which are responsible for vowel production.

  1. Interference Elimination: The system first filters out environmental noise and hardware distortions from the raw signals.
  2. Speech Feature Extraction: This module analyzes the speech signal to extract features related to tongue position, vocal tract size, and pitch. It uses linear predictive coding (LPC) and formant frequency analysis, which are known to reflect these articulatory characteristics.
  3. Ultrasonic Feature Extraction: This module focuses on the ultrasonic signals reflected off the mouth. The Doppler effect, caused by the movement of the lips and jaw, alters the ultrasonic signal's frequency. By analyzing the envelope of these reflected ultrasonic signals, the system captures lip movement patterns. Features extracted include mean frequency, power bandwidth, occupied bandwidth, and median frequency.
  4. User Identification: An ensemble classifier, specifically a bagged tree kernel, is used to combine the extracted features from both speech and ultrasonic signals. This model is trained to distinguish between legitimate users and attackers.

The system operates in two stages: a Register Stage where legitimate users provide training samples (monophthongs), and a Login Stage where users provide a single utterance for authentication.

Datasets

The study collected data from 20 native Mandarin Chinese speakers aged 21-25. They were asked to pronounce six monophthongs (/a/, /i/, /r/, /o/, /u/, /y/) in three different noisy environments: an office (≈20 dB), a noisy room (≈50 dB), and outdoors (≈30 dB).

For attack simulations, the study included:

  • Mimicry Attacks: Ten attackers attempted to replicate the articulatory gestures of participants by watching recorded videos.
  • Playback Attacks: Pre-recorded voice samples from participants were replayed using a speaker.

The paper does not explicitly mention using pre-existing public datasets for training or evaluation.

Key Insights and Ideas

  • Single Utterance Power: The most significant insight is that even a single, simple utterance (a monophthong) contains enough unique physiological and behavioral information to reliably authenticate a user. This drastically reduces the burden on the user compared to systems requiring longer passphrases or multiple gestures.
  • Synergy of Speech and Ultrasound: Combining speech and ultrasonic signals provides a more comprehensive picture of articulation. Speech captures internal vocal tract dynamics, while ultrasound captures external lip and jaw movements, making the system more robust and harder to spoof.
  • Doppler Effect for Lip Dynamics: Leveraging the Doppler effect on ultrasonic signals to analyze lip movement is an innovative way to capture behavioral biometrics that are difficult to replicate.
  • Environmental Robustness: The system demonstrates remarkable resilience to environmental noise and variations, achieving high accuracy across different acoustic conditions. This is partly due to the distinct frequency bands of speech and ultrasound, allowing for effective filtering.

Key Results

  • High Accuracy: ArtiLock achieved an average accuracy of 99.48% for user identification across three different environments.
  • Low Error Rates: The system reported an average equal error rate (EER) of 0.5% under the tested conditions.
  • Attack Resistance: The system demonstrated strong resistance to mimicry and playback attacks, with false acceptance rates (FAR) as low as 2.17% and true acceptance rates (TAR) as high as 97.70% in various environments.
  • Minimal Training Data: The system requires very few training samples (as few as 1-5) to achieve high accuracy, significantly improving user experience compared to other biometric systems. For instance, accuracy reaches approximately 98% with just five training samples.
  • Comparison to State-of-the-Art: ArtiLock outperformed existing single-utterance articulation-based systems like LipPass (90.2% ACC) and SilentKey (82.5% ACC) in terms of accuracy and sample requirements.

Limitations and Future Work

  • Limited Subject Demographics: The study was conducted with subjects of limited age and education levels. Future work should involve a more diverse participant pool to ensure broader applicability and understanding of performance across different demographics.
  • Long-Term Stability: The evaluation period was relatively short (a few months). Individual characteristics like mouth cavities or beards can change over time. A long-term study is needed to assess the system's stability and the necessity for periodic user profile updates.
  • Auxiliary Authentication: ArtiLock is currently considered a research prototype. It could potentially serve as an auxiliary channel for two-factor authentication.
  • Proximity Requirement: The system requires users to hold their phones close to their mouths for reliable capture. This limits its applicability in scenarios where the phone is not held directly by the user (e.g., placed on a table).
  • Commercial Viability: The system is not yet a commercial product and requires further development to address these limitations.

Summary updated

Abstract

Although voice authentication is generally secure, voiceprint-based authentication methods have the drawback of being affected by environmental noise, long passphrases, and large registered samples. Therefore, we present a breakthrough idea for smartphone user authentication by analyzing articulation and integrating the physiology and behavior of the vocal tract, tongue position, and lip movement to expose the uniqueness of individuals while making utterances. The key idea is to leverage the smartphone speaker and microphone to simultaneously transmit and receive speech and ultrasonic signals, construct identity-related features, and determine whether a single utterance is a legitimate user or an attacker. Physiological authentication methods prevent other users from copying or reproducing passwords. Compared to other types of behavioral authentication, the system is more accurately able to recognize the user’s identity and adapt accordingly to environmental variations. The proposed system requires a smaller number of samples because single utterances are utilized, resulting in a user-friendly system that resists mimicry attacks with an average accuracy of 99% and an equal error rate of 0.5% under the three different surroundings.
BibTeX
@inproceedings{wong2023artilock,
  author = {Aslan B. Wong and Ziqi Huang and Xia Chen and Kaishun Wu},
  title = {{ArtiLock: Smartphone User Identification Based on Physiological and Behavioral Features of Monosyllable Articulation}},
  booktitle = {Sensors},
  year = {2023},
  url = {https://openreview.net/forum?id=HAdoRe67S2},
}
Figure