Beyond Speech: Generalizing D-Vectors for Biometric Verification

Jacob Baldwin, Ryan Burnham, Andrew Meyer, Robert Dora, Robert WrightAAAI 2019 · DOI

Introduction

Biometric verification—the process of confirming an individual's identity—has traditionally relied on "hard" physiological traits like fingerprints or iris scans. However, behavioral biometrics, such as keystroke dynamics and walking gait, offer the advantage of passive and continuous authentication. Despite their potential, modeling human behavior is challenging due to inherent noise and the limitations of handcrafted features. Historically, researchers relied on manually engineered statistics (e.g., digraph timings or accelerometer variance), which are often brittle and require extensive data to be effective.

This research presents a significant shift by generalizing the D-Vector framework—originally a cornerstone of Google's speaker recognition technology—to behavioral biometrics. By leveraging deep learning to automatically extract discriminative latent features, this approach achieves superior accuracy with significantly less data than previous state-of-the-art methods.

The Generalized D-Vector Framework

The core innovation lies in treating "identity" as a latent feature that can be learned from a large population. The system operates in two distinct phases: Training and Execution.

The D-Vector framework consists of a Training Phase to learn latent features and an Execution Phase for user enrollment and verification.
  1. Training Phase: A Deep Neural Network (DNN) is trained on a large, diverse pool of users to perform classification. Through this process, the model learns to map raw sensor data into a high-dimensional "latent space." Once training is complete, the final classification layer is removed. The remaining network serves as a feature extractor that can transform any behavioral sequence into a compact numerical "D-Vector."
  2. Execution Phase: During enrollment, a user provides a few samples of their behavior. These are processed into D-Vectors and averaged to create a unique Signature. For subsequent verification, a new Test Vector is generated from a live sample and compared against the stored signature using a multi-metric scoring system (combining cosine similarity, $L^2$ distance, and z-scores).

Domain-Specific Adaptations

While the D-Vector engine is generalizable, the input layers must be tailored to specific data modalities. For keystroke dynamics, the researchers converted sequences into 250x20 "images," where rows represent time and columns represent the most frequently used keys, allowing a Convolutional Neural Network (CNN) to capture spatial-temporal typing patterns. For gait analysis, they adopted an audio-inspired approach, transforming accelerometer and gyroscope data into Power Spectral Densities (PSD) using a triangle filterbank before feeding it into the DNN.

Performance and Results

The D-Vector approach was evaluated against three datasets: the custom MultiMod dataset (keystroke and gait), the Clarkson keystroke benchmark, and the UCI Human Activity Recognition gait dataset. The results demonstrated a dramatic improvement over existing state-of-the-art (SOA) baselines.

Comparison of Equal Error Rates (EER %) between the D-Vector approach and the state-of-the-art baseline across different datasets and test vector counts.

As shown in the table above, the D-Vector framework consistently outperformed the baseline (labeled "Intel"). On the MultiMod dataset, the system achieved an Equal Error Rate (EER) of 7.0% for gait and 7.7% for keystrokes, compared to over 24% for the baseline. Notably, the model maintained high performance (11.4% EER) even when tested on the UCI dataset, which it had not seen during training, proving the robustness and generalizability of the learned features.

Furthermore, the system exhibited remarkable data efficiency. While traditional methods often require thousands of keystrokes, the D-Vector framework reached high accuracy with as few as five samples. When keystroke and gait data were fused using a Bayesian ensemble, the error rate dropped to an impressive 2.5% EER.

Conclusion

By moving away from handcrafted features and embracing deep latent representations, the generalized D-Vector framework sets a new standard for behavioral biometrics. Its ability to create accurate signatures from minimal data makes it a practical solution for real-world applications, such as continuous authentication on mobile devices. Future work will focus on increasing robustness to environmental noise and extending the framework to other time-series biometrics like heart rate or mouse movement.

Summary updated

BibTeX
@inproceedings{baldwin2019beyond,
  author = {Jacob Baldwin and Ryan Burnham and Andrew Meyer and Robert Dora and Robert Wright},
  title = {{Beyond Speech: Generalizing D-Vectors for Biometric Verification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.3301842},
}
Figure