Learning Heuristically-Selected and Neurally-Guided Feature for Age Group Recognition Using Unconstrained Smartphone Interaction

Yingmao Miao, Qiwei Tian, Chenhao Lin, Tianle Song, Yajie Zhou, Junyi Zhao, Shuxin Gao, Minghui Yang, Chao ShenIJCAI 2023 · OpenReview

Abstract

As smartphone usage expands to children and the elderly, new challenges regarding addiction, fraud, and accessibility emerge. We present AgeCare, the first implicit and continuous system for age group recognition that operates without interaction restrictions. By leveraging a massive new corpus of 1.44 million unrestricted actions from 2,100 subjects, AgeCare combines heuristically-selected and neurally-guided features to achieve state-of-the-art accuracy in identifying user age groups, enabling privacy-preserving, age-tailored assistance.

Introduction

The ubiquity of smartphones across generations necessitates systems that can adapt to the specific needs of different age groups. Children require protection from addictive content, while the elderly benefit from simplified interfaces and fraud prevention. Traditional age-recognition methods often rely on privacy-invasive data like facial images or voice recordings, or require users to perform specific tasks within constrained applications.

AgeCare addresses these limitations by identifying age groups through "unconstrained" interaction—analyzing how users naturally touch and move their phones across various apps and postures.

The AgeCare System

The AgeCare architecture is designed to operate implicitly in the background, fusing human intuition with deep learning capabilities.

Overview of the AgeCare system pipeline, from data acquisition to age-tailored assistance.

Hybrid Feature Extraction

The core innovation of AgeCare lies in its dual-path feature extraction approach:

  1. Heuristically-selected Features (HSF): 231 hand-crafted features capturing hand characteristics (stability, dexterity), action patterns (interaction frequency), and frequency-domain information from motion sensors.
  2. Neurally-guided Features (NGF): Deep embeddings extracted using a 3-layer LSTM network to capture subtle, human-imperceptible patterns in raw sensor and touch sequences.
Detail of the Feature Analysis & Learning module, highlighting the combination of HSF and NGF.

Classification and Assistance

These features are fused and processed by a Deep Neural Network (DNN) classifier. The model is trained using a combination of cross-entropy loss for classification and triplet loss to maximize the separability between different age groups in the feature space. The resulting classification triggers "Age-tailored Assistance," such as automatically enabling "Kid Mode" or "Easy Mode."

The AgeCare Corpus

To train and evaluate the system, we built the largest and most diverse dataset for smartphone-based age recognition to date:

  • Scale: 2,100 subjects and over 1.44 million operations.
  • Diversity: Includes 7 age groups (3 to 70+), 4 body postures, and 14 smartphone models.
  • Realism: Data was collected during unrestricted use of over 50 popular apps, reflecting real-world behavioral variety.

Key Findings and Results

AgeCare demonstrates significant improvements over existing methods:

  • Performance: Achieved a Macro-F1 score of 0.778 and a Macro-AUC of 0.91, substantially outperforming previous state-of-the-art models (which hovered around 0.65 and 0.82 respectively).
  • Multi-Interaction Benefit: Analyzing sequences of interactions (multi-interaction) proved far more effective than single-action analysis, as it captures temporal correlations in user habits.
  • Posture Sensitivity: The system's accuracy increases when users hold phones "unsupported," as motion sensors more clearly detect age-related differences in hand stability.
  • Privacy Preservation: By focusing on how a user interacts rather than what content they access, AgeCare provides a privacy-respecting solution suitable for on-device deployment.

Conclusion

AgeCare represents a significant step toward personalized and secure mobile computing. By combining behavioral heuristics with neural feature learning on a massive, unconstrained dataset, the system provides a robust, privacy-preserving mechanism for age-tailored smartphone assistance. Future work will focus on optimizing the model for real-time commercial deployment and addressing dataset imbalances.

Summary updated

Abstract

Owing to the boom of smartphone industries, the expansion of phone users has also been significant. Besides adults, children and elders have also begun to join the population of daily smartphone users. Such an expansion indeed facilitates the further exploration of the versatility and flexibility of digitization. However, these new users may also be susceptible to issues such as addiction, fraud, and insufficient accessibility. To fully utilize the capability of mobile devices without breaching personal privacy, we build the first corpus for age group recognition on smartphones with more than 1,445,087 unrestricted actions from 2,100 subjects. Then a series of heuristically-selected and neurally-guided features are proposed to increase the separability of the above dataset. Finally, we develop AgeCare, the first implicit and continuous system incorporated with bottom-to-top functionality without any restriction on user-phone interaction scenarios, for accurate age group recognition and age-tailored assistance on smartphones. Our system performs impressively well on this dataset and significantly surpasses the state-of-the-art methods.
BibTeX
@inproceedings{miao2023learning,
  author = {Yingmao Miao and Qiwei Tian and Chenhao Lin and Tianle Song and Yajie Zhou and Junyi Zhao and Shuxin Gao and Minghui Yang and Chao Shen},
  title = {{Learning Heuristically-Selected and Neurally-Guided Feature for Age Group Recognition Using Unconstrained Smartphone Interaction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=aiPUpo6svT},
}
Figure