FOSS: Multi-Person Age Estimation with Focusing on Objects and Still Seeing Surroundings

Masakazu Yoshimura, Satoshi OgataarXiv 2020 · OpenReview

Abstract

Age estimation from images is a critical task for real-world applications such as retail analytics and human-robot interaction. While most existing research focuses on single-person images with tightly cropped faces, practical scenarios often involve multiple people in unconstrained environments. Traditional approaches utilize two separate models—a face detector and an age estimator—which often leads to a loss of contextual information. In this work, we propose FOSS, a unified model that detects and estimates the age of multiple people simultaneously. By focusing on facial regions while preserving surrounding visual context, FOSS achieves state-of-the-art accuracy on both multi-person and single-person datasets.

Introduction

Most age estimation datasets and models are designed for single, centered faces. However, "in the wild" scenarios require processing multiple individuals at once. The standard two-step pipeline (detection followed by cropping and estimation) suffers from two primary issues: loss of context and data scarcity. Tightly cropping a face discards environmental clues—like lighting or camera noise—that help the model distinguish between actual skin texture and image artifacts.

Visualizing the "Still Seeing Surroundings" concept: (a) original image, (b) traditional tight crop, (c) FOSS approach retaining context.

To address these challenges, we introduce the FOSS architecture, which integrates detection and estimation while maintaining a connection to the surrounding pixels. Furthermore, we propose a "tiling" training strategy to enable multi-person estimation using widely available single-person datasets.

The FOSS Architecture

The FOSS model utilizes a dual-backbone approach to balance the different resolution requirements of detection and age estimation. While face detection can be performed on lower-resolution inputs, age estimation requires high-resolution data to identify fine details like wrinkles.

The FOSS Architecture Overview: Integrating detection and age estimation sub-networks with intermediate feature connections.

Key Architectural Features:

  • Dual Backbones: A detection sub-network (CenterNet/DLA-34) processes low-res images, while an age estimation sub-network (TResNet) processes high-res crops.
  • Focusing on Objects: The model crops high-resolution facial regions based on the detector's predicted bounding boxes.
  • Still Seeing Surroundings: Crucially, the model bridges the two sub-networks. Intermediate feature maps from the detection backbone are transformed and concatenated into the age estimation backbone. This provides the estimator with visual context from outside the immediate facial crop.

Tiling Augmentation for Training

To overcome the lack of annotated multi-person datasets, we developed a tiling augmentation strategy. We create pseudo-multi-person images by stitching together standard single-person images into a grid. To ensure the model generalizes to real-world crowds, we mix in unannotated images from face detection datasets (like WiderFace) and mask the age-loss calculation for those specific faces. This allows the model to learn multi-instance detection and estimation without requiring expensive new datasets.

Experimental Results

We evaluated FOSS against traditional two-step pipelines and current state-of-the-art (SOTA) models.

Multi-Person Performance

On the Images of Groups (IoG) dataset, FOSS outperformed the baseline two-step model. The "1-off" accuracy (predicting within one age group category) improved from 79.04% to 81.07%, demonstrating the value of preserving surrounding context.

Single-Person Performance

Despite being designed for multi-person scenarios, FOSS achieved SOTA results on standard benchmarks:

  • IMDB-WIKI: Achieved a Mean Absolute Error (MAE) of 5.64, surpassing previous top models (5.89–6.93).
  • UTKFace: Achieved an MAE of 4.49, edging out the previous best of 4.55.

Efficiency

The model is highly efficient, running at 14.6 FPS on a Tesla T4 GPU while processing up to 20 faces simultaneously, making it suitable for real-time video applications.

Conclusion

FOSS demonstrates that integrating face detection and age estimation into a single model—while specifically preserving environmental context—significantly improves accuracy. Our tiling augmentation strategy further proves that multi-person models can be effectively trained using existing single-person data. Future work will explore applying this "focus and surround" architecture to other tasks, such as multi-person human pose estimation.

Summary updated

Abstract

Age estimation from images can be used in many practical scenes. Most of the previous works targeted on the estimation from images in which only one face exists. Also, most of the open datasets for age estimation contain images like that. However, in some situations, age estimation in the wild and for multi-person is needed. Usually, such situations were solved by two separate models; one is a face detector model which crops facial regions and the other is an age estimation model which estimates from cropped images. In this work, we propose a method that can detect and estimate the age of multi-person with a single model which estimates age with focusing on faces and still seeing surroundings. Also, we propose a training method which enables the model to estimate multi-person well despite trained with images in which only one face is photographed. In the experiments, we evaluated our proposed method compared with the traditional approach using two separate models. As the result, the accuracy could be enhanced with our proposed method. We also adapted our proposed model to commonly used single person photographed age estimation datasets and it is proved that our method is also effective to those images and outperforms the state of the art accuracy.
BibTeX
@misc{yoshimura2020foss,
  author = {Masakazu Yoshimura and Satoshi Ogata},
  title = {{FOSS: Multi-Person Age Estimation with Focusing on Objects and Still Seeing Surroundings}},
  year = {2020},
  eprint = {2010.07544},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.07544},
}
Figure