TL;DR
This paper presents a real-time face annotation system for smartphones, leveraging the embedded GPU to overcome the computational demands of face recognition. By offloading the computationally intensive Gabor feature extraction to the GPU, they achieve a significant speedup, making accurate face annotation feasible on off-the-shelf mobile devices.
What Problem Does This Paper Address?
Face annotation, the process of identifying and labeling faces in photos and videos, has numerous applications in photo management and sharing. While face recognition algorithms can achieve high accuracy, their computational intensity makes real-time implementation on resource-constrained embedded platforms like smartphones a significant challenge. Previous attempts at real-time face annotation on PDAs often sacrificed accuracy for speed, using less robust methods. This paper aims to bridge this gap by demonstrating that modern smartphones, equipped with embedded GPUs, can accelerate these demanding algorithms to achieve both real-time performance and high accuracy.
Methods
The authors developed a face annotation system implemented on an Android-powered Motorola Droid phone. The system's pipeline involves face detection, landmark detection and alignment, and face recognition. The key innovation lies in their CPU-GPU co-design. While face/landmark detection and similarity comparison are handled by the CPU, the computationally intensive Gabor-based face feature extraction is offloaded to the embedded GPU. This is achieved by implementing the Gabor wavelet convolution, which involves Fast Fourier Transforms (FFTs), on the GPU using OpenGL ES 2.0. The Gabor features are then further processed using Linear Discriminant Analysis (LDA) for dimensionality reduction.
Datasets
The paper mentions evaluation on an "uncontrolled family album dataset which consists of 18 people." No specific details about the dataset's creation, size, or characteristics beyond the number of individuals are provided.
Key Insights and Ideas
- GPU as a viable accelerator for mobile vision: The paper demonstrates that even low-power embedded GPUs in smartphones are powerful enough to significantly accelerate computationally demanding computer vision tasks, moving beyond their traditional graphics roles.
- Targeted GPU offloading is crucial: Identifying the most computationally intensive part of the pipeline (Gabor feature extraction) and strategically offloading it to the GPU is key to achieving overall system speedup.
- Balancing accuracy and performance on embedded systems: The work highlights the possibility of achieving both high accuracy (90%) and real-time performance on mobile devices, overcoming limitations of previous embedded systems that prioritized speed over robustness.
- Challenges of embedded GPU programming: The authors note that embedded GPUs are less powerful and harder to program than their desktop counterparts, and that high-level APIs like CUDA or OpenCL are not yet supported, necessitating the use of graphics libraries like OpenGL ES 2.0.
Key Results
The face recognition system achieves an accuracy of 90% on their family album dataset. For a CPU-only implementation, face feature extraction takes approximately 15 seconds on a Droid phone. With GPU acceleration, this process is sped up by 4 times, bringing the overall response time within real-time requirements.
Limitations and Future Work
The authors acknowledge the limitations of embedded systems, specifically limited memory and storage. They also mention that the current implementation is a standalone application and will be available for download on the Android Market, implying future work on wider distribution and compatibility. The paper does not explicitly detail other limitations or extensive future work beyond the availability of the application.
Summary updated