Introduction
The development of self-recognition is a fundamental milestone in human psychology. Researchers often distinguish between contingency—the immediate temporal link between one's actions and visual feedback—and familiarity—the recognition of one's own facial features. While infants detect movement contingency shortly after birth, they typically do not recognize their own features until approximately 18 months of age.
To investigate this transition, psychologists require tools that can "decouple" identity from motion. For instance, showing an infant a stranger's face that moves in perfect synchrony with their own head. Previous attempts at this were hindered by high latency or the need for invasive markers. This paper presents a markerless, high-speed face-swapping system designed to bridge this gap.
System Architecture
The proposed system operates in real-time by combining a 3D visual tracker with a pose-dependent face swapper.
The pipeline consists of three primary stages:
- 3D Visual Tracking: Instead of complex mesh modeling, the system utilizes a generic 3D ellipsoid model. A particle filter based on sparse template matching estimates a 10-dimensional state vector, capturing the subject's head position and orientation.
- Face Swapping: A library of "replacement faces" (Subject B) is pre-captured across various orientations. The system selects the image closest to the estimated pose of Subject A and applies temporal interpolation for smooth transitions.
- GPGPU Acceleration: To maintain the illusion of contingency, the system must operate with minimal delay. By parallelizing tracking and rendering tasks using NVIDIA CUDA, the researchers achieved a total system latency of just 99ms.
Performance and Robustness
The system was validated against a professional motion capture system, yielding an average orientation error of only 9°. Beyond accuracy, the system demonstrates significant robustness in environments typical of developmental studies.
Key performance highlights include:
- Occlusion Handling: The ellipsoid model remains stable even when infants touch their faces or hold toys, which frequently causes traditional trackers to fail.
- Wide Range of Motion: The tracker maintains lock at pitch angles up to 70°.
- Environmental Stability: The system effectively ignores background distractors, such as other people moving in the camera's field of view.
Conclusion and Future Directions
By providing a non-invasive way to manipulate visual feedback, this tool opens new avenues for studying infant development and conditions like Autistic Spectrum Disorder (ASD). Future work aims to incorporate dynamic facial expression mapping, allowing the system to transfer smiles or frowns from the source to the target face. Additionally, the framework may inform the development of "self-consciousness" in robotic systems by helping AI correlate motor commands with visual outcomes.
Summary updated