Introduction
Current Augmented Reality (AR) teleconferencing faces a significant "fidelity vs. co-presence" trade-off. High-fidelity 3D reconstruction remains prohibitively expensive and complex for consumer use. Conversely, mobile-friendly solutions often rely on cartoonish virtual avatars that lack realism or "video grids"—floating 2D boxes that feel detached from the user's physical environment.
This research proposes a middle ground: life-size 2D video avatars. By using background-removed video streams rendered at a 1:1 scale within the user's local space, the system aims to provide the photorealism of a video call with the spatial presence of a 3D avatar.
Methodology
The researchers investigated how hardware constraints, specifically the limited Field of View (FoV) of current Head-Mounted Displays (HMDs), influence social proxemics. The study followed a three-stage approach:
- Pilot Study: Using a Microsoft HoloLens 2, 18 participants manually adjusted the placement (Radius and Radian) of up to four remote video avatars to find the most comfortable configuration.
- System Prototype: A functional AR teleconferencing system was built using MediaPipe for real-time background removal and pose estimation. This was compared against traditional 2D video (PC), 3D cartoon avatars, and floating video grids.
- VR-Simulated AR Study: To anticipate future hardware, the team used an HTC Vive Pro to simulate AR environments with varying FoVs, ranging from 10° to a human-like 180°.
Key Insights: FoV and Social Geometry
The study revealed that a headset's FoV significantly dictates where users want remote participants to "stand."
- The "Bunching" Effect: In headsets with a narrow FoV, users tend to place remote participants closer together (smaller Radian) to keep them within the visible window.
- The "Step Back" Effect: When vertical FoV is limited, users prefer to move avatars further away (larger Radius). This allows them to see the remote person's full body without constant vertical head movement.
- Fidelity vs. Co-presence: Video avatars were the most preferred mode. They maintained high fidelity scores (~5.1/7) similar to traditional video, while achieving co-presence scores (~5.2/7) nearly equal to 3D avatars (~5.3/7), far exceeding the performance of floating video boxes (~2.9/7).
Mathematical Modeling for Auto-Placement
To assist developers, the authors derived regression models to automatically calculate optimal avatar placement based on headset specifications. For example, the model for two remote users (RU) is defined as:
$$Radian_{2RU} = -0.0012FoV^2 + 0.45FoV + 20.20$$
Conclusion and Future Work
Life-size 2D video avatars offer a compelling, low-cost alternative for AR teleconferencing that balances realism and spatial presence. While the current "billboard" approach breaks the 3D illusion if a user walks around the avatar, future work involving neural rendering (e.g., NeRFs) and automatic lighting estimation could further bridge the gap between 2D video and true 3D presence. The study successfully demonstrates that VR simulations are a reliable method for predicting user behavior for future AR hardware iterations.
Summary updated