Towards Collaborative Photorealistic VR Meeting Rooms

Alexander Schäfer, Gerd Reis, Didier StrickerMuC 2019 · DOI · OpenReview

Abstract When designing 3D applications, it is necessary to find a compromise between cost (e.g., money, time) and the achievable realism of the virtual environment. Reusing existing assets impacts the uniqueness of the application, while creating high-quality 3D assets is both time-consuming and expensive. This paper presents a low-cost, high-quality, and time-efficient solution for creating virtual environments. The main contribution is a novel method for developing virtual meeting applications by utilizing augmented spherical images to achieve photorealistic virtual environments.

Introduction

While remote work and global collaboration have become standard, traditional video conferencing often fails to provide a sense of "social presence"—the feeling of actually being in the same room as others. Virtual Reality (VR) offers a solution, but creating high-quality, believable 3D environments is a major bottleneck. Developers typically face a trade-off between low cost, high realism, and minimal effort. This paper proposes bypassing the labor-intensive 3D modeling phase by using 360-degree photography as the foundation for interactive, collaborative VR spaces.

The Augmented Spherical Image Pipeline

The proposed system utilizes a three-pillar approach to create immersive meeting spaces rapidly, moving from static photography to an interactive experience.

The proposed workflow: capturing 360-degree images, augmenting them with virtual objects, and enabling gesture interaction.
  1. Spherical Image Environments: Instead of modeling a room, the system captures high-resolution (5376 x 2688) 360-degree photos from specific seating positions. These images are applied as textures to spheres in the Unity game engine, providing instant photorealism.
  2. Shared Interactive Elements: To make the static photo feel "alive," 3D models of interactive tools—such as a virtual TV and a projector screen—are overlaid. These objects are synchronized across all participants' views, allowing everyone to interact with the same data simultaneously.
  3. Natural Interaction via Hand Tracking: To maximize immersion, the system avoids standard VR controllers in favor of the Leap Motion Controller (LMC). This allows for intuitive gestures: navigation between seats, swiping to change slides, and accessing menus by looking at one's palm.

Collaborative Architecture and Layout

To support multiple users, the system calibrates different spherical viewpoints into a common coordinate system. While each user is at the center of their own photo-sphere, they see other participants as minimalistic avatars with fully rigged, moving hands, which facilitates non-verbal communication.

Top-down schematic of the virtual environment showing the arrangement of users and shared interactive objects.

Key Insights

  • "Good Enough" Realism: The authors argue that a 2D photo of a real room provides a higher sense of immersion and "co-presence" than a technically complex but visually "uncanny" 3D-modeled room.
  • Viewpoint-Dependent Interaction: By using the VR headset's forward vector to simulate eye gaze, the system ensures that gestures (like swiping) only trigger actions when the user is actually attending to the target object, preventing accidental inputs.

Results and Future Work

This proof-of-concept demonstrates that a functional, believable meeting room can be deployed in minutes rather than the days or weeks required for high-fidelity 3D modeling. Qualitative observations suggest that seeing the exact hand movements of colleagues significantly improves the collaborative experience.

Future research will focus on several key areas:

  • Avatar Realism: Moving beyond minimalistic avatars to more sophisticated representations.
  • Automation: Automating the calibration of shared interactive elements using camera pose estimation.
  • Occlusion: Solving the technical hurdle where avatars do not currently "hide" behind physical objects in the 360-degree photo, which requires depth mapping.
  • Mobility: Addressing the static nature of the environment, which currently limits users to jumping between predefined seating positions.

Summary updated

Abstract

When designing 3D applications it is necessary to find acompromise between necessary cost (e.g.money, time) andachievable realism of the virtual environment. Reusing exist-ing assets has an impact on the uniqueness of the applicationand creating high quality 3D assets is very time consumingand expensive. Therefore, typical 3D applications have limitson graphical quality. In general one could say that applica-tions are not deliberately kept simple in terms of graphicalquality, the simplified quality is rather the consequence ofcost and time savings. We aim for a low cost, high quality andminimal time effort solution to create virtual environments.We explore the scenario of virtual meetings in context of VRsince it is of great interest not only in research but also inindustry. This paper’s main contribution is a novel way ofcreating a virtual meeting application by utilizing augmentedspherical images for photo realistic virtual environments.
BibTeX
@inproceedings{schafer2019towards,
  author = {Alexander Schäfer and Gerd Reis and Didier Stricker},
  title = {{Towards Collaborative Photorealistic VR Meeting Rooms}},
  booktitle = {MuC},
  year = {2019},
  doi = {10.1145/3340764.3344466},
}
Figure