Abstract
In this paper, the goal is to investigate how manipulating the camera can minimize motion sickness and maximize immersion when a user moves in a virtual environment that requires vertical movement. In general, since a user uses virtual reality in a flat space, the actual movement of the user and the virtual movement are different, resulting in sensory conflict, which has the possibility of causing virtual reality motion sickness. Therefore, we propose three powerful camera manipulation techniques, implement them, and then propose which model is most appropriate through user experiments.
Introduction
The core challenge in Virtual Reality (VR) locomotion is Sensory Conflict Theory. When a user navigates a virtual environment that involves verticality—such as climbing stairs or ascending hills—while remaining on a flat physical floor, a mismatch occurs between visual stimuli and the vestibular system. The eyes perceive ascent, while the inner ear reports level ground, frequently resulting in "VR Sickness." While horizontal movement techniques like "Redirected Walking" are well-documented, vertical locomotion remains a significant barrier to immersion. This study explores whether data-driven machine learning models or heuristic algorithms can effectively manipulate the virtual camera to mitigate these effects.
Methodology
The researchers evaluated three distinct approaches to controlling virtual camera height during vertical movement:
1. Real-world Data Model (Test 1)
Motion capture data was collected from a participant using a VIVE HMD and trackers on the waist and feet while navigating a physical three-step staircase. This data was used to train a Multi-Layer Perceptron (MLP) to predict head and waist height based on foot positions over the preceding 10 frames.
2. Virtual Data Model (Test 2)
To overcome the limitations of physical data collection, a synthetic dataset was generated using Unity’s "Foot IK" (Inverse Kinematics). A virtual character was simulated walking on diverse terrains. To ensure robustness, the character's height and width were dynamically scaled using Gaussian random variables, creating a more generalized dataset than the real-world capture.
3. Heuristic Algorithm (Test 3)
A non-learning approach was implemented where the camera height was adjusted simply based on the height of the lower foot on the virtual terrain.
Model Architecture
The MLP models utilized an Exponential Linear Unit (ELU) activation function and were optimized using the Adam algorithm to minimize Mean Squared Error (MSE). The input layer consisted of 20 nodes representing foot positions, feeding into two hidden layers of 128 nodes each.
Results and Discussion
The study evaluated seven participants using the Simulator Sickness Questionnaire (SSQ) and the Igroup Presence Questionnaire (IPQ).
The "Jitter" Trap: Surprisingly, the sophisticated MLP models (Tests 1 and 2) induced higher levels of motion sickness than the heuristic algorithm. The neural networks produced subtle vertical oscillations—micro-vibrations—as they predicted height frame-by-frame. In VR, these high-frequency camera shakes are highly provocative for nausea.
Synthetic vs. Real Data: Between the two machine learning approaches, the model trained on virtual data (Test 2) outperformed the real-world model (Test 1). The synthetic dataset's diversity, achieved through random scaling of body types, allowed for better generalization across different users.
Consistency and Immersion: The heuristic algorithm (Test 3) was rated as the most "natural" and immersive. By following the lower foot, the camera movement remained predictable and smooth, lacking the "micro-stutter" inherent in the AI-driven predictions.
Conclusion
While the real-world ML model achieved the highest mathematical accuracy (0.86cm error for head height), it failed to provide the best user experience due to temporal instability. The heuristic approach proved superior for current applications. Future research will focus on implementing temporal smoothing filters or utilizing recurrent architectures like LSTMs to eliminate jitter, as well as testing on more complex, irregular terrains.
Summary updated