3D Geology Scene Exploring Base on Hand-Track Somatic Interaction

Wei Zhang, Fang Zhu, Ping Lu, Ping Li, Bin Sheng, Lijuan MaoCGI 2020 · DOI

Introduction

Geological research frequently encounters barriers such as extreme distances, inaccessible terrain, and the limitations of physical scale. While 3D digital terrain models offer a solution for remote analysis, traditional interaction methods—primarily mouse and keyboard—often fail to provide the immersion and fluid manipulation necessary for detailed geological study. This paper presents a gesture-controlled 3D exploration platform designed to bridge this gap, offering a more natural human-computer interaction (HCI) experience tailored for geological scenes.

System Architecture and Methodology

The proposed system leverages the Intel RealSense SR300 depth camera, which operates on the "Time of Flight" (ToF) principle to calculate distances by measuring light reflection. The software stack integrates 3DMax for environmental modeling and the Unity engine for real-time rendering and interaction.

The system architecture and data processing pipeline, from initialization to resource release.

The workflow begins with the PXCMSenseManager, which initializes the pipeline and configures data input. The system then enters a loop of updating hand data, identifying body side, hand position, and specific gestures to control the camera within the virtual environment.

Gesture Interaction and Mapping

A critical component of the platform is the intuitive mapping of hand poses to navigation events. The design philosophy emphasizes "natural mapping," where digital interactions mimic real-world physical actions. For instance, expanding two hands mimics the act of stretching a physical map to zoom in.

Mapping of geological exploration functions to specific hand gestures and their corresponding scene views.

As shown in the mapping table, five core functions are supported:

  • Move View Point: Controlled by moving a single hand with a "V-sign."
  • Adjust Pitch Angle: Managed by moving two hands up or down.
  • Scaling: Achieved by opening or closing two hands.
  • Switch View Point: Triggered by a two-hand "thumb-up" gesture.
  • Character Movement: Executed by moving a single hand with a "V-sign" in specific contexts.

Shaking Optimization Algorithm

One of the primary challenges in gesture-based control is "jitter" or hand tremor, which can lead to unstable camera movement. To mitigate this, the authors developed a Shaking Optimization Algorithm based on a moving average filter. The system maintains a queue of the five most recent hand positions. Each request for position data returns the arithmetic mean of this queue, effectively smoothing out minor fluctuations.

The researchers found that a queue size of 5 represents the "sweet spot," providing sufficient stability without introducing perceptible latency or excessive computational overhead. If the sensor loses track of the hand, the queue is immediately cleared to prevent stale data from affecting subsequent interactions.

Results and Discussion

Quantitative analysis demonstrated that the optimization algorithm significantly reduced variance in hand position data, resulting in a much smoother user experience compared to raw sensor input. The system successfully integrated complex 3D desert scenes with high visual fidelity and responsive controls.

However, the system has notable limitations. The SR300's reliance on near-infrared light makes it sensitive to bright sunlight, restricting its use to indoor laboratory environments. Additionally, the effective recognition range for a full palm is limited to approximately 20–30 cm, which defines a relatively small active interaction zone for the user.

Conclusion

This research demonstrates a low-cost, accessible alternative to high-end somatic interaction devices like the Microsoft Kinect. By combining affordable hardware with custom optimization algorithms, the authors have created a functional tool for immersive geological exploration. Future work will focus on expanding these gesture-based controls into broader IoT and smart home applications.

Summary updated

BibTeX
@inproceedings{zhang2020d,
  author = {Wei Zhang and Fang Zhu and Ping Lu and Ping Li and Bin Sheng and Lijuan Mao},
  title = {{3D Geology Scene Exploring Base on Hand-Track Somatic Interaction}},
  booktitle = {CGI},
  year = {2020},
  doi = {10.1007/978-3-030-61864-3\_30},
}
Figure