Introduction
The "reality gap" remains a primary obstacle in robotic learning, where policies trained in simulation often fail when deployed in the physical world due to discrepancies in visual fidelity and physical dynamics. Traditional approaches to mitigate this gap include Domain Randomization (DR)—which introduces artificial noise to the simulator to force the model to learn invariant features—or extensive real-world fine-tuning. However, DR can be computationally expensive and often fails to capture the nuanced temporal dynamics required for stable flight. This research explores a novel alternative: combining photorealistic simulation via 3D Gaussian Splatting with the temporal robustness of Liquid Neural Networks to achieve zero-shot sim-to-real transfer.
Methodology
The proposed pipeline integrates high-fidelity visual reconstruction with biologically inspired neural architectures to create a robust navigation policy.
Photorealistic Simulation
Instead of relying on traditional mesh-based graphics, the researchers utilized 3D Gaussian Splatting (3DGS) to reconstruct an indoor flight environment from smartphone imagery. 3DGS enables high-frame-rate, photorealistic rendering. This visual environment was integrated with the PyBullet physics engine, ensuring that the drone's visual perspective updated realistically in response to its physical movements.
Liquid Neural Networks
For the navigation policy, the team employed Closed-form Continuous-time (CfC) networks, a class of Liquid Neural Networks. Unlike standard Recurrent Neural Networks (RNNs) that process time in discrete steps, Liquid networks model the world as a continuous flow, making them inherently more adaptable to varying temporal scales. To further enhance robustness, the models were trained using Irregular Sampling Augmentation, teaching the network to handle the jittery and inconsistent timing typical of real-world hardware sensors.
Experimental Setup
The study utilized a custom dataset derived from 430 iPhone images of an indoor flight room to create the 3DGS model. Training data consisted of 600 simulated trajectories—specifically approach and turn maneuvers—generated by a PID controller within the GS environment.
Results and Discussion
The Liquid policy was benchmarked against a standard Long Short-Term Memory (LSTM) baseline across indoor and outdoor environments:
- Indoor Transfer: In the environment used for the GS model, the Liquid policy achieved a 75% success rate, significantly outperforming the LSTM's 12.5%.
- Zero-Shot Outdoor Generalization: When deployed on an unseen outdoor campus lawn characterized by harsh lighting and complex shadows, the Liquid policy maintained a 50% success rate, while the LSTM failed completely (0%).
- Computational Efficiency: The Liquid models were approximately 25 times smaller in parameter count than the LSTM baseline, demonstrating high suitability for edge deployment on resource-constrained drone hardware.
These results suggest that high-fidelity visuals can serve as a viable alternative to heavy domain randomization. By aligning the simulation's visual features closely with reality, the network learns representations that are directly transferable. Furthermore, the temporal adaptability of Liquid networks allows them to generalize from simple indoor maneuvers to complex outdoor navigation tasks.
Conclusion and Future Work
This research demonstrates that combining photorealistic 3D Gaussian Splatting with Liquid Neural Networks provides a powerful framework for bridging the reality gap in drone navigation. While currently limited to static scenes and relatively simple maneuvers, future work will focus on dynamic environments (e.g., moving obstacles) and more complex trajectory planning. Additionally, addressing the remaining "physics gap" through dynamics randomization alongside visual photorealism remains a key area for development.
Summary updated