Abstract
Reliable state estimation in visually degraded environments—such as smoke-filled corridors or lightless mines—remains a significant hurdle for autonomous systems. Traditional Visual Odometry (VO) and LiDAR-based methods often fail due to occlusion or lack of features. We present TP-TIO, a robust thermal-inertial odometry framework that utilizes a lightweight deep-learning feature detector, ThermalPoint, and a novel radiometric tracking method. By processing raw 16-bit thermal data and employing a noise-aware training strategy, TP-TIO achieves high-precision motion estimation that outperforms state-of-the-art laser and visual-inertial systems in challenging conditions.
Introduction
Standard navigation sensors like visible-light cameras and LiDAR face severe limitations in "denied" environments. Smoke, dust, and total darkness render RGB cameras useless, while LiDAR beams often reflect off airborne particles, leading to catastrophic drift. Thermal cameras offer a solution by sensing long-wave infrared radiation, but they introduce new challenges: low contrast, high fixed-pattern noise (FPN), and sudden photometric shifts during non-uniformity correction (NUC) cycles. TP-TIO addresses these issues by combining deep-learning-based feature detection with a tightly coupled thermal-inertial backend.
Methodology
The TP-TIO framework is built upon three core pillars: a specialized feature detector, radiometric tracking, and a robust optimization backend.
ThermalPoint Network
At the heart of the system is ThermalPoint, a lightweight convolutional neural network (CNN) based on the MobileNet architecture, optimized for real-time performance on embedded hardware. Unlike general-purpose detectors, ThermalPoint is trained using a noise-augmentation strategy that includes synthetic Fixed-Pattern Noise (FPN). This allows the network to become "blind" to common thermal sensor artifacts like vertical or horizontal stripes. The network employs an independent logistic loss function to maximize feature repeatability across frames.
Radiometric Feature Tracking
To preserve the maximum amount of information, TP-TIO tracks features using raw 16-bit radiometric values rather than compressed 8-bit images. We utilize an IMU-aided Kanade-Lucas-Tomasi (KLT) tracker that is invariant to intensity scaling. This approach ensures stability even during sudden brightness jumps caused by temperature shifts or sensor recalibration.
Tightly Coupled Backend
The system employs a sliding-window optimization that jointly minimizes visual reprojection errors and IMU preintegration errors. This tight coupling ensures that the IMU maintains state estimation during brief thermal sensor interruptions, such as NUC cycles.
Experimental Results
The system was evaluated across synthetic datasets, public underground mine data, and custom indoor smoke environments.
Feature Repeatability
ThermalPoint demonstrated superior performance compared to both traditional and deep-learning baselines. It achieved a repeatability score of 0.7674 on thermal data, significantly higher than SuperPoint (0.7166) and SIFT (0.4976).
Performance in Degraded Environments
In dense smoke environments, TP-TIO maintained a stable trajectory where LiDAR-based methods (LOAM) failed. LiDAR beams reflected off smoke particles, causing massive point cloud misalignment and z-axis drift. In contrast, TP-TIO's thermal-based tracking remained unaffected by the smoke.
Accuracy and Real-time Viability
In indoor testing, TP-TIO achieved an Absolute Trajectory Error (ATE) of 0.359m, outperforming VINS-Mono (2.406m) and OKVIS (3.199m) when applied to thermal imagery. The entire pipeline runs in real-time on a Jetson AGX Xavier, confirming its suitability for deployment on robotic platforms.
Conclusion and Future Work
TP-TIO provides a robust solution for odometry in environments where traditional sensors fail. By integrating a noise-aware deep detector with raw radiometric tracking, the system achieves high accuracy and reliability. Future work will focus on developing a loop closure module to enable global consistency and large-scale mapping capabilities.
Summary updated