Traditional computer vision relies on the visible spectrum—the same narrow band of light our eyes perceive. While this works well in brightly lit, unobstructed environments, it fails in total darkness, through smoke, or when trying to detect internal physiological states. To overcome these limits, researchers have turned to Thermal Infrared (TIR) imaging, which captures the heat emitted by objects rather than the light reflected off them.
However, moving from visible light to thermal radiation isn't just a change in camera; it’s a fundamental shift in how materials interact with sensors. The core of this research direction lies in understanding and exploiting these spectral discrepancies to build robust systems for biometrics, health monitoring, and robotics.
The Glass Ceiling of Thermal Imaging
The most striking intuition in this field is that transparency is wavelength-dependent. In the visible spectrum, glass is transparent. In the Long-Wave Infrared (LWIR) spectrum used by thermal cameras, glass is an opaque, reflective barrier.
As Diego A. Socolinsky points out in Design and Deployment of Visible-Thermal Biometric Surveillance Systems, this creates a "failure mode" for thermal-only systems: a person wearing eyeglasses or standing behind a window becomes partially or fully invisible to the sensor. Ognjen Arandjelovic and colleagues note in Infrared face recognition: A comprehensive review of methodologies and databases that this occlusion hides the periocular region—the area around the eyes—which is one of the most discriminative regions for identifying a face.
Conversely, this "weakness" is a strength for robotics. In TRansPose: Large-Scale Multispectral Dataset for Transparent Object, Ayoung Kim demonstrates that because thermal sensors see the "solid shape" of glass that standard cameras miss, they can help robots manipulate transparent objects like bottles or lab equipment that are otherwise "invisible" to RGB-D sensors.
Seeing Through the Surface
While glass blocks thermal radiation, other materials behave the opposite way. Thermal sensors can sometimes "see" through thin plastic layers, such as vinyl bags, that are opaque to visible light. More importantly for human sensing, thermal imaging allows us to see "under the skin."
In Vesselness Features and the Inverse Compositional AAM for Robust Face Recognition Using Thermal IR, Arandjelovic and Reza Shoja Ghiass describe how thermal sensors capture subcutaneous vascular patterns. These heat signatures from blood vessels act as a unique, internal biometric that is much harder to spoof than a 2D photograph.
This ability to sense internal states extends to physiology:
- Deception: Pradeep Buddharaju and Ioannis T. Pavlidis developed ATHEMOS, which detects lies by tracking temperature spikes in the lacrimal puncta (eye corners) caused by increased blood flow during stress.
- Fatigue: Constantino Alvarez Casado and colleagues showed in Estimating exercise-induced fatigue from thermal facial images that a single thermal snapshot can estimate physical exhaustion by analyzing facial heat patterns.
- Respiration: Zheng Jiang used thermal cameras to monitor breathing patterns for COVID-19 screening, though they noted that face masks—much like eyeglasses—act as an insulating barrier that obscures the nostrils.
Overcoming Insulation and Occlusion
The primary challenge in this research is the "insulation problem." Just as glass blocks thermal radiation, thick clothing and blankets disperse heat signatures, making it difficult to see the human form.
In the context of sleep monitoring, Sara Mohammadi and Kevin Wells demonstrated in Transfer Learning for Clinical Sleep Pose Detection that IR cameras can identify sleep poses through blankets, but only up to a certain thickness. To push this further, Mohamed Afham and colleagues in Towards Accurate Cross-Domain In-Bed Human Pose Estimation used self-supervised knowledge distillation to help models "infer" body positions even when heat signatures are heavily muffled by duvets.
The Power of Multimodal Fusion
The consensus across these 26 papers is that neither visible nor thermal imaging is sufficient on its own for "in the wild" deployment. The solution is Multimodal Fusion.
By combining the two, systems can leverage the high resolution and "glass-transparency" of visible light with the "darkness-immunity" and "physiological-insight" of thermal IR. Ciarán Ó Conaire and Noel E. O'Connor demonstrated this in Background Modelling in Infrared and Visible Spectrum Video, where visible features "fill in" the gaps when a subject moves behind a window, while thermal features maintain tracking when the lights go out.
Ultimately, this research direction is about building a more complete "perceptual envelope" around human subjects, using the physics of different light wavelengths to see what was previously hidden.