Traditional computer vision relies heavily on high-resolution RGB cameras. While effective, these sensors capture an unnecessary surplus of information—such as fine facial features, skin textures, and environmental details—that can compromise personal privacy. As Yongping Guo et al. point out, traditional RGB-based methods risk leaking sensitive personal data, sparking a critical need for alternative imaging modalities.
To protect identity while still extracting meaningful behavioral data, a growing body of research explores a hardware-first solution: replacing high-resolution visible light cameras with low-resolution thermal and infrared sensors. By understanding the physical and informational limits of these sensors, researchers can design systems that are private by design.
The Hardware-Level Privacy Bottleneck
Rather than capturing high-resolution data and attempting to anonymize it later via software—a process vulnerable to security breaches or reconstruction attacks—privacy-preserving human sensing enforces an information bottleneck at the hardware level.
[Physical World] ---> [Low-Res Thermal Sensor] ---> [Anonymized Heat Blobs] ---> [Deep Learning Model]
(Identity Filtered)
By restricting spatial resolution or shifting the spectrum to long-wave infrared (LWIR), the sensor acts as a physical low-pass filter. High-frequency spatial details required for facial recognition are completely lost, while low-frequency thermodynamic signatures remain intact.
The extremes of this hardware constraint are highly effective. For instance, Ouday Hanosh et al. utilize an ultra-low-resolution $8 \times 8$ thermopile array to detect convulsive seizures during sleep, demonstrating that high-fidelity imagery is not a prerequisite for capturing complex physical movements. Similarly, Thomas Dubail et al. leverage a $32 \times 24$ infrared camera for person detection, ensuring that individuals appear only as unidentifiable thermal blobs at the exact point of data acquisition. This hardware-constrained paradigm extends beyond thermal imaging; for example, Svetlana Seliunina et al. demonstrate that utilizing hemispheric field-of-view LiDAR sensors with inherent resolution limits provides a similar hardware-level privacy guarantee, preventing the exposure of facial features in training datasets or model weights.
Disentangling Identity from Activity
Why does this coarse data remain highly functional for complex downstream tasks? The intuition lies in the separation of identity features from structural and temporal features.
As Farzan Erlik Nowruzi et al. explain, thermal imaging captures heat signatures that lack the high-frequency spatial details necessary for facial recognition, yet retain sufficient structural information for object and passenger counting. Human bodies naturally emit distinct thermal radiation that contrasts sharply with ambient indoor or vehicular backgrounds. This high contrast simplifies the segmentation problem.
Rikke Gade et al. (2013) exploit this thermodynamic contrast to perform long-term occupancy analysis in crowded indoor spaces, using graph-based optimization to track people based purely on their heat signatures. Even when spatial details are minimal, the temporal dynamics—how a thermal blob moves, accelerates, or changes shape over time—provide a rich signal. For example, Rikke Gade et al. (2015) combine short player trajectories ("tracklets") derived from thermal cameras with ambient audio to classify sports types with high accuracy, proving that motion patterns alone carry sufficient semantic meaning.
Overcoming the Lack of Texture
While low-resolution thermal data preserves privacy, it introduces significant algorithmic challenges. The absence of rich surface textures makes tasks like occlusion handling, feature alignment, and signal enhancement much more difficult. Researchers have developed several distinct strategies to overcome this lack of texture:
Simulation-Based Priors
When multiple human thermal signatures overlap, they merge into a single, ambiguous "blob." To resolve this without relying on high-resolution visual cues, Noor ul Huda et al. train models on thousands of simulated player overlaps within a virtual environment. This allows the system to learn the underlying geometric and statistical properties of occluded thermal shapes, enabling accurate player counting without high-resolution data.
Denoising and Enhancement Pipelines
Ultra-low-resolution sensors often suffer from low signal-to-noise ratios. To combat this, Tianyi Gong et al. implement a deep learning pipeline for real-time fall detection that uses a RetinexNet architecture to clean up and enhance low-resolution infrared array data before passing it to an object detection network. This step restores structural boundaries without introducing identity-revealing details.
Physics-Based Modality Fusion
Thermal sensors can also capture subtle physiological changes that are invisible to the naked eye. Satish Kumar et al. developed StressNet, a framework that detects human stress by combining a physics-based heat emission model with a spatio-temporal neural network. By reconstructing a specific physiological marker—the Initial Systolic Time Interval (ISTI)—from thermal videos, the authors achieve non-contact health monitoring that respects privacy by bypassing facial structure entirely. Similarly, Martin Møller Jensen et al. combine thermal temperature distributions with optical flow analysis to track body heat radiation during exercise, providing a non-intrusive method for estimating energy expenditure without wearables.
The Generalization Frontier
As the field matures from simple occupancy counting to complex tasks like Human Pose Estimation (HPE), the limitations of thermal data become more pronounced. Yongping Guo et al. established the Identity-Preserved Human Posture Detection (IPHPDT) benchmark, noting that the lack of rich texture information makes accurate manual annotation difficult and hinders the direct transfer of standard RGB algorithms to the thermal domain.
Furthermore, models trained on thermal data often struggle with domain adaptation. Mickael Cormier et al. identified a critical "generalization gap" in thermal pose estimation: while state-of-the-art models perform exceptionally well on familiar thermal sensors, their accuracy drops significantly when exposed to new sensors or environmental conditions. Because thermal signatures are highly dependent on ambient temperatures and sensor calibrations, developing cross-domain robustness remains one of the most vital open challenges in privacy-preserving sensing.
Surveyed References
- Audio-Visual Classification of Sports Types
- Long-Term Occupancy Analysis Using Graph-Based Optimisation in Thermal Imagery
- Convulsive Movement Detection using Low-Resolution Thermopile Sensor Array
- StressNet: Detecting Stress in Thermal Videos
- Estimating the Number of Soccer Players Using Simulation-Based Occlusion Handling
- In-Vehicle Occupancy Detection With Convolutional Networks on Thermal Images
- UPPET: Unified Pedestrian Pose Estimation in Thermal Imaging
- Identity-Preserved Human Posture Detection in Infrared Thermal Images: A Benchmark
- Real-time fall detection system based on deep learning and infrared array sensors
- Privacy-Preserving Person Detection Using Low-Resolution Infrared Cameras
- Estimation of Energy Expenditure during Treadmill Exercise via Thermal Imaging
- Person Segmentation and Action Classification for Multi-Channel Hemisphere Field of View LiDAR Sensors
Key Foundational Work
Audio-Visual Classification of Sports Types (2015) — contributed 2 concept elements to this direction.
Long-Term Occupancy Analysis Using Graph-Based Optimisation in Thermal Imagery (2013) — contributed 1 concept elements to this direction.