TL;DR
This review paper explores how computer vision (CV) and AI are revolutionizing the monitoring of indoor and outdoor environments, as well as HVAC equipment. By leveraging non-contact sensing, CV offers a more efficient, precise, and less intrusive way to understand thermal comfort, urban conditions, and equipment health, paving the way for smarter, more energy-efficient buildings and cities.
What Problem Does This Paper Address?
Traditional methods for monitoring indoor environments, outdoor conditions, and HVAC equipment often rely on contact sensors, manual surveys, or fixed sensor placements. These approaches can be intrusive, labor-intensive, limited in scope, and unable to capture real-time, dynamic changes. For instance, assessing human thermal comfort traditionally involves questionnaires (subjective and prone to recall bias) or environmental sensors that don't account for individual physiological responses. Outdoor monitoring faces challenges in capturing the complex interplay of urban microclimates and human perception. HVAC equipment monitoring often relies on scheduled maintenance or reactive fault detection, leading to inefficiencies and potential failures. This paper addresses the need for advanced, non-contact, and intelligent monitoring solutions that overcome these limitations.
Methods
The paper reviews a wide range of computer vision techniques applied to three key areas:
- Indoor Environmental Monitoring:
- Skin Temperature Measurement: Utilizes infrared thermal imaging (TIR) cameras and visible light cameras. Techniques include analyzing facial regions of interest (ROIs), combining thermal and visible images for better facial detection, and employing Eulerian Video Magnification (EVM) to detect subtle changes in blood flow from visible light images.
- Human Position and Posture: Employs human skeleton keypoint models (e.g., OpenPose) to recognize body posture and movements, which are correlated with thermal comfort.
- Occupancy Detection: Passive Infrared (PIR) sensors are used for basic location detection, often in conjunction with other sensors.
- Outdoor Environmental Monitoring:
- Urban Environmental Monitoring: Leverages remote sensing (infrared thermal, visible light, LiDAR) and street view imagery (SVI). CV techniques like object detection, semantic segmentation, and deep learning are used to extract features such as land surface temperature (LST), green space percentage, building geometry, and sky view factor (SVF).
- Building Construction Safety: Combines CV with robotics. Techniques include target detection, tracking, and action recognition using algorithms like YOLO and Faster R-CNN, often deployed on mobile robots or drones.
- HVAC Equipment Monitoring:
- Thermal Infrared Imaging: TIR cameras are used to detect temperature anomalies in equipment and piping, aiding in fault diagnosis and predictive maintenance.
- Visible Light Imaging: Applied to monitor phenomena like frost formation on heat pumps and condensation on heat exchangers. Image processing techniques (binarization, edge detection) and deep learning are used to analyze frost growth and condensation patterns.
Datasets
The paper discusses the use of various datasets, often generated from experiments or collected from public sources:
- Facial Images: Used for skin temperature analysis and thermal comfort prediction.
- Hand Skin Feature Data: Large datasets (e.g., 1.44 million sets) used for training models to predict thermal comfort from subtle skin color changes.
- Street View Imagery (SVI): Publicly available datasets like Google Street View (GSV), Tencent Street View (TSV), and Baidu Street View (BSV) are crucial for urban environmental analysis.
- Construction Site Images/Videos: Used for training models for safety monitoring, often collected from surveillance cameras, fixed cameras, or RGB-D sensors.
- HVAC Equipment Images: Thermal and visible images of mechanical equipment, circuit boards, transformers, etc., used for fault diagnosis.
- Frost/Condensation Images: Collected from high-speed cameras and microscopic imaging systems for analyzing frost growth and condensation phenomena.
The paper highlights the need for high-quality, diverse datasets, particularly for complex scenarios like construction sites and urban environments.
Key Insights and Ideas
- Bridging the Gap: CV offers a powerful way to bridge the gap between objective environmental measurements and subjective human perception, especially for thermal comfort.
- Beyond Temperature: While skin temperature is a key indicator, human posture and movement also provide valuable insights into thermal discomfort.
- Urban Complexity: Street view imagery, combined with deep learning, is unlocking new ways to quantify urban morphology, greening, and microclimates at a pedestrian level.
- Predictive Maintenance: Thermal imaging is proving invaluable for early detection of equipment faults in HVAC systems, moving beyond reactive repairs.
- Energy Efficiency: Non-contact monitoring directly contributes to energy savings by enabling on-demand ventilation, optimized HVAC operation, and smarter defrosting strategies.
- The Rise of Robotics: The integration of CV with robotics is transforming safety monitoring in hazardous environments like construction sites.
Key Results
- Thermal Comfort Prediction: Models based on skin temperature and posture have shown promising accuracy, with some achieving around 70% for thermal state prediction and over 90% for sleep posture classification.
- Urban Analysis: CV techniques applied to SVI can achieve high accuracy in tasks like pedestrian detection (95%), shade detection (80%), and building feature extraction.
- HVAC Monitoring: Methods for frost detection have demonstrated improved accuracy and energy savings (e.g., reducing defrost energy consumption by up to 10%).
- Ventilation Control: On-demand ventilation systems using CV for occupancy detection have shown effectiveness in balancing comfort and energy consumption.
Limitations and Future Work
- Indoor Monitoring:
- Posture Recognition: Algorithms need improvement to detect a wider variety of human postures accurately.
- Control Integration: While monitoring is advancing, integrating CV insights with control systems for real-time automated regulation of indoor environments needs further development.
- Condensation Data: Current research on condensation is limited, particularly regarding lateral reflection and dew coverage.
- Outdoor Monitoring:
- Dataset Quality: Creating high-quality, diverse image datasets for urban environmental monitoring, especially in complex construction scenarios, remains a challenge.
- Data Fusion: Further integration of SVI with other data sources (social media, weather, IoT) is needed for comprehensive urban analysis.
- HVAC Monitoring:
- Defrosting Precision: Developing more intelligent defrosting strategies that precisely quantify frosting using new image-derived parameters is crucial.
- Dynamic Droplet Features: Research on condensation heat transfer on external surfaces needs to better extract and analyze dynamic droplet features.
- Generic Models: Developing universal condensation heat transfer performance prediction models using CV and AI is an important future direction.
- Generalization: Current fault diagnosis methods for HVAC are often specific; broader applicability to various equipment types is needed.
The paper concludes by emphasizing the continued potential of CV and AI in creating more intelligent, efficient, and comfortable built environments.
Summary updated