TL;DR
This paper tackles the challenge of accurately predicting occupant count and activities in smart buildings using low-cost sensors. They propose a novel deep learning approach combining infrared grid-eye sensor data with sound sensor data, achieving high accuracy. Furthermore, they introduce an efficient data compression method using Truncated SVD to reduce storage requirements without significant loss of performance.
What Problem Does This Paper Address?
Efficiently managing energy consumption in smart buildings hinges on understanding how many people are present and what they are doing. Traditional methods, like using cameras, are accurate but prohibitively expensive and generate massive amounts of data. This paper addresses the need for a more cost-effective and storage-efficient solution by leveraging readily available, low-cost sensors like infrared grid-eye sensors and sound sensors. The core problem is to accurately infer occupant count and activity types from these sensor streams while minimizing the storage footprint of the collected data.
Methods
The authors propose two main deep learning models based on Convolutional Neural Networks (CNNs), specifically leveraging the ResNet architecture.
- 2D ResNet Model: This model combines data from an 8x8 infrared grid-eye sensor (which captures temperature variations and can be treated as image-like data) with sound sensor data. To balance the different data types and prevent the sound data from being ignored due to its lower dimensionality, the grid-eye sensor data is processed through a Deep Residual Network to extract refined features. These features are then concatenated with the sound sensor data and fed into fully connected layers for classification.
- 3D ResNet Model: This model extends the 2D approach by incorporating temporal information. It takes sequential data from both the grid-eye and sound sensors as input, allowing the model to learn from the temporal dynamics of activities. This is achieved by using 3D convolutional kernels in the ResNet architecture.
For data compression, the paper utilizes Truncated Singular Value Decomposition (SVD). SVD is a matrix factorization technique. Truncated SVD approximates the original matrix by keeping only the most significant singular values and their corresponding vectors. This effectively reduces the dimensionality of the data, leading to significant storage savings while aiming to minimize the loss of information critical for accurate classification.
Datasets
The authors collected their own dataset for this study.
- Data Sources: Infrared Grid-Eye sensor (8x8 array, measuring temperature) and a sound sensor (measuring ambient noise levels).
- Data Collection Environment: The data was collected in a laboratory setting at Seoul National University, specifically in a room within the New Engineering Building.
- Data Points: A total of 20,000 data points were collected, with each label (occupant count and activity type) having 5,000 data points.
- Labels:
- Occupant Count: 0, 1, 2, 3+ people.
- Activity Types: 1. Absence, 2. Office work/Rest, 3. Conversation, 4. Walking, 5. Exercise.
Key Insights and Ideas
- Sensor Fusion Benefits: Combining infrared grid-eye sensor data with sound sensor data significantly improves the accuracy of activity recognition compared to using the grid-eye sensor alone. This highlights the complementary nature of different sensor modalities.
- Temporal Information is Crucial: Incorporating temporal data (recent sensor readings) through the 3D ResNet model leads to a substantial increase in accuracy for both occupant count and activity prediction. This suggests that understanding the sequence of events is vital for accurate activity inference.
- Low-Cost Sensors are Viable: The study demonstrates that relatively inexpensive sensors can be effectively used for sophisticated tasks like activity prediction, making smart building solutions more accessible.
- SVD for Efficient Compression: Truncated SVD proves to be an effective method for compressing sensor data, achieving significant storage reduction with minimal impact on classification accuracy. This is crucial for managing the vast amounts of data generated by IoT devices.
- Sound Sensor Nuances: While sound sensors help, their effectiveness for distinguishing between different numbers of people can be limited if the sound patterns are not distinct (e.g., multiple people talking vs. one person talking).
Key Results
- Occupant Count Prediction:
- The 3D ResNet model achieved an accuracy of 95.3% for predicting the number of occupants.
- The 2D ResNet model achieved an accuracy of 85.5% (Grid-Eye only) and 85.25% (Grid-Eye + Sound).
- Activity Prediction:
- The 3D ResNet model achieved an accuracy of 90.9% for predicting human activities.
- The 2D ResNet model achieved an accuracy of 73.775% (Grid-Eye only) and 75.425% (Grid-Eye + Sound).
- Data Compression:
- Using Truncated SVD, the authors achieved a data compression rate of 6.25% (meaning the compressed data is 6.25% of the original size, or a reduction of 93.75%).
- With this compression, the accuracy for occupant count dropped by 0-5%, and for activity type by 0-5% compared to the original data.
Limitations and Future Work
- Limitations:
- The study acknowledges that the sound sensor's effectiveness in distinguishing between different numbers of people can be limited.
- The accuracy improvements from temporal data plateaued after a certain number of consecutive data points (around 32-44), suggesting diminishing returns.
- Future Work:
- The paper implicitly suggests exploring more advanced sensor fusion techniques and potentially different deep learning architectures to further improve accuracy and robustness.
- Investigating the impact of different compression ratios and exploring other compression algorithms could be future research directions.
- The authors' work lays the groundwork for more sophisticated smart building management systems that can dynamically adapt to occupant presence and activities.
Summary updated