TL;DR
This paper introduces a novel, non-invasive method for estimating energy expenditure (EE) during walking and running using thermal imaging and optical flow analysis. The technique accurately predicts oxygen uptake and EE with a remarkably fast response time, offering a promising alternative to traditional methods that require wearable sensors.
What Problem Does This Paper Address?
Accurately measuring energy expenditure (EE) during human locomotion is crucial for various applications, from sports science and performance analysis to health monitoring and rehabilitation. Traditional methods for quantifying EE fall into two categories: direct measurement (e.g., oxygen uptake via indirect calorimetry) and indirect estimation (e.g., using heart rate monitors, accelerometers, or GPS).
Direct methods, while accurate, often involve cumbersome equipment that can interfere with natural movement patterns and require significant data collection time (2-3 minutes) to achieve steady-state measurements. Indirect methods, while more convenient, can be prone to inaccuracies due to factors like emotional stress, environmental conditions, or individual training status (especially for heart rate). Accelerometers, while useful, have limitations in accurately capturing speed changes during running.
The core problem this paper addresses is the need for a fast, accurate, and non-invasive method to estimate energy expenditure during locomotion, specifically walking and running, without the constraints of attached equipment or lengthy measurement periods.
Methods
The study proposes and validates a novel approach using thermal imaging and optical flow analysis.
- Thermal Imaging: A thermal camera captures the heat radiation emitted by the subject, providing a visual representation of their body's temperature distribution. This allows for non-intrusive detection and tracking of the person without privacy concerns, as the thermal data doesn't reveal identity.
- Optical Flow: This computer vision technique analyzes the apparent motion of objects between consecutive frames in a video sequence. In this context, thermal optical flow estimates the movement of pixels representing the subject's body parts across successive thermal images.
- Global and Local Movement Analysis:
- Global Movement: This refers to the overall movement of the person's center of gravity, approximated by tracking the chest area using techniques like Speeded Up Robust Features (SURF) and Kanade-Lucas-Tomasi (KLT) trackers.
- Local Movement: This focuses on the movement of body segments (limbs) relative to the torso. This is calculated using optical flow techniques, specifically the Large Displacement Optical Flow (LDOF) method, which combines optical flow with descriptor matching to handle both large and small movements.
- Energy Expenditure Estimation: The calculated optical flow values are used to estimate energy expenditure. This involves creating a grid over the subject's image, calculating local changes in flow (acceleration) within grid tiles, and then aggregating these accelerations to derive an energy value per frame. This value is then normalized by body size and ROI to ensure comparability.
The method was validated against indirect calorimetry (measuring oxygen uptake) during a controlled treadmill exercise test.
Datasets
The study utilized data collected from 14 endurance-trained subjects (13 male, 1 female) performing a discontinuous incremental exercise test on a treadmill. The dataset consists of measurements of:
- Heart rate
- Gas exchange (oxygen uptake and carbon dioxide production)
- Optical flow (derived from thermal imaging)
- Accelerations of ankle, thigh, wrist, and hip
No new benchmarks were introduced; the study used existing validated methods (indirect calorimetry) as the ground truth for validation.
Key Insights and Ideas
- Thermal imaging as a proxy for biomechanics: The core insight is that the thermal signature of human movement, when analyzed through optical flow, can effectively capture the biomechanical changes associated with energy expenditure. This bypasses the need for direct physiological measurements or attached sensors.
- Speed of estimation: The method achieves a remarkably fast response time, estimating energy expenditure within 3-4 seconds of a given walking or running velocity. This is a significant improvement over traditional methods that require minutes to establish steady-state conditions.
- Non-invasiveness and freedom of movement: The absence of wearable sensors is a major advantage, allowing for more natural movement and opening up possibilities for research in less controlled environments.
- Correlation across intensities: The strong linear correlation between optical flow and oxygen uptake across a wide range of exercise intensities (walking and running) highlights the robustness of the method.
- Potential for real-time applications: The speed and non-invasive nature of the technique suggest its potential for real-time EE monitoring in various applications.
Key Results
- Strong correlation with oxygen uptake: Average optical flow values for all 14 subjects showed a strong correlation with oxygen uptake across all exercise intensities (R² = 0.96, P<0.0001).
- Accurate prediction of EE: The optical flow measurements provided an accurate estimation of energy expenditure during horizontal walking and running.
- Fast response time: Less than 100 frames (approximately 3-4 seconds) were required to obtain a valid calculation of optical flow, reflecting EE.
- Linear relationship with speed: Optical flow values increased with increasing treadmill velocities, mirroring the increase in oxygen uptake and energy expenditure.
- Validation against accelerometry: Summarizing acceleration data from multiple body sites showed a strong correlation with optical flow (R² = 0.93; P < 0.0001), providing an additional validation criterion.
Limitations and Future Work
- Limitations at higher speeds: The study noted that while optical flow increased with speed, it was not significantly different between 16 and 18 km/h, suggesting potential limitations at very high running speeds. This is further supported by the observation that both VO2 and accelerations at distal placed accelerometers also showed limitations at higher speeds.
- Gross vs. Net EE: The study measured gross energy expenditure and estimated net EE by subtracting standard resting EE. Measuring actual resting EE during the lab session could have provided more precise net EE values.
- Substrate utilization: The current methodology cannot identify the fractional rate of substrate utilization, meaning small differences in EE between subjects moving at the same velocity might not be detected.
- Walking and running economy differences: Detecting subtle differences in walking and running economy might be challenging with this method.
Future Work suggested by the authors:
- Estimating EE in free-living conditions: The next step involves estimating EE using optical flow with individuals performing specific movement patterns outside the laboratory.
- Addressing camera angle challenges: Future investigations should involve optical measurements during free-living activities, addressing challenges related to camera angles and the direction of the person.
- Combining thermal cameras for 3D motion: One approach could be to combine two thermal cameras to estimate 3D motion and then correlate optical flow changes with oxygen consumption obtained by a portable metabolic cart.
Summary updated