RISE Video Dataset: Recognizing Industrial Smoke Emissions

Yen-Chia Hsu, Ting-Hao K. Huang, Ting-Yao Hu, Paul Dille, Sean Prendi, Ryan Hoffman, Anastasia Tsuhlares, Randy Sargent, Illah R. Nourbakhsh

Introduction

Industrial air pollution remains a critical public health concern, yet monitoring compliance often relies on antiquated methods. Regulatory bodies frequently depend on "Visual Opacity Readings," where certified human observers manually grade smoke thickness. This process is infrequent, expensive, and difficult to scale. While computer vision offers a potential solution, existing datasets often lack the realism needed for deployment, featuring synthetic smoke or controlled laboratory environments. Project RISE (Recognizing Industrial Smoke Emissions) addresses this gap by introducing a large-scale, real-world video dataset for automated industrial smoke recognition.

Methodology

The researchers developed a comprehensive pipeline integrating hardware deployment, community-driven data labeling, and deep learning baselines.

Hardware Deployment

Ten custom camera systems, utilizing Nikon 1 J5 cameras managed by Raspberry Pis, were installed near three industrial coke plants in Pennsylvania. These systems captured high-resolution images every 10 seconds, which were processed into panoramas and timelapse videos to provide a continuous record of emissions.

Overview of the RISE project including the camera system and examples of smoke and steam emissions.

Citizen Science Labeling

To annotate the massive volume of data, the project leveraged "citizen science." Collaborating with local air quality advocates, the researchers developed a web-based tool for volunteers to label clips as "smoke" or "no smoke." Quality control was maintained through "gold standard" clips pre-labeled by experts to filter unreliable contributors.

The web-based labeling interface used by citizen scientists to classify video clips.

Deep Learning Baselines

Smoke recognition was framed as an action recognition task. The primary baseline utilized an Inflated 3D ConvNet (I3D) architecture. The study also explored various temporal modeling techniques, including LSTMs, Temporal Shift Modules (TSM), and Timeception, to better capture the fluid dynamics of industrial emissions.

The RISE Dataset

The RISE dataset represents a significant advancement in smoke detection resources:

  • Scale: 12,567 labeled video clips comprising 452,412 frames.
  • Diversity: 19 distinct viewpoints across three facilities, recorded over two years to capture all seasons and diverse weather conditions.
  • Complexity: Includes challenging edge cases such as steam (often confused with smoke) and low-opacity emissions.
  • Reliability: High inter-rater agreement (Cohen’s kappa of 0.80) between researchers and citizen scientists.

Key Insights and Results

A major hurdle identified was the "Steam Challenge"—distinguishing regulated smoke from non-regulated steam. Interactive tutorials were essential for training volunteers to make this distinction accurately. Interestingly, the study observed a "long tail" distribution in volunteer contributions, where 12% of participants provided 86% of the labels.

In terms of model performance, the RGB-TC (using Timeception) achieved the highest average F-score of 0.823. Data augmentation proved critical, raising the F-score from 0.777 to 0.817. Surprisingly, complex temporal modules did not significantly outperform the standard I3D baseline, suggesting that current architectures struggle to differentiate smoke motion from environmental noise like moving trees or clouds.

Conclusion and Future Work

Project RISE demonstrates that community-engaged citizen science can successfully power large-scale AI datasets for social good. While the current dataset is limited to daylight classification, it provides a robust foundation for future work in nighttime monitoring and spatial localization (detection). By providing a "data-driven voice" to local communities, RISE bridges the gap between technical innovation and environmental advocacy.

Summary updated

BibTeX
@inproceedings{hsu2021rise,
  author = {Yen-Chia Hsu and Ting-Hao K. Huang and Ting-Yao Hu and Paul Dille and Sean Prendi and Ryan Hoffman and Anastasia Tsuhlares and Randy Sargent and Illah R. Nourbakhsh},
  title = {{RISE Video Dataset: Recognizing Industrial Smoke Emissions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i17.17739},
}
Figure