TL;DR
This paper introduces a novel deep learning approach for classifying cigarette usage from images by focusing on extracted face and hand regions. Their method achieves a high accuracy of 96.74% on a new dataset, demonstrating effectiveness in challenging scenarios and offering a foundation for automated smoking behavior monitoring.
What Problem Does This Paper Address?
The paper tackles the challenge of automatically detecting and classifying cigarette usage from images. This is motivated by the significant negative impacts of smoking, including pollution, health issues, and accidents. Existing methods often rely on image-based analysis of smoke or color, or sensor-based approaches. However, the authors identify a gap in research that focuses on individual-level surroundings and the specific difficulty of detecting the small visual footprint of a cigarette, especially in diverse environmental conditions and varying poses. They aim to develop a more robust and accurate system to address these limitations.
Methods
The proposed methodology consists of three main components:
- Region Extraction Module: To overcome the challenge of small cigarette size and similar appearances between smokers and non-smokers, this module preprocesses images by extracting relevant regions. It uses the "Faced" algorithm to detect face proposals and a fine-tuned YOLO-v3 model to detect hand proposals. These extracted regions (face and hand bounding boxes) are then adjusted to better encompass potential cigarette locations. This focused approach reduces computational load and improves performance by avoiding processing the entire image.
- Classification Module: This module takes the extracted face and hand proposals and classifies them. Instead of training a CNN from scratch, the authors leverage transfer learning by using an ensemble of pre-trained ResNet-18 and ResNet-34 models. The fully-connected layer of these models is modified to classify images into two categories: smoker or non-smoker. A softmax classifier with a cross-entropy loss function is employed.
- Conditionally Active Detection Module: Based on the classification result, a real-time cigarette detector, a YOLO-v3 model trained on cigarette images, is conditionally activated. If the classification module identifies an image as containing a smoker, the detection module then attempts to locate the cigarette. This conditional activation is crucial for reducing false positives and improving overall detection performance, as the detector only runs when there's a higher probability of a cigarette being present.
Datasets
The paper introduces and evaluates its approach on a newly curated dataset: "Dataset containing smoking and not-smoking images (smoker vs. non-smoker)". This dataset comprises 2,400 images with an approximately equal distribution of smokers and non-smokers captured in various environmental settings. The dataset was split into an 80%-20% train-test ratio for evaluation.
Key Insights and Ideas
- Region of Interest (ROI) Focus is Crucial: By concentrating on extracted face and hand regions, the model significantly improves its ability to detect small objects like cigarettes, outperforming methods that process raw images. This is a key takeaway from their ablation study.
- Transfer Learning Boosts Performance: Utilizing pre-trained ResNet models for classification, rather than training from scratch, provides a substantial performance gain, especially with a relatively small dataset.
- Conditional Detection Reduces False Positives: Activating the cigarette detector only after a positive classification of "smoker" is a clever strategy to minimize unnecessary computations and reduce the rate of incorrect detections.
- Dataset Novelty: The paper highlights the use of a new dataset for this specific problem, which they claim is the first of its kind for this task, and emphasizes its diverse environmental settings.
Key Results
The proposed approach achieved a classification accuracy of 96.74% on the Mendeley smoker dataset.
Quantitative Performance Measures: * Precision: 95% * Recall: 98% * Accuracy: 96.74% * True Positives: 197 * True Negatives: 190 * False Positives: 10 * False Negatives: 3
Qualitative results demonstrate the approach's effectiveness in challenging scenarios, including dark backgrounds, side-face poses, low visibility of cigarettes, and multi-subject environments. The paper also shows qualitative improvements in detection compared to a baseline.
Limitations and Future Work
The authors acknowledge that the current approach is primarily image-based. They suggest future work could involve:
- Incorporating More Modalities: Extending the approach to include information from videos and feeds from night-vision cameras to enhance its robustness.
- Broader Applications: Applying the methodology to anomalous human activity recognition and the detection of other small objects.
- Further Refinement: Continuously improving the model's effectiveness through ongoing research and development.
Summary updated