Abstract
Video classification in outdoor environments remains a significant challenge due to dynamic background motions such as swaying trees and weather conditions. This paper introduces LSFNet (Loss Switching Fusion Network), a lightweight system that integrates traditional handcrafted features with a specialized neural network architecture. By employing an alternating training strategy between reconstruction and classification losses, LSFNet learns robust feature representations. Experimental results on industrial surveillance datasets demonstrate that LSFNet outperforms heavy deep learning models like C3D and I3D, achieving over 90% accuracy in foreground motion detection while maintaining computational efficiency.
1. Introduction
Outdoor video classification is complicated by "dynamic background motions"—environmental factors like rain, illumination changes, and camera shake—that often mimic foreground human activities. Standard deep learning models (CNNs) typically require massive labeled datasets and high computational power, making them difficult to deploy in specific industrial contexts. This work proposes a hybrid approach that leverages spatiotemporal features and a novel loss-switching mechanism to create a unified, efficient representation for both background and foreground classification.
2. Methodology
2.1 Spatiotemporal Feature Extraction
The system utilizes "Improved Trajectories" (iDT) as the primary input. These handcrafted descriptors capture motion through Spatio-Temporal Histograms of Optical Flow (ST-HOF), Motion Boundary Histograms (ST-MBH), and Trajectory descriptors (ST-DT), alongside appearance via Histograms of Oriented Gradients (ST-HOG).
2.2 LSFNet Architecture and Loss Switching
The core of the system is the Loss Switching Fusion Network (LSFNet), a 5-layer autoencoder integrated with a Multi-Layer Perceptron (MLP).
The innovation lies in the training phase, where the network alternates between two loss functions:
- Mean Squared Error (MSE) Loss: Forces the network to learn how to reconstruct the input, preserving general structural information.
- Cross-Entropy Loss: Forces the network to make the features more discriminative for specific classes.
This dual-steering approach produces low-dimensional features that are both representative and highly separable.
2.3 Feature Selection and Classification
To further refine the representation, Fisher Scores are used to select the top 50% of feature components. For final classification, the system employs Locality Sensitive Hashing (LSH) for similarity search. Instead of a standard softmax output, the system identifies the "nearest neighbors" in the training set and uses soft voting to assign labels, enhancing robustness in noisy environments.
3. Experimental Results
3.1 Datasets
Evaluation was conducted on two private industrial datasets from iCetana, containing over 9,000 videos from outdoor environments like car parks and train stations. Labels included six background classes (e.g., tree waving, rainy) and three foreground human motion classes.
3.2 Performance Comparison
LSFNet significantly outperformed fine-tuned state-of-the-art models. As shown in the table below, the system achieved a substantial lead in both background and foreground classification tasks.
| Algorithm | Background Env. Motion | Foreground Human Motion |
|---|---|---|
| C3D (Fine-tuned) | 75.9% | 77.8% |
| I3D RGB (Fine-tuned) | 77.0% | 79.9% |
| LSFNet (Whole System) | 88.9% | 90.7% |
3.3 Feature Visualization
The effectiveness of the loss-switching strategy is qualitatively demonstrated through UMAP visualizations.
As seen in Figure 5, LSFNet produces significantly tighter and more distinct clusters for both background types (e) and human motion detection (f) compared to baseline methods like PCA or standalone autoencoders.
4. Conclusion
LSFNet demonstrates that combining handcrafted features with a targeted neural fusion strategy can surpass heavy-duty deep learning models in specialized, noisy domains. By using a single feature representation for multiple tasks, the system offers a computationally efficient solution for real-time industrial surveillance. Future work will explore integrating higher-level semantic features into the fusion pipeline.
Summary updated