Introduction
Deploying Deep Neural Networks (DNNs) on edge devices, such as smart cameras and IoT sensors, requires a delicate balance between accuracy and power consumption. To extend battery life, these devices often capture data at lower resolutions or operate sensors in low-power modes, which introduces significant electronic noise. While data augmentation can improve robustness, it often incurs a "regularization penalty," where performance on clean images degrades. Furthermore, standard denoising filters can blur critical features like edges, which are essential for tasks like object detection.
The MoPE Framework
To address these challenges, researchers at the Georgia Institute of Technology proposed the Mixture of Pre-processing Experts (MoPE) framework. MoPE is a lightweight architectural add-on designed to "clean up" images before they reach a primary neural network, such as Faster R-CNN.
The framework consists of three main components:
- Gating Network: A shallow CNN that classifies incoming images as "clean" or "noisy." It acts as a router, ensuring that computational resources are only spent on denoising when necessary.
- Identity Unit: For clean or low-resolution images, the data passes through unchanged, preserving original features and avoiding unnecessary distortion.
- Denoise Neural Network: For noisy images, a U-Net-style autoencoder with skip connections is employed to remove artifacts.
Adversarial Training Strategy
A key innovation in MoPE is the use of Generative Adversarial Network (GAN) objectives during training. Unlike standard denoisers that minimize pixel-wise L2 loss—often resulting in blurred images—MoPE's denoiser is trained against a discriminator.
This adversarial approach forces the denoiser to preserve sharp edges and textures. The goal is not just visual appeal for humans, but the preservation of mathematical features required for downstream machine vision tasks like object detection and tracking.
Experimental Results
The authors evaluated MoPE across object detection (MS-COCO), multiple object tracking (MOT), and human activity classification (UCF 101).
- Robustness: On images with high Gaussian noise (σ = 0.15), the baseline Faster R-CNN's performance dropped to 0.226 mAP. With MoPE, the model maintained a significantly higher 0.359 mAP.
- Efficiency: MoPE is exceptionally efficient, adding only 0.67% more parameters and 0.17% more operations (GFLOPs) to the baseline model, making it ideal for resource-constrained platforms.
- Clean Image Performance: By selectively bypassing the denoiser for clean images, MoPE achieved 0.456 mAP on clean data, nearly matching the 0.461 baseline and avoiding the typical regularization penalty seen in standard robust training.
Conclusion
MoPE provides a highly efficient, modular solution for noise robustness on edge devices. By combining a lightweight gating network with task-specific pre-processing experts, it enables high-performance deep learning in challenging real-world environments without significant computational overhead.
Summary updated