Optimizing Backbone Networks Through Hybrid–Modal Fusion: A New Strategy for Waste Classification

Houkui Zhou, Qifeng Ding, Chen Chang, Qingbo Liao, Qun Wang, Huimin Yu, Haoji Hu, Guangqun Zhang, Junguo Hu, Tao HeSensors 2025 · DOI

TL;DR

This paper introduces HFWC-Net, a novel hybrid deep learning architecture that fuses CNNs and Transformers for highly accurate and efficient waste classification. By incorporating an "Agent Attention" mechanism and a "LionBatch" optimization strategy, HFWC-Net significantly reduces computational demands and training time while achieving state-of-the-art accuracy on multiple datasets, paving the way for more practical automated waste management systems.

What Problem Does This Paper Address?

The ever-increasing volume of urban waste presents a significant challenge for effective management and recycling. Traditional manual waste sorting is labor-intensive, costly, and prone to errors, leading to reduced accuracy and compromised recycling efforts. While deep learning, particularly Convolutional Neural Networks (CNNs) and more recently Transformer architectures, have shown promise in automating this process, existing methods still struggle with:

  • Diverse Waste Types: Differentiating between a vast array of waste items with subtle visual similarities.
  • Complex Environments: Handling variations in lighting, image quality, and cluttered backgrounds often found in real-world waste streams.
  • Computational Demands: Many advanced models require substantial computational resources, limiting their deployment in resource-constrained environments or real-time applications.
  • Generalization Issues: Poor performance on underrepresented or novel waste categories.

This paper aims to address these limitations by developing a more accurate, efficient, and robust waste classification system.

Methods

The core innovation of this paper is the HFWC-Net (Hybrid–Modal Fusion Waste Classification Network). This architecture combines the strengths of two powerful deep learning paradigms:

  • CNNs for Local Features: Traditional CNNs are adept at capturing fine-grained local features like edges and textures within an image.
  • Transformers for Global Context: Transformer architectures, particularly the CSWin Transformer used here, excel at understanding long-range dependencies and global relationships between different parts of an image.

HFWC-Net achieves hybrid-modal fusion by integrating these two through a hierarchical, four-stage pyramid structure. This allows for a coarse-to-fine feature extraction and integration process.

Key novel components include:

  • Agent Attention Mechanism: This mechanism introduces "agent tokens" to optimize the attention calculation process. Instead of directly comparing all query and key tokens, agent tokens act as intermediaries, aggregating global information from key-value pairs and then broadcasting it back to the query tokens. This reduces computational complexity from quadratic to linear, improving efficiency without sacrificing global context understanding.
  • LionBatch Optimization Strategy: This strategy combines the Lion optimizer (known for its resource efficiency and simplified hyperparameter tuning) with InfoBatch pruning. InfoBatch dynamically prunes less informative samples during training based on their loss, reducing computational load and memory usage while maintaining gradient expectations. This leads to faster training and reduced resource consumption.

Datasets

The authors utilized a combination of public and a newly created dataset:

  • Garbage Classification Dataset: A public dataset used for evaluation.
  • TrashNet Dataset: A well-known public dataset for waste classification, comprising 2527 images across six categories.
  • MixTrash Dataset (Newly Introduced): This dataset was created by the authors to address the limitations of existing benchmarks. It features a significantly larger scale (52,324 images) and a much higher diversity of waste categories (135 subclasses). The images are designed to reflect real-world conditions with varied backgrounds and balanced class distributions, aiming to improve model generalization.

Key Insights and Ideas

  • Synergy of CNNs and Transformers: The paper strongly demonstrates that combining local feature extraction (CNNs) with global context understanding (Transformers) in a hybrid architecture leads to superior performance compared to using either alone.
  • Agent Attention for Efficiency: The introduction of agent tokens in attention mechanisms is a clever way to reduce computational burden by focusing on salient information, a critical step for making advanced models practical.
  • Dynamic Pruning for Speed: The LionBatch strategy highlights the effectiveness of dynamically adjusting the training process by pruning less informative data points, leading to significant speedups without compromising accuracy.
  • Dataset Diversity is Crucial: The creation and use of the MixTrash dataset underscore the need for more comprehensive and realistic benchmarks to truly push the boundaries of waste classification models.

Key Results

HFWC-Net achieved state-of-the-art results across all tested datasets:

  • Garbage Classification Dataset: Top-1 Accuracy of 98.89%.
  • TrashNet Dataset: Top-1 Accuracy of 96.88%.
  • MixTrash Dataset: Top-1 Accuracy of 94.35%.

Beyond accuracy, HFWC-Net also demonstrated significant improvements in efficiency:

  • Reduced Training Time: The LionBatch strategy significantly reduced training time compared to baseline models. For example, on the MixTrash dataset, HFWC-Net's training time was reduced by approximately 5 times compared to VMamba.
  • Lower Computational Demands: The Agent Attention mechanism contributes to linear complexity, making the model more scalable.
  • Parameter Efficiency: HFWC-Net achieved high accuracy with a competitive number of parameters, making it more suitable for deployment. For instance, it achieved higher accuracy than Swin Transformer with about 13% fewer parameters.

Limitations and Future Work

The authors acknowledge several limitations:

  • Low-Resolution Images: The model's feature extraction capabilities may be insufficient for very low-resolution images or those with missing fine details, potentially leading to reduced accuracy.
  • Underrepresented Categories: Performance might be limited for garbage categories that are rare or absent in the training set, indicating challenges with generalization to long-tail distributions and open-set environments.
  • Training Batch Stability: The LionBatch strategy's performance can be sensitive to the quality and stability of training batches, especially with imbalanced data.

Future research directions include:

  • Adaptive Learning Mechanisms: Incorporating techniques like meta-learning to enable faster adaptation to new categories and improve generalization in dynamic environments.
  • Advanced Data Augmentation: Utilizing methods like Generative Adversarial Networks (GANs) and Mixup to create more realistic synthetic data and improve robustness to rare categories.
  • Computational Efficiency Enhancements: Exploring parallel computing frameworks, distributed training, model pruning, and quantization for deployment on resource-constrained edge devices.

Summary updated

Abstract

With rapid urbanization, effective waste classification is a critical challenge. Traditional manual methods are time-consuming, labor-intensive, costly, and error-prone, resulting in reduced accuracy. Deep learning has revolutionized this field. Convolutional neural networks such as VGG and ResNet have dramatically improved automated sorting efficiency, and Transformer architectures like the Swin Transformer have further enhanced performance and adaptability in complex sorting scenarios. However, these approaches still struggle in complex environments and with diverse waste types, often suffering from limited recognition accuracy, poor generalization, or prohibitive computational demands. To overcome these challenges, we propose an efficient hybrid-modal fusion method, the Hybrid-modal Fusion Waste Classification Network (HFWC-Net), for precise waste image classification. HFWC-Net leverages a Transformer-based hierarchical architecture that integrates CNNs and Transformers, enhancing feature capture and fusion across varied image types for superior scalability and flexibility. By incorporating advanced techniques such as the Agent Attention mechanism and the LionBatch optimization strategy, HFWC-Net not only improves classification accuracy but also significantly reduces classification time. Comparative experimental results on the public datasets Garbage Classification, TrashNet, and our self-built MixTrash dataset demonstrate that HFWC-Net achieves Top-1 accuracy rates of 98.89%, 96.88%, and 94.35%, respectively. These findings indicate that HFWC-Net attains the highest accuracy among current methods, offering significant advantages in accelerating classification efficiency and supporting automated waste management applications.
BibTeX
@inproceedings{zhou2025optimizing,
  author = {Houkui Zhou and Qifeng Ding and Chen Chang and Qingbo Liao and Qun Wang and Huimin Yu and Haoji Hu and Guangqun Zhang and Junguo Hu and Tao He},
  title = {{Optimizing Backbone Networks Through Hybrid–Modal Fusion: A New Strategy for Waste Classification}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25103241},
}
Figure