Recycling Waste Classification Using Vision Transformer on Portable Device

Kai Huang, Huan Lei, Zeyu Jiao, Zhenyu ZhongSustainability 2021 · DOI

TL;DR

This paper introduces a Vision Transformer (ViT) model for automatic waste classification, achieving a new state-of-the-art accuracy of 96.98% on the TrashNet dataset. By deploying this model on a cloud server and accessing it via portable devices, the authors enable efficient and convenient waste sorting, crucial for resource conservation and recycling efforts.

What Problem Does This Paper Address?

The ever-increasing volume of global waste presents a significant challenge for effective recycling. Manual waste sorting is costly, inefficient, and can even be hazardous. While Convolutional Neural Networks (CNNs) have advanced automatic waste classification, their inherent limitations in capturing global contextual information have led to an accuracy bottleneck. This hinders the development of robust and scalable waste management systems. The authors aim to overcome this by leveraging the power of Vision Transformers, which excel at understanding global relationships within an image, and making this technology accessible on portable devices.

Methods

The core of this research is the application of a Vision Transformer (ViT) for waste classification. Unlike traditional CNNs that process images hierarchically through convolutional layers, ViT treats an image as a sequence of patches. These patches are then processed using a self-attention mechanism, which allows the model to weigh the importance of different parts of the image globally.

Key methodological aspects include:

  • Patching and Linear Projection: Images are divided into fixed-size patches, which are then linearly projected into a D-dimensional space. This reduces dimensionality while preserving spatial information.
  • Position Embedding: To retain positional information lost during patching, trainable random initialization position coding is employed.
  • Transformer Encoder: The core of the ViT, consisting of Multi-Head Self-Attention (MSA) blocks and Multi-Layer Perceptron (MLP) blocks. Layer normalization and residual connections are used to facilitate training.
  • Transfer Learning: To address the limited size of the waste dataset, the ViT model is pre-trained on the large-scale ImageNet dataset and then fine-tuned on the TrashNet dataset. This leverages general image features learned from ImageNet.
  • Cloud Deployment: The trained ViT model is deployed on a cloud server. Portable devices (like smartphones) capture images of waste, upload them to the server, and receive real-time classification results. This approach avoids the need for high-performance computing on the portable device itself. A lightweight web application framework (Flask) is used for this deployment.

Datasets

  • TrashNet Dataset: This dataset, containing 2527 images of six common waste categories (glass, paper, cardboard, plastic, metal, and trash), was used for training and evaluating the proposed ViT model. The images were captured using portable devices. The authors specifically chose TrashNet over other datasets like TACO or OpenLitterMap due to its more suitable background for waste sorting station scenarios and its focus on classification rather than detection.
  • ImageNet: Used for pre-training the Vision Transformer model.
  • GLASSENSE-VISION Dataset: Used to evaluate the generalization capabilities of the ViT model.
  • Drinking Waste Classification Dataset: Also used to assess the model's generalization performance.

Key Insights and Ideas

  • ViT's Global Context Advantage: The self-attention mechanism in ViT allows it to capture long-range dependencies and global context within an image, which is crucial for distinguishing between visually similar waste items. This directly addresses the limitations of CNNs' receptive fields.
  • Transfer Learning is Essential: Pre-training on ImageNet significantly boosts performance (an ~8% accuracy improvement), highlighting the importance of leveraging large, general datasets for training models on smaller, specialized datasets like waste classification.
  • Self-Attention Visualization Matters: Visualizing the attention maps reveals that the ViT model focuses on relevant areas of the waste items, mimicking human visual attention and explaining its improved performance.
  • Cloud Deployment for Accessibility: Offloading the computationally intensive classification task to a cloud server makes advanced AI models accessible on low-power portable devices, democratizing the application of waste classification technology.

Key Results

  • State-of-the-Art Accuracy: The proposed Vision Transformer model achieved an accuracy of 96.98% on the TrashNet test set, outperforming existing CNN-based methods.
  • Comparison with CNNs: The ViT model significantly surpassed traditional machine learning methods (e.g., HOG+SVM) and various CNN architectures (e.g., DenseNet, ResNet, RecycleNet), with the closest competitor achieving 94.2% accuracy.
  • Generalization: The model demonstrated strong generalization capabilities, achieving 100% accuracy on the GLASSENSE-VISION dataset and 99.29% on the Drinking Waste Classification dataset.
  • Inference Speed: While slightly slower than some CNN methods (average of 100ms more per image), the inference speed is still considered sufficient for real-time waste classification on portable devices.

Limitations and Future Work

  • Misclassification Causes: The authors identified that misclassifications often occur due to:
  • Inability to perceive the material (e.g., reflective surfaces on cardboard).
  • One waste item being covered by another.
  • Visual similarity between items (e.g., plastic bottles vs. glass bottles).
  • Labels on waste items confusing the model (e.g., a paper label on a metal can).
  • Future Directions:
    • Investigate clearer causes of misclassification.
  • Develop methods for multiple target detection or segmentation of waste items.
  • Further optimize the model to accelerate inference speed for even more efficient deployment.
  • Expand the focus beyond image classification to include object detection or segmentation for more accurate waste localization in natural scenes.

Summary updated

Abstract

Recycling resources from waste can effectively alleviate the threat of global resource strain. Due to the wide variety of waste, relying on manual classification of waste and recycling recyclable resources would be costly and inefficient. In recent years, automatic recyclable waste classification based on convolutional neural network (CNN) has become the mainstream method of waste recycling. However, due to the receptive field limitation of the CNN, the accuracy of classification has reached a bottleneck, which restricts the implementation of relevant methods and systems. In order to solve the above challenges, in this study, a deep neural network architecture only based on self-attention mechanism, named Vision Transformer, is proposed to improve the accuracy of automatic classification. Experimental results on TrashNet dataset show that the proposed method can achieve the highest accuracy of 96.98%, which is better than the existing CNN-based method. By deploying the well-trained model on the server and using a portable device to take pictures of waste in order to upload to the server, automatic waste classification can be expediently realized on the portable device, which broadens the scope of application of automatic waste classification and is of great significance with respect to resource conservation and recycling.
BibTeX
@inproceedings{huang2021recycling,
  author = {Kai Huang and Huan Lei and Zeyu Jiao and Zhenyu Zhong},
  title = {{Recycling Waste Classification Using Vision Transformer on Portable Device}},
  booktitle = {Sustainability},
  year = {2021},
  doi = {10.3390/su132111572},
}
Figure