Fungi Recognition: A Practical Use Case

Milan Sulc, Lukas Picek, Jiri Matas, Thomas Jeppesen, Jacob Heilmann-ClausenWACV 2020

TL;DR

This paper presents a deep learning system for recognizing 1394 species of fungi from images, achieving state-of-the-art results in a major competition. The system has been deployed in a citizen-science project, demonstrating its practical utility in assisting both amateur enthusiasts and professional mycologists in data collection and species identification.

What Problem Does This Paper Address?

Accurate identification of species is fundamental for biological research, especially for understanding biodiversity, climate change, and extinction patterns. Citizen science initiatives are invaluable for large-scale data collection, but they often face challenges with species identification accuracy, particularly for less charismatic groups like fungi. While some projects simplify identification or rely heavily on human experts, this paper explores how advanced computer vision, specifically deep learning, can automate and assist this process, thereby improving data quality and encouraging greater public participation in mycological research.

Methods

The core of the system is a deep convolutional neural network (CNN) ensemble. Specifically, the authors trained an ensemble of Inception-v4 and Inception-ResNet-v2 networks, inspired by winning solutions in similar fine-grained visual classification tasks. They employed several techniques to boost performance:

  • Ensembling: Combining predictions from multiple trained CNNs.
  • Test-time Image Augmentation: Applying various transformations (crops, mirrors) to input images at inference time to create more robust predictions.
  • Class Prior Adjustment: Calibrating the CNN's output probabilities based on known class priors (e.g., species rarity, geographical distribution) to improve accuracy, especially when the test data distribution differs from the training data.

The system was deployed using TensorFlow Serving, accessible via a REST API for web and mobile applications.

Datasets

  • FGVCx Fungi Dataset: This dataset, used for training and evaluation, comprises 1394 fungal species. It was split into a training set (85,578 images), a validation set (4,182 images), and a competition test set (9,758 images). A key characteristic noted is the imbalanced class distribution in the training set versus a uniform distribution in the validation set.

Key Insights and Ideas

  • The Power of "Tricks": Simple techniques like test-time augmentation and adjusting predictions based on class priors significantly boost CNN performance, especially in fine-grained classification tasks where class distributions can be uneven.
  • Citizen Science Synergy: Automating species identification not only helps users but also generates valuable, verified data for scientific databases like GBIF, addressing taxonomic biases and enriching our understanding of biodiversity.
  • Practical Deployment Matters: Successfully integrating a high-performing model into user-friendly web and mobile interfaces is crucial for real-world impact and widespread adoption.

Key Results

  • The system achieved 1st place in the FGVCx Fungi Classification Kaggle competition, demonstrating state-of-the-art performance.
  • The ensemble of 6 CNNs with test-time augmentation and class prior adjustment achieved 79% Recall@3 on the competition test set.
  • In the Danish Fungal Atlas application, 89% of approved annotations were based on the top-2 predictions of the model, indicating high utility for users.
  • The system's AI suggestions were approved by community/expert validation in 84% of cases, significantly outperforming non-expert users.

Limitations and Future Work

  • Mobile Inference: While the server-based inference is efficient, the paper suggests exploring on-device mobile inference for real-time recognition in areas with limited connectivity. This would require reducing model size and complexity through techniques like weight pruning and quantization.
  • Online Learning: Deeper integration with mycological information systems could enable online learning, allowing the classifier to continuously improve by learning from new data.
  • Location-Specific Priors: Utilizing GPS information to estimate location-specific class priors could further calibrate predictions and improve accuracy.
  • User Variability: The accuracy of user identification using the tool varied significantly (17% to 100%), highlighting that while the tool is helpful, it's not a perfect replacement for expert knowledge and user skill development is still important.

Summary updated

BibTeX
@inproceedings{sulc2020fungi,
  author = {Milan Sulc and Lukas Picek and Jiri Matas and Thomas Jeppesen and Jacob Heilmann-Clausen},
  title = {{Fungi Recognition: A Practical Use Case}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
}
Figure