Annotation-free learning of plankton for classification and anomaly detection

Vito Paolo Pastore, Thomas G. Zimmerman, Sujoy Biswas, Simone Bianco

TL;DR

This paper introduces an "annotation-free" machine learning pipeline for plankton classification and anomaly detection, significantly reducing the need for manual data labeling. Their novel approach achieves performance comparable to supervised methods, paving the way for more efficient and scalable environmental monitoring of aquatic ecosystems.

What Problem Does This Paper Address?

Accurately identifying and classifying plankton species is crucial for understanding aquatic ecosystems and their response to environmental changes. However, the sheer volume and rapid acquisition of plankton image data pose a significant bottleneck for traditional machine learning approaches, which rely heavily on extensive manual annotation. This manual labeling process is time-consuming, expensive, and often a limiting factor in utilizing these large datasets for real-time analysis and environmental monitoring. The authors aim to overcome this challenge by developing a system that can effectively classify plankton and detect anomalies with minimal to no human supervision.

Methods

The core of the proposed solution is a pipeline named "plankton classifier" that operates in an unsupervised manner. It consists of four main modules:

  • Image Processor: This module takes raw video data and extracts individual plankter images, ensuring only fully visible organisms are processed.
  • Feature Extractor: A set of 131 hand-engineered features are extracted from each plankter image, covering geometric, invariant moments, texture, and Fourier descriptors. This aims to capture diverse morphological characteristics.
  • Unsupervised Partitioning Module: This module uses the extracted features to cluster plankton samples into distinct groups without prior knowledge of species labels. They specifically employ Fuzzy K-Means clustering, finding it more robust to class imbalance and the inherent fuzziness of plankton morphology compared to standard K-Means or Gaussian Mixture Models. The number of clusters is automatically determined using a Partition Entropy (PE) algorithm.
  • Classification Module: This module performs two key functions:
  • Supervised Classification: It uses the clusters identified by the unsupervised module to train a supervised classifier (an Artificial Neural Network or ANN is preferred due to its performance). This allows for classification based on the inferred labels.
  • Anomaly Detection: A novel "Delta-Enhanced Class (DEC) detector" is introduced. This deep neural network is trained for each identified plankton class. It identifies anomalies as samples whose features significantly deviate from the established class average. This detector is designed to flag both intra-species variations and entirely new, unseen species.

For comparison and benchmarking, they also explore using deep features extracted from a Convolutional Neural Network (CNN) trained on the plankton data, finding it achieves high accuracy but opting for the interpretable hand-engineered features for the main pipeline.

Datasets

The authors evaluate their pipeline on two datasets:

  • Custom Lensless Digital Microscope Dataset: This is a newly generated dataset consisting of videos of ten freshwater plankton species captured using their custom-built lensless microscope. This dataset is used for training and initial evaluation of the unsupervised pipeline.
  • Woods Hole Oceanographic Institution (WHOI) Dataset: A larger, publicly available dataset comprising millions of static images of microscopic marine plankton. This dataset is used to benchmark the performance of their pipeline on a more diverse and extensive collection of species.

Key Insights and Ideas

  • Unsupervised Learning for Plankton: The paper demonstrates that unsupervised learning can achieve classification accuracy comparable to supervised methods, significantly reducing the burden of manual annotation.
  • Fuzzy K-Means for Plankton: Fuzzy K-Means proves to be a more effective clustering algorithm for plankton data than crisp clustering methods, likely due to the inherent variability and overlapping characteristics of plankton species.
  • Automatic Cluster Number Determination: The Partition Entropy (PE) algorithm provides a robust way to automatically determine the optimal number of clusters, eliminating another manual parameter.
  • DEC Detector for Anomaly Detection: The novel Delta-Enhanced Class (DEC) detector is a key contribution, enabling the system to identify not only known plankton classes but also to flag unknown or anomalous samples, which is critical for real-time environmental monitoring.
  • Interpretable Features vs. Deep Features: While deep features from CNNs offer high accuracy, the paper highlights the value of interpretable hand-engineered features for understanding and potentially adapting the classifier.

Key Results

  • Lensless Dataset Performance: The Fuzzy K-Means unsupervised partitioning achieved a purity of 0.934. The ANN classifier trained on these unsupervised labels reached a validation accuracy of 99% and a testing accuracy of 94.5%.
  • WHOI Dataset Performance: The unsupervised pipeline achieved an overall purity of 0.715 on a subset of 40 species. A random forest trained on these labels achieved ~63% accuracy, while a supervised random forest achieved ~79%.
  • Anomaly Detection Performance: The DEC detector achieved an average testing accuracy of 98.8% for classification and 99.2% for anomaly detection on real data. It demonstrated high accuracy in detecting synthetic species with up to 30% similarity to known classes.
  • Unseen Species Detection: The pipeline showed an average detection accuracy of 98.3% for identifying new, unseen species.

Limitations and Future Work

  • Static Image Focus: The current pipeline primarily uses static images. The authors acknowledge that incorporating spatio-temporal (dynamic) information from video data could significantly improve classification and anomaly detection, especially for species with similar morphologies.
  • Feature Engineering: While hand-engineered features are interpretable, they still require some domain knowledge. The authors suggest that deep feature extraction is a viable alternative for achieving high performance.
  • Human-in-the-Loop: For newly detected "unseen species," an expert is still required for initial examination and potential retraining of the system. The authors envision a human-in-the-loop paradigm where the system learns and expands its knowledge base over time.
  • Generalizability: While the approach shows promise for other biological imaging fields, further validation on diverse datasets is needed.

The future work suggested includes leveraging spatio-temporal data, further refining the DEC detector, and developing more sophisticated methods for automatically incorporating newly discovered species into the classification system, moving towards fully autonomous environmental monitoring.

Summary updated

Abstract

The acquisition of increasingly large plankton digital image datasets requires automatic methods of recognition and classification. As data size and collection speed increases, manual annotation and database representation are often bottlenecks for utilization of machine learning algorithms for taxonomic classification of plankton species in field studies. In this paper we present a novel set of algorithms to perform accurate detection and classification of plankton species with minimal supervision. Our algorithms approach the performance of existing supervised machine learning algorithms when tested on a plankton dataset generated from a custom-built lensless digital device. Similar results are obtained on a larger image dataset obtained from the Woods Hole Oceanographic Institution. Additionally, we introduce a new algorithm to perform anomaly detection on unclassified samples. Here an anomaly is defined as a significant deviation from the established classification. Our algorithms are designed to provide a new way to monitor the environment with a class of rapid online intelligent detectors.
BibTeX
@inproceedings{pastore2020annotation,
  author = {Vito Paolo Pastore and Thomas G. Zimmerman and Sujoy Biswas and Simone Bianco},
  title = {{Annotation-free learning of plankton for classification and anomaly detection}},
  booktitle = {Scientific},
  year = {2020},
  doi = {10.1038/s41598-020-68662-3},
}
Figure