Generating segmentation masks of herbarium specimens and a data set for training segmentation models using deep learning

Alexander E. White, Rebecca B. Dikow, Makinnon Baugh, Abigail Jenkins, Paul B. FrandsenApplications 2020 · DOI

TL;DR

This paper introduces a robust workflow for generating high-quality segmentation masks of herbarium specimens using deep learning, specifically a U-Net architecture. They address the challenge of noisy and biased image data in digitized collections by creating a curated dataset of fern specimens and their corresponding masks, enabling more accurate and efficient analysis of botanical data.

What Problem Does This Paper Address?

Digitizing natural history collections has created vast, open datasets, offering unprecedented opportunities for research linking biological patterns across space and time. However, these datasets are plagued by noise and bias stemming from diverse digitization, mounting, and collection protocols. This variability can skew analyses, even those leveraging advanced machine learning techniques. Specifically, for analyses relying on herbarium specimen images, it's crucial to isolate the plant material from extraneous elements like labels, color charts, and mounting artifacts. This paper tackles the challenge of systematically and efficiently segmenting plant tissues from these complex images to enable more reliable downstream analyses.

Methods

The authors developed a two-stage workflow to generate segmentation masks. First, they employed an automatic segmentation method using Otsu's binarization technique on grayscale images to create initial foreground (plant tissue) and background masks. This was followed by a crucial manual editing step using Adobe Photoshop to meticulously remove non-plant elements like labels and color palettes from the foreground masks. Finally, a post-processing step involved binarizing and blurring the masks to ensure consistent pixel values and smooth out any remaining disparities.

For training the deep learning model, they utilized a U-Net architecture, a standard for image segmentation tasks, implemented using the fastai library in Python. The model was trained on 80% of their curated dataset, with images resized to 256x256 pixels to optimize training efficiency. Data augmentation techniques, including flipping, rotation, zooming, lighting adjustments, and warping, were applied to enhance model generality.

Datasets

The authors created a new dataset consisting of 400 high-resolution (approx. 9100 x 6800 pixels) digitized images of fern specimens from the U.S. National Herbarium at the Smithsonian National Museum of Natural History. These images were manually curated to represent a broad diversity of fern leaf shapes. For each of these 400 images, they generated corresponding "ground-truth" segmentation masks, where each pixel is classified as either plant tissue or background. This dataset, along with the trained U-Net model and the Python scripts, is made publicly available on GitHub and Figshare.

Key Insights and Ideas

  • The "human-in-the-loop" is critical: While automation is key, manual curation and editing of segmentation masks are essential for achieving high-quality ground truth, especially for complex biological structures.
  • Standardization is paramount for ML in herbaria: The paper highlights the need for transparent and systematic protocols for generating training data to ensure deep learning models are generalizable and reliable across diverse herbarium collections.
  • Sharing is caring (for science): By openly sharing their dataset, masks, and model, the authors aim to accelerate research by providing a valuable resource for training and transfer learning in herbarium science.
  • Ferns as a challenging but representative case: The authors chose ferns due to their diverse leaf morphology and the fact that they often consist primarily of leaf material in herbarium specimens, making them a good test case for segmentation challenges.

Key Results

The trained U-Net model achieved a high Sørensen-Dice coefficient of 0.95 on the validation set, indicating excellent performance in accurately segmenting plant tissues from herbarium images. The model demonstrated consistent performance across most fern families, with slight variations observed for families with less typical specimen representations (e.g., thin stems or yellowed sheets). The workflow allows for the efficient generation of segmentation masks for massive datasets, significantly reducing the labor-intensive nature of manual annotation.

Limitations and Future Work

The authors acknowledge that their trained model is specific to ferns and may not perform as well on plant tissues not represented in their dataset, such as flowers and fruits. They suggest that botanists specializing in other plant taxa can adapt their workflow to create similar datasets and train models for their specific needs. Future work could involve developing models that can handle more complex segmentation tasks, such as distinguishing between different plant parts or identifying specific features within the plant tissue. The paper also points to the ongoing need for solutions to challenges like identifying specimens that are primarily fragments or contained within envelopes, which are not visible in standard images.

Summary updated

Abstract

The application of deep learning in herbarium sciences requires transparent and systematic protocols for generating training data so that these labor-intensive resources can be generalized to other deep learning applications. Segmentation ground-truth masks are hard-won data, and we share these data and the model openly in the hopes of furthering model training and transfer learning opportunities for broader herbarium applications.
BibTeX
@inproceedings{white2020generating,
  author = {Alexander E. White and Rebecca B. Dikow and Makinnon Baugh and Abigail Jenkins and Paul B. Frandsen},
  title = {{Generating segmentation masks of herbarium specimens and a data set for training segmentation models using deep learning}},
  booktitle = {Applications},
  year = {2020},
  doi = {10.1002/aps3.11352},
}
Figure