FewMedical-XJAU: A Challenging Benchmark for Fine-Grained Medicinal Plant Classification

Tao Zhang, Sheng Huang, Gulimila Kezierbieke, Yeerjiang Halimu, Hui LiSensors 2025 · DOI

TL;DR

This paper introduces FewMedical-XJAU, a challenging new dataset for fine-grained medicinal plant classification featuring rare species, complex backgrounds, and high intra-class variability. They also propose BDCC, a novel multimodal approach that fuses visual and textual information using class-aware prompts and dynamic fusion, achieving state-of-the-art results in few-shot learning scenarios.

What Problem Does This Paper Address?

Accurately classifying plant species, especially those with subtle visual differences (fine-grained classification), is crucial for ecological conservation and medicinal research. However, existing plant image datasets often fall short. They typically suffer from limited species diversity, overly simple backgrounds, and lack of environmental variation, making them unsuitable for real-world applications. This is particularly problematic for rare medicinal plants, which are often underrepresented in datasets, leading to poor model performance in few-shot learning scenarios. The authors aim to bridge this gap by creating a more realistic dataset and developing a robust classification method capable of handling these challenges.

Methods

The core of the proposed solution is the BDCC (Bilinear Deep Cross-modal Composition) framework. It combines two key components:

  • DeepBDC: A visual feature extraction module that uses second-order statistics (covariance modeling) to capture intra-class distributions, offering a more nuanced representation than traditional methods.
  • CLIP (Contrastive Language–Image Pretraining): This model is leveraged to incorporate textual priors. The authors introduce a novel Class-Aware Structured Text Prompt Construction strategy. Instead of generic prompts, they generate rich, semantically structured descriptions for each plant species, covering aspects like appearance, growth habits, and functional attributes. This enhances the model's understanding of subtle inter-class differences.

Finally, a Task-Adaptive Fusion Strategy dynamically weights the visual and textual modalities based on their performance on the support set during inference. This adaptive weighting, combined with a weight smoothing mechanism, ensures robustness and stability, especially in low-data scenarios.

Datasets

  • FewMedical-XJAU (New Dataset): This is the primary contribution. It focuses on rare medicinal plants from Xinjiang, China. Key features include:
  • High intra-class variability.
  • Complex and diverse natural backgrounds.
  • Varied shooting angles and lighting conditions.
  • Rigorous expert annotations.
  • 540 plant species and 4992 images.
  • Naturally suited for few-shot learning due to species scarcity.
  • Existing Plant Datasets (for comparison): The paper references and compares against several established datasets like Flavia, Swedish Leaf, LeafSnap, ImageCLEF 2013, Pl@ntNet-300K, MED117, Bangladeshi medicinal plant dataset, DIMPSAR, and Karst Landform Herbs.

Key Insights and Ideas

  • The Importance of Realistic Datasets: The FewMedical-XJAU dataset highlights that real-world plant classification is far more complex than what's captured in many existing benchmarks. The combination of complex backgrounds and high intra-class variability is a significant hurdle.
  • Textual Priors are Crucial for Fine-Grained Distinctions: Simply relying on visual features is insufficient when inter-class differences are subtle. Rich, semantically detailed textual descriptions, as generated by their Class-Aware Structured Text Prompt Construction, are vital for disambiguating similar species.
  • Dynamic Fusion is Key to Robustness: Adapting the contribution of visual and textual modalities based on their performance in a specific few-shot task is more effective than fixed fusion strategies. This allows the model to lean on the more reliable modality for a given sample.
  • Weight Smoothing Prevents Overfitting to Support Sets: In few-shot learning, the small support set can lead to unstable fusion weights. The proposed smoothing mechanism helps mitigate this, leading to more consistent performance.

Key Results

  • Dataset Performance: The FewMedical-XJAU dataset demonstrates significantly higher intra-class variability compared to nine other mainstream plant datasets, as measured by various statistical metrics.
  • Few-Shot Classification: The proposed BDCC method achieves state-of-the-art results on the FewMedical-XJAU dataset for both 5-way 1-shot (83.02% accuracy) and 5-way 5-shot (89.15% accuracy) settings, outperforming existing methods like ProtoNet, DeepBDC, SRN, and HMDRN.
  • Ablation Studies:
    • Using Class-Aware Structured Text Prompts instead of generic ones improves CLIP's accuracy from 64.93% to 71.35% in a zero-shot setting.
  • The proposed adaptive fusion strategy (AWSS) and weight smoothing (Ours) significantly improve performance over simpler fusion methods.
  • Inference Speed: The method is computationally efficient, achieving approximately 29 FPS in the 1-shot scenario and 9.5 FPS in the 5-shot scenario on consumer-grade hardware, making it practical for real-world deployment.

Limitations and Future Work

  • Reliance on Textual Prompts: The current method still relies on predefined structured textual prompts. The authors suggest future work to explore automatic prompt generation strategies that can incorporate domain-specific knowledge without manual crafting.
  • Generalization to Novel Classes: While the method shows promise, further research is needed to ensure its effectiveness in recognizing entirely new classes with limited or no prior exposure.
  • Computational Cost: While efficient, the multimodal nature and dynamic fusion still introduce computational overhead compared to unimodal approaches. Future work could focus on further optimizing this.

Summary updated

Abstract

Fine-grained plant image classification (FPIC) aims to distinguish plant species with subtle visual differences, but existing datasets often suffer from limited category diversity, homogeneous backgrounds, and insufficient environmental variation, limiting their effectiveness in complex real-world scenarios. To address these challenges, a novel dataset, FewMedical-XJAU, is presented, focusing on rare medicinal plants native to Xinjiang, China. This dataset offers higher intra-class variability, more complex and diverse natural backgrounds, varied shooting angles and lighting conditions, and more rigorous expert annotations, providing a realistic testbed for FPIC tasks. Building on this, an improved method called BDCC (Bilinear Deep Cross-modal Composition) is proposed, which incorporates textual priors into a deep metric learning framework to enhance semantic discrimination. A Class-Aware Structured Text Prompt Construction strategy is introduced to improve the model's semantic understanding, along with a dynamic fusion mechanism to address high inter-class similarity and intra-class variability. In few-shot classification experiments, the method demonstrates superior accuracy and robustness under complex environmental conditions, offering strong support for practical applications of fine-grained classification.
BibTeX
@inproceedings{zhang2025fewmedical,
  author = {Tao Zhang and Sheng Huang and Gulimila Kezierbieke and Yeerjiang Halimu and Hui Li},
  title = {{FewMedical-XJAU: A Challenging Benchmark for Fine-Grained Medicinal Plant Classification}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25175499},
}
Figure