The changing landscape of text mining: a review of approaches for ecology and evolution

Maxwell J. Farrell, Nicolas Le Guillarme, Liam Brierley, Sara Bronwen Hunter, Daan Scheepens, Anna Willoughby, Andrew Yates, Nicole MideoProceedings of the Royal Society B Biological Sciences 2024 · DOI

TL;DR

This review explores the evolving landscape of text mining for ecology and evolution, highlighting how advancements in Natural Language Processing (NLP), particularly deep learning and large language models (LLMs), can revolutionize data synthesis from scientific literature. It provides a roadmap for researchers to adopt these powerful tools, addressing the challenges of data scarcity and computational demands in this field.

What Problem Does This Paper Address?

The synthesis and modeling of data from published literature are crucial for generating insights and testing theories in ecology and evolutionary biology. However, manually sifting through vast amounts of research papers for systematic reviews, meta-analyses, and dataset compilation is incredibly time-consuming and labor-intensive. As the volume of scientific literature explodes, researchers face an increasing challenge in efficiently extracting and utilizing this information. This paper addresses the need for accessible and effective computer-based approaches to text mining, bridging the gap between the rapid advancements in NLP and their adoption within the ecological and evolutionary research communities.

Methods

The paper categorizes text mining approaches into three main paradigms:

  • Frequency-based approaches (Bag-of-words): These methods focus on the occurrence of words, ignoring their order or context. Techniques like word clouds and TF-IDF (Term Frequency-Inverse Document Frequency) are discussed for tasks like topic modeling and document classification.
  • Traditional Natural Language Processing (NLP): This paradigm involves a pipeline of linguistic and computational techniques to understand text structure and meaning. Key steps include:
  • Tokenization: Breaking text into smaller units (words, sentences).
  • Part-of-Speech (POS) Tagging: Identifying the grammatical role of words.
  • Lemmatization/Stemming: Reducing words to their base form.
  • Dependency Parsing: Analyzing grammatical relationships between words.
  • Named Entity Recognition (NER): Identifying specific entities like species names, locations, or genes.
  • Coreference Resolution: Linking different mentions of the same entity.
  • Deep Learning-based Language Models: This is the most rapidly evolving area, leveraging neural networks to learn complex patterns and representations from text. This includes:
  • Word Embeddings: Representing words as dense vectors that capture semantic meaning.
  • Transformer Architectures (e.g., BERT): Models that excel at understanding context and long-range dependencies in text.
  • Large Language Models (LLMs): Powerful, pre-trained models that can be fine-tuned or used directly via prompting for various NLP tasks. The paper emphasizes transfer learning and prompt-based interaction as key strategies for applying LLMs in ecology.

Datasets

The paper discusses the challenge of data scarcity in ecology and evolution, noting the lack of large, open-access text databases for training foundation models. However, it highlights the potential of:

  • Existing ecological databases: These can be used to generate labeled training data through rule-based methods or pattern matching.
  • Comparative databases: Databases of species traits, biodiversity responses, and experimental data are valuable resources for creating labeled training data.
  • Specialized corpora: Examples like COPIOUS (for species occurrences) and BiodivNERE (for NER and RE) are mentioned as existing gold-standard datasets.
  • General-domain corpora: LLMs are often pre-trained on vast datasets like Wikipedia, BookCorpus, PubMed abstracts, and scientific literature (e.g., SciBERT, BioBERT).

The paper notes that a specific LLM trained explicitly on biodiversity texts, BiodivBERT, is a recent development.

Key Insights and Ideas

  • The "Low-Resource" Challenge: Ecology and evolution are considered "low-resource" domains for NLP due to a lack of large, publicly available text datasets and gold-standard labeled data, hindering the development of custom deep learning models.
  • Transfer Learning as a Solution: Adapting existing pre-trained LLMs (even those trained on general or biomedical texts) through transfer learning is a promising strategy to overcome data limitations for specific ecological tasks.
  • Prompt-Based Interaction for Accessibility: Generative LLMs, when prompted effectively, can perform complex information extraction tasks without requiring extensive model training or fine-tuning, making advanced NLP more accessible to researchers without deep technical expertise.
  • The Importance of "Human-in-the-Loop": Despite the power of LLMs, human oversight remains critical for validating outputs, identifying errors, and ensuring the ethical and reproducible use of these technologies.
  • Environmental Impact of LLMs: The significant computational resources and energy required for training and running LLMs are a growing concern that researchers need to consider.

Key Results

The paper is a review and does not present novel quantitative results from its own experiments. Instead, it synthesizes existing knowledge and discusses the potential of various text mining approaches. It highlights that:

  • Frequency-based approaches can be useful for tasks like topic modeling and document classification, with naive Bayes classifiers achieving high accuracy in some scenarios.
  • Traditional NLP pipelines, with steps like NER and relation extraction, are essential for structuring information from text.
  • Deep learning models, particularly transformers and LLMs, offer significant advantages in capturing complex linguistic patterns and context.
  • Prompt-based interaction with LLMs has shown promise in tasks like identifying pest controllers, hosts, pathogens, and species locations.

Limitations and Future Work

Limitations:

  • Data Scarcity: The primary limitation for developing domain-specific NLP models in ecology and evolution is the lack of large, open-access text corpora and labeled datasets.
  • Computational Resources: Training and deploying large language models require significant computational power, which can be a barrier for many researchers.
  • Reproducibility Concerns: The reliance on closed-source LLMs poses challenges for reproducible research due to lack of transparency in training data and model architecture.
  • Potential for Errors: LLMs can generate false or harmful information, necessitating careful validation.
  • Prompt Engineering Complexity: Crafting effective prompts for LLMs can be challenging and highly task-dependent.

Future Work:

  • Developing Domain-Specific LLMs: Investigating the necessity and feasibility of creating LLMs specifically for ecology and evolution.
  • Ensuring Open-Access and Transparency: Promoting the development and use of open-source LLMs and transparent methodologies.
  • Mitigating Environmental Impact: Exploring ways to reduce the computational and environmental costs associated with LLMs.
  • Leveraging Retrieval-Augmented Generation (RAG): Integrating LLMs with external databases to improve accuracy and reduce misinformation.
  • Balancing Model Complexity and Resource Use: Determining when simpler NLP approaches are sufficient and more resource-efficient than complex deep learning models.
  • Continued Human Oversight: Emphasizing the ongoing need for human involvement in validating outputs and ensuring responsible AI use.

Summary updated

Abstract

In ecology and evolutionary biology, the synthesis and modelling of data from published literature are commonly used to generate insights and test theories across systems. However, the tasks of searching, screening, and extracting data from literature are often arduous. Researchers may manually process hundreds to thousands of articles for systematic reviews, meta-analyses, and compiling synthetic datasets. As relevant articles expand to tens or hundreds of thousands, computer-based approaches can increase the efficiency, transparency and reproducibility of literature-based research. Methods available for text mining are rapidly changing owing to developments in machine learning-based language models. We review the growing landscape of approaches, mapping them onto three broad paradigms (frequency-based approaches, traditional Natural Language Processing and deep learning-based language models). This serves as an entry point to learn foundational and cutting-edge concepts, vocabularies, and methods to foster integration of these tools into ecological and evolutionary research. We cover approaches for modelling ecological texts, generating training data, developing custom models and interacting with large language models and discuss challenges and possible solutions to implementing these methods in ecology and evolution.
BibTeX
@inproceedings{farrell2024changing,
  author = {Maxwell J. Farrell and Nicolas Le Guillarme and Liam Brierley and Sara Bronwen Hunter and Daan Scheepens and Anna Willoughby and Andrew Yates and Nicole Mideo},
  title = {{The changing landscape of text mining: a review of approaches for ecology and evolution}},
  booktitle = {Proceedings of the Royal Society B Biological Sciences},
  year = {2024},
  doi = {10.1098/rspb.2024.0423},
}
Figure