Seeds of Discourse: A Multilingual Corpus of Direct Quotations from African Media on Agricultural Biotechnologies

Patricia Chiril, Trevor Spreadbury, Joeva Rock, Brian Dowd-Uribe, David UminskyNAACL 2025 · DOI

Introduction

The discourse surrounding genetically modified organisms (GMOs) in Africa is a multifaceted intersection of science, politics, and social identity. While proponents highlight the potential for increased crop yields and resilience against climate change, critics raise concerns regarding food sovereignty and the influence of multinational agribusiness. As media outlets serve as the primary arena for these competing narratives, understanding "who says what" is crucial for researchers. However, existing Natural Language Processing (NLP) resources often lack the domain specificity and multilingual coverage necessary to capture the nuances of African media.

To bridge this gap, researchers have introduced the Seeds of Discourse corpus, the first large-scale multilingual dataset of direct quotations from African news media focused on agricultural biotechnologies.

Methodology: From Archives to Annotated Corpus

The creation of the Seeds of Discourse corpus involved a rigorous multi-stage pipeline designed to ensure both scale and high fidelity.

The data processing pipeline, from raw news archives to annotated quotation-speaker pairs using Label Studio and various NLP models.

The process began with the retrieval of 1.2 million articles from the Dow Jones Factiva archive, spanning from 1997 to 2023. These were filtered to include only Africa-based publishers and articles containing relevant biotechnology keywords, resulting in a pool of approximately 804,000 English and 300,000 French articles.

For quotation extraction, the team compared two distinct approaches:

  1. Fine-tuned Encoder Models: Smaller transformer models, including distilBERT, mBERT, and XLM-RoBERTa, were fine-tuned on existing quotation datasets.
  2. Large Language Models (LLMs): Llama-3.1-8B was evaluated using a three-shot prompting strategy to extract quotation-speaker pairs directly.

Finally, a subset of the data underwent manual annotation using the Label Studio platform. Eight annotators, including native speakers, performed Aspect-Based Sentiment Analysis (ABSA), identifying specific entities (e.g., "Crops"), attributes (e.g., "Productivity"), and sentiment polarities.

The Seeds of Discourse Dataset

The resulting corpus provides a unique resource for both NLP and social science researchers. It consists of:

  • Manually Annotated Set: 1,657 high-quality direct quotations (1,299 English, 358 French).
  • Automatically Extracted Set: Over 102,000 quotations where multiple models reached consensus, providing a massive scale for longitudinal analysis.
  • ABSA Subset: 665 instances annotated for fine-grained sentiment across 11 entity types and 9 attribute labels.

By focusing on direct quotations, the dataset ensures traceability, allowing researchers to attribute specific claims to identifiable stakeholders within the biotechnology sector.

Results: The "LLM Gap" in Structural Tasks

A key finding of this research is the performance disparity between specialized fine-tuned models and general-purpose LLMs. While LLMs like Llama-3.1-8B are highly versatile, they struggled with the structural precision required for quotation extraction.

Comparison of model performance (accuracy) for quotation extraction across English and French datasets, highlighting the superiority of fine-tuned transformers.

As shown in the performance data, fine-tuned encoder models like mBERT and XLM-RoBERTa achieved superior accuracy, reaching up to 98.6% in French. In contrast, Llama-3.1-8B lagged behind, often "hallucinating" by splitting quotes or failing to capture complete substrings. This suggests that for specific, well-defined NLP tasks, specialized fine-tuning remains more effective than general-purpose prompting.

Complexity in Discourse Annotation

The manual annotation process revealed the inherent complexity of the GMO debate. While inter-annotator agreement was nearly perfect for identifying the presence of a quote, it decreased significantly for the Aspect-Based Sentiment Analysis (78.2% for English, 61.6% for French). This reflects the nuanced nature of the discourse; a single statement might simultaneously express positive sentiment regarding a crop's productivity while voicing negative concerns about its economic impact or safety.

Conclusion and Future Directions

The Seeds of Discourse corpus provides a foundational tool for analyzing the evolving narrative of agricultural biotechnology in Africa. While currently limited to text-based news, future work aims to incorporate radio, television, and social media data to capture a more holistic view of the media landscape. By providing a traceable, multilingual dataset, this research enables a deeper understanding of how public opinion is shaped by various stakeholders in the global agricultural sector.

Summary updated

BibTeX
@inproceedings{chiril2025seeds,
  author = {Patricia Chiril and Trevor Spreadbury and Joeva Rock and Brian Dowd-Uribe and David Uminsky},
  title = {{Seeds of Discourse: A Multilingual Corpus of Direct Quotations from African Media on Agricultural Biotechnologies}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.findings-naacl.473},
}
Figure