VerbaNexAI at SemEval-2025 Task 9: Advances and Challenges in the Automatic Detection of Food Hazards

Andrea Menco Tovar, Edwin Puertas, Juan Carlos Martinez-SantosACL 2025

TL;DR

This paper details VerbaNexAI's participation in SemEval-2025 Task 9, focusing on automatically identifying and classifying food hazards from text descriptions. They achieved respectable results using a Random Forest classifier with TF-IDF and character n-grams, highlighting the challenges of underrepresented categories and semantic ambiguity in this domain.

What Problem Does This Paper Address?

Ensuring global food safety is a critical challenge, and the timely detection of potential hazards in food products is paramount. This paper tackles the problem of automatically identifying and classifying these hazards directly from descriptive texts, such as incident reports and product warnings. The sheer volume of data generated in the food supply chain necessitates automated systems to monitor for risks, prevent public health incidents, and improve food safety standards. SemEval-2025 Task 9 specifically aims to advance this capability by providing a benchmark for systems that can process and interpret textual data related to food safety incidents.

Methods

VerbaNexAI employed a machine learning-based strategy centered on supervised classification. Their core approach involves:

  • TF-IDF Vectorization with Character N-grams: Textual data is transformed into numerical feature vectors using TF-IDF. Crucially, they incorporated character n-grams (ranging from 2 to 5 characters) to better capture local and contextual linguistic patterns, which is particularly useful for handling variations and potential typos. Parameters like strip_accents='unicode', max_df=0.5, and min_df=5 were used to manage linguistic variability and focus on relevant terms.
  • Random Forest Classifier: A Random Forest classifier was chosen for the classification stage. The authors highlight its ability to handle high-dimensional data, provide accurate results without extensive hyperparameter tuning, and reduce overfitting by averaging multiple decision trees. Its interpretability through feature importance was also noted as a benefit.
  • Pipeline Approach: The system integrates text vectorization and classification into a streamlined pipeline, allowing for independent experimentation and tuning of each component.

Datasets

The paper mentions the use of datasets provided for the SemEval-2025 Task 9 challenge, which include training and validation divisions. The training set contained approximately 5,082 samples, while the test set comprised about 5,984 examples. The data encompasses various textual genres related to food safety incidents from multiple sources. The task itself is divided into two sub-tasks: ST1 (text classification for food hazard prediction) and ST2 (food hazard and product vector detection).

Key Insights and Ideas

  • Character N-grams are Key for Nuance: The use of character n-grams (2-5) proved effective in capturing subtle linguistic patterns and contextual information, which is vital for distinguishing between similar hazard categories or products. This approach helps overcome semantic ambiguity and linguistic variability.
  • Random Forest Offers Robustness: The Random Forest classifier demonstrated robustness in handling data variability and provided a good balance between accuracy and recall, even with potentially unbalanced classes.
  • Underrepresented Categories are a Major Hurdle: A significant challenge identified is the difficulty in classifying categories with limited data. This "data shortage" issue impacts the model's ability to learn representative patterns, leading to confusion between semantically similar but distinct categories (e.g., food allergies vs. bacterial contamination).
  • Semantic Ambiguity Requires Context: The inherent ambiguity in natural language, where terms can have multiple meanings, poses a substantial challenge. While character n-grams help, fully disambiguating terms in complex texts remains an area for improvement.

Key Results

VerbaNexAI achieved the following macro average F1 scores:

  • Sub-task ST1 (Hazard and Product Category Classification): 0.5165, ranking 24th out of 27 participating teams.
  • Sub-task ST2 (Food Hazard and Product Vector Detection): 0.3223, ranking 16th out of 26 teams.

These results are compared to the "Task Best System," which achieved 0.8223 (ST1) and 0.5473 (ST2), indicating a significant performance gap but positioning VerbaNexAI competitively among other participants.

Limitations and Future Work

The authors acknowledge several limitations:

  • Under-represented Categories: Despite parameter adjustments like min_df=5, under-represented classes continue to influence model accuracy.
  • Semantic Ambiguity: While character n-grams assist, fully disambiguating terms in complex texts remains a challenge.
  • Linguistic Variability: Synonyms, grammatical variations, and typos still pose difficulties, even with preprocessing.
  • Product Category Variability: Predictions for product categories showed higher variability, likely due to the diverse and specific nature of food products mentioned.

Future work proposed includes:

  • Incorporating more advanced NLP techniques, such as contextualized embeddings, to better capture semantic subtleties.
  • Expanding the training dataset.
  • Exploring architectures based on deep neural networks.
  • Evaluating new NLP methods to address the complexity of textual data more effectively.
  • Optimizing text representation and exploring more sophisticated classifiers.

Summary updated

BibTeX
@inproceedings{tovar2025verbanexai,
  author = {Andrea Menco Tovar and Edwin Puertas and Juan Carlos Martinez-Santos},
  title = {{VerbaNexAI at SemEval-2025 Task 9: Advances and Challenges in the Automatic Detection of Food Hazards}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}
Figure