Scoping natural language processing in Indonesian and Malay for education applications

Zara Maxwell-Smith, Michelle Kohler, Hanna SuominenACL 2022

TL;DR

This paper provides a comprehensive overview of the current state of Natural Language Processing (NLP) research for Indonesian and Malay, highlighting a significant underrepresentation of these languages in NLP development. The authors identify key trends, research gaps, and propose recommendations to foster collaboration and advance NLP applications, particularly in education.

What Problem Does This Paper Address?

Despite the large number of speakers in Indonesia and Malaysia, Indonesian and Malay languages are significantly underrepresented in the development of NLP technologies and available resources. This scarcity makes it difficult for researchers to find existing work, leading to duplicated efforts and hindering the formulation of effective research questions and study designs. The authors aim to address this by providing a clear picture of the existing research landscape, which can invigorate and inform future projects, especially within the education sector where NLP tools could greatly enhance teaching and learning.

Methods

The researchers conducted a scoping review of Indonesian and Malay NLP literature, applying a simplified PRISMA approach. They searched multiple databases using both English and Indonesian/Malay keywords, including terms related to education. A total of 657 studies published from 2016 onwards were included. These studies were classified according to Hirschberg and Manning's 2015 NLP characterization, with an added "Statistical Work" category. The classification was performed by a single reviewer, with a portion double-checked by another to ensure agreement. The study also involved manual normalization of author names and affiliations to identify research hubs.

Datasets

The paper does not explicitly list specific datasets used for training or evaluation in a dedicated section. However, it mentions that many studies involved the creation of custom datasets due to the scarcity of publicly available resources. The authors also highlight the development of several notable NLP resources and benchmarks for Indonesian and Malay, including:

  • IndoLEM (Indonesian Language Evaluation Montage)
  • IndoBERT (Indonesian Bidirectional Encoder Representations from Transformers)
  • IndoNLU (Indonesian Natural Language Understanding)
  • IndoNLG (Indonesian Natural Language Generation)
  • IndoNLI (Indonesian Natural Language Inference)

Key Insights and Ideas

  • Underrepresentation is a major hurdle: The core challenge is the lack of readily available NLP resources and tools for Indonesian and Malay, forcing researchers to often compile their own domain-specific corpora.
  • Exploratory corpus work dominates: The field is largely characterized by exploratory corpus work, machine reading of online text, and sentiment analysis.
  • Education sector potential is untapped: While NLP has advanced, its application in the Indonesian and Malay education sector is limited, with a need for more sophisticated language tools.
  • Cross-linguistic synergy is crucial: Given the close relationship between Indonesian and Malay, leveraging research and resources from one language can significantly benefit the other.
  • Data scarcity impacts evaluation: A recurring theme across reviews is the difficulty in benchmarking performance due to a lack of clear evaluation methods and reference datasets.
  • Emerging trend towards zero/few-shot methods: The paper notes the growing adoption of zero and few-shot methods, supported by pipelines like HuggingFace, as a way to address data limitations.

Key Results

The study identified 657 relevant studies, with statistical and corpus work being the dominant areas. Machine reading and sentiment analysis of online text (news and social media) were also popular. A notable increase was observed in "Spoken Dialogue Systems" research, with a boom in publications in 2019. The majority of the included studies were written in English or had English titles/abstracts. The authors also identified key researchers and research hubs, with Malaysia showing a slightly higher affiliation ratio among the top authors.

Limitations and Future Work

The authors acknowledge several limitations:

  • 'Light-touch' coding: The subjective coding method used for classification might have led to unintentional misinterpretations.
  • Title and abstract screening: The vast majority of references were screened only by title and abstract, limiting the depth of analysis.
  • Author name variations: Normalizing author names proved challenging due to prevalent variations.
  • Exclusion of non-peer-reviewed work: The initial search strategy might have missed valuable research published in non-traditional venues.

Future work suggested includes:

  • Broader adoption of best practices: Encouraging the use of FAIR principles (Findable, Accessible, Interoperable, Reusable) for data and tools.
  • Expanded evaluation methods and datasets: Developing more robust evaluation methods and publicly available datasets, including negative result publications.
  • Collaboration and connection: Fostering stronger ties between Indonesian and Malay NLP research communities.
  • Flexible author name formats: Encouraging publishers to accommodate diverse name traditions and authors to use consistent naming conventions.
  • Investment in spoken language data: Significant investment is needed to develop NLP applications for spoken Indonesian and Malay, including robust handling of code-mixing.
  • Investment in other languages: Simultaneous investment in other languages of Indonesia and Malaysia is crucial for equitable access to NLP advancements.
  • Interdisciplinary collaboration: Encouraging education researchers to consider datasets from other fields and NLP researchers to engage with educational applications.

Summary updated

BibTeX
@inproceedings{maxwellsmith2022scoping,
  author = {Zara Maxwell-Smith and Michelle Kohler and Hanna Suominen},
  title = {{Scoping natural language processing in Indonesian and Malay for education applications}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure