CaMEL: Case Marker Extraction without Labels

Leonie Weissweiler, Valentin Hofmann, Masoud Jalili Sabet, Hinrich Schütze

Abstract

We introduce CaMEL (Case Marker Extraction without Labels), a novel and challenging task in computational morphology that is especially relevant for low-resource languages. We propose a first model for CaMEL that uses a massively multilingual corpus to extract case markers in 83 languages based only on a noun phrase chunker and an alignment system. To evaluate CaMEL, we automatically construct a silver standard from UniMorph. The case markers extracted by our model can be used to detect and visualise similarities and differences between the case systems of different languages as well as to annotate fine-grained deep cases in languages in which they are not overtly marked.

Introduction

In linguistics, "case" refers to the grammatical system that marks the relationship between a noun and other elements in a sentence—identifying roles such as the subject, direct object, or recipient. While languages like English primarily rely on word order or prepositions, many of the world's languages utilize morphological markers, such as suffixes or prefixes, to convey these relationships.

For the vast majority of the world’s 7,000+ languages, manually labeled morphological analyzers (gold standards) do not exist. Furthermore, the mapping between a surface marker and its grammatical meaning is rarely one-to-one; languages often exhibit polysemy (one marker with multiple meanings) or syncretism (multiple cases sharing a single marker). CaMEL (Case Marker Extraction without Labels) addresses these challenges by providing an unsupervised framework to automatically identify these markers from raw text, facilitating research into low-resource languages and cross-linguistic typology.

Methodology

The CaMEL pipeline leverages the "parallel nature" of multilingual datasets to identify morphological signals without human intervention. The process consists of four primary stages:

1. NP Annotation and Projection

The system begins with English text, where Noun Phrases (NPs) can be reliably identified using mature NLP tools like spaCy. Using word-level alignments, these NP boundaries are "projected" from the English source onto parallel texts in 83 target languages. This creates a noisy but functional set of NP labels in languages that lack native chunkers.

2. Candidate Generation

Within the projected NPs of the target languages, the model generates all possible character n-grams (substrings) from the constituent words. These substrings serve as the initial pool of potential case markers.

3. The "Inside/Outside" Statistical Filter

The core innovation of CaMEL is the "Inside/Outside" heuristic. Because case is a grammatical property of nouns, a valid case marker should appear significantly more frequently within an NP than outside of it (where verbs, adverbs, or other parts of speech dominate). The model employs a Fisher’s Exact Test to statistically evaluate each candidate substring. Substrings that cluster significantly within NP boundaries are retained, while common character sequences that appear uniformly across the text are filtered out.

4. Morphological Constraints

Guided by the linguistic observation that case is predominantly marked via suffixes rather than prefixes across the world's languages, the model restricts its final output to word-final substrings. This constraint helps prune the candidate list and improves the precision of the extracted markers.

Data and Evaluation

To test the model across a diverse linguistic landscape, the researchers utilized the Parallel Bible Corpus (PBC). The PBC provides high-quality alignments across 83 languages, making it an ideal environment for cross-linguistic projection.

For evaluation, the authors constructed a "silver standard" using UniMorph, a database of morphological paradigms. By comparing the unsupervised output against the suffixes documented in UniMorph for 19 specific languages, the researchers were able to quantify the model's ability to recover known morphological structures.

Insights and Results

Distribution as a Proxy for Labels

A key takeaway from CaMEL is that distributional statistics can effectively substitute for expert labels. If a specific suffix (e.g., the Latin -ibus) consistently appears within projected NP boundaries across thousands of verses, it is mathematically probable that the suffix serves a nominal grammatical function.

Cross-Linguistic Triangulation

The model demonstrates that ambiguous markers in one language can be disambiguated by looking at their counterparts in other languages. For instance, a single marker in Latin might represent three different semantic roles. By analyzing the alignments to languages like Russian or Polish—which may use distinct markers for those same roles—the system can "triangulate" and visualize the underlying semantic categories hidden behind surface-level syncretism.

Quantitative Performance

The model achieved an average F1 score of 45% across the 19 evaluation languages. While lower than supervised benchmarks, this represents a significant baseline for a completely unsupervised approach in a zero-resource setting. The system generally exhibits higher precision than recall; it is effective at identifying valid markers but occasionally struggles with "segmentation noise," such as including a portion of the word root within the identified suffix.

Limitations and Future Work

Despite its successes, CaMEL faces several hurdles:

  • Segmentation Accuracy: Precisely determining the boundary between a root and a suffix remains difficult, often resulting in substrings that are slightly too long or short.
  • Suffix Bias: The current reliance on word-final markers means the system would fail to identify case in languages that utilize prefixes or infixes.
  • Language Diversity: While tested on 83 languages, the formal evaluation focused largely on Indo-European families. Future work will involve testing the "Inside/Outside" heuristic on more diverse language families, such as Turkic or Sino-Tibetan.

Conclusion

CaMEL represents a significant step toward the automated analysis of unstudied languages. By combining cross-linguistic projection with statistical filtering, it provides a pathway to map out the declension systems of low-resource languages using only parallel corpora, offering a powerful tool for both computational and descriptive linguistics.

Summary updated

Abstract

We introduce CaMEL (Case Marker Extraction without Labels), a novel and challenging task in computational morphology that is especially relevant for low-resource languages. We propose a first model for CaMEL that uses a massively multilingual corpus to extract case markers in 83 languages based only on a noun phrase chunker and an alignment system. To evaluate CaMEL, we automatically construct a silver standard from UniMorph. The case markers extracted by our model can be used to detect and visualise similarities and differences between the case systems of different languages as well as to annotate fine-grained deep cases in languages in which they are not overtly marked.
BibTeX
@inproceedings{weissweiler2022camel,
  author = {Leonie Weissweiler and Valentin Hofmann and Masoud Jalili Sabet and Hinrich Schütze},
  title = {{CaMEL: Case Marker Extraction without Labels}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=JlvO97QOlE},
}
Figure