Predicting Degrees of Technicality in Automatic Terminology Extraction

Anna Hätty, Dominik Schlechtweg, Michael Dorna, Sabine Schulte im WaldeACL 2020 · DOI

The Spectrum of Technicality

Traditional Automatic Terminology Extraction (ATE) often simplifies the task into a binary classification: a word is either a technical term or it is not. However, in practice, language exists on a spectrum of technicality. For instance, in a DIY context, "drill" is a basic term, whereas "countersink bit" is highly specialized. Distinguishing these degrees—technical, basic, and non-term—is essential for downstream tasks like text simplification or educational tool development.

A major challenge in this multi-class classification is ambiguity. A word like "Licht" (light) carries a general meaning for most speakers but refers specifically to the eyes of game animals in a hunting context. Capturing these nuances requires models that can explicitly compare a word's usage across different linguistic environments.

Examples of semantic shifts between general and domain-specific contexts.

Methodology: Contrastive Word Embeddings

The authors propose a contrastive approach leveraging word embeddings to capture these semantic shifts. They train two distinct sets of embeddings: one on a general-language corpus (GEN) and another on domain-specific corpora (SPEC).

To make these independently trained vector spaces comparable, the authors employ Procrustes Alignment, which rotates the general-language space to align with the domain-specific space. Once aligned, the relationship between the two vectors for a single word can be analyzed. The study evaluates two primary neural architectures:

  1. MLP, CON$\oplus$DIFF: A Multi-Layer Perceptron that takes the concatenated general and domain vectors, along with their absolute difference vector ($|GEN - SPEC|$), as input.
  2. Multi-Channel Model: A more advanced architecture featuring three channels. Two channels process the GEN and SPEC vectors independently, while a third Siamese network component uses shared weights to map both into a shared space, calculating their internal difference.

Dataset and Labeling

To evaluate their approach, the authors developed a new German gold standard for technicality across four domains: Automotive, Cooking, DIY, and Hunting. The general-language baseline was provided by SdeWaC, a large-scale web-crawled corpus. The dataset was labeled using a semi-automatic process:

  • Technical Terms: Sourced from domain-specific glossaries.
  • Basic Terms: Sourced from basic vocabulary lists.
  • Non-terms: Words appearing in neither list.

The resulting dataset comprises over 14,000 term candidates, providing a robust benchmark for multi-class technicality prediction.

Results and Key Insights

The experimental results demonstrate that explicitly modeling the "difference" between general and specialized usage is a powerful signal for technicality. As shown in the performance metrics, models that incorporated the difference vector significantly outperformed traditional baselines and simple concatenation models.

Performance comparison of different models on the technicality prediction task.

The Multi-Channel model achieved the highest performance (F1: 0.89), suggesting that learning the comparison internally through shared weights is more effective than using pre-computed difference vectors. Interestingly, the authors noted that web-crawled general corpora often contain technical snippets. This "pollution" means that highly technical, unambiguous terms often look similar in both GEN and SPEC spaces, while "basic" terms—which are used differently by experts versus laypeople—show the most significant variation.

Conclusion and Future Directions

This research highlights the importance of moving beyond binary ATE. By aligning embedding spaces and focusing on semantic shifts, the proposed models can accurately categorize words by their degree of technicality.

Future work remains, particularly in addressing multi-word expressions (e.g., "variable valve timing"), which are prevalent in technical domains but were excluded from this single-word study. Additionally, the "Hunting" domain proved particularly challenging due to its heavy use of metaphorical language, suggesting that domain-specific linguistic characteristics play a significant role in model performance.

Summary updated

BibTeX
@inproceedings{hatty2020predicting,
  author = {Anna Hätty and Dominik Schlechtweg and Michael Dorna and Sabine Schulte im Walde},
  title = {{Predicting Degrees of Technicality in Automatic Terminology Extraction}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.258},
}
Figure