Web-based Annotation Interface for Derivational Morphology

Lukáš KyjánekNAACL 2022

TL;DR

This paper introduces a web-based visual interface designed to streamline the manual annotation of derivational morphology, a complex linguistic task. By offering a more intuitive and interactive approach compared to traditional text-based methods, the interface significantly speeds up the annotation process, reducing costs and potential errors, and is also adaptable for syntactic annotation.

What Problem Does This Paper Address?

Creating high-quality language resources often requires extensive manual annotation, which can be a time-consuming and expensive process, especially for complex linguistic phenomena like derivational morphology. Derivational morphology involves forming new words from existing ones (e.g., "read" -> "reader," "readable," "readability"), and annotating these relationships, particularly in languages with rich morphology, requires annotators to make numerous interconnected decisions. This complexity can lead to prolonged annotation times and increased errors. Existing methods often rely on simple text files (like .tsv), which lack visual guidance and interactivity, making the process tedious and prone to mistakes. The paper aims to address this by providing a more efficient and user-friendly annotation tool.

Methods

The core contribution is a novel web-based visual interface for annotating derivational morphology. This interface is built using standard web technologies (HTML5, CSS3, JavaScript, jQuery, CytoScape.js, Notify.js) and runs entirely on the client's side, ensuring broad accessibility across different browsers and devices without requiring complex installation.

Key features of the interface include:

  • Visual Representation: Derivational families are displayed as rooted trees, aligning with linguistic theory.
  • Interactive Editing: Annotators can directly manipulate the tree structure by adding or removing derivational relationships (edges) represented by solid or dotted lines, respectively.
  • Data Format: The interface primarily uses the JSON format for input and output, with built-in converters for .tsv files, bridging the gap with existing data formats.
  • User Feedback Integration: The interface has been iteratively developed based on feedback from real annotators, incorporating features like keyboard shortcuts, search functionality, and buttons to manage complex families (e.g., removing/restoring all edges).
  • Adaptability: The paper demonstrates the interface's potential for annotating syntactic data by converting Universal Dependencies .conllu files into the required JSON format.

Datasets

The paper does not introduce new datasets for training. Instead, it uses existing derivational morphology data, specifically mentioning Czech DeriNet, for the annotation experiment. For the syntactic annotation demonstration, data from the Universal Dependencies collection was used.

Key Insights and Ideas

  • Visualisation is Key: The most significant insight is that a visual, interactive interface dramatically simplifies and accelerates the annotation of complex relational data like derivational morphology, compared to purely text-based methods.
  • Simplicity Can Be Powerful: The interface is built with relatively basic web technologies, demonstrating that sophisticated functionality and significant time savings can be achieved without resorting to highly complex or specialized tools.
  • Annotator Feedback is Crucial: Iterative development based on direct feedback from annotators is essential for creating tools that are not only functional but also genuinely useful and efficient in practice.
  • Cross-Domain Applicability: The interface's underlying principles of visualising and editing tree-like structures make it adaptable beyond derivational morphology, showing promise for other annotation tasks like syntax.

Key Results

The primary hypothesis that the annotation process is faster with the visual interface was validated.

  • Speedup: Annotators using the visual interface were consistently faster than those using the traditional .tsv file format.
  • Smaller Families: For smaller derivational families, the time difference was less pronounced, suggesting the .tsv method is comparably efficient for simpler cases.
  • Larger Families: For larger and more complex families, the visual interface showed a significant speed advantage. Figure 7 illustrates that the total time spent annotating 20 derivational families was noticeably less with the visual interface.
  • Qualitative Feedback: Annotators found the new interface more attractive, easier to use, and faster, leading to greater savings in time and potentially cost. They also confirmed that the interface made their work easier and faster.
  • Error Reduction: While not explicitly quantified, the paper implies that the visual nature and interactive feedback mechanisms of the interface help reduce errors.

Limitations and Future Work

  • Browser Limitations: The client-side nature of the web application means it is subject to the memory limitations of web browsers, although the authors note that current limits are generally sufficient for manual annotation.
  • Continuous Improvement: The authors acknowledge that the interface is a work in progress and that there are always ways to improve or extend its functionalities. Future work could involve further enhancements based on ongoing user feedback and exploring its application to an even wider range of annotation tasks.

Summary updated

BibTeX
@inproceedings{kyjanek2022web,
  author = {Lukáš Kyjánek},
  title = {{Web-based Annotation Interface for Derivational Morphology}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure