Using Formulaic Expressions in Writing Assistance Systems

Kenichi Iwatsuki, Akiko AizawaCOLING 2018

TL;DR

This paper introduces a novel semantic search framework for formulaic expressions (FEs) in academic writing, enabling writing assistance systems to suggest relevant phrases even with incomplete user input. By leveraging annotated FE dictionaries and domain-specific corpora, their method significantly improves the accuracy of predicting the correct FE category, addressing a key limitation of existing keyword-based systems.

What Problem Does This Paper Address?

Non-native English speakers often struggle with academic writing, relying on repetitive phrases or lacking confidence in their word choices. While existing writing assistance systems can suggest phrases, they typically rely on keyword matching, which fails to capture the nuances of academic language and often misses relevant formulaic expressions (FEs) that don't directly contain the user's input keywords. Furthermore, manual searching of FE dictionaries is time-consuming. This paper tackles two main challenges: the lack of contextual information in existing FE dictionaries for accurate semantic representation, and the difficulty users face in manually selecting the correct semantic category for their intended phrase.

Methods

The authors propose a new framework for semantic FE search. The core idea is to move beyond keyword matching to a category-based search, mimicking how humans use FE dictionaries. This involves:

  • Semantic Search Framework: The system first predicts a user's intent (represented by an incomplete phrase or sentence) and then searches for candidate FEs within the predicted category.
  • Annotated Dictionary Creation: They address the lack of context by creating an annotated dictionary. This involves distinguishing formulaic from non-formulaic parts of FEs and enriching them with example sentences extracted from actual corpora.
  • Category Prediction: User input is converted into vector representations (averaging skip-gram vectors of words) and classified into categories using a Random Forest classifier.
  • FE Classification: FEs themselves are also classified into categories using a similar methodology, leveraging vector representations of FEs and context information.
  • FE Extraction: They reformulate FE extraction as a sequential labeling problem using Conditional Random Fields (CRFs), learning formulaicity from the annotated dictionary and sentence corpora. This allows for the extraction of both continuous and discontinuous FEs.

Datasets

The study utilizes two primary types of linguistic resources:

  • Existing FE Dictionary: Academic Phrasebank (Morley, 2018) is used as the initial dictionary.
  • Sentence Corpora:
    • ACL Anthology: A corpus of scientific papers from the computational linguistics domain.
  • PubMed: A corpus of scientific papers from the life science and medicine domain.

The authors also created an annotated dictionary by enriching Academic Phrasebank with example sentences from the corpora and distinguishing formulaic/non-formulaic parts. They also developed a paraphrased dataset to increase the number of FEs and example sentences.

Key Insights and Ideas

  • Context is King for FEs: The scarcity of example sentences in traditional FE dictionaries severely hinders the ability of systems to understand and predict FE usage. Real-world corpus data is crucial for learning effective semantic representations.
  • Category-Based Search is Superior: Mimicking human behavior, where users first select a category and then an expression, proves more effective than simple keyword matching for FE retrieval.
  • Discontinuous FEs are Tricky: Extracting FEs that are not contiguous (e.g., "X plays a vital role in the metabolism of...") is significantly more challenging than extracting continuous ones, and current methods still struggle with this.
  • Domain Specificity Matters: While some FEs are common across disciplines, domain-specific technical terms and the frequency of certain FEs can differ significantly between fields like computational linguistics and medicine.

Key Results

  • Improved Category Prediction: The proposed method, trained with example sentences from corpora, achieved significantly higher accuracy in predicting FE categories from incomplete user input compared to using only the limited examples in Academic Phrasebank. For instance, the "PickedWords w/SC" (user input with selected words, trained on corpus sentences) achieved an accuracy of 81.7% for ACL Anthology, compared to 11.2% for "PickedWords w/AP" (same input, trained on Academic Phrasebank examples).
  • Enhanced FE Classification: Using context information (example sentences and section labels) improved the classification of FEs into categories.
  • Effective FE Extraction: The CRF-based sequential labeling approach for FE extraction outperformed methods relying solely on high frequency or mutual information, particularly in extracting continuous FEs.
  • Dataset Characteristics: The annotated dictionary contained 39 categories for ACL Anthology and 45 for PubMed. After paraphrasing, the number of FEs increased, but the final count of usable FEs in the annotated dictionary was 225 for ACL and 334 for PubMed.

Limitations and Future Work

  • Nonsensical Errors: The most frequent error type in FE extraction was "nonsensical" expressions, indicating that the system sometimes generates phrases that are not helpful or grammatically sound.
  • Spanning Errors: Errors related to unnecessary words or missing words in FEs (spanning errors) were also observed, particularly in the PubMed dataset.
  • Domain-Specific Terms: The method is not designed to extract domain-specific technical terms.
  • Future Work: The authors suggest improving vector representations for FEs, exploring advanced machine learning algorithms to reduce nonsensical FEs, and incorporating syntactic information to better handle spanning errors. They also highlight the need for domain-specific FE re-ranking based on user discipline.

Summary updated

BibTeX
@inproceedings{iwatsuki2018using,
  author = {Kenichi Iwatsuki and Akiko Aizawa},
  title = {{Using Formulaic Expressions in Writing Assistance Systems}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}
Figure