Sentence Generation by Semantic Concordance

Toshiyuki Sakai, Makoto NagaoCOLING 1965

TL;DR

This 1965 paper introduces a novel approach to English sentence generation by focusing on "semantic concordance" – ensuring that generated words and phrases align semantically. It proposes a three-stage generation process: kernel sentence generation via phrase structure rules, application of transformational rules to internal tree structures, and final morphophonemic modifications. This work lays foundational ideas for how meaning can guide syntactic construction in natural language processing.

What Problem Does This Paper Address?

In 1965, the field of Machine Translation (MT) and natural language processing faced significant challenges in generating grammatically correct and semantically meaningful sentences. While syntactic rules were being developed, grasping the semantics of sentences and their interrelationship with syntax remained a major hurdle. The authors aimed to address this by developing a sentence generation method that explicitly incorporated semantic considerations from the outset, moving beyond purely structural approaches. They recognized that a truly effective translation system needed to understand and generate sentences with "complete semantic consistency."

Methods

The paper outlines a three-stage process for generating English sentences:

  • Kernel Sentence Generation: This initial stage uses phrase structure grammar to create a basic sentence structure. Crucially, semantic categories attached to words in a dictionary are utilized to select word pairs (e.g., subject-verb, verb-object) that are semantically compatible. This ensures that even at this early stage, the generated sentence has a coherent conceptual basis.
  • Transformational Rules: Unlike immediate constituent analysis, this stage treats sentences as having an internal tree structure. Transformational rules are then applied to this structure. The authors also propose a method for sentence generation that can start from any point in the tree, not just the axiom, allowing for more flexible construction.
  • Morphophonemic Modifications: The final stage involves applying rules to modify the generated string into its final phonetic form. These rules are conceptualized as operators that act on words in their neighborhood, handling aspects like negation, tense, and case.

A key innovation is the concept of "semantic concordance," where word selection is guided by semantic categories. This is formalized by defining semantic distances between categories and using a condition fx to ensure compatibility between words in different syntactic roles.

Datasets

Not discussed in the paper. The paper focuses on the theoretical framework and algorithmic approach to sentence generation rather than empirical evaluation on specific datasets.

Key Insights and Ideas

  • Semantic Concordance as a Guiding Principle: The core idea is that semantic compatibility should drive syntactic generation, not just be an afterthought. This is a departure from purely structural grammar approaches.
  • Three-Stage Generation Process: The proposed model breaks down sentence generation into distinct, manageable steps, each with a specific role (syntax, transformation, phonology).
  • Internal Tree Structures for Transformations: Treating sentences as having internal tree structures allows for more sophisticated transformations than simple string manipulation.
  • Generation from Arbitrary Points: The ability to generate sentences starting from any point in a syntactic tree offers greater flexibility and can handle complex sentence structures more naturally.
  • Semantic Categories and Distances: The formalization of semantic categories and their relationships (via distances and compatibility conditions) provides a computational basis for semantic reasoning in generation.
  • Operators for Morphophonemic Rules: Conceptualizing morphophonemic rules as operators acting on word neighborhoods offers a structured way to handle inflectional and derivational morphology.

Key Results

The paper presents a theoretical framework and algorithmic proposals for sentence generation. It illustrates the concepts with examples and figures (Fig. 1-5) demonstrating the generation process, transformations, and application of morphophonemic rules. While no quantitative performance metrics are provided, the paper demonstrates the feasibility of generating syntactically and semantically coherent sentences using the proposed methods. The generated examples, though simple, showcase the intended functionality.

Limitations and Future Work

The authors acknowledge several limitations and areas for future work:

  • Defining "Conceivable Sentence Structures": While the rules aim to generate "conceivable sentence structures," the actual constraints on these structures (depth, coordination, asymmetry) require further investigation, possibly by analyzing frequency ratios in actual sentences.
  • Semantic Sentence Anomaly Detection: Detecting semantic anomalies is noted as more difficult than detecting grammatical errors, and the paper proposes three levels of criteria for meaningful sentences, focusing on "complete semantic consistency."
  • Formalizing Transformational Rules: Representing transformational rules formally without contradiction for all generated structures is acknowledged as difficult.
  • Transformations Involving Part-of-Speech Changes: Transformations that alter the part of speech or morphophonemic forms of words are difficult to handle and require word dictionaries with information on such interchanges.
  • Semantic Dependence of Transformations: Transformational rules cannot always be applied unconditionally; their application often depends on the semantics of the sentence.
  • Complexity of Morphophonemic Rules: The paper notes that while morphophonemic rules can be represented as operators, "many complicated situations" arise after applying transformational rules, and the extent to which this approach can be pursued remains to be seen.
  • Experimental Scope: The experiments mentioned were conducted on a medium-sized computer with a limited vocabulary, suggesting that scaling up the system would be a significant undertaking.

Future work suggested includes exploring the reverse procedure of sentence generation (decomposition into semantic units) and applying these techniques to machine translation, particularly for languages like Japanese that have different sentence structures than English.

Summary updated

BibTeX
@inproceedings{sakai1965sentence,
  author = {Toshiyuki Sakai and Makoto Nagao},
  title = {{Sentence Generation by Semantic Concordance}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1965},
}
Figure