A Hybrid Natural Language Generation System Integrating Rules and Deep Learning Algorithms

Wei Wei, Bei Zhou, Georgios Leontidis

Abstract

This paper proposes an enhanced natural language generation system combining the merits of both rule-based approaches and modern deep learning algorithms, boosting its performance to the extent where the generated textual content is capable of exhibiting agile human-writing styles and the content logic of which is highly controllable. We also come up with a novel approach called HMCU to measure the performance of the natural language processing comprehensively and precisely.

Introduction

Natural Language Generation (NLG) has long struggled with a fundamental trade-off between factual reliability and stylistic fluidity. Rule-based systems offer high controllability and logical consistency by following strict templates, but they often produce repetitive, "machine-like" text. Conversely, deep learning models like LSTMs or Transformers generate creative, human-like prose but are prone to "hallucinations" and logical drift, making them risky for high-stakes, data-driven applications like real-time news reporting.

This research introduces a hybrid system designed to bridge this gap, ensuring high factual controllability while achieving a natural, human-like writing style. To evaluate this balance, the authors introduce the HMCU Analysis Model, a novel framework for mapping AI text across dimensions of style and logic.

Methodology

The proposed system treats NLG as a multi-stage refinement process, moving from structured data to polished prose.

  1. Rule-Based Generation: Structured data (e.g., news attributes) is processed by a traditional rule-based engine to ensure all core facts are accurately captured.
  2. Deep Learning Paraphraser: The output from the rule-based stage is fed into an LSTM-based sequence-to-sequence model. This model, trained on the extensive ParaNMT-50M dataset, rewrites the factual sentences into more varied and agile human styles.
  3. Grammar Correction: A final verification step ensures that the paraphrasing process has not introduced grammatical errors.
Pipeline of the Hybrid NLG System

The HMCU Evaluation Framework

To measure performance, the authors developed the HMCU model, which evaluates text based on four key metrics:

  • H (Human language): Similarity to natural human writing.
  • M (Machine style): Similarity to repetitive, template-driven writing.
  • C (Controllable context logic): The degree to which factual logic remains accurate.
  • U (Uncontrollable context logic): The degree of logical drift or meaninglessness.
The HMCU Analysis Model Quadrant

The goal of the hybrid system is to achieve a "Quadrant Shift." While rule-based systems typically reside in the MC (Machine-style/Controllable) quadrant and pure deep learning models in the HU (Human-style/Uncontrollable) quadrant, this hybrid approach aims for the elusive HC (Human-style/Controllable) quadrant.

Experimental Results

The hybrid system was tested against pure rule-based and pure LSTM models using a dataset of 900 news items. The results highlighted a successful middle ground:

  • Logic Retention: While the rule-based system remained the gold standard for logic (0.744 similarity), the pure LSTM was nearly unusable (0.059). The Hybrid system achieved 0.173, a significant improvement over pure deep learning.
  • Style Flexibility: The rule-based system exhibited a high "Machine Style" score (0.242). The Hybrid system reduced this to 0.098, moving significantly closer to the pure LSTM's human-like score of 0.043.

Qualitative case studies further demonstrated the system's ability to transform dry, repetitive reports into sophisticated paragraphs while maintaining factual integrity.

Conclusion

The hybrid approach successfully mitigates the "black box" risks of deep learning by using it as a paraphrasing safety net for factually sound, rule-generated text. While challenges like semantic drift remain—where paraphrasing occasionally alters specific meanings—the system represents a significant step toward controllable, high-quality AI writing. Future work will focus on refining the HMCU quantitative indicators and improving logical stability during the paraphrasing stage.

Summary updated

Abstract

This paper proposes an enhanced natural language generation system combining the merits of both rule-based approaches and modern deep learning algorithms, boosting its performance to the extent where the generated textual content is capable of exhibiting agile human-writing styles and the content logic of which is highly controllable. We also come up with a novel approach called HMCU to measure the performance of the natural language processing comprehensively and precisely.
BibTeX
@misc{wei2020hybrid,
  author = {Wei Wei and Bei Zhou and Georgios Leontidis},
  title = {{A Hybrid Natural Language Generation System Integrating Rules and Deep Learning Algorithms}},
  year = {2020},
  eprint = {2006.09213},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.09213},
}
Figure