Abstract
Generating long and informative review text is a challenging natural language generation task. Previous work focuses on word-level generation, neglecting the importance of topical and syntactic characteristics from natural languages. In this paper, we propose a novel review generation model by characterizing an elaborately designed aspect-aware coarse-to-fine generation process. First, we model the aspect transitions to capture the overall content flow. Then, to generate a sentence, an aspect-aware sketch will be predicted using an aspect-aware decoder. Finally, another decoder fills in the semantic slots by generating corresponding words. Our approach is able to jointly utilize aspect semantics, syntactic sketch, and context information. Extensive experiments results have demonstrated the effectiveness of the proposed model.
Introduction
Generating long-form text, such as product reviews, is notoriously difficult for standard Natural Language Generation (NLG) models. Most existing methods predict text one word at a time, which often results in repetitive or incoherent output for longer sequences. The core challenge lies in the lack of high-level planning. Humans typically decide which features to discuss and follow a general sentence structure before selecting specific words. This paper addresses this gap by incorporating topical and syntactic constraints into a hierarchical decoding process.
Methodology
The proposed Aspect-Aware Coarse-to-Fine Decoder framework mimics human writing through three distinct stages:
- Aspect Sequence Generation: The model encodes context information (User ID, Item ID, and Rating) and predicts a sequence of topics (e.g., Sound → Service → Price). These aspects are discovered in an unsupervised manner using a modified Twitter-LDA model.
- Aspect-Aware Sketch Generation: For each predicted aspect, the model generates a "sketch"—a sentence skeleton containing semantic slots filled with Part-of-Speech (POS) tags or frequent n-grams (e.g., "the NN sounds RB great").
- Sketch-based Review Generation: A final sentence decoder fills the semantic slots with actual words, utilizing aspect-specific vocabulary distributions to ensure topical relevance.
Experiments
The model was evaluated on three large-scale, real-world review datasets: AMAZON (Electronics) with ~681k reviews, YELP (Restaurants) with ~1.06M reviews, and RATEBEER with ~2.48M reviews. Preprocessing ensured that users and items had sufficient interaction history to provide meaningful context for the generation process.
Results
The proposed model consistently outperformed several strong baselines, including context-aware models (gC2S, Attr2Seq) and GAN-based models (SeqGAN, LeakGAN).
As shown in the table above, the model achieved the best results across all metrics. On the Amazon dataset, it reached a Perplexity of 26.55 and a BLEU-1 of 28.22, significantly better than the LeakGAN baseline. Furthermore, human evaluations on a 5-point Likert scale rated the model at 3.54 for fluency, significantly closer to the gold standard (4.01) than competing models (~3.2).
Conclusion
This research demonstrates that breaking down the generation task into hierarchical stages—planning "what to say," "how to structure it," and "which words to use"—makes the generation of long text much more controllable and coherent. By providing a grammatical "rail" through syntactic sketches, the model avoids the common pitfalls of word-level generation. Future work will explore integrating more advanced linguistic features, such as dependency parsing, to further enhance linguistic variety.
Summary updated