PopALM: Popularity-Aligned Language Models for Social Media Trendy Response Prediction

Erxin Yu, Jing Li, Chunpu Xu

Abstract

Large Language Models (LLMs) often struggle to replicate the specific nuances of social media discourse, frequently defaulting to neutral or generic responses that fail to capture the "vibe" of viral content. This paper introduces PopALM (Popularity-Aligned Language Models), a framework designed to predict trendy social media responses by aligning model outputs with public engagement metrics. By utilizing a novel Curriculum Learning-Enhanced Proximal Policy Optimization (CL-PPO) strategy, PopALM effectively navigates the inherent noise in social media popularity signals. Experimental results on a large-scale Weibo dataset demonstrate that PopALM significantly outperforms standard fine-tuning and reinforcement learning baselines in generating diverse, viewpoint-specific, and popular responses.

Introduction

While standard LLMs excel at being helpful and harmless, they often lack the stylistic flair and opinionated nature characteristic of social media platforms. When prompted to respond to trending news or controversial events, models like ChatGPT typically provide polite, macro-level summaries. However, real-world "trendy" responses—those that garner the most engagement—are often characterized by specific emotions, sharp viewpoints, or unique cultural references.

The primary obstacle in training models to emulate this behavior is that popularity metrics, such as "likes," are extremely noisy labels. A comment's popularity is not solely a function of its content; it is heavily influenced by external factors like the author's follower count or the timing of the post. This noise makes it difficult for standard reward models to discern what truly constitutes a high-quality, popular response based on text alone. PopALM addresses this by integrating curriculum learning into the Reinforcement Learning from Human Feedback (RLHF) pipeline.

Methodology: Curriculum Learning-Enhanced PPO

PopALM follows the traditional RLHF workflow—Supervised Fine-Tuning (SFT), Reward Modeling (RM), and Reinforcement Learning (RL)—but introduces three key innovations within a framework called Curriculum Learning-Enhanced PPO (CL-PPO):

  1. Reward Enhancement: To provide a more grounded training signal, the authors supplement the learned reward model with a task-specific metric. They calculate the ROUGE-L overlap between the model's generated response and the actual "golden" trendy responses from the dataset, ensuring the model stays anchored to real-world popular discourse.
  2. Reward Ranking and Filtering: To mitigate the impact of noisy labels, the framework ranks generated samples by their reward scores and discards the bottom-performing percentage. This prevents the model from attempting to learn from low-confidence samples that may represent noise rather than genuine popularity.
  3. Self-paced Sampling (Curriculum Learning): The training process follows a curriculum that starts with "easy" samples—those where the reward signal is high and the confidence is clear. As the model matures, the threshold is gradually lowered to include "harder," more ambiguous samples. This mimics human learning patterns, allowing the model to establish a stable foundation before tackling the complexities of social media sentiment.

Dataset and Benchmarking

The researchers developed a comprehensive benchmark using data from Weibo, a leading Chinese social media platform. The dataset consists of approximately 70,000 posts and 24 million comments, reflecting the "long-tail" distribution typical of social media where a small fraction of content receives the vast majority of engagement.

  • SFT Set: 25,140 samples using the top 10 comments per post as references.
  • RM Set: 9,985 pairs of responses (high-popularity vs. low-popularity) to train the reward model.
  • RL Set: 2,514 posts for policy optimization.

Key Insights and Results

PopALM was evaluated against several strong baselines, including LLaMA and ChatGLM. The results highlight several critical findings:

  • Superior Performance: Using ChatGLM as a base, PopALM achieved a ROUGE-L of 16.42 (Top-1), outperforming the SFT version (15.94) and standard PPO.
  • Popularity vs. Genericity: A significant finding of the study is that generic responses are actually less effective at reflecting public viewpoints. Popularity is closely tied to specific, concrete viewpoints. Human raters scored PopALM higher on both "Specification" (presence of specific viewpoints) and "Popularity" (potential to be liked) compared to standard models.
  • Diversity and Stability: Unlike some RL training methods that suffer from "mode collapse" (where the model repeats a few safe, high-reward phrases), PopALM maintained high diversity in its outputs, as measured by M-Distinct metrics.
  • Cross-Task Utility: The study found that learning to predict trendy responses provides "social signals" that benefit other tasks, such as Poll Question Generation and Social Emotion Prediction, suggesting that popularity alignment helps the model understand broader public concerns.

Limitations and Future Directions

Despite its success, PopALM faces challenges inherent to social media data. The "rich get richer" bias—where existing fame drives likes regardless of content—remains difficult to fully decouple from text quality. Additionally, as the current study focused on Weibo, the generalizability of these popularity features across different cultures and platforms (such as X/Twitter or Reddit) remains an area for future exploration. Finally, given the rapid evolution of social media trends, future iterations of PopALM may incorporate real-time online learning to keep pace with shifting public discourse.

Summary updated

BibTeX
@inproceedings{yu2024popalm,
  author = {Erxin Yu and Jing Li and Chunpu Xu},
  title = {{PopALM: Popularity-Aligned Language Models for Social Media Trendy Response Prediction}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2024},
}
Figure