A Genetic Algorithm for Finding a Small and Diverse Set of Recent News Stories on a Given Subject: How We Generate AAAI's AI-Alert

Joshua Eckroth, Eric SchoenAAAI 2019 · DOI

Abstract

This paper presents a practical application of evolutionary computation for the curation of the Association for the Advancement of Artificial Intelligence (AAAI) weekly "AI-Alert" newsletter. While traditional deep learning often struggles with "small data" problems where content changes rapidly, a genetic algorithm (GA) proves effective at balancing topic relevance with editorial diversity. By optimizing for a multi-criteria fitness function, the developed system, NewsFinder, achieves significantly higher reader engagement than both random selection and standard heuristics.

Introduction

Curation is a labor-intensive task, particularly for niche technical subjects. For years, AAAI employed a human webmaster to manually sift through hundreds of weekly news stories to select the 10 most relevant items. Automating this process presents two primary challenges: the "moving target" of news, which precludes simple supervised learning, and the "diversity trap," where high-scoring algorithms may over-select multiple versions of the same trending story. A successful automated curator must model editorial intuition by ensuring a breadth of topics, publishers, and timing.

The NewsFinder Pipeline

The researchers developed a multi-stage pipeline to transform a vast stream of raw data into a curated newsletter. The process begins with discovery, crawling approximately 100 sources including journals, social media, and news sites to identify roughly 1,500 candidate stories.

The NewsFinder pipeline, illustrating the flow from discovery and processing to final delivery to readers.

Following discovery, the system applies NLP-based pre-filtering to remove job advertisements, blog posts, and low-confidence content, narrowing the pool to approximately 200 high-quality candidates.

Optimization via Genetic Algorithm

The core of the system is a Genetic Algorithm that treats newsletter curation as a multicriteria optimization problem. Each "individual" in the population represents a set of 10 stories. The "fitness" of these sets is calculated based on two primary factors:

  1. Relevance: The confidence score of the AI classifier.
  2. Diversity: Measured using the Gini-Simpson index to ensure variety in publication dates, domains, and vocabulary.

The GA starts with 100 random sets and evolves them over several hundred generations through crossover and mutation. This process allows the system to converge on a "fittest" set that satisfies both relevance and diversity constraints.

Fitness optimization over 200 generations, showing the convergence of multiple GA runs toward an optimal set of stories.

Results and Evaluation

The system was evaluated through a six-month A/B/C test involving nearly 11,000 subscribers. Readers were randomly assigned to newsletters generated by a random selector, a "TopClass" heuristic (picking top stories from common categories), or the GA.

The Genetic Algorithm was the clear winner, achieving a 5.55% mean click rate, compared to 4.87% for Random and 4.39% for TopClass. Interestingly, the rigid TopClass heuristic performed worse than random selection, suggesting that over-engineered heuristics can miss the nuanced "interestingness" of news. The GA-based system also significantly outperformed industry benchmarks for "Computer and Electronics" marketing, which typically see a 2.0% click rate.

Conclusion

By treating curation as an evolutionary search problem rather than a simple classification task, the NewsFinder system successfully automates a complex editorial process. While a human-in-the-loop remains for a final 10-minute sanity check, the GA provides a robust, efficient (1.5–2 minute runtime), and highly engaging solution for specialized news delivery.

Summary updated

BibTeX
@inproceedings{eckroth2019genetic,
  author = {Joshua Eckroth and Eric Schoen},
  title = {{A Genetic Algorithm for Finding a Small and Diverse Set of Recent News Stories on a Given Subject: How We Generate AAAI's AI-Alert}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33019357},
}
Figure