TL;DR
This paper introduces "Inflo," a system designed to automatically categorize news articles and extract keyphrases, aiming to enhance content discovery in news aggregation platforms. By leveraging a neural network for categorization and a multi-method approach for keyphrase extraction, Inflo promises to make finding relevant information faster and more personalized.
What Problem Does This Paper Address?
The proliferation of online news and aggregation platforms presents a challenge for users trying to find relevant and interesting content efficiently. Existing solutions often fall into two camps: broad, pre-selected categories that lack specificity, or personalized recommendations that can lead to "echo chambers." This paper tackles the problem of making news navigation more nuanced by providing automatic, granular labels (categories and keyphrases) for news articles, thereby improving the user's ability to filter and discover content.
Methods
Inflo employs a two-stage process. First, it uses a neural network, specifically a fine-tuned ULMFIT model, to classify news articles into one of 12 discrete categories. This classification is crucial for the second stage: keyphrase extraction. The keyphrase extraction module utilizes three methods:
- Statistically Extracted Keyphrases: This method uses a tool called KP-Miner, leveraging document frequencies, term frequencies, and positional occurrences. Crucially, it uses category-specific document frequencies to improve relevance. This method is primarily used for multi-word keyphrases.
- Statistically Extracted Entities: Similar to the above, this method also uses KP-Miner but focuses on named entities (persons, organizations, places). It also employs category-specific document frequencies for entities. Techniques like singularization and "nounification" (converting verbs to nouns using WordNet) are used for de-duplication.
- Graphically Extracted Topics: This approach uses graphical methods to identify key topics within an article, treating topics as nodes and semantic relations as edges. The paper highlights MultipartiteRank as a successful algorithm for this task, though it notes that de-duplication and nounification are still necessary.
The final keyphrases are an aggregation of the results from these three methods after de-duplication.
Datasets
The authors collected a dataset of over 500,000 news articles from 2000 to 2018 using APIs from The New York Times, CNN, and News API. They defined 12 discrete news categories. To improve the models' understanding of recent vocabulary and styles, they doubled the dataset from 2017-2018. The data was split into 80% for training, 10% for validation, and 10% for testing. Articles that lacked explicit categories were manually assigned to one of the 12 defined categories based on their metadata.
Key Insights and Ideas
- Category-Specific Context is Key: The paper strongly emphasizes that using category-specific document frequencies for both phrase and entity extraction significantly improves the relevance of the extracted keyphrases. This acknowledges that language and important terms vary greatly across different news domains.
- Named Entities as Rich Indicators: The authors hypothesize and demonstrate that using named entity-specific UNK tokens (instead of generic UNK tokens) in language models can improve classification accuracy, as entities often signal trends and specific topics within an article.
- Ambiguity is Expected, but Useful: The authors acknowledge that news categorization can be ambiguous (e.g., an article about Apple could be "business" or "technology"). Their goal isn't perfect accuracy but rather to provide labels that facilitate better keyphrase extraction.
Key Results
- Classification Accuracy: A well-tuned CNN achieved 62.4% accuracy, a regular ULMFiT model achieved 73.5%, and a named-entity-based ULMFiT model achieved 77.4% accuracy on the test dataset.
- Keyphrase Extraction: The system aggregates results from statistical (phrase and entity) and graphical methods. While specific quantitative results for keyphrase extraction are not detailed, the paper states that the methods were found to recommend phrases well, with de-duplication and nounification being crucial for refining the output.
Limitations and Future Work
- Ambiguity in Categorization: The authors explicitly state that perfect accuracy in news categorization is difficult due to inherent ambiguity. Their focus is on improving keyphrase extraction rather than achieving flawless classification.
- Potential for Redundancy: While de-duplication and nounification are employed, the paper notes that single-word keywords from the statistical method were often redundant, leading them to focus on multi-word keyphrases from that method.
- Future Work: The paper suggests that the Inflo system lays the groundwork for more sophisticated aggregation platforms. The ability to instantly label articles could enable more personalized feeds and better clustering of related news, allowing users to gain diverse perspectives on events.
Summary updated