TL;DR
This paper tackles the challenge of ranking online news topics by proposing a novel algorithm that combines "media focus" (how much a topic is reported) and "user attention" (how much users engage with it). By analyzing the inconsistency between these two factors using an "aging theory" to model topic lifecycles, the authors present a more nuanced and user-centric approach to news topic prioritization.
What Problem Does This Paper Address?
The internet bombards us with an overwhelming number of news topics generated by Topic Detection and Tracking (TDT) systems. While TDT helps organize news, users can't possibly consume everything. The core problem is how to effectively rank these topics to present the most timely and important ones to users. Existing methods often rely on simple metrics like update frequency or topic size, or independently consider media coverage and user interest. However, these approaches fail to capture the dynamic interplay and potential disconnect between what the media emphasizes and what users actually find engaging. This paper aims to bridge this gap by developing an automatic ranking system that accounts for both media attention and user engagement, acknowledging that these two signals don't always align.
Methods
The authors propose an automatic online news topic ranking algorithm built upon several key components:
- Topic Detection and Tracking (TDT) with Aging Theory: They leverage existing TDT techniques to group news stories into topics. Crucially, they incorporate an "aging theory" to model the lifecycle of a topic, treating it like a living entity that grows in popularity and eventually decays. This theory is applied to calculate a topic's "energy," representing its liveliness.
- Burstiness of Terms: To improve TDT, they utilize the concept of "burstiness" in keywords, assuming that new topics often emerge with a sudden surge of specific terms.
- Media Focus Calculation: This is directly derived from the topic's "energy" value calculated using the aging theory. A higher energy value signifies greater media attention.
- User Attention Calculation: This is a novel aspect, leveraging anonymous web page view data collected via browser toolbars. Similar to media focus, user attention also decays over time and is modeled using an aging theory, with "view value" acting as nutrition.
- Inconsistency Measurement: A key contribution is the quantitative measure of inconsistency between media focus and user attention. They use the Pearson correlation coefficient on cumulative media focus and user attention values of "dead" topics (those no longer actively reported) to quantify this gap.
- Topic Ranking: The final ranking is achieved by combining media focus and user attention values using a weighted sum, where the weight
acan be adjusted. This allows for a flexible balance between the two factors.
Datasets
- TrainingSet: 14,602 news stories published from January 1 to March 31, 2007. This set was used for parameter tuning.
- TestingSet: 38,767 news pages published from April 1 to October 31, 2007. This set was used for evaluating the ranking algorithm.
- User Attention Data: Over 10.64 billion page views from 1.51 billion web pages and 78.2 million user sessions, collected from January 1 to December 31, 2007. A subset of 0.6 million user page views was extracted for calculating user attention to generated topics.
The paper focuses on news related to search engine companies, collected from various Chinese news websites.
Key Insights and Ideas
- Media vs. User Attention Disconnect: The most significant insight is the demonstration that media focus and user attention are not always aligned. Topics heavily covered by the media might not capture user interest, and vice-versa. This inconsistency is a crucial factor for effective ranking.
- Aging Theory for Dynamic Topics: Applying an aging theory to model the lifecycle of news topics provides a robust mechanism for tracking their relevance and decay, both from a media and user perspective.
- User Behavior as a Ranking Signal: Incorporating real user engagement data (page views) offers a more grounded and user-centric approach to news prioritization than relying solely on editorial or media-driven metrics.
- Practicality of Toolbar Data: The paper highlights the feasibility of collecting user attention data through widely adopted web browser toolbars, suggesting the practical applicability of their approach.
Key Results
- Quantified Inconsistency: The study quantitatively measured the inconsistency between media focus and user attention, finding a Pearson correlation coefficient generally between 0.4 and 0.6, indicating a significant gap.
- Improved Ranking: Experiments showed that ranking using a combination of media focus and user attention produced more relevant results compared to using media focus alone. For instance, "Sogou Pinyin 3.0 released" was a top-ranked topic when user attention was considered, despite not being in the top 10 based solely on media focus.
- User Feedback: User evaluations indicated that the proposed algorithm's results were more reasonable and reflected the influence of time, media, and users, compared to media-focus-only ranking.
- Parameter Tuning: Preliminary experiments on a training set were used to determine optimal parameters for the aging theory and weighting factors.
Limitations and Future Work
- Limitations:
- The paper acknowledges that the dataset was filtered to focus on search engine related companies, which might limit generalizability to broader news domains.
- While user attention data was collected from a major commercial website, the authors did not consider user behavior on their own system, which could provide a positive feedback loop.
- The evaluation of TDT results was limited due to the lack of complete annotation for the practical web data.
- Future Work:
- Analyze topic trends in continuous time slots for more granular insights.
- Discover and incorporate additional user behaviors that can further enhance topic ranking.
Summary updated