Online High-Quality Topic Detection for Bulletin Board Systems

Jungang Xu, Hui Li, Yan Zhao, Ben HeIEICE Trans. Information and Systems 2014 · OpenReview

Abstract

Bulletin Board Systems (BBS) remain a cornerstone for niche communities and vertical discussions despite the rise of modern social media. However, the sheer volume of data and the prevalence of "noise"—low-quality or off-topic replies—make it difficult for users to navigate these platforms. This paper proposes a novel Online Topic Detection (OTD) framework tailored for BBS. By utilizing a Markov chain-based model to select representative posts and applying Aging Theory to optimize clustering, the proposed method achieves high detection accuracy and nearly linear time efficiency, enabling real-time processing of massive discussion threads.

The Challenge of BBS Data

Unlike edited news articles or character-limited microblogs, BBS threads are characterized by organizational flexibility and high noise levels. A single thread can span hundreds of posts, many of which are trivial "me too" comments or off-topic banter.

Structure of a typical BBS thread showing parent-child reply relationships.

Traditional OTD algorithms face two primary hurdles in this environment:

  1. Noise Dilution: Low-quality replies obscure the core topic, reducing clustering precision.
  2. Computational Complexity: Standard clustering methods often scale quadratically ($O(n^2)$). As the number of threads grows, the system becomes too slow for real-time use because it compares every new post against the entire history of topics.

Proposed Methodology

1. Representative Post Selection via Markov Chains

To filter out noise, the authors treat a BBS thread as a directed graph where nodes are posts and edges represent reply relationships. They employ a Markov chain model—similar to PageRank—to simulate a "random reader" navigating the thread. Posts that receive more replies or initiate deeper sub-threads accumulate higher steady-state probabilities. By selecting only the top $m$ highest-ranking posts to represent a thread (experimentally set to 7), the system effectively summarizes the discussion while discarding irrelevant content.

2. Efficient Clustering with Aging Theory

To address scalability, the researchers introduced Aging Theory into the clustering process. Topics are modeled as biological entities with a life cycle:

  • Energy Gain: New relevant posts act as "nutrition," increasing a topic's energy.
  • Decay: Over time, a topic's energy naturally dissipates if it is not discussed.
  • Active Candidate Set: The algorithm only compares new posts against "active" topics (those with positive energy). "Dead" topics are archived, preventing the search space from expanding indefinitely.

Experimental Results

The framework was tested using data from Newsmth, a prominent Chinese BBS, comprising over 327,000 posts.

Accuracy and Parameter Optimization

The Markov chain-based selection significantly outperformed baseline methods like keyword extraction or simple chronological selection. The researchers found that the detection performance is sensitive to the number of representative posts selected ($r$).

Normalized detection cost (C_DET) as a function of parameter r, showing the optimization of topic detection accuracy.

As shown in the figure above, the Normalized Detection Cost reaches its minimum when the number of representative posts is optimized (around 7), confirming that a small subset of high-quality posts is sufficient to characterize a thread.

Efficiency

While standard Single Pass Clustering (SPC) exhibited a steep quadratic increase in processing time as data volume grew, the proposed method maintained nearly linear time complexity. This efficiency is directly attributable to the Aging Theory-based candidate set generation, which keeps the number of active topics manageable.

Conclusion

This research demonstrates that the topological structure of a BBS thread is a powerful indicator of post quality. By combining structural analysis with biological aging models, the authors created a system that is both accurate and fast enough for real-world deployment. Future work will look into integrating semantic content analysis with these structural features to further refine post selection in even noisier environments like microblog comment sections.

Summary updated

Abstract

Even with the recent development of new types of social networking services such as microblogs, Bulletin Board Systems (BBS) remains popular for local communities and vertical discussions. These BBS sites have high volume of traffic everyday with user discussions on a variety of topics. Therefore it is difficult for BBS visitors to find the posts that they are interested in from the large amount of discussion threads. We attempt to explore several main characteristics of BBS, including organizational flexibility of BBS texts, high data volume and aging characteristic of BBS topics. Based on these characteristics, we propose a novel method of Online Topic Detection (OTD) on BBS, which mainly includes a representative post selection procedure based on Markov chain model and an efficient topic clustering algorithm with candidate topic set generation based on Aging Theory. Experimental results show that our method improves the performance of OTD in BBS environment in both detection accuracy and time efficiency. In addition, analysis on the aging characteristic of discussion topics shows that the generation and aging of topics on BBS is very fast, so it is wise to introduce candidate topic set generation strategy based on Aging Theory into the topic clustering algorithm.
BibTeX
@article{xu2014online,
  author = {Jungang Xu and Hui Li and Yan Zhao and Ben He},
  title = {{Online High-Quality Topic Detection for Bulletin Board Systems}},
  journal = {IEICE Trans. Information and Systems},
  year = {2014},
  url = {https://openreview.net/forum?id=vQ1uhC08Tz},
}
Figure