Incremental web page template detection

Yu Wang, Binxing Fang, Xueqi Cheng, Li Guo, Hongbo XuWWW 2008 · OpenReview

Abstract

Most template detection methods process web pages in batches that a newly crawled page can not be processed until enough pages have been collected. This results in large storage consumption and a huge delay of data refreshing. In this paper, we present an incremental framework to detect templates in which a page is processed as soon as it has been crawled. In this framework, we don't need to cache any web page. Experiments show that our framework consumes less than 7% storage than traditional methods. And also the speed of data refreshing is accelerated because of the incremental manner.
BibTeX
@inproceedings{wang2008incremental,
  author = {Yu Wang and Binxing Fang and Xueqi Cheng and Li Guo and Hongbo Xu},
  title = {{Incremental web page template detection}},
  booktitle = {The Web Conference},
  year = {2008},
  url = {https://openreview.net/forum?id=Hk4MZfbdbS},
}

Loading related content...

Figure