Entropy as an Indicator of Context Boundaries: An Experiment Using a Web Search Engine
Lists
Only you see this
Abstract
Previous works have suggested that the uncertainty of tokens coming after a sequence helps determine whether a given position is at a context boundary. This feature of language has been applied to unsupervised text segmentation and term extraction. In this paper, we fundamentally verify this feature. An experiment was performed using a web search engine, in order to clarify the extent to which this assumption holds. The verification was applied to Chinese and Japanese.
BibTeX
@inproceedings{tanakaishii2005entropy,
author = {Kumiko Tanaka-Ishii},
title = {{Entropy as an Indicator of Context Boundaries: An Experiment Using a Web Search Engine}},
booktitle = {IJCNLP},
year = {2005},
doi = {10.1007/11562214\_9},
}Loading related content...