A Discriminative Reranking Approach to Spelling Correction
Lists
Only you see this
Abstract
提出一種基于判別模型的拼寫校正方法.它針對已有拼寫校正系統Aspell的輸出進行重排序,使用判別模型Ranking SVM來改進其性能.將現今較為成熟的拼寫校正技術(包括編輯距離、基于字母的n元語法、發音相似度和噪音信道模型)以特征的形式整合到該模型中來,顯著地提高了基準系統Aspell的初始排序質量,同時性能也超過了一些商用系統(如Microsoft Word 2003)的拼寫校正模塊.此外,還提出了一種在搜索引擎查詢日志鏈中自動抽取拼寫校正訓練對的方法.基于這種方法訓練的模型獲得了基于人工標注數據所得結果相近的性能,它們分別將基準系統的錯誤率降低了32.2%和32.6%. This paper proposes an approach to spelling correction. It reranks the output of an existing spelling corrector, Aspell. A discriminative model (Ranking SVM) is employed to improve upon the initial ranking, using additional features as evidence. These features are derived from state-of-the-art techniques in spelling correction, including edit distance, letter-based n-gram, phonetic similarity and noisy channel model. This paper also presents a method to automatically extract training samples from the query log chain. The system outperforms the baseline Aspell greatly, as well as the previous models and several off-the-shelf systems (e.g. spelling corrector in Microsoft Word 2003). The experimental results based on query chain pairs are comparable to that based on manually-annotated pairs, with 32.2%/32.6% reduction in error rate, respectively.
BibTeX
@article{zhang2008discriminative,
author = {Yang Zhang},
title = {{A Discriminative Reranking Approach to Spelling Correction}},
journal = {Journal of Software},
year = {2008},
doi = {10.3724/sp.j.1001.2008.00557},
}Loading related content...