Optimizing two stage bigram language models for IR
Lists
Only you see this
Abstract
Although higher order language models (LMs) have shown benefit of capturing word dependencies for Information retrieval(IR), the tuning of the increased number of free parameters remains a formidable engineering challenge. Consequently,in many real world retrieval systems, applying higher order LMs is an exception rather than the rule. In this study, we address the parameter tuning problem using a framework based on a linear ranking model in which different component models are incorporated as features. Using unigram and bigram LMs with 2 stage smoothing as examples, we show that our method leads to a bigram LM that outperforms significantly its unigram counterpart and the well-tuned BM25 model.
BibTeX
@inproceedings{javanmardi2010optimizing,
author = {Sara Javanmardi and Jianfeng Gao and Kuansan Wang},
title = {{Optimizing two stage bigram language models for IR}},
booktitle = {The Web Conference},
year = {2010},
url = {https://openreview.net/forum?id=S1V7SZ-dWS},
}Loading related content...