TL;DR
This thesis extends the Bradley-Terry-Luce (BTL) model to handle ordinal responses and incorporate subject-specific covariates and order effects. A novel component-wise boosting algorithm is proposed to address the over-parameterization issue arising from subject-specific covariates, enabling efficient variable selection.
What Problem Does This Paper Address?
Pairwise comparison data is common in many fields, from marketing to sports, where the goal is to rank a set of objects based on preferences. Traditional BTL models are limited to binary outcomes. This paper tackles the challenge of modeling ordinal responses (e.g., "strongly prefer," "mildly prefer," "no preference") in pairwise comparisons. Furthermore, it addresses the complexity introduced by subject-specific covariates (e.g., demographics of the judge) and order effects (e.g., home advantage in sports), which can lead to an over-parameterized model where too many parameters need to be estimated, making it difficult to identify the most influential factors.
Methods
The core of this work lies in extending the BTL model and developing a new algorithm.
- Ordinal Responses: The paper introduces extensions to the BTL model to accommodate ordinal responses using cumulative logit and adjacent categories logit models.
- Subject-Specific Covariates and Order Effects: It shows how to incorporate these factors into the models. A key challenge is that including subject-specific covariates as interactions can lead to an explosion of parameters.
- Component-wise Boosting Algorithm (BTLboost): To combat over-parameterization, a novel component-wise boosting algorithm is proposed. This algorithm iteratively selects the most important subject-object interactions (candidate covariates) and updates their parameters, effectively performing variable selection. This is framed as a "likelihood-based boosting" approach.
- R Package
ordBTL: The authors have developed an R package,ordBTL, to implement these models and the boosting algorithm.
Datasets
The paper uses three real-world datasets for illustration:
- Typewriter Ribbon Data: This dataset involves 5 typewriter ribbon brands compared by 30 secretaries, with an ordinal response of 7 categories.
- CEMS Data: This dataset comes from a survey of 303 students comparing 6 European universities, with ordinal responses and 8 subject-specific covariates.
- German Football League 2005/2006 Data: This dataset is used to model match outcomes (win, tie, lose) and investigate home advantages for 18 teams.
Key Insights and Ideas
- Ordinality Matters: Moving beyond binary preferences to ordinal scales provides richer information and allows for more nuanced rankings.
- Subject-Specific Covariates are Powerful but Tricky: While these covariates can reveal important insights into why certain subjects prefer specific objects, their inclusion can easily lead to models that are too complex to estimate reliably.
- Boosting as a Solution for Over-parameterization: The component-wise boosting algorithm offers an elegant way to handle the high-dimensional nature of models with subject-specific covariates by intelligently selecting the most relevant interactions.
- Home Advantage is Real (and Measurable): The application to football data demonstrates that home advantage is a statistically significant factor in match outcomes.
Key Results
- Simulation Study: The BTLboost algorithm showed promising performance in simulation studies, demonstrating its ability to identify relevant subject-object interactions while controlling for false alarms, especially when using BIC as a stopping criterion. The choice between AIC and BIC depends on whether identifying "small effects" is crucial.
- Typewriter Ribbon Data: The analysis revealed a clear ranking of brands, with brand 3 being the most preferred and brand 4 the least preferred.
- CEMS Data: The analysis confirmed initial visual findings, identifying London as the most preferred university. The BTLboost algorithm successfully identified key subject-object interactions and interactions with the "no preference" term, providing insights into how student characteristics influence university preferences.
- German Football League Data: A likelihood ratio test confirmed that home advantage is a significant factor in match outcomes. Both models with and without home advantage provided similar rankings, but the model including home advantage was statistically superior.
Limitations and Future Work
- Linear Effects for Covariates: The current implementation of the BTLboost algorithm uses linear effects for covariates. The authors suggest that extending it to include non-linear effects would be a valuable area for future research.
- Object-Specific Covariates: The thesis did not directly consider object-specific covariates, though it notes that they could be incorporated by replacing object parameters with linear combinations of these covariates.
- Interpretation of Threshold Covariates: While threshold covariates were implicitly used in some models, their interpretation becomes more complex with more than three response categories.
- Team-Specific Home Advantages: The paper notes that the inclusion of team-specific home advantages can be controversial due to the increased number of parameters. The proposed boosting algorithm can help address this by selecting only the most important team-specific advantages.
Summary updated