Introduction
In online auctions, trust is the primary currency. However, legacy reputation systems, such as eBay’s, are often "massively biased" due to reputation inflation. Users frequently leave positive star ratings even when dissatisfied to avoid retaliatory feedback, burying their true grievances within free-text comments. This paper introduces AuctionRules, an NLP-based algorithm designed to extract granular, feature-based trust metrics from these comments and visualize them as a social network to facilitate better-informed decision-making.
The AuctionRules Methodology
The authors developed a classification pipeline tailored for the specific linguistic constraints of online auctions. Unlike general sentiment analysis, the algorithm focuses on eight core "feature sets" salient to auctions: Item, Person, Cost, Shipping, Response, Packaging, Payment, and Transaction.
As illustrated in the pipeline, the system utilizes Porter stemming to normalize words and a specialized "stop-word" list to filter noise. It identifies intensifiers (e.g., "very"), articles, and negators (e.g., "not") to determine sentiment polarity. A critical component is the Query Expander, which uses an online thesaurus to find synonyms or antonyms when specific keywords are missing, significantly increasing the algorithm's coverage from 62% to over 90%.
Data and Evaluation
The researchers crawled eBay to collect 10,000 user comments, specifically from the "Egyptian antiques" domain. To establish a "gold standard," 1,000 comments were manually rated by humans on a 5-point Likert scale.
AuctionRules demonstrated superior performance, achieving 97.5% accuracy and outperforming seven popular classification algorithms (including Naive Bayes and C4.5) by up to 21%. Notably, the system achieved a 0% false negative rate, ensuring that good sellers are not unfairly penalized while reliably identifying problematic behavior.
Visualizing Multi-dimensional Trust
A key insight of this research is that trust is not binary. By breaking trust down into specific features, users can assess risks more accurately. The system calculates "interpersonal trust" between users and "contextual trust" based on historical performance.
The visualization component integrates these metrics directly into the browsing experience. Users can view a breakdown of a seller's performance across different categories—such as packaging quality versus communication speed—alongside a social network graph where edge thickness represents trust value and length represents the volume of supporting evidence. This allows for a personalized trust model where a buyer might prioritize item quality over shipping speed.
Conclusion and Future Work
AuctionRules effectively uncovers the "silent" negatives hidden within inflated positive ratings. In tests, it identified that 16% of comments rated as 93-100% positive by eBay actually contained negative indicators. While the algorithm currently struggles with extreme noise and misspellings, it provides a significantly more useful scale for buyers. Future work will focus on refining trust propagation mathematics to better calculate trust across extended social networks in a commercial context.
Summary updated