Abstract
This study investigates the traditional accuracy-interpretability trade-off in machine learning by applying it to the task of predicting product ratings from text reviews. By introducing a "Composite Interpretability" (CI) score, the authors quantify the transparency of multi-module NLP pipelines. The findings suggest that while a general trend exists where black-box models outperform glass-box models, the relationship is non-monotonic. Strategic combinations of black-box features and glass-box classifiers can often achieve superior performance while maintaining a degree of interpretability.
Introduction
As AI systems permeate high-stakes decision-making, the tension between high-performing "black-box" models and inherently interpretable "glass-box" models has intensified. Regulatory frameworks like the GDPR emphasize a "right to explanation," yet the trade-off between accuracy and transparency is often assumed to be a zero-sum game. Most existing research focuses on tabular data; this paper shifts the focus to a complex NLP task: inferring numerical product ratings from textual reviews. Real-world systems often utilize composite pipelines, necessitating a fair method to rank the interpretability of hybrid architectures.
Methods
The researchers evaluated 26 model configurations across four balanced Amazon product review datasets: Cell Phones and Accessories (CPA), Office Products (OP), Automotive (AM), and Video Games (VG). Each dataset contained 10,000 reviews (2,000 per star rating).
Composite Interpretability (CI) Score
To quantify transparency, the authors developed the CI score. This metric combines expert rankings (based on simplicity and explainability) with quantitative data (parameter counts). For composite models, the score is the sum of the interpretability scores of its individual components.
Model Architectures
The study compared various embeddings (CountVectorizer, TF-IDF, Word2Vec) and classifiers (Logistic Regression, Naive Bayes, SVM, Neural Networks). Crucially, "composite models" were created by extracting sentiment scores from a fine-tuned BERT model and using them as features for traditional classifiers, such as Logistic Regression (LR-BS).
Key Insights
A primary finding is that the accuracy-interpretability trade-off is not a linear progression. Increasing model complexity does not guarantee higher accuracy. For instance, the synergy between components matters more than individual complexity; pairing Naive Bayes with Word2Vec resulted in poor performance due to conflicting algorithmic assumptions (feature independence vs. dense contextual vectors).
Furthermore, the study highlights the importance of domain-specific sentiment analysis. While rule-based tools like VADER are highly interpretable, they correlate poorly with product ratings compared to fine-tuned models like BERT.
Results
Accuracies for the 5-class rating task typically ranged between 40% and 60%. The highly interpretable VADER baseline performed the worst (approx. 25-32% accuracy). Pure BERT achieved 53.5% to 55.7%.
The standout performer was the composite LR-BS model (TF-IDF + BERT Sentiment + Logistic Regression), which reached 58.97% accuracy on the Office Products dataset. This hybrid approach outperformed the pure black-box BERT model while keeping the final decision layer transparent.
Conclusion
The research demonstrates that the accuracy-interpretability trade-off is nuanced. Composite models offer a promising path forward, allowing practitioners to leverage the power of deep learning for feature extraction while utilizing interpretable layers for final classification. Future work aims to refine the CI score by replacing human expert surveys with objective analytical factors like feature importance and decision boundary complexity.
Summary updated