Introduction
In the realm of sequential learning, machine learning models are expected to adapt continuously as new data arrives. However, a significant yet often ignored side effect of this adaptation is the "negative flip": a phenomenon where a model suddenly misclassifies a data point it had previously identified correctly. While standard optimization techniques treat all errors equally, human users do not. This paper introduces a novel framework that aligns machine learning updates with human cognitive biases, specifically the endowment effect.
The Problem: Negative Flips and Human Trust
From a purely mathematical standpoint, a negative flip might be seen as a necessary trade-off for improving overall model accuracy. However, in human-centric systems, these flips are detrimental to user trust. Drawing from behavioral economics, the authors highlight the endowment effect—the psychological tendency for individuals to value an object they already possess more highly than a potential gain of equal value.
In the context of ML, a "correct classification" is an endowment. If an email filter correctly flags a spam message today but fails on a similar one tomorrow, the user perceives a disproportionate loss of utility. Traditional algorithms like Online Gradient Descent (OGD) ignore this "divestiture loss," leading to erratic behavior that frustrates users even if the final accuracy is high.
Methods: Endowment-induced Online Gradient Descent (E-OGD)
To address this, the authors propose Endowment-induced Online Gradient Descent (E-OGD). This algorithm internalizes human bias into the optimization process through three key components:
- Divestiture Loss: A new cost function specifically designed to penalize the model when it loses a previously held "correct" state on a datum.
- Asymmetric Updates: Unlike standard OGD, which uses uniform step sizes, E-OGD categorizes examples dynamically. When the model predicts a label correctly, it updates the weight vector more aggressively (scaled by a trade-off parameter $\gamma$). This "locks in" correct classifications, preventing the model from drifting away from these successful states in subsequent rounds.
- Importance-aware Stability: To prevent overshooting when $\gamma$ is high, the authors integrate an importance-aware update framework. This ensures the model remains stable and converges reliably even with large weightings on correct examples.
Crucially, E-OGD does not require storing the entire history of past data, making it suitable for large-scale, memory-constrained applications.
Experimental Evaluation
The researchers evaluated E-OGD using both subjective human studies and large-scale benchmarks. The human study involved a scene recognition task where 100 participants rated the "learnability" of systems. Results showed that users gave significantly lower ratings to systems exhibiting negative flips ($p=0.0093$), confirming that consistency is as vital as accuracy for human satisfaction.
For technical validation, the authors tested the algorithm on five standard datasets: news20, rcv1, algebra, BtA, and webspam-t.
Key Results
The empirical results demonstrate that E-OGD effectively manages the trade-off between overall accuracy and model stability.
As shown in the loss progression plots, E-OGD (solid lines) consistently maintains lower divestiture loss (red lines) compared to standard OGD (dashed lines). This reduction in "forgetting" does not come at the cost of convergence; the authors proved that E-OGD maintains a sublinear regret bound of $O(\sqrt{T})$, matching the theoretical reliability of standard methods.
Quantitative analysis across all five datasets (as seen in the table above) confirms that E-OGD significantly reduces divestiture loss across various trade-off parameters ($r$). For instance, on the news20 dataset, divestiture loss was nearly halved in several configurations compared to the OGD baseline.
Conclusion and Future Work
This research bridges the gap between machine learning optimization and human psychology. By treating correctness as an endowment, E-OGD creates models that are not only accurate but also consistent and trustworthy. Future work may involve extending this framework to non-linear deep learning architectures and exploring automated methods for tuning the bias parameter $\gamma$ based on specific user preferences.
Summary updated