Abstract
In real-world machine learning deployments, classifiers often require periodic retraining using feedback from end-users to combat data drift. However, this feedback loop is vulnerable to "noisy" or "confused" users whose labels may be incorrect. This paper, by researchers at JP Morgan AI Research, proposes PRUNE (Periodically Retraining Using Native Entities), a framework that maintains individual "trust scores" for users. By filtering out feedback from unreliable agents, PRUNE prevents model degradation and significantly outperforms naive retraining strategies, matching the performance of an ideal system with perfect knowledge of user reliability.
Introduction
Document classifiers in production environments, such as financial institutions, inevitably face data drift or novel inputs. A common solution is the "feedback-retrain loop," where analysts provide corrections for misclassified items. Traditional crowdsourcing relies on redundant labeling (e.g., majority voting), but business workflows typically assign a document to a single person. This creates a risk: if the user is unreliable, their feedback can actively degrade the model. The authors categorize unreliable users into two types: noisy agents, who provide random labels, and confused agents, who consistently misinterpret specific categories.
The PRUNE Framework
The authors introduce the PRUNE wrapper to manage the retraining process. The core innovation is the Trust Score ($T$), which quantifies an agent's reliability based on their agreement with the current classifier:
$$T(i) = \frac{\text{Agreement count between classifier and agent } A_i}{\text{Total labels provided by agent } A_i}$$
Two filtering strategies are evaluated:
- MEANPRUNE: Accepts feedback only from agents with trust scores above the group average.
- THRESHOLDPRUNE: Accepts feedback only if the trust score exceeds a fixed threshold (e.g., 0.3).
To avoid the "cold start" problem—where an initially weak classifier might unfairly distrust a competent user—the system employs a "burn-in" period. During this phase, all feedback is accepted to allow the classifier to stabilize before it begins evaluating user reliability.
Experimental Results
The framework was tested on the 20 Newsgroups (20N) and Reuters-21578 (reu) datasets. The authors compared PRUNE against TRUSTINGNB (accepts all feedback) and DISCERNINGNB (an oracle with perfect knowledge of agent reliability).
Resilience to Confused Agents
Confused agents, who provide "structured misinformation," were found to be significantly more damaging than noisy agents. Naive retraining (TRUSTINGNB) often led to immediate performance drops when faced with confused feedback. In contrast, PRUNE algorithms successfully identified and muted these agents.
On the Reuters dataset, the filtering was so effective that it reached the same performance level as the perfect-knowledge benchmark by the later stages of the trial.
Resilience to Noisy Agents
While noisy feedback acts as statistical jitter, it can still hinder learning. In scenarios with a high ratio of noisy agents (e.g., 4 out of 5), PRUNE consistently outperformed the "trust everyone" baseline.
Discussion and Limitations
A key insight is the "Misplaced Trust" trap: if a classifier makes an error and a reliable agent corrects it, the disagreement lowers the agent's trust score. The "burn-in" period is critical to mitigate this. However, some trials still exhibited bimodal results, where the model either successfully identified reliable agents or failed completely.
Future work could extend this model to handle dynamic user reliability (e.g., learning or fatigue) and larger agent pools. Additionally, trust scores could serve as a diagnostic tool to identify users in need of further training.
Conclusion
PRUNE offers a robust solution for maintaining classifiers in human-in-the-loop systems. By treating users as persistent agents with measurable reliability, the system can automatically clean its training data stream, ensuring that "some people aren't worth listening to" doesn't lead to model failure.
Summary updated