Abstract
Hybrid Decision-Making (HDM) systems often struggle with rigid hierarchies that fail to adapt as AI models evolve. This paper introduces Bridget, a framework that dynamically shifts control between human-led "Skeptical Learning" and machine-led "Learning-to-Defer." By utilizing a novel temporal weighting metric called Fading Empirical Accuracy (FEA), Bridget ensures that the most capable agent leads the decision process, allowing for a seamless transition between Human-in-Command and Machine-in-Command modes as model performance fluctuates.
Introduction
In high-stakes domains such as medicine and law, AI is rarely deployed in isolation. Instead, Hybrid Decision-Making (HDM) systems are employed to combine human expertise with machine efficiency. Currently, these systems typically adhere to one of two paradigms: Learning-to-Defer (LtD), where the machine leads and defers to humans only when uncertain; or Skeptical Learning (SL), where the human leads and the machine intervenes only to correct perceived errors. Bridget addresses the gap between these static approaches by providing a flexible framework that evolves alongside the model's capabilities.
Methodology
The core of Bridget is its ability to transition between two distinct operational modes based on real-time performance metrics.
Dynamic Status Switching
Bridget operates in two primary modes:
- Human-in-Command (HiC): In this initial phase, the human is the primary decision-maker. The machine acts as a "skeptical" observer, calculating its own confidence and offering suggestions or explanations only when it strongly disagrees with the human's choice.
- Machine-in-Command (MiC): As the model improves, it can be promoted to the lead role. In MiC mode, the machine handles the bulk of the workload to maximize efficiency. However, it continues to monitor its internal "belief" and calls for human intervention if it encounters ambiguous cases or if its accuracy begins to decline.
Fading Empirical Accuracy (FEA)
The transition between these modes is governed by Fading Empirical Accuracy (FEA). Unlike standard metrics that treat all historical data equally, FEA weights recent decisions more heavily. This allows the system to de-emphasize early training errors and more accurately reflect the model's current state of knowledge. Promotion to MiC occurs only when a minimum record threshold is met, FEA crosses a reliability target, and the human provides explicit approval.
Key Insights and Discussion
- The Bridge Concept: Bridget treats HDM as a dynamic loop rather than a static hierarchy. This "co-evolution" allows the system to adapt to changes in data patterns (concept drift) by reverting control to the human if the model's performance degrades.
- Uncertainty and Overreliance: Since internal epistemic uncertainty is often difficult to calculate, Bridget uses FEA as a practical, model-agnostic proxy. To prevent "machine overreliance"—where humans stop scrutinizing AI outputs—the system includes a random check mechanism in the MiC phase, forcing human verification even when the machine is confident.
- Integrated Explainability: Bridget incorporates Explainable AI (XAI) as a fundamental component. In HiC mode, the machine explains its skepticism; in MiC mode, it explains why it feels unreliable for a specific task, fostering trust and clarity in the decision-making process.
Conclusion and Future Work
Bridget represents a conceptual shift in hybrid AI systems, moving away from rigid roles toward a collaborative, adaptive partnership. While empirical validation on benchmark datasets remains a focus for future work, the architectural design provides a robust foundation for systems that can safely and efficiently scale from human-led training to machine-led production. Future research will explore the system's performance against standalone LtD systems and its applicability to complex data types like time series.
Summary updated