Traditional sequential decision-making frameworks—whether they are multi-armed bandits, reinforcement learning (RL) agents, or network admission controllers—frequently operate under a convenient fiction: the assumption of infinite horizons or fixed interaction budgets. In these idealized environments, if an algorithm makes a suboptimal choice, it receives a low reward but retains the freedom to try again. The system's horizon is treated as an exogenous parameter, completely decoupled from the quality of the system's own decisions.
In reality, human interactions are governed by a finite, volatile resource: patience. When a conversational agent asks too many redundant questions, a recommendation feed surfaces irrelevant content, or a ride-sharing app takes too long to dispatch a driver, users do not wait indefinitely. They grow fatigued, disengage, or permanently abandon the platform.
To bridge this gap, a growing body of research replaces static interaction assumptions with dynamic user patience models. By treating session termination as an endogenous, stochastic function of user frustration and system performance, these models fundamentally reshape how algorithms balance risk, exploration, and long-term utility.
Modeling Patience as a Depleting Resource
To understand why dynamic patience models work, we must first look at how they formalize human attrition. Rather than treating abandonment as a sudden, unpredictable timeout, recent frameworks model patience as a latent state variable that fluctuates based on real-time feedback.
Consider a user exploring a digital platform. In KuaiSim: A Comprehensive Simulator for Recommender Systems, Kesen Zhao et al. track this via a latent patience variable that acts as a dynamic reservoir. High-quality recommendations replenish or sustain this reservoir, while poor recommendations drain it. The moment this variable crosses a critical threshold, the session automatically terminates.
[System Action] ---> [Perceived Quality / Delay] ---> [Patience Reservoir]
|
(Crosses Threshold?)
|
+-----------------+-----------------+
| |
[ Yes ] [ No ]
| |
(Session Terminates) (Interaction Continues)
Mathematically, this attrition can be captured using continuous probability distributions rather than rigid time windows. For instance, Erica Coppolillo et al. employ a Weibull distribution within their recommendation framework in Relevance meets Diversity: A User-Centric Framework for Knowledge Exploration through Recommendations. Here, the baseline probability of a user ending a session increases naturally with each interaction step (capturing natural weariness), but high content relevance acts as a mitigating factor that actively lowers the immediate probability of attrition.
Crucially, these models must account for memory and cumulative frustration. As Rafid Mahmood demonstrates in Routing, Cascades, and User Choice for LLMs, static abandonment models fail because they assume a user's tolerance for failure or latency is fixed before the interaction begins. In practice, a user's willingness to continue decreases non-linearly with each subsequent delay, meaning that a system's history of errors directly accelerates the rate of patience depletion.
The Information-Gathering Risk in Conversational Systems
The tension between system exploration and user patience is most acute in conversational systems, such as conversational recommenders or search engines. To understand a user's precise intent, the system needs to ask clarifying questions. However, every question carries an implicit cognitive load and an opportunity cost.
If an algorithm prioritizes information gathering without considering user fatigue, it risks driving the user away entirely before a solution is reached. Jinhang Zuo et al. highlight this vulnerability in Hierarchical Conversational Preference Elicitation with Bandit Feedback, noting that standard conversational frameworks often fail by ignoring the cumulative cost of interaction. Similarly, Hang Li et al. argue in Conversational Contextual Bandit: Algorithm and Application that mathematical models must explicitly account for the psychological trade-offs between the information gain of a question and the risk of user disengagement.
To solve this, researchers frame conversational interaction as a constrained risk-control problem. In Simulating and Modeling the Risk of Conversational Search, Qingyao Ai et al. model user tolerance using two distinct parameters: a total question limit (general patience) and a failure threshold (tolerance for irrelevant queries). By integrating these parameters into a reinforcement learning framework, the agent learns to predict when a user is approaching their breaking point.
This risk-aware modeling changes the optimal policy behavior:
- High Patience State: The system asks broad, high-level clarifying questions to aggressively narrow down the search space.
- Low Patience State: The system abandons information gathering and immediately surfaces its best available, safest recommendation to prevent a total session bounce.
This dynamic transition is also vital in preference elicitation. As Shuhua Yang et al. observe in Conversational Dueling Bandits in Generalized Linear Models, modeling user patience allows dueling bandit algorithms to dynamically calculate the exact inflection point where the system should stop asking the user to compare items and instead deliver the final recommendations.
From Micro-Interactions to Macro-System Stability
While patience modeling optimizes individual user sessions, its mathematical implications are equally profound at the macro-system level, particularly in network routing, queueing theory, and resource allocation.
In traditional queueing systems, items or requests remain in a queue until served. However, human-centric systems are "leaky." In K competing queues with customer abandonment: optimality of a generalised $$c \mu $$-rule by the Smoothed Rate Truncation method, Sandjai Bhulai analyzes systems where customers exhibit stochastic abandonment, proving that optimal service strategies change completely when queues can spontaneously lose traffic due to delays.
This leakage can actually act as an unintended stabilizing mechanism during network strain. In their foundational work Performance modeling of elastic traffic in overload, Thomas Bonald et al. demonstrate that when a network link is overloaded, user impatience—characterized by abandoning connections that take too long—prevents an infinite buildup of flows. However, this stabilization comes at a steep price: it severely degrades network goodput and efficiency because resources are wasted on transfers that are interrupted midway.
To prevent this degradation, modern admission control systems explicitly bake stochastic impatience functions into their logic. For example, in PSACCF: Prioritized Online Slice Admission Control Considering Fairness in 5G/B5G Networks, Miao Dai et al. incorporate models of balking (users refusing to join a long queue) and reneging (users leaving the queue after waiting). By accounting for these behaviors, the admission control system avoids aggressive preemption and achieves higher resource stability across varying workloads.
+---------------------------------------+
| Incoming Request |
+---------------------------------------+
|
(Check Queue Length)
|
+-------------------+-------------------+
| |
[ Long Queue ] [ Short Queue ]
| |
(Risk of Balking) (Join Queue)
| |
[ Reject/Reroute Early ] |
(Monitor Wait Time)
|
+-----------+-----------+
| |
[ Exceeds Patience ] [ Served in Time ]
| |
(Risk of Reneging) (Successful Flow)
|
[ Abort Connection ]
This predictive abandonment logic is also what makes real-time matching markets viable. In ridesourcing platforms, assuming that users will wait indefinitely or until a fixed deadline leads to high cancellation rates. To solve this, Hao Wang 0214 et al. introduce a continuous "quitting function" into request assignment algorithms in Real-Time Driver-Request Assignment in Ridesourcing. By knowing the continuous probability of a user canceling at any given second, the dispatch algorithm can strategically decide whether to match a rider immediately with a suboptimal driver or risk waiting a few seconds for a significantly closer driver to become available.
Why It Works: Shifting to Survival-Aware Optimization
The core reason dynamic patience modeling works across all these domains boils down to a fundamental mathematical shift: it transforms an unconstrained optimization problem into a survival-aware optimization problem.
In a standard multi-armed bandit or MDP, the expected total reward is a simple sum of discounted future rewards:
$$V^{\pi}(s) = \mathbb{E} \left[ \sum_{t=0}^{\infty} \gamma^t R(s_t, a_t) \right]$$
In this formulation, a dangerous or highly frustrating action $a_t$ merely yields a low immediate reward $R_t$.
When we introduce a dynamic patience model, the interaction horizon itself becomes a random variable $T$ that depends directly on the history of choices. If a single outcome falls below a user's satisfaction threshold, the user permanently exits ($T = t$). As Ping-Chun Hsieh et al. formalize in Heteroscedastic Bandits with Reneging, this risk of interaction termination makes the availability of all future rewards strictly contingent on the safety of every individual decision. The objective function effectively mutates into a lifetime maximization problem under survival constraints:
$$V^{\pi}(s) = \mathbb{E} \left[ \sum_{t=0}^{T(\pi)} \gamma^t R(s_t, a_t) \right]$$
By making the horizon endogenous, the algorithm is naturally forced to internalize the long-term cost of user frustration. It learns that exploration is not free; exploration costs patience. Whether it is an online recommendation engine balancing diversity against immediate relevance, or a 5G network provider optimizing slice allocation, accounting for the stochastic nature of human impatience ensures that systems maximize long-term engagement and revenue without inadvertently driving their users away.
Key Foundational Work
Performance modeling of elastic traffic in overload (2001) — contributed 4 concept elements to this direction.
Statistical bandwidth sharing: a study of congestion at flow level (2001) — contributed 3 concept elements to this direction.
Scalable Live Streaming Service Based on Interoverlay Optimization (2007) — contributed 1 concept elements to this direction.