Clinical trials are inherently constrained by time. A typical trial might track a new drug or intervention for two or three years, yet chronic conditions like glaucoma, epilepsy, or cancer unfold over decades. To evaluate whether a medical intervention is truly effective and financially viable over a patient's lifetime, researchers must bridge the gap between short-term empirical data and long-term clinical reality.
Multi-state Markov modeling provides a mathematical framework to solve this problem. By discretizing a continuous, complex disease process into a set of distinct health states, these models simulate how a cohort or individual transitions through different stages of health, sickness, and mortality over an extended horizon.
Mapping Disease as a Finite State Machine
The core mechanism of a Markov state-transition model relies on defining a set of mutually exclusive health states. A patient can occupy exactly one state during any given time cycle (e.g., a month or a year).
In oncology evaluations, for example, researchers frequently employ a three-state model consisting of progression-free survival, progressive disease, and death. This structure is evident in the work on esophageal carcinoma by Jia Hu et al. in Cost-effectiveness analysis of pembrolizumab vs. chemotherapy as second-line treatment for advanced esophageal carcinoma in the United States and the study on pleural mesothelioma by Zhuo‐miao Ye et al. in Cost-effectiveness of nivolumab plus ipilimumab as first-line treatment for American patients with unresectable malignant pleural mesothelioma.
[Progression-Free Survival] ----> [Progressive Disease] ----> [Death]
| ^
+---------------------------------------------------+
The dynamics of the disease are governed by transition probabilities—the likelihood of moving from one state to another during a single cycle. As demonstrated by Yiping Xie et al. in Cost-effectiveness analysis of advanced radiotherapy techniques for post-mastectomy breast cancer patients, this structure allows researchers to integrate short-term clinical trial endpoints with long-term projections of morbidity and mortality.
Crucially, these models can capture bidirectional pathways. Unlike simple survival curves that only track a downward trajectory toward death, multi-state Markov models can simulate recovery or fluctuating chronic conditions. For instance, Chengbei Hou et al. in Disability Transitions and Health Expectancies among Elderly People Aged 65 Years and Over in China: A Nationwide Longitudinal Study utilized a multistate model to capture bidirectional movement between independence, mild disability, and severe disability, allowing the authors to quantify the probability of functional recovery among elderly populations.
Quantifying Lifetime Value and Cost-Effectiveness
The primary reason to model these transitions is to calculate cumulative clinical outcomes and economic costs over a lifetime horizon. Each health state in the model is assigned a "utility" weight (representing the patient's quality of life in that state) and a financial cost per cycle.
When a cohort is simulated over many cycles, the model tracks the total time the population spends in each state. Summing these values yields the total Quality-Adjusted Life Years (QALYs) and total healthcare expenditures. This mechanism is central to health technology assessments and cost-utility analyses.
For example, Xuan Xiao et al. in Health care cost and benefits of artificial intelligence-assisted population-based glaucoma screening for the elderly in remote areas of China: a cost-offset analysis used a state-transition model to track transitions between healthy states, various glaucoma stages, and blindness to determine if population-based screening offsets long-term costs. Similarly, Donger Zhang et al. in Value of Perampanel as Adjunctive Treatment for Partial-Onset Seizures in Epilepsy: Cost-Effectiveness and Budget Impact Analysis calculated cumulative costs and health outcomes over a lifetime horizon to evaluate anti-epileptic treatments, demonstrating how tracking cohorts until a specific mortality threshold is reached reveals the true economic impact of a therapy.
Hybrid Architectures and Microsimulations
Standard Markov models assume the cohort is homogeneous and that transition probabilities depend only on the current state (the classic Markov property). To address more complex clinical scenarios, researchers expand this toolkit using two primary variations:
Hybrid Decision Tree-Markov Models
Acute clinical events—such as a surgical procedure or an initial screening outcome—involve immediate, branching decisions that do not repeat recursively. To capture this, researchers append a short-term decision tree to a long-term Markov model.
As Ivandito Kuntjoro et al. demonstrated in Cost-Effectiveness of Transcatheter Aortic Valve Implantation in Intermediate and Low Risk Severe Aortic Stenosis Patients in Singapore, a short-term decision tree models immediate procedural risks, while a subsequent Markov model tracks chronic health states and long-term mortality over multiple years. Yehuan Yang et al. in Decision tree-Markov model of perinatal depression screening: a cost–utility analysis similarly combined these structures to capture both immediate clinical branching and the recursive nature of chronic conditions.
Markov Microsimulations
Instead of moving an entire cohort simultaneously based on average probabilities, microsimulations track individual patient trajectories one by one. This allows the model to retain a patient's unique history and characteristics (such as age, prior complications, or treatment adherence), which can dynamically influence their transition probabilities over time.
Stewart B. Harris et al. in Cost-Effectiveness of FreeStyle Libre for Glucose Self-Management Among People with Diabetes Mellitus: A Canadian Private Payer Perspective and Wendy Babidge et al. in THC volume 36 issue 4 Cover and Front matter highlight how person-level microsimulations extrapolate health outcomes and economic impacts by capturing individual variability over extended time horizons, offering a granular view that cohort-level models cannot achieve.