Demystifying the Markov Model

Demystifying the Markov Model

A Markov model is a type of model which leverages an analytic framework designed to model pseudo-randomly changing systems. To accomplish this, Markov models represent the scenario as a series of mutually exclusive states with transition probabilities determining the transition between any two states over a certain time period. The probability of transitioning from 1 state to another only depends on the current state and does not consider any previous states as the model does not retain any memory of previous states. Markov models are used in many applications including predictive modeling, such as predicting market crashes, health economics (such as tracking changes in cost of chronic disease over time), environmental management (such as monitoring polar ice sheets), and social sciences (such as describing consumer behavior). While Markov models have wide-ranging applications in many fields, when used in health economics they are mostly applied to cost-effectiveness research where the mutually exclusive states represent health states specific to the disease being modeled. 

Why Use Markov Models in Health Economics?

Markov models are an exceptionally useful methodology for modeling long-term disease progression in indications with characteristics such as multiple stages, remission and recurrence, or risk of events occurring over time where the events can happen multiple times. Markov models are a proven methodology for cost-effectiveness analysis and are frequently used in health technology assessments (HTAs) across the world. 

Concepts Used in Health Economics Markov Models

• Health States: As described above, these represent all possible disease states within the condition being studied, which are mutually exclusive and exhaustive. For example, a Markov model for cancer treatment might include health states such as progression-free, post-progression, and dead. Over time, patients will transition between connected states as their condition changes. Not all disease states are necessarily connected to all other disease states, as this depends on the disease being modeled. For example, a basic methodology for a Markov model in cancer might include disease states representing the different stages of the cancer. A patient may be able to transition from stage 1 to stage 2 but may not have any probability of transitioning from stage 2 back to stage 1. 

• Transition Probabilities: These probabilities represent the likelihood that a patient will move from 1 health state to another connected state or remain in their current health state at either the start or end of each Markov cycle. A patient can only transition from 1 state to another once during each cycle. 

• Cost and Health Outcome: For each health state within the model, a distinct cost and health outcome are associated for the duration of that cycle. Additionally, the transition between health states may have a cost associated with it, such as the cost of diagnostic testing when transitioning from a healthy state to a disease state. In Markov-based cost-effectiveness models, health outcomes are most often represented by utility values. Utility values are used to represent the patient’s quality of life associated with being in that disease state. The utility values for each patient in each stage in each cycle can then be summed to determine the total quality-adjusted life years (QALYs) experienced in the entire scenario. 

• Markov Cycle: These cycles divide the study time interval into equal time intervals that represent specific points during treatment for the studied condition. Within each cycle, each health state is assumed to be constant for the duration of the cycle. At the end of each cycle, the hypothetical condition of the patient is evaluated, and the health state will change based on transition probability.  

Interested in Learning More About Markov Models in Health Economics?

Markov models are a powerful tool in healthcare decision-making, providing insights into cost-effectiveness, patient outcomes, and long-term healthcare planning. Whether you’re looking to optimize your market access strategy or understand how these models can impact your decision-making process, our team of experts is here to help. Contact us to explore how we can support your organization with tailored health economic models, value propositions, and market access solutions.