Introduction to Markov Chains and Stochastic Systems
A Markov chain is a mathematical model used to describe a sequence of events where the probability of transitioning to the next state depends solely on the current state, not on the path taken to reach it. This property is known as the Markov property or memorylessness. First studied by Andrey Markov in the early 20th century, Markov chains are used in physics, chemistry, economics, genetics, and computer science (for example, the PageRank algorithm used by search engines is based on a Markov chain model).
A discrete-time Markov chain consists of a set of states and a transition matrix containing the probabilities of moving from one state to another. Since the system must always transition to some state, the rows of a transition matrix must sum exactly to 1. Such matrices are called stochastic matrices.
This calculator solves Markov chain problems for 2-state or 3-state systems. By entering the transition probabilities and initial state distribution, it computes the system's state probabilities after n steps and solves the steady-state distribution representing the long-term balance of the system. Matrix rows you enter that don't already sum to 1 are automatically renormalized before any calculation runs.