This observation led to the discovery of Markov chains, which may be regarded as the precursors of stochastic processes. By studying the sequence of Cyrillic characters in Alexander Pushkin’s verse novel Eugene Onegin, the Russian mathematician Andrey Andreyevich Markov (1856–1922) noticed that each letter depended, according to a certain probability distribution, on the letter immediately preceding it. Before they became standard mathematical objects, these sequences of letters came to be called Markov chains. In mathematics, Markov chains describe an evolving system that can occupy a finite or countable number of states E1, E 2… The system’s state is observed at each (assumed discrete) time step. Markov postulates that the transition from state E *i to state E j occurs with probability pi, j . He describes a memoryless process: the transition probabilities depend only on the system’s state before the transition (E i ) and its state afterward (E j ). More generally, a system that evolves randomly over time is called a Markov process* if the conditional probability distribution of its future states, given its past and present states, depends only on its present state. For such processes, the best prediction of their future based on their past and present is identical to the best prediction based solely on knowledge of their current state.
