Markov chains and kernels

Markov chains, and their generalizations to Markov kernels and Markov processes, are fundamental objects in probability and statistics. Markov chain Monte Carlo (MCMC) is the workhorse of Bayesian statistics.

Cosma Shalizi’s notebooks on Markov models , hidden Markov models , and inference for such offer a much more comprehensive annotated bibliography.

Literature

General

Several experts have written nice perspective pieces on MCMC:

  • Häggström, 2007: Problem solving is often a matter of cooking up an appropriate Markov chain (doi)
  • Diaconis, 2009: The Markov chain Monte Carlo revolution (doi, pdf)
  • Diaconis, 2013: Some things we’ve learned (about Markov chain Monte Carlo) (doi, arxiv, pdf)

Mixing times

  • Levin & Peres, 2017: Markov Chains and Mixing Times, 2nd ed. (doi, pdf)
  • Vilnis, 2013: Markov chain Monte Carlo, mixing, and the spectral gap (pdf)
  • Montenegro & Tetali, 2006: Mathematical aspects of mixing times in Markov chains (doi, pdf)

Intertwinings

Intertwinings, a kind of Markov kernel, are morphisms between Markov chains.

  • Diaconis & Fill, 1990: Strong stationary times via a new form of duality (doi)
    • First systematic study of intertwinings
    • Fill, 1992: Strong stationary duality for continuous-time Markov chains (doi)
  • Jansen & Kurt, 2012: On the notion(s) of duality for Markov processes (doi, arxiv)
  • Swart, 2013, lecture notes: Duality and intertwining of Markov chains (pdf)
  • Sturm, Swart, Völlering, 2018, lecture notes: The algebraic approach to duality: An introduction (arxiv)

Category theory

For literature on Markov kernels from the categorical perspective, see categorical probability theory.