Semi-Markov processes
Semi-Markov processes, which are equivalent to Markov renewal processes , are stochastic processes generalizing continuous-time Markov chains (CTMCs). They are Markov (memoryless) with respect to states, but not with respect to time.
Theory
- Feller, 1964: On semi-Markov processes (doi)
- Generalizes Kolmogorov’s backward equation (aka the master equation) from CTMCs to semi-Markov processes
- Mentioned in (Feller, Volume II, Chapter XIV, Problem 14)
- Cinlar, 1969: Markov renewal theory (doi)
- Cinlar, 1975: Markov renewal theory: a survey (doi)
- Cinlar, 1975: Introduction to Stochastic Processes, Chapter 10: Markov renewal theory
Statistical applications
- Titman & Sharples, 2010: Semi-Markov models with phase-type sojourn distributions (doi)
- Asanjarani et al, 2021: Estimation of semi-Markov multi-state models: a comparison of the sojourn times and transition intensities approaches (doi, arxiv)
Software
- Incerti, 2020, talk: Multistate semi-Markov modeling in R (slides)
Matrix-exponential distributions
Phase-type distributions , assembled from mixtures and convolutions of exponential distributions, and their generalization to matrix-exponential distributions , are analytically tractable classes of probability distributions for waiting times. They are building blocks from which semi-Markov processes can be constructed.
Books
- Neuts, 1981: Matrix-geometric solutions in stochastic models: an algorithmic
approach
- Early reference by a pioneer of phase-type distributions
- Latouche & Ramaswami, 1999: An introduction to matrix analytic methods in stochastic modeling (doi)
- Bladt & Nielsen, 2017: Matrix-exponential distributions in applied
probability (doi)
- A good book, encyclopedic but still readable
- Extensive bibliographic notes at end of book
Theory
- O’Cinneide, 1990: Characterization of phase-type distributions (doi)
- Main theorem, according to the abstract: “A distribution with rational Laplace transform is of phase type if and only if it is either the point mass at zero, or it has a continuous positive density on the positive reals and its Laplace transform has a unique pole of maximal real part (which is therefore real).”
- Main theorem is Theorem 4.7.45 in (Bladt & Nielsen, 2017)
Statistical applications