Continuous-time Markov State Model¶

API Documentation¶

ContinuousTimeMSM([lag_time, n_timescales, ...]) Reversible first order master equation model

Theory¶

Consider an n-state time-homogeneous Markov process, \(X(t)\) At time \(t\), the \(n\)-vector \(P(t) = Pr[ X(t) = i ]\) is probability that the system is in each of the \(n\) states. These probabilities evolve forward in time, governed by an \(n \times n\) transition rate matrix \(K\)

\[dP(t)/dt = P(t) \cdot K\]

The solution is

\[P(t) = \exp(tK) \cdot P(0)\]

Where \(\exp(tK)\) is the matrix exponential. Written differently, the state-to-state lag-\(\tau\) transition probabilities are

\[Pr[ X(t+\tau) = j \;|\; X(t) = i ] = \exp(\tau K)_{ij}\]

For this model, we observe the evolution of one or more chains, \(X(t)\) at a regular interval, \(\tau\). Let \(C_{ij}\) be the number of times the chain was observed at state \(i\) at time \(t\) and at state \(j\) at time \(t+\tau\) (the number of observed transition counts). Suppose that \(K\) depends on a parameter vector, \(\theta\). The log-likelihood is

\[\mathcal{L}(\theta) = \sum_{ij} \left[ C_{ij} \log\left(\left[\exp(\tau K(\theta))\right]_{ij}\right)\right]\]

The ContinuousTimeMSM model finds a rate matrix that fits the data by maximizing this likelihood expression.

Estimation¶

ContinuousTimeMSM uses L-BFGS-B to find a maximum likelihood estimate (MLE) rate matrix, \(\hat{\theta}\) and \(K(\hat{\theta})\).

Uncertainties¶

Analytical estimates of the asymptotic standard deviation in estimated parameters like the stationary distribution, rate matrix, eigenvalues, and relaxation timescales can be computed by calling methods on the ContinuousTimeMSM object. See [1] for more detail.

References¶

[1]	McGibbon, R. T. and V. S. Pande, “Efficient maximum likelihood parameterization of continuous-time Markov processes.” J. Chem. Phys. 143 034109 (2015) http://dx.doi.org/10.1063/1.4926516

[2]	Kalbfleisch, J. D., and Jerald F. Lawless. “The analysis of panel data under a Markov assumption.” J. Am. Stat. Assoc. 80.392 (1985): 863-871.