# Continuous-time MSMs¶

`MarkovStateModel`

estimates a series of
transition *probabilities* among states that depend on the discrete
lag-time. Physically, we are probably more interested in a sparse set of
transition *rates* in and out of states, estimated by
`ContinuousTimeMSM`

.

## Theory¶

Consider an n-state time-homogeneous Markov process, \(X(t)\). At time \(t\), the \(n\)-vector \(P(t) = Pr[ X(t) = i ]\) is the probability that the system is in each of the \(n\) states. These probabilities evolve forward in time, governed by an \(n \times n\) transition rate matrix \(K\)

The solution is

Where \(\exp(tK)\) is the matrix exponential. Written differently, the state-to-state lag-\(\tau\) transition probabilities are

For this model, we observe the evolution of one or more chains, \(X(t)\) at a regular interval, \(\tau\). Let \(C_{ij}\) be the number of times the chain was observed at state \(i\) at time \(t\) and at state \(j\) at time \(t+\tau\) (the number of observed transition counts). Suppose that \(K\) depends on a parameter vector, \(\theta\). The log-likelihood is

The `ContinuousTimeMSM`

model finds a rate matrix that fits the data
by maximizing this likelihood expression. Specifically, it uses L-BFGS-B
to find a maximum likelihood estimate (MLE) rate matrix,
\(\hat{\theta}\) and \(K(\hat{\theta})\).

### Uncertainties¶

Analytical estimates of the asymptotic standard deviation in estimated
parameters like the stationary distribution, rate matrix, eigenvalues, and
relaxation timescales can be computed by calling methods on the
`ContinuousTimeMSM`

object. See [1] for more detail.

## Algorithms¶

`ContinuousTimeMSM` ([lag_time, n_timescales, ...]) |
Reversible first order master equation model |

## References¶

[1] | McGibbon, R. T. and V. S. Pande, “Efficient maximum likelihood parameterization of continuous-time Markov processes.” J. Chem. Phys. 143 034109 (2015) http://dx.doi.org/10.1063/1.4926516 |

[2] | Kalbfleisch, J. D., and Jerald F. Lawless. “The analysis of panel data under a Markov assumption.” J. Am. Stat. Assoc. 80.392 (1985): 863-871. |