Regime Switching and Hidden Markov Models [/rəˈʒim sˈwɪʧɪŋ ənd ˈhɪdən ˈmarkov ˈmɑdəlz/] n - Markets have not one temperament but several, and they trade them back and forth. A low-volatility trending regime is a different country from a high-volatility mean-reverting one; a model built for the first will bleed out in the second. Regime switching models endeavor to learn which country you stand in before you size the trade.
A Hidden Markov Model supposes the market abides always in one of a few unobserved states. Passage between states follows a probability matrix , where . What we observe (returns, volatility, correlations) is drawn from a distribution that depends upon the present state. The task is to infer the state from the data.
In equations:
The inferred regime probabilities, laid over time, resemble a stack of bands:
Inferred regime probabilities
The yield is a distribution over regimes at each moment. If the probability of the high-volatility regime spikes, you lessen your size; if the low-volatility trending regime holds sway, you let positions run. The worth is no better point forecast, but a truer forecast of uncertainty.
Estimation is a subtle business. The likelihood surface is non-convex, and a model granted too many states will overfit. Baum-Welch and expectation-maximization are the standard instruments, yet initialization weighs heavily. A model that finds states only because you bade it find three has not discovered structure; it has imposed it.
These models serve best in company with others. A trend-following signal may be weighted by the probability that the market is in a trending state; a mean-reversion signal may be silenced whilst the crash regime holds. The model is an overlay, not a strategy.
Other roads exist. Threshold models switch upon observable variables, volatility or credit spreads. Smooth transition models admit gradual change. Bayesian changepoint detection finds structural breaks without fixing the number of regimes beforehand. The right tool turns on whether regimes are discrete or continuous.
The hardest part is timing. Regimes are known only in arrears; a model that tells you yesterday was a crash serves less than one that warns today may become one. Some delay abides always. The art is to lessen the delay without multiplying false alarms.
Regime switching is, at the last, an exercise in humility: it writes down formally that no model works at all times. See Alpha Decay and Model Lifecycle for what to do when the regime you modeled vanishes.