Regime-Conditioned Performance: Measuring ML Robustness Across Market States
Most backtests still report a single Sharpe, a single drawdown, a single equity curve. But ML models in finance almost never fail "on average" – they fail by regime.
This post outlines a research framework for regime-conditioned performance analysis that we are building directly into volarixs.
1. Why Average Metrics Are Misleading
Let a strategy produce daily returns (r_t) over T days. Standard practice: compute a single Sharpe ratio:
Sharpe = E[r_t] / √Var(r_t) × √252Suppose we segment the sample into regimes k ∈ {1, ..., K}, e.g.:
- Regime 1: Low-vol bull
- Regime 2: High-vol sideways
- Regime 3: Crisis
Two models can have the same overall Sharpe, but very different regime profiles:
- Model A: Sharpe₁ = 3.0, Sharpe₂ = -1.5, Sharpe₃ = -2.0
- Model B: Sharpe₁ = 1.2, Sharpe₂ = 0.8, Sharpe₃ = 0.3
If you're allocating real capital, Model B is far more attractive. The usual backtest won't tell you this.
2. Defining Market Regimes
We need objective regime labels that:
- are computed without peeking into the future
- can be reproduced across experiments
- can be applied across asset classes
2.1 Volatility-Based Regimes
Compute rolling realised volatility σ_t over a window w, then bucket by quantiles:
- 0–20%: Low-vol
- 20–60%: Normal
- 60–85%: Elevated
- 85–100%: Crisis
Regime Segmentation Playground
Adjust the volatility thresholds and rolling window to see how regime definitions change. Notice how different parameters create different regime patterns, affecting model performance evaluation.
2.2 Hidden Markov Model (HMM) Regimes
Fit a 2–3 state Gaussian HMM to infer most likely state sequence. More flexible but heavier to compute.
2.3 Cluster-Based Regimes
Define a feature vector (vol, correlation, dispersion, etc.) and cluster using k-means or Gaussian mixtures.
3. Regime-Conditioned Metrics
Once we have regime labels, we compute for each model:
- Regime Sharpe: Sharpe_k for each regime
- Regime Max Drawdown
- Regime Hit Ratio (% r_t^(k) > 0)
- Regime Turnover
- Regime PnL contribution
We can then define robustness scores that penalize dispersion of performance across regimes.
4. Research Use Cases
4.1 Model Comparisons
Compare LSTM vs Ridge vs Random Forest by regime, not just overall. Analyze which models collapse during crisis states.
4.2 Portfolio Construction
Build ensembles where constituent models dominate in different regimes. Allocate capital dynamically based on current regime probability.
4.3 Production Monitoring
A live model suddenly deteriorates only in one regime → diagnosis, not panic. Rolling regime-conditioned Sharpe becomes part of your health dashboard.
5. How This Maps to volarixs
Regime-conditioned analysis only works if the underlying material is kept together. That part is real today: each experiment is anchored to the data this framework needs.
- the feature sets, models, and target horizon the run was built on
- a stored prediction history, with multi-horizon signals per ticker
- the realised outcomes those predictions are scored against
- the macro state (inflation, policy, liquidity) and regime context each run carried
The metrics in this post — regime Sharpe, regime drawdown, robustness scores — are the direction that data is built to support. A polished “Performance by Regime” breakdown is design intent rather than a shipped dashboard today; what exists now is the run-level history and regime context it would be computed from.
Either way, the point is to shift the conversation from “What's the Sharpe?” to “How does this thing behave when the world changes?” That's where robustness lives — and it's the lens volarixs is built around.