Feature Selection for HMM Regime Detection: An Empirical Characterisation of Volatility and Trend Features in Equity Markets
Abstract
Feature-saliency methods such as Feature-saliency Hidden Markov Models (FSHMM) ask which features an HMM should retain from a candidate financial time-series feature pool. This paper addresses the complementary empirical question: given a candidate feature, what regime structure does it produce? We argue that this must be answered before algorithmic feature selection can be meaningful, and examine it systematically across 53 US-listed equities. Stacking features within conceptual groups, we measure regime structure using minority-state occupancy. In the volatility group, daily return alone produces a valid variance regime, and adding rolling volatility preserves structure. Adding absolute return, however, collapses regimes for all 53 stocks because it introduces redundant return-magnitude information into the volatility feature set. In contrast, stacking trend features based on 20-, 60-, and 100-day cumulative returns preserves regime structure across the panel. We also confirm that multivariate Gaussian HMMs are effectively invariant to feature ordering. Per-feature analysis shows that volatility features universally produce variance-dominated regimes, while trend features produce direction-dominated regimes when the asset exhibits sufficient trend variation. These findings complement saliency-based selection by showing which financial time-series features are worth composing into candidate pools before algorithmic pruning. Code is available at https://github.com/axxx129-ops/HMMfeatureselection.git
Authors: Jiakang Xu, Kaung Myat Kyaw, Jonathan H. Chan