Correlations among variables in complex ecosystems such as weather systems and financial markets result in large amounts of dynamic and co-evolving time sequences. The benefits of discovering and predicting intricate patterns (aka regimes) in time sequences are multifold, including better understanding of the ecosystem dynamics, optimizing model selection, and improving interpretability of results. Despite recent advancements, existing methods primarily emphasize predictive accuracy, which might overshadow the need to comprehend the structural dynamics within the series. Additionally, these methods often encounter one or more of the following limitations: (1) difficulty in identifying regimes within domain-dependent segmentations; (2) inability to integrate nonlinear relationships across time series; (3) lack of an effective method to encapsulate the temporal behaviors. To tackle these challenges, we introduce a twin learning regime-switch model to simultaneously learn domain-agnostic segmentation and regime switch in a principled way. Specifically, we devise a kernel-based method that determines the duration of regime and captures dynamic switches through potent representations, accounting for the non-linear interactions between series. With this model, it is feasible to automatically achieve the two subtasks of identifying the optimal regimes and determining the most suitable segmentation. Experimental results on synthetic and real-world datasets indicate that our method is capable of revealing the structures that underpin the behavior of co-evolving ecosystems, which display different dynamics. These structures can be leveraged to better define regimes with superior predictive capabilities compared to widely used traditional models and state-of-the-art neural network models. • We propose a novel regime-switch model to identify domain-agnostic regimes in time series. • We develop a kernel representation learning approach to capturing dynamic regime switches. • This representation allows effectively modeling nonlinear interactions and co-evolving patterns in time series. • Our model transforms heavy sets of time series into a lighter and meaningful structure, enabling a deeper understanding of structural dynamics. • The extensive experimental results showcase superior predictive performance compared to traditional and state-of-the-art methods.

Time sequences are essential in fields such as finance, healthcare, and environmental science, where understanding temporal dependencies and making accurate predictions are crucial. These sequences often exhibit complexities like nonlinearity, noise, and concept drift. Traditional models struggle to capture the intricate dynamics of multivariate and co-evolving sequences, particularly in contexts where relationships between variables shift unpredictably. This thesis introduces a range of Kernel Representation Learning (KRL) methodologies to address these challenges. We develop kernel self-representation learning to capture the temporal dependencies and hidden structures, while identifying concept drift in co-evolving sequences. Additionally, we explore theoretical connections between KRL and advanced deep-learning models. The proposed methods are validated through real-world applications, showing improvements in predictive accuracy, interpretability, and robustness.

Time series modeling is essential in finance, healthcare, and environmental science, yet nonlinear patterns, noise, and concept drift pose challenges. Although deep learning models, such as Transformer-based and recent pre-trained models, have achieved good performance across various time series tasks, they often lack interpretability, especially in co-evolving time series. This work introduces a kernel representation learning (KRL) perspective, rethinking time series modeling through kernel-induced self-representation to effectively capture temporal structures and dynamic transitions. Additionally, we establish theoretical connections between KRL and advanced deep-network models, demonstrating how kernel methods provide a principled approach to capturing complex time series behaviors.

We investigate the problem of discovering and forecasting regular regime switches in a financial ecosystem comprising multiple time series. Such regime switches, indicative of varying market behaviors across distinct time intervals, are pivotal for a nuanced understanding of market dynamics, which in turn allows informed model selection for forecasting and enhanced interpretability of predictive outcomes. Despite strides in this domain, prevailing methodologies often falter due to: (1) an inability to effectively model the temporal behaviors inherent in financial series; and (2) neglecting the interdependencies among series when discovering regimes. In this paper, we propose RHINE, a Regime-switcHIng model with Nonlinear rEpresentation. RHINE stands out with its kernel-based representation, adept at capturing the dynamic shifts in market regimes. This representation encapsulates the nonlinear interplay across multiple financial time series. By leveraging the kernel representation, we introduce an eigengap thresholding measure, designed to automatically discern the optimal number of financial market regimes, enhancing the model's adaptability to market fluctuations. Empirical assessments on both synthetic and real-world stock market datasets underscore RHINE's prowess. The findings illuminate that the inherent structures governing financial market behaviors are dynamic, and harnessing these dynamics via RHINE leads to a regime-based model that outperforms both conventional and state-of-the-art neural network models in predictive capabilities.

We investigate issues related to dynamic cross-sectional regime identification for financial market prediction. A financial market can be viewed as an ecosystem regulated by regimes that may switch at different time points. In most existing regime-based prediction models, regimes can only switch, according to a static transition probability matrix, among a fixed set of regimes identified on training data due to the fact that they lack in mechanism of identifying new regimes on test data. This prevents them from being effective as the financial markets are time-evolving and may fall into a new regime at any future time. Moreover, most of them only handle single time series, and are not capable of dealing with multiple time series. These shortcomings prompted us to devise a dynamic cross-sectional regime identification model for time series prediction. The new model is defined on a multi-time-series system, with time-varying transition probabilities, and can identify new cross-sectional regimes dynamically from the time-evolving financial market. Experimental results on real-world financial datasets illustrate the promising performance and suitability of our model.