Hidden Markov models (HMM) have been widely used by scientists to model stochastic systems: the underlying process is a discrete Markov chain, and the observations are noisy realizations of the underlying process. Determining the number of hidden states for an HMM is a model selection problem which is yet to be satisfactorily solved, especially for the popular Gaussian HMM with heterogeneous covariance. In this paper, we propose a consistent method for determining the number of hidden states of HMM based on the marginal likelihood, which is obtained by integrating out both the parameters and hidden states. Moreover, we show that the model selection problem of HMM includes the order selection problem of finite mixture models as a special case. We give rigorous proof of the consistency of the proposed marginal likelihood method and provide an efficient computation method for practical implementation. We numerically compare the proposed method with the Bayesian information criterion (BIC), demonstrating the effectiveness of the proposed marginal likelihood method.

In this paper, we examine the existence of the Rényi divergence between two time invariant hidden Markov models with arbitrary positive initial distributions. By making use of a Markov chain representation of the probability distribution for the hidden Markov model and eigenvalue for the associated Markovian operator, we obtain, under some regularity conditions, convergence of the Rényi divergence. By using this device, we also characterize the Rényi divergence and obtain the Kullback–Leibler divergence as $$\alpha \rightarrow 1$$ of the Rényi divergence. Several examples, including classical finite state hidden Markov models, Markov switching models, and recurrent neural networks, are given for illustration. Moreover, we develop a non-Monte Carlo method that computes the Rényi divergence of two-state Markov switching models via the underlying invariant probability measure, which is characterized by the Fredholm integral equation.

To characterize the Kullback-Leibler divergence and Fisher information in general parametrized hidden Markov models, in this paper, we first show that the log likelihood and its derivatives can be represented as an additive functional of a Markovian iterated function system, and then provide explicit characterizations of these two quantities through this representation. Moreover, we show that Kullback-Leibler divergence can be locally approximated by a quadratic function determined by the Fisher information. Results relating to the Cramér-Rao lower bound and the Hájek-Le Cam local asymptotic minimax theorem are also given. As an application of our results, we provide a theoretical justification of using Akaike information criterion (AIC) model selection in general hidden Markov models. Last, we study three concrete models: a Gaussian vector autoregressive-moving average model of order$(p,q)$, recurrent neural networks, and temporal restricted Boltzmann machine, to illustrate our theory.

This paper investigates change point detection in state space models, in which the pre-change distribution fθ0is given, while the post-distribution fθafter change is unknown. The problem is to raise an alarm as soon as possible after the distribution changes from fθ0to fθ, under a restriction on the false alarms. We investigate theoretical properties of a weighted Shiryayev-Roberts-Pollak (SRP) change point detection rule in state space models. By making use of a Markov chain representation for the likelihood function, exponential embedding of the induced Markovian transition operator, nonlinear Markov renewal theory, and sequential hypothesis testing theory for Markov random walks, we show that the weighted SRP procedure is second-order asymptotically optimal. To this end, we derive an asymptotic approximation for the expected stopping time of such a stopping scheme when the change time w = 1. To illustrate our method we apply the results to two types of state space models: general state Markov chains and linear state space models. Simulation study is given to illustrate the method.

In the 1960s, Shiryaev developed a Bayesian theory of change-point detection in the i.i.d. case, which was generalized in the early 2000s by Tartakovsky and Veeravalli and recently by Tartakovsky (2017) for general stochastic models assuming a certain stability of the log-likelihood ratio process. Hidden Markov models represent a wide class of stochastic processes in a variety of applications. In this paper, we investigate the performance of the Bayesian Shiryaev change-point detection rule for hidden Markov models. We propose a set of regularity conditions under which the Shiryaev procedure is first-order asymptotically optimal in a Bayesian context, minimizing moments of the detection delay up to certain order asymptotically as the probability of false alarm goes to zero. The developed theory for hidden Markov models is based on Markov chain representation for the likelihood ratio and r-quick convergence for Markov random walks. In addition, applying Markov nonlinear renewal theory, we present a high-order asymptotic approximation for the expected delay to detection and a first-order asymptotic approximation for the probability of false alarm of the Shiryaev detection rule. We also study asymptotic properties of another popular change detection rule, the Shiryaev-Roberts rule, and provide some interesting examples.

The quickest change detection problem is studied in two-state hidden Markov models (HMM), where the vector parameter θ of the HMM may change from θ0to θ1at some unknown time, and one wants to detect the true change as quickly as possible while controlling the false alarm rate. It turns out that the generalized likelihood ratio (GLR) scheme, while theoretically straightforward, is generally computationally infeasible for the HMM. To develop efficient but computationally simple schemes for the HMM, we first show that the recursive CUSUM scheme proposed in Fuh (Ann. Statist., 2003) can be regarded as a quasi-GLR scheme for some suitable pseudo post-change hypotheses. Next, we extend the quasi-GLR idea to propose recursive score schemes in a more complicated scenario when the post-change parameter θ1of the HMM involves a real-valued nuisance parameter. Finally, our research provides an alternative approach that can numerically compute the Kullback-Leibler (KL) divergence of two-state HMMs via the invariant probability measure and the Fredholm integral equation.

Degradation analysis is a tool for assessing the lifetime distribution in reliability analysis. The lifetime of a product, in degradation analysis, is defined as the time when the value of a chosen degradation characteristic reaches a predetermined threshold. This kind of failure is called a soft failure, in contrast with a hard failure which means that the product is not functioning at any positive performance level. In this article, we introduce the idea of considering the threshold for the degradation characteristic as random. The difference between the time to soft failure and hard failure is then modeled. We modify Lu & Meeker's two-stage method, and a new approach named the three-stage method is developed. We apply both the two- and three-stage methods to a simulation study. From the simulation study, we conclude that when the time to soft failure is very different from the time to hard failure, the three-stage method leads to a better performance than the two-stage method. Fatigue-crack growth data are analysed at the end.

It is well known that an intuitionistic fuzzy relation is a generalization of a fuzzy relation. In fact there are situations where intuitionistic fuzzy relations are more appropriate. This paper discusses the fuzzy clustering based on intuitionistic fuzzy relations. On the basis of max -t & min -s compositions, we discuss an n-step procedure which is an extension of Yang and Shih's [17] n-step procedure. A similarity-relation matrix is obtained by beginning with a proximity-relation matrix using the proposed n-step procedure. Then we propose a clustering algorithm for the similarity-relation matrix. Numerical comparisons of three critical max -t & min -s compositions: max -t1 & min -s1, max -t2 & min -s2 and max -t3 & min -s3, are made. The results show that max -t1 & min -s1 compositions has better performance. Sometimes, data may be missed with an incomplete proximity-relation matrix. Imputation is a general and flexible method for handling missing-data problem. In this paper we also discuss a simple form of imputation is to estimate missing values by max -t & min -s compositions.