Interested in estimating the order of a finite-state Hidden Markov Model (HMM) with non-parametric emission distributions from a single observed sequence, we introduce a new method that only requires full rank transition matrix and linear independence between the emission distributions. This method relies on the equality between the order of the HMM and the rank of a specific integral operator. Since only the empirical counter-part of the singular values of the operator can be obtained, a thresholding procedure is proposed. At a non-asymptotic level, an upper-bound on the probability of overestimating the order of the HMM is provided. At an asymptotic level, the consistency of the estimator is established. In addition, we introduce a general heuristic that can be successfully applied to several problems in spectral analysis for designing a data-driven procedure for the threshold. The approach has the advantage of not requiring any knowledge of an upper-bound on the order of the HMM. Moreover, different types of data (including circular or mixed-type data) can be managed. The relevance of the approach is illustrated on numerical experiments and on real data considering multivariate data with directional variables.

SUMMARY: VarSelLCM allows a full model selection (detection of the relevant features for clustering and selection of the number of clusters) in model-based clustering, according to classical information criteria. Data to be analyzed can be composed of continuous, integer and/or categorical features. Moreover, missing values are managed, without any pre-processing, by the model used to cluster with the assumption that values are missing completely at random. Thus, VarSelLCM also allows data imputation by using mixture models. A Shiny application is implemented to easily interpret the clustering results. AVAILABILITY AND IMPLEMENTATION: VarSelLCM is available to download at https://CRAN.R-project.org/package=VarSelLCM/. TUTORIAL: vignette is available online at http://varsellcm.r-forge.r-project.org/. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.