In recent years, axial data, where observations are treated as axes of direction, has gained prominence in a range of complex tasks, including gene expression data clustering, blind speech separation, and depth image analysis. However, prevailing methods for axial data modeling mainly rely on shallow probabilistic models, which often overlook the hidden and hierarchical dependencies in the latent space. These methods also require a separate, human-engineered feature extractor to obtain features from raw axial data for downstream tasks. This work introduces a novel framework, Axial Variational Autoencoders (AVAEs), for modeling and representation learning of axial data by leveraging a deep generative model, the Variational Autoencoder (VAE). Unlike existing approaches, our method can autonomously learn more expressive representations from axial data by designing a VAE that uses the Watson distribution as the latent prior. Furthermore, we introduce a tailored reparameterization technique to support stable training. We validate the effectiveness of our model through experiments on simulated axial datasets and a real-world application.

Clustering high-dimensional directional data (i.e., L2 normalized vectors) presents significant challenges due to the intricate spherical representations of latent embeddings and the limitations of classical (non-deep) clustering techniques. Moreover, dynamically inferring the number of clusters remains a fundamental issue in existing deep clustering methods, especially those involving complex model-selection criteria. This paper addresses these challenges by introducing a novel deep nonparametric clustering framework that employs hyperspherical latent embeddings within a Variational Autoencoder architecture, enhanced by an infinite Von Mises-Fisher Mixture Model as a dynamic prior. This approach enables automatic adaptation of cluster numbers during training, eliminating the need for predefined clusters and traditional model selection processes. Our scalable architecture effectively integrates In-vMFMM with hyperspherical embeddings to tackle the complexities of directional data. Utilizing a joint training strategy, our method alternates between updating neural network parameters and adjusting mixture model priors via nonparametric variational Bayes. Empirical evaluations on benchmark datasets, including complex ImageNet-50, demonstrate that our approach significantly outperforms state-of-the-art deep nonparametric clustering methods. It also robustly estimates the number of clusters, showcasing its effectiveness and versatility in handling high-dimensional directional data.

We introduce a mixture of mixtures of bounded asymmetric generalized Gaussian and uniform distributions. Based on this framework, we propose model-based classification and model-based clustering algorithms. We develop an objective function for the minimum message length (MML) model selection criterion to discover the optimal number of clusters for the unsupervised approach of our proposed model. Given the crucial attention received by Explainable AI (XAI) in recent years, we introduce a method to interpret the predictions obtained from the proposed model in both learning settings by defining their boundaries in terms of the crucial features. Integrating Explainability within our proposed algorithm increases the credibility of the algorithm’s predictions since it would be explainable to the user’s perspective through simple If-Then statements using a small binary decision tree. In this paper, the proposed algorithm proves its reliability and superiority to several state-of-the-art machine learning algorithms within the following real-world applications: fault detection and diagnosis (FDD) in chillers, occupancy estimation and categorization of residential energy consumers.

We consider the problem of anomaly detection in proportional data by investigating the Libby-Novick Beta-Liouville distribution, a novel distribution merging the salient characteristics of Liouville and Libby-Novick Beta distributions. Its main benefit, compared to the typical distributions dedicated to proportional data such as Dirichlet and Beta-Liouville, is its adaptability and explanatory power when dealing with this kind of data. Our goal is to exploit this appropriateness for modeling proportional data to achieve great performance in the anomaly detection task. First, we develop generative models, namely finite mixture models of Libby-Novick Beta-Liouville distributions. Then, we propose two discriminative techniques: Normality scores based on selecting the given distribution to approximate the softmax output vector of a deep classifier and an improved version of Support Vector Machine (SVM) by suggesting a feature mapping approach. We demonstrate the benefits of the presented approaches through a variety of experiments on both image and non-image datasets. The results demonstrate that the proposed anomaly detectors based on the Libby-Novick Beta-Liouville distribution outperform the classical distributions as well as the baseline techniques.

In topic models, collections are organized as documents where they arise as mixtures over latent clusters called topics. A topic is a distribution over the vocabulary. In large-scale applications, parametric or finite topic mixture models such as LDA (latent Dirichlet allocation) and its variants are very restrictive in performance due to their reduced hypothesis space. In this article, we address the problem related to model selection and sharing ability of topics across multiple documents in standard parametric topic models. We propose as an alternative a BNP (Bayesian nonparametric) topic model where the HDP (hierarchical Dirichlet process) prior models documents topic mixtures through their multinomials on infinite simplex. We, therefore, propose asymmetric BL (Beta-Liouville) as a diffuse base measure at the corpus level DP (Dirichlet process) over a measurable space. This step illustrates the highly heterogeneous structure in the set of all topics that describes the corpus probability measure. For consistency in posterior inference and predictive distributions, we efficiently characterize random probability measures whose limits are the global and local DPs to approximate the HDP from the stick-breaking formulation with the GEM (Griffiths-Engen-McCloskey) random variables. Due to the diffuse measure with the BL prior as conjugate to the count data distribution, we obtain an improved version of the standard HDP that is usually based on symmetric Dirichlet (Dir). In addition, to improve coordinate ascent framework while taking advantage of its deterministic nature, our model implements an online optimization method based on stochastic, at document level, variational inference to accommodate fast topic learning when processing large collections of text documents with natural gradient. The high value in the predictive likelihood per document obtained when compared to the performance of its competitors is also consistent with the robustness of our fully asymmetric BL-based HDP. While insuring the predictive accuracy of the model using the probability of the held-out documents, we also added a combination of metrics such as the topic coherence and topic diversity to improve the quality and interpretability of the topics discovered. We also compared the performance of our model using these metrics against the standard symmetric LDA. We show that online HDP-LBLA (Latent BL Allocation)’s performance is the asymptote for parametric topic models. The accuracy in the results (improved predictive distributions of the held out) is a product of the model’s ability to efficiently characterize dependency between documents (topic correlation) as now they can easily share topics, resulting in a much robust and realistic compression algorithm for information modeling.