Recent advances in rotation-invariant (RI) learning for 3D point clouds typically replace raw coordinates with handcrafted RI features to ensure robustness under arbitrary rotations. However, these approaches often suffer from the loss of global pose information, making them incapable of distinguishing geometrically similar but spatially distinct structures. We identify that this limitation stems from the restricted receptive field in existing RI methods, leading to Wing–tip feature collapse, a failure to differentiate symmetric components (e.g., left and right airplane wings) due to indistinguishable local geometries. To overcome this challenge, we introduce the Shadow-informed Pose Feature (SiPF), which augments local RI descriptors with a globally consistent reference point (referred to as the “shadow”) derived from a learned shared rotation. This mechanism enables the model to preserve global pose awareness while maintaining rotation invariance. We further propose Rotation-invariant Attention Convolution (RIAttnConv), an attention-based operator that integrates SiPFs into the feature aggregation process, thereby enhancing the model’s capacity to distinguish structurally similar components. Additionally, we design a task-adaptive shadow locating module based on the Bingham distribution over unit quaternions, which dynamically learns the optimal global rotation for constructing consistent shadows. Extensive experiments on 3D classification and part segmentation benchmarks demonstrate that our approach substantially outperforms existing RI methods, particularly in tasks requiring fine-grained spatial discrimination under arbitrary rotations.

Topic modeling plays a critical role in organizing and understanding large-scale web content. While neural topic models (NTMs) based on variational autoencoders (VAEs) have achieved notable success in analyzing textual data, they remain limited in addressing the multimodal nature of modern web content. Existing unimodal or multimodal extensions often suffer from posterior collapse and fail to capture the directional semantics inherent in both text and images, resulting in incoherent topics and limited interpretability. To address these challenges, we propose MM-vNTM (MultiModal Neural Topic Model with von Mises-Fisher Mixtures), a framework for web-scale topic discovery over multimodal data. MM-vNTM leverages pre-aligned cross-modal embeddings as inputs and jointly models document-level representations of text and image modalities in a shared hyperspherical latent space. Furthermore, it defines topics as mixtures of von Mises-Fisher (vMF) distributions in the L2-normalized word embedding space, explicitly capturing directional similarity. Experiments on multimedia web datasets demonstrate that MM-vNTM consistently outperforms state-of-the-art unimodal and multimodal baselines in terms of overall topic quality, highlighting its effectiveness for real-world web scenarios.

Joint blind source separation (JBSS) is commonly used to uncover latent structures across multiple datasets. However, it often struggles with high-dimensional data and inaccurate latent dimensionality estimation, limiting its scalability and separation accuracy. To address these challenges, we propose a novel common subspace independent vector analysis model based on a bounded multivariate generalized Gaussian mixture distribution, referred to as BMIVA-CS. The model extracts low-rank common sources across datasets while modeling subject-specific variations. It incorporates a bounded indicator function and leverages spatial correlations to improve robustness, particularly under noisy, high-dimensional conditions. We validate the model through simulations and experiments on resting-state fMRI datasets from individuals with schizophrenia and autism. BMIVA-CS reliably identifies clinically relevant sources and consistently localizes affected brain regions across subjects. These results demonstrate the effectiveness of BMIVA-CS and its potential as a diagnostic tool for neurological and psychiatric disorders using rs-fMRI data.

Timely fault detection in HVAC systems is crucial for preventing energy waste and maintaining thermal comfort in commercial buildings. This paper introduces the adaptive adversarial autoencoder (AdaAAE), an innovative anomaly detection approach designed for unsupervised fault detection within air handling units (AHUs) of HVAC systems. By combining the adversarial autoencoder (AAE) with deep support vector data description (DSVDD), AdaAAE efficiently trains the reconstruction, regularization, and compactness phases to produce a spherical and compact latent representation of the training data, enabling effective fault detection within the latent space. A key feature of AdaAAE is its ability to accurately determine the anomaly threshold at the end of the training phase, which is particularly beneficial in scenarios where there is no prior knowledge of the anomaly ratio for test instances—a common challenge in real-world applications. AdaAAE’s performance was evaluated on a real-world AHU system developed as part of the ASHRAE research project 1312 (RP-1312). Experimental results, especially with regard to AUC-ROC and AUC-PR, highlight AdaAAE’s superior fault detection capabilities across AHU datasets with varying complexities. Notably, AdaAAE achieved outstanding performance, with an average AUC-ROC of 97.5% and AUC-PR of 94.4%, significantly outperforming the baseline methods in this study. Additionally, the experimental findings demonstrate AdaAAE’s proficiency in accurately estimating the anomaly threshold without prior knowledge, with minimal performance degradation when shifting from a scenario with a known anomaly ratio to one requiring estimation.Note to Practitioners—The concept of anomaly detection through subspace learning has attracted significant interest, with a focus on achieving a compact latent distribution. In this study, we introduce a new anomaly detection technique called AdaAAE, which is designed to produce a spherical and compact latent representation of training data. This feature enhances the method’s ability to identify anomalies within the latent space. Moreover, AdaAAE offers a distinct advantage in accurately determining the anomaly threshold at the end of the training phase, which is especially useful in practical scenarios where prior information about anomaly ratios for test instances is unavailable. To ensure reproducibility and allow for future enhancements by other researchers, the entire source code for this study is available in the following repository: https://github.com/viettra-xai/Ada-AAE.

Bus passenger flow prediction is a crucial task in bus transportation management and optimization, characterized by strong spatiotemporal dependencies and influenced by external factors such as weather and holidays. However, existing methods face challenges in responding to sudden events, modeling dynamic changes in bus network topology, and improving model generalization. To address these issues, this paper proposes a graph transfer learning-based approach for bus passenger flow prediction. First, a dynamic graph neural network is used to construct the passenger flow network, and Framelet Transform along with Self-Expressiveness Regularization is applied for denoising, enhancing data quality. Second, an adaptive neighborhood-aware dynamic graph convolutional network is introduced, integrating random mask enhancement, hop count perception fusion, and multi-channel spatiotemporal feature extraction for accurate passenger flow modeling. Furthermore, a combination of source data and source free transfer learning is leveraged to optimize feature distribution alignment through pseudo-label generation, graph diffusion, and consistency loss. Finally, a GRU is used for passenger flow prediction, and extensive experiments on real-world datasets test the proposed method. Results demonstrate superior accuracy and generalization performance compared to existing state of the art (SOTA) models.

Short-term passenger flow prediction is critical for intelligent scheduling and efficient operation of public transportation systems. However, existing methods often struggle to maintain robustness and generalizability when facing complex and heterogeneous noise sources, such as passenger flow noise including sensor-induced missing values and abrupt ridership fluctuations caused by unexpected events, and graph structural noise resulting from dynamic changes in the network topology due to route modifications or stop closures. To address these challenges, this paper proposes a novel deep learning framework: Robust Dynamic Graph Neural Networks (RDGNN), that integrates noise cleaning, dynamic graph modeling, and more efficient prediction. The proposed noise cleaning employs K-Nearest Neighbor imputation and Gaussian Mixture Models with a penalty function to mitigate data-level noise, while a graph structure denoising module is introduced to correct topological anomalies in the transit network and enhance the reliability of spatial representation. For feature modeling, the framework constructs a dynamic graph neural network from a subgraph perspective to capture both spatial dependencies and temporal dynamics, particularly suited to modeling interactions among transfer stations. A Liquid Neural Network is adopted as the prediction module, leveraging its strong adaptability and memory capacity to handle irregular and non-stationary time series, while remaining computationally efficient. The RDGNN model was trained and validated on real-world bus passenger flow data from Ames, Iowa, and further evaluated on the large-scale EXO dataset. Compared to state-of-the-art models, RDGNN consistently achieves better prediction accuracy, higher robustness to noise and missing data, and greater computational efficiency. Its strong and stable performance across both datasets, including in scenarios such as route disruptions and seasonal variation, demonstrates excellent generalization capability in diverse urban transit systems. The code is available at:https://github.com/XinyiZhou0318/A-Noise-Robust-Approach-Using-Dynamic-Graph-Neural-Networks-for-Bus-Passenger-Flow-Prediction

In recent years, axial data, where observations are treated as axes of direction, has gained prominence in a range of complex tasks, including gene expression data clustering, blind speech separation, and depth image analysis. However, prevailing methods for axial data modeling mainly rely on shallow probabilistic models, which often overlook the hidden and hierarchical dependencies in the latent space. These methods also require a separate, human-engineered feature extractor to obtain features from raw axial data for downstream tasks. This work introduces a novel framework, Axial Variational Autoencoders (AVAEs), for modeling and representation learning of axial data by leveraging a deep generative model, the Variational Autoencoder (VAE). Unlike existing approaches, our method can autonomously learn more expressive representations from axial data by designing a VAE that uses the Watson distribution as the latent prior. Furthermore, we introduce a tailored reparameterization technique to support stable training. We validate the effectiveness of our model through experiments on simulated axial datasets and a real-world application.

Link prediction (LP) is fundamental to graph-based applications, yet existing graph autoencoders (GAEs) and variational GAEs (VGAEs) often struggle with intrinsic graph properties, particularly the presence of negative eigenvalues in adjacency matrices, which limits their adaptability and predictive performance. To address this limitation, we propose Hyperspherical Kolmogorov-Arnold Networks for LP (HKANLP), a novel framework that combines multiple graph neural network (GNN)-based representation learning strategies with Kolmogorov-Arnold networks (KANs) in a hyperspherical embedding space. Specifically, our model leverages the von Mises-Fisher (vMF) distribution to impose geometric consistency in the latent space and employs KANs as universal function approximators to reconstruct adjacency matrices, thereby mitigating the impact of negative eigenvalues and enhancing spectral diversity. Extensive experiments on homophilous, heterophilous, and large-scale graph datasets demonstrate that HKANLP achieves superior LP performance and robustness compared to state-of-the-art baselines. Furthermore, visualization analyses illustrate the model's effectiveness in capturing complex structural patterns. The source code of our model is publicly available at https://github.com/zxj8806/HKANLP/.

Topic modeling plays a vital role in uncovering latent semantic structures from large-scale biomedical corpora. While classical probabilistic models such as Latent Dirichlet Allocation (LDA) have been widely used, they often struggle with scalability and capturing complex semantic relationships in domain-specific contexts. Neural topic models (NTMs) based on variational autoencoders (VAEs) provide a more flexible and scalable alternative. However, existing NTMs face two key challenges: the neglect of the underlying geometric structure of semantic spaces and the lack of domain-specific adaptation, both of which contribute to suboptimal topic coherence and weak document clustering. To address these limitations, we pro-pose BiovMNVTM, a geometry-aware variational topic modeling framework tailored for biomedical text analysis. BiovMNVTM leverages the von Mises-Fisher (vMF) mixture distribution to model directional relationships on the hypersphere and integrates contextualized biomedical embeddings from BioBERT to capture rich, domain-specific semantics. Comprehensive experiments on multiple biomedical datasets show that BiovMNVTM achieves superior performance in terms of topic quality and clustering ability, demonstrating the effectiveness of incorporating geomet-ric and domain-aware modeling in biomedical topic discovery.

Occupancy estimation (OE) and activity recognition (AR), predicted using Internet of Things (IoT) sensor data, are among the most important outcomes of artificial intelligence (AI) applied to buildings. The unavailability of enough labeled data is a serious problem for building researchers which is why they came up with domain adaptation (DA). Unsupervised domain adaptation (UDA) solves the problem of the unavailability of labeled data in target domains. In this research, we present a novel application for OE and AR that considers UDA methods. We have adapted 3 UDA methods called Joint Adaptation Networks (JAN), Deep Adaptation Networks (DAN), and Deep Correlation Alignment (D-CORAL). Our research application offers a practical solution for smart building management, enabling accurate OE and AR using adapted UDA techniques.

Clustering high-dimensional directional data (i.e., L2 normalized vectors) presents significant challenges due to the intricate spherical representations of latent embeddings and the limitations of classical (non-deep) clustering techniques. Moreover, dynamically inferring the number of clusters remains a fundamental issue in existing deep clustering methods, especially those involving complex model-selection criteria. This paper addresses these challenges by introducing a novel deep nonparametric clustering framework that employs hyperspherical latent embeddings within a Variational Autoencoder architecture, enhanced by an infinite Von Mises-Fisher Mixture Model as a dynamic prior. This approach enables automatic adaptation of cluster numbers during training, eliminating the need for predefined clusters and traditional model selection processes. Our scalable architecture effectively integrates In-vMFMM with hyperspherical embeddings to tackle the complexities of directional data. Utilizing a joint training strategy, our method alternates between updating neural network parameters and adjusting mixture model priors via nonparametric variational Bayes. Empirical evaluations on benchmark datasets, including complex ImageNet-50, demonstrate that our approach significantly outperforms state-of-the-art deep nonparametric clustering methods. It also robustly estimates the number of clusters, showcasing its effectiveness and versatility in handling high-dimensional directional data.

Accurate electric power load forecasting is critical for power utility companies as it increases control over the relevant infrastructure, resulting in significant improvements in energy management and scheduling. However, point forecasting appears to fall short of providing these businesses with enough information to prepare for the worst. This paper proposes an encoder–decoder model that takes advantage of the expressiveness of Transformer-based encoders to produce probabilistic forecasts, i.e., a distribution over future predictions. Two real-world datasets are utilized to incorporate the performance of the proposed model on two different types of data: hourly load data from the power supply company of the city of Johor in Malaysia and hourly load consumption data from one of Grenoble Institute of Technology’s buildings. The former represents aggregated data, which makes identifying patterns and trends easier, but the latter was taken from a single building (non-aggregated), which increases the difficulty of forecasts. The model’s performance is discussed across multiple time horizons, including 24-hour, 1-week, and 1-month predictions. It achieved notable improvements compared to the used baseline, Amazon DeepAr. For 24 h ahead forecasting, accuracy was increased from 87.2 percent to 96.2 percent for Malaysian data and from 52.3 percent to 68.2 percent for Grenoble data. And for 1 month ahead forecasting, it was improved from 84.7 percent to 89.7 percent for Malaysian data, and from 45.5 percent to 57.2 percent for Grenoble data.

ABSTRACT This paper presents an extension of the Dirichlet multinomial regression (DMR) and deep Dirichlet multinomial regression (dDMR) topic modelling approaches by incorporating the generalised Dirichlet (GD) and Beta‐Liouville (BL) distributions using collapsed Gibbs sampling for parameter inference. The DMR and dDMR approaches have been shown to be effective in discovering latent topics in text corpora. However, these approaches have limitations when it comes to handling complex data structures and overfitting issues. To address these limitations, we introduce the GD and BL distributions, which have more flexibility in modelling complex data structures and handling sparse data. Additionally, we use collapsed Gibbs sampling to estimate the model parameters, which provides a computationally efficient method for inference. Experimental results on benchmark datasets demonstrate the effectiveness of the proposed approach in improving topic modelling performance, particularly in handling complex data structures and reducing overfitting. The proposed models also exhibit good interpretability of the learned topics, making them suitable for various applications in natural language processing and machine learning.

This article proposes a novel deep clustering model based on the variational autoencoder (VAE), named GamMM-VAE, which can learn latent representations of training data for clustering in an unsupervised manner. Most existing VAE-based deep clustering methods use the Gaussian mixture model (GMM) as a prior on the latent space. We employ a more flexible asymmetric Gamma mixture model to achieve higher quality embeddings of the data latent space. Second, since the Gamma is defined for strictly positive variables, in order to exploit the reparameterization trick of VAE, we propose a transformation method from Gaussian distribution to Gamma distribution. This method can also be considered a Gamma distribution reparameterization trick, allows gradients to be backpropagated through the sampling process in the VAE. Finally, we derive the evidence lower bound (ELBO) based on the Gamma mixture model in an effective way for the stochastic gradient variational Bayesian (SGVB) estimator to optimize the proposed model. ELBO, a variational inference objective, ensures the maximization of the approximation of the posterior distribution, while SGVB is a method used to perform efficient inference and learning in VAEs. We validate the effectiveness of our model through quantitative comparisons with other state-of-the-art deep clustering models on six benchmark datasets. Moreover, due to the generative nature of VAEs, the proposed model can generate highly realistic samples of specific classes without supervised information.

With the prevalence of short texts in various forms such as news headlines, tweets, and reviews, short text analysis has gained significant interest in recent times. However, modeling short texts remains a challenging task due to its sparse and noisy nature. In this paper, we propose a new Spherical Correlated Topic Model (SCTM), which takes into account the correlation between topics. Our model integrates word and knowledge graph embeddings to better capture the semantic relationships among short texts. We adopt the von Mises-Fisher distribution to model the high-dimensional word and entity embeddings on a hypersphere, enabling better preservation of the angular relationships between topic vectors. Moreover, knowledge graph embeddings are incorporated to further enrich the semantic meaning of short texts. Experimental results on several datasets demonstrate that our proposed SCTM model outperforms existing models in terms of both topic coherence and document classification. In addition, our model is capable of providing interpretable topics and revealing meaningful correlations among short texts.

In the field of human-computer interaction and psychological assessment, speech emotion recognition (SER) plays an important role in deciphering emotional states from speech signals. Despite advancements, challenges persist due to system complexity, feature distinctiveness issues, and noise interference. This paper introduces a new end-to-end (E2E) deep learning multi-resolution framework for SER, addressing these limitations by extracting meaningful representations directly from raw waveform speech signals. By leveraging the properties of the fast discrete wavelet transform (FDWT), including the cascade algorithm, conjugate quadrature filter, and coefficient denoising, our approach introduces a learnable model for both wavelet bases and denoising through deep learning techniques. The framework incorporates an activation function for learnable asymmetric hard thresholding of wavelet coefficients. Our approach exploits the capabilities of wavelets for effective localization in both time and frequency domains. We then combine one-dimensional dilated convolutional neural networks (1D dilated CNN) with a spatial attention layer and bidirectional gated recurrent units (Bi-GRU) with a temporal attention layer to efficiently capture the nuanced spatial and temporal characteristics of emotional features. By handling variable-length speech without segmentation and eliminating the need for pre or post-processing, the proposed model outperformed state-of-the-art methods on IEMOCAP and EMO-DB datasets.

In this paper, we propose a Human-Machine Inter-face offering an efficient and privacy-conscious approach to train Occupancy Estimation Machine Learning models by interacting with users to request the occupancy level of a room. Although there are existing works that optimize the frequency of interactions, these techniques assume that the data is clean. Hence, anomalies in sensor measurements may lead to unnecessary interactions with users, resulting in their disengagement. To address this problem, we employ an Autoencoder neural network to detect anomalies in univariate time series sensor data using the reconstruction error. Furthermore, we tackle the autoencoder architecture selection challenge by utilizing a Reinforcement Learning-based Neural Architecture Search (RLNAS) approach, where an agent explores a predefined search space and identifies the optimal neural configuration by learning through trial and error. Experiments conducted on a custom anomaly detection dataset demonstrate competitive performance, and illustrate how this technique discovers effective architectures that may not be immediately apparent or intuitive.

This work presents a nonparametric Bayesian approach that utilizes a mixture of multivariate Libby-Novick Beta distributions to address clustering challenges. When using mixtures, model selection is a significant obstacle. As a solution to this problem, we extend the finite Libby-Novick Beta mixture model (FLNBMM)to the infinite case. This enables us to accurately represent the data distribution by accommodating an unspecified number of mixture components. We develop a Bayesian inference strategy based on Markov Chain Monte Carlo to estimate the posterior distribution, which provides strong power and flexibility for modeling and analyzing complicated data. Our suggested method’s effectiveness is assessed on three applications and contrasted with that of FLNBMM, the infinite Gaussian mixture model (IGMM), and the finite Gaussian mixture model (FGMM) to show the efficacy of our methodology. It is evident from the results that our proposed model is a good alternative.

In human-computer interaction and psychological evaluation, speech emotion recognition (SER) is crucial for interpreting emotional states from spoken language. Although there have been advancements, challenges such as system complexity, issues with feature distinctiveness, and noise interference continue to persist. This paper presents a novel end-to-end (E2E) deep learning multi-resolution framework for SER, which tackles these limitations by deriving significant representations directly from raw speech waveform signals. By leveraging the properties of wavelet packet transform (WPT), our approach introduces a learnable model for both wavelet bases and denoising through deep learning techniques. Unlike discrete wavelet transform (DWT), WPT offers a more detailed analysis by decomposing both approximation and detail coefficients, providing a finer resolution in the time-frequency domain. This capability enhances feature extraction by capturing more nuanced signal characteristics across different frequency bands. The framework incorporates a learnable activation function for asymmetric hard thresholding of wavelet packet coefficients. Our approach exploits the capabilities of wavelet packets for effective localization in both time and frequency domains. We then combine one-dimensional dilated convolutional neural networks (1D dilated CNN) with a spatial attention layer and bidirectional gated recurrent units (Bi-GRU) with a temporal attention layer to efficiently capture emotional features' nuanced spatial and temporal characteristics. By handling variable-length speech without segmentation and eliminating the need for pre or post-processing, the proposed model outperforms state-of-the-art methods on our NSPL-CRISE dataset and on the public IEMOCAP dataset. The source code of this paper is shared on this repository: https://github.com/alaaNfissi/WPT-Deep-Learning-SER-for-Suicide-Monitoring.

Storing data is particularly a challenge when dealing with image data which often involves large file sizes due to the high resolution and complexity of images. Efficient image compression algorithms are crucial to better manage data storage costs. In this paper, we propose a novel region-based lossy image compression technique, called PatchSVD, based on the Singular Value Decomposition (SVD) algorithm. We show through experiments that PatchSVD outperforms SVD-based image compression with respect to three popular image compression metrics. Moreover, we compare PatchSVD compression artifacts with those of Joint Photographic Experts Group (JPEG) and SVD-based image compression and illustrate some cases where PatchSVD compression artifacts are preferable compared to JPEG and SVD artifacts.

In this paper, we present a multivariate bounded Kotz mixture model (BKMM) for data modeling when the data lies in a bounded support region. In BKMM, parameter estimation is performed by maximizing the log-likelihood through Expectation-Maximization (EM). Model selection in mixtures is considered essential for determining the optimal number of mixture components. Thus, we propose a model selection criterion for BKMM using minimum message length (MML). Initially, we validate the proposed model selection criteria for identifying the exact number of components using five medical disease diagnosis data. Additionally, to further assess its performance, we examined the model using various image datasets such as Alzheimer, lung tissue, gastrointestinal tract and object categorization. The results of MML are compared with seven different model selection criteria to examine the effectiveness of the proposed model. The experimental results demonstrate the effectiveness of the BKMM and MML.

This paper introduces the Author Dirichlet Multinomial Allocation Model with Generalized Distribution (ADMAGD), a novel probabilistic topic model aimed at enhancing the representation of complex author-topic relationships. ADMAGD integrates a generalized Dirichlet distribution into the Author-Topic Model (ATM), enabling more nuanced modeling of authors topical preferences and word distributions within topics. We also propose a Gibbs sampling method for efficient parameter estimation. We evaluate ADMAGD on the 20-newsgroups and NIPS datasets, demonstrating its effectiveness over existing models such as LDA and ATM in scenarios involving intricate authorship patterns and topic-word relationships. The proposed model offers significant advantages for tasks such as authorship identification, content recommendation, and analysis of document creation patterns.

Abstract Latent Dirichlet allocation (LDA) is one of the major models used for topic modelling. A number of models have been proposed extending the basic LDA model. There has also been interesting research to replace the Dirichlet prior of LDA with other pliable distributions like generalized Dirichlet, Beta‐Liouville and so forth. Owing to the proven efficiency of using generalized Dirichlet (GD) and Beta‐Liouville (BL) priors in topic models, we use these versions of topic models in our paper. Furthermore, to enhance the support of respective topics, we integrate mixture components which gives rise to generalized Dirichlet mixture allocation and Beta‐Liouville mixture allocation models respectively. In order to improve the modelling capabilities, we use variational inference method for estimating the parameters. Additionally, we also introduce an online variational approach to cater to specific applications involving streaming data. We evaluate our models based on its performance on applications related to text classification, image categorization and genome sequence classification using a supervised approach where the labels are used as an observed variable within the model.

The proliferation of Internet of Things (IoT) sensors in smart buildings has generated vast amounts of time series data, offering valuable insights when properly leveraged. We propose to use this data to identify abnormal behaviors and deviations in temporal data which will enable the detection of anomalies related to power consumption, control system failures, and sensor malfunctions. To achieve this, we propose a reconstruction-based anomaly detection framework utilizing autoencoders where we train the model on anomaly-free samples, minimizing the error between the original and reconstructed sequences. Then, by setting a threshold on the reconstruction error, abnormal sequences can be distinguished from the predominant regular patterns observed in the majority of the time windows. Moreover, to address the challenge of selecting a suitable autoencoder architecture, a Reinforcement Learning-based Neural Architecture Search (RLNAS) approach is employed to explore a manually defined search space and discover the best neural configuration by learning through trial and error. Experimental results on two custom anomaly detection datasets demonstrate competitive performance, showcasing the effectiveness of this approach in discovering effective architectures that may not be immediately apparent or intuitive.

Incomplete data is the most common but tricky problem for data-driven energy and building solutions. Due to sensor errors or communication failures, raw building data with missing data points are rarely satisfactory for fault detection and diagnosis (FDD) applications. In this paper, a new framework named a deep variational mixture of principal component analyzers (DV-MPPCA) is proposed to address the building FDD problem with incomplete data. DV-MPPCA is the combination of a variational autoencoder (VAE) model for data compression and a mixture of principal component analyzers (MPPCA) for density estimation. To construct an integrated framework comprising both VAE and MPPCA, we introduce a novel methodology that represents the algebraic model of MPPCA within the architecture of a neural network. This innovative architecture undergoes optimization through the minimization of a designated loss function. Subsequently, the refined and optimized framework is harnessed as an unsupervised fault detection model for a real-world air handling unit (AHU) system designed by the ASHRAE research project 1312 (RP-1312). Furthermore, by incorporating the modified evidence lower bound (ELBO) loss function within the VAE, the resulting DV-MPPCA framework exhibits exceptional performance when confronted with incomplete AHU datasets, even with high missing rates. Empirical findings substantiate the supremacy of DV-MPPCA over other contemporary classic and deep models. Impressively, even with a missing rate as modest as 10%, DV-MPPCA consistently delivers outstanding performance, achieving F1-scores of 98.10%, 93.50%, and 81.57% for the Summer, Winter, and Spring datasets, respectively.Note to Practitioners—This article was motivated by the fact that raw building data with missing data points are not qualitative enough for direct use in fault detection and diagnosis (FDD) applications. To resolve this problem, existing studies manipulated imputation algorithms in the preprocessing step to prepare the data for constructing FDD models and impute missing points in test instances during the online monitoring process. However, this step makes an overall FDD framework cumbersome. Therefore, instead of utilizing a data imputation method as a separately operating model, we adopt VAE in our framework to exclude the contribution of missing points during the offline modeling and online monitoring processes. This modification of VAE helps our framework be immune to incomplete data. For reproducibility and future improvement by other researchers, the complete source code of this study is provided in the following repository: https://github.com/viettra-xai/DV-MPPCA.

We introduce a mixture of mixtures of bounded asymmetric generalized Gaussian and uniform distributions. Based on this framework, we propose model-based classification and model-based clustering algorithms. We develop an objective function for the minimum message length (MML) model selection criterion to discover the optimal number of clusters for the unsupervised approach of our proposed model. Given the crucial attention received by Explainable AI (XAI) in recent years, we introduce a method to interpret the predictions obtained from the proposed model in both learning settings by defining their boundaries in terms of the crucial features. Integrating Explainability within our proposed algorithm increases the credibility of the algorithm’s predictions since it would be explainable to the user’s perspective through simple If-Then statements using a small binary decision tree. In this paper, the proposed algorithm proves its reliability and superiority to several state-of-the-art machine learning algorithms within the following real-world applications: fault detection and diagnosis (FDD) in chillers, occupancy estimation and categorization of residential energy consumers.

We consider the problem of anomaly detection in proportional data by investigating the Libby-Novick Beta-Liouville distribution, a novel distribution merging the salient characteristics of Liouville and Libby-Novick Beta distributions. Its main benefit, compared to the typical distributions dedicated to proportional data such as Dirichlet and Beta-Liouville, is its adaptability and explanatory power when dealing with this kind of data. Our goal is to exploit this appropriateness for modeling proportional data to achieve great performance in the anomaly detection task. First, we develop generative models, namely finite mixture models of Libby-Novick Beta-Liouville distributions. Then, we propose two discriminative techniques: Normality scores based on selecting the given distribution to approximate the softmax output vector of a deep classifier and an improved version of Support Vector Machine (SVM) by suggesting a feature mapping approach. We demonstrate the benefits of the presented approaches through a variety of experiments on both image and non-image datasets. The results demonstrate that the proposed anomaly detectors based on the Libby-Novick Beta-Liouville distribution outperform the classical distributions as well as the baseline techniques.

normalized vectors) modeling has become a promising research topic in various real-world applications (such as gene expression data analysis, document categorization, and gesture recognition). In this work, we propose a hierarchical nonparametric Bayesian model based on von Mises-Fisher (VMF) distributions for modeling spherical data that involve multiple groups, where each observation within a group is sampled from a VMF mixture model with an infinite number of components allowing them to be shared across groups. Our model is formulated by employing a hierarchical nonparametric Bayesian framework known as the hierarchical Pitman-Yor (HPY) process mixture model, which possesses a power-law nature over the distribution of the components and is particularly useful for data distributions with heavy tails and skewness. To learn the proposed HPY process mixture model with VMF distributions, we systematically develop a closed-form optimization algorithm based on variational Bayes (VB). The merits of the proposed hierarchical Bayesian nonparametric model for modeling grouped spherical data are demonstrated through experiments on both synthetic data and a real-world application about resting-state functional magnetic resonance imaging (fMRI) data analysis.

Mixture models are widely used in unsupervised machine learning applications where annotating a large amount of data is not feasible. They have succeeded in various real-world problems, including medical applications, human activity recognition, and anomaly detection. This paper proposes a fully Bayesian analysis of the multivariate McDonald's Beta mixture model (McDBMM) using Gibbs sampling method and Metropolis-Hastings to estimate parameters. In addition, we integrated a feature selection technique which simultaneously determines the most relevant features for our mixture model. This allows for the simultaneous selection of the most relevant features, improving the accuracy and efficiency of the unsupervised learning process. Our approach is evaluated on challenging applications, including lung cancer image analysis and human activity recognition. Experimental results indicate that our proposed method is an effective solution compared to the Gaussian mixture model (GMM).

In this paper, we propose an unsupervised learning algorithm for Natural Language Processing (NLP). In particular, we present a sentiment analysis solution using probabilistic models. We propose two new smoothed probabilistic-based models that incorporate the benefits of smoothing techniques and scaling word vectors to address sparseness and high-dimensionality challenges. We introduce the smoothed scaled Dirichlet and the smoothed shifted scaled Dirichlet mixture models, the learning approach for the mixture parameters, the clustering algorithms, and the sentiment analysis framework. We consider in our experiments different benchmarks of sentiment analysis, namely, Stanford Twitter sentiment (STD), Stanford sentiment gold standard (STS-Gold), SemEval2014 Task9, and Sentiment Strength Twitter (SentiStrength). The results are compared with the baselines and state-of-the-art (SOTA) models in the literature where our proposed approaches outperform all the other models.

This research explores the Beta-Liouville (BL) distribution as an alternative to the classic Dirichlet distribution in Dirichlet multinomial regression (DMR) and topic modelling. We examine the inherent limitations presented by the Dirichlet distribution in DMR, particularly in handling sparse data and complex correlations, then propose that the BL distribution, with its inherent flexibility and ability to handle higher-dimensional data, presents a substantial upgrade. We demonstrate the advantages of this new approach through a series of simulated and real-world datasets, highlighting improvements in model performance and inference quality. We discuss the impact of this change in the realm of topic modelling, as well as how the BL distribution enhances the interpretability and specificity of topics extracted. We anticipate our work to inspire further research and development in this field, aiming for continued refinement and enhancement of predictive modelling.

In speech emotion recognition (SER), using predefined features without considering their practical importance may lead to high dimensional datasets, including redundant and irrelevant information. Consequently, high-dimensional learning often results in decreasing model accuracy while increasing computational complexity. Our work underlines the importance of carefully considering and analyzing features in order to build efficient SER systems. We present a new supervised SER method based on an efficient feature engineering approach. We pay particular attention to the explainability of results to evaluate feature relevance and refine feature sets. This is performed iteratively through feature evaluation loop, using Shapley values to boost feature selection and improve overall framework performance. Our approach allows thus to balance the benefits between model performance and transparency. The proposed method outperforms human-level performance (HLP) and state-of-the-art machine learning methods in emotion recognition on the TESS dataset. The source code of this paper is publicly available at Iterative-Feature-Boosting-for-Explainable-Speech-Emotion-Recoanition.

While semantic segmentation has been extensively studied in the realm of regular perspective images, its application to fisheye images remains relatively unexplored. Existing literature on fisheye semantic segmentation mostly revolves around multi-task or multi-modal models, which are computationally intensive. This motivated us to assess the performance of current segmentation methods, specifically on fisheye images. Surprisingly, we discover that these methods do not yield satisfactory results when directly trained on fisheye datasets using a fully supervised approach. This can be attributed to the fact that the models are not designed to handle fisheye images, and the available fisheye datasets are not sufficiently large to effectively train complex models. To overcome these challenges, we propose a novel training method by employing Transfer Learning (TL) on existing semantic segmentation models that concentrate on a single task and modality. To achieve this, we investigate six different fine-tuning configurations using the WoodScape fisheye image segmentation dataset. Furthermore, we introduce a pre-training stage that learns from perspective images by applying a fisheye transformation before employing transfer learning. As a result, our proposed training pipeline demonstrates a remarkable 18.29% improvement in mean Intersection over Union (mIoU) compared to directly adopting the best existing segmentation methods for fish eye images.

The Dirichlet Compound Negative Multinomial (DCNM) distribution is a probabilistic model used to model multivariate count data. The presence of the Gamma function in the density probability distribution increases significantly the time and computation complexity, especially for high-dimensional datasets. In this paper, we use an alternative parametrization of DCNM using rising polynomials. Then, we use the Minorization-Maximization approach, to learn the parameter estimates of the DCNM mixture model, which brings higher numerical stability, lower tendency of poor local maxima, initialization independence, and avoids cumbersome derivative calculations. Moreover, we propose a new learning method to simultaneously update the mixture’s parameters and find the optimal number of clusters, using the Minimum Description Length Criterion. The proposed approach is validated in two interesting applications, namely, topic detection and scene recognition, where the results demonstrate the robustness and efficiency of our new learning approach.

Projection and Clustering are two main approaches in text mining. The goal of projection is to map the high-dimensional data into a lower-dimensional latent space, where the clustering task is to categorize data into different groups based on their similarity features. Several methods have been proposed to retrieve relevant information based on the co-occurrence of the data. However, the majority of works do not examine the joint effect of the projection and clustering, especially the effect of the prior distribution in the case of discrete data. For this purpose, in this paper, we propose a novel approach using a probabilistic clustering-projection framework where Dirichlet distribution, generalized Dirichlet distribution, and Beta-Liouville distribution are implemented as priors to study the impacts of prior knowledge in the perplexity of the model. Using a variational EM algorithm we estimate latent variables associated with clustering and projection parameters, iteratively updating the lower bound of the log-likelihood until convergence. Our experimental results demonstrate reduced perplexity using generalized Dirichlet and Beta-Liouville priors compared to Dirichlet. Moreover, we evaluate the model's performance in word projection and document clustering tasks, finding that both generalized Dirichlet and Beta-Liouville outperform Dirichlet in these domains.

Human action recognition (HAR) is a crucial research field that necessitates the implementation of advanced mathematical concepts to recognize human activities from sequences of observations. This paper presents a novel framework employing a mixture-based Hidden Markov Model (HMM) that capitalizes on the advantages of asymmetric modeling, bounded support, and robustness in sensor-based HAR. To accommodate variations in observations within each human activity class, we propose an asymmetric generalized Gaussian mixture model (AGGM) to model the emission probabilities. Subsequently, we propose incorporating the bounded asymmetric generalized Gaussian mixture model (BAGGM) to address the constraints inherent in real-life data. The parameters of the corresponding HMM are estimated using the Baum-Welch algorithm, and the most probable sequence of hidden states is inferred using the Viterbi algorithm. We validate our proposed framework using four datasets for human activities. Experimental results demonstrate that our proposed model outperforms all state-of-the-art HAR models that are HMM-based, thereby emphasizing the superiority of our proposed frameworks in HAR systems to understand behaviours, predict potential actions, and facilitate sports applications for the general population and independent living applications for vulnerable populations.

The widespread deployment of smart meters in residential settings has led to a wealth of high-resolution electrical power consumption data, providing the opportunity to discover valuable insights into energy consumption patterns. Mixture models are essential for revealing hidden patterns in data, enabling accurate insights and informed decision-making across diverse applications. In this paper, we introduce the mixture of mixtures of bounded asymmetric generalized Gaussian and Uniform distributions (BAGGUMM) and investigate its potential for characterizing residential energy users, thereby enhancing energy management applications, including demand response and energy efficiency programs. We investigate the potential for improved clustering efficacy by incorporating an inner mixture containing the Uniform distribution to enhance robustness against outliers. Additionally, we integrate a decision tree algorithm for model explainability to define pattern boundaries using if-then statements. We validate our proposed model using three real-life datasets. Additionally, The performance of BAGGUMM is compared against several state-of-the-art mixture models.

Nonparametric mixture models are a powerful and flexible approach to data clustering; they account for uncertainty by using Bayesian inference to obtain a posterior distribution over the model's parameters and a better fit to the data. Additionally, nonparametric mixture models can adaptively adjust the number of components to fit the data. Incorporating asymmetric generalized Gaussian distribution (AGGD) within the mixture framework extends the capabilities of the widely used Gaussian mixture model (GMM) by adding parameters that control the shape and the skewness of the per-component distribution, which enables the model to capture diverse and complex data patterns. Furthermore, we incorporate explainability within our proposed infinite asymmetric generalized Gaussian mixture model (IAGGMM) to provide interpretable insights into the clustering results, enhancing the model's practicality and transparency. This integration facilitates a deeper understanding of the underlying data structures and the rationale behind the model's decisions, fostering trust and promoting the adoption of our approach in various real-world scenarios. In this study, we explore the application of occupancy estimation for optimizing energy efficiency and facility management in smart buildings. Our approach demonstrates superior performance in modelling complex and asymmetric data distributions, resulting in improved accuracy and adaptability for occupancy level estimation. Therefore, we achieve the optimal trade-off between model complexity and accuracy.

In this work, we propose a novel anomaly detection approach in images based on normality scores using transformations. By applying various transformations to the input image such as rotation and flipping, we train a classifier to predict the transformation label applied to the images. Then, we represent the output of the classifier by a softmax vector. Thanks to the flexibility of multivariate Beta in fitting the data compared to other conventional distributions such as the Dirichlet distribution, we approximate the softmax vector by this general form of the Beta distribution to construct the normality scores. Moreover, we use the Maximum Likelihood to estimate the parameters of the proposed distribution. To show the power and the effectiveness of our approach, we conduct experiments of detecting anomalies in various public datasets. Furthermore, the proposed method is compared with state-of-the-art techniques and results demonstrate its superiority in terms of Area Under Receiver Operating characteristics (AUROC).

Learning statistical models successfully is both an essential and a challenging task for various pattern recognition and knowledge discovery applications. In particular, generative models such as finite and infinite mixture models have demonstrated to be efficient in terms of overall performance. In this paper, a robust framework based on an expectation propagation (EP) inference is developed to learn inverted Beta-Liouville (IBL) mixture models which is proper choice for positive data classification. Within the proposed EP learning method, the full posterior distribution is estimated accurately, the model complexity and all related parameters are evaluated simultaneously in a single optimization scheme. Extensive experiments using challenging real-world applications including recognition of facial expression, automatic human action categorization, and hand gesture recognition show the merit of our approach in terms of achieving better results than comparable techniques.

Abstract Short text clustering is one of the fundamental tasks in natural language processing. Different from traditional documents, short texts are ambiguous and sparse due to their short form and the lack of recurrence in word usage from one text to another, making it very challenging to apply conventional machine learning algorithms directly. In this article, we propose two novel approaches for short texts clustering: collapsed Gibbs sampling infinite generalized Dirichlet multinomial mixture model infinite GSGDMM) and collapsed Gibbs sampling infinite Beta‐Liouville multinomial mixture model (infinite GSBLMM). We adopt two flexible and practical priors to the multinomial distribution where in the first one the generalized Dirichlet distribution is integrated, while the second one is based on the Beta‐Liouville distribution. We evaluate the proposed approaches on two famous benchmark datasets, namely, Google News and Tweet. The experimental results demonstrate the effectiveness of our models compared to basic approaches that use Dirichlet priors. We further propose to improve the performance of our methods with an online clustering procedure. We also evaluate the performance of our methods for the outlier detection task, in which we achieve accurate results.

Abstract Currently, no classical clustering algorithm is efficient on its own. The predefined number of clusters required for their operation does not consistently produce satisfactory segmentation results. They exhibit cluster instability, are vulnerable to the local optimum trap, and are sensitive to noise and imaging artefacts. Most contributions designed to overcome these drawbacks incorporate prior knowledge such as cluster label information and statistic measures that demand minimal labelled training data. Although these approaches improve the segmentation accuracy, they tend to diminish the advantages of clustering algorithms over the supervised learning methods. This study proposes a shift from the use of a predefined number of clusters to a clustering tree‐based method for performance enhancement of classical clustering algorithms. The proposed method is a three‐stage algorithm. It begins with the extraction of low‐level features from a clustering tree. Clustering trees are sets of labelled clusters of an image at multiple clustering resolutions. The second stage extracts high‐level features by coupling the clustering tree to a single‐layer feedforward neural network. The third stage is the classification stage, where the basic model of a neural network extracts the tumour from a high‐level feature map. Because neither of the neural networks requires training, the proposed method is both fully unsupervised and fully automated and retains all its advantages over supervised methods. A performance evaluation using FLAIR MRI images of brain tumour patients from the BRATS2015 and BRATS2020 databases demonstrates significant performance enhancement over four classical clustering algorithms and two of the four proposed techniques were comparable to deep learning methods.

In this article, we propose a predictive model based on Beta-Liouville (BL) and inverted BL (IBL) mixture models for occupancy estimation in smart buildings. This model gives better results than point estimate methods, when the training data is small, because it is based on data-driven predictive distribution. The Liouville-based mixture models were chosen because of their flexibility in fitting symmetric and asymmetric distributions. However, the large number of parameters of BL and IBL increases the uncertainty in approximating the upper bound of the predictive distribution, hence we propose an optimization scheme in which reliability is investigated and verified. In addition, we extend our work presented by giving more details about the predictive model and by studying the occupancy estimation in smart buildings problems in depth. Indeed, different occupancy scenarios are considered to show the merits of our predictive framework. This article aims to address the problem of occupancy estimation with a focus on scenarios where small training data sets are available. By developing robust predictive models that can generalize well with limited data, this research seeks to facilitate the early adoption and practical application of occupancy models in various domains.

Graph neural networks (GNNs) have been extensively used in a wide variety of domains in recent years. Owing to their power in analyzing graph-structured data, they have become broadly popular in intelligent transportation systems (ITS) applications as well. Despite their widespread applications in different transportation domains, there is no comprehensive review of recent advancements and future research directions that covers all transportation areas. Accordingly, in this survey, for the first time, we provide an overview of GNN studies in the general domain of ITS. Unlike previous surveys, which have been limited to traffic forecasting problems, we explore how GNN frameworks have evolved for different ITS applications, including traffic forecasting, demand prediction, autonomous vehicles, intersection management, parking management, urban planning, and transportation safety. Also, we micro-categorize the studies based on their transportation application to identify domain-specific research directions, opportunities, and challenges, which have been missing in previous surveys. Moreover, we identify unique and undiscussed research opportunities and directions, which is the result of reviewing a wide range of transportation applications. The neglected role of edge and graph learning in ITS applications, developing multi-modal models, and exploiting the power of unsupervised and reinforcement learning methods for developing more powerful GNNs are some examples of such new discussions in this survey. Finally, we have identified popular baseline models and datasets in each transportation domain, which facilitate the development and evaluation of future GNN-based frameworks.

Bayesian non-negative matrix factorization (BNMF) has been widely used in different applications. In this article, we propose a novel BNMF technique dedicated to semibounded data where each entry of the observed matrix is supposed to follow an Inverted Beta distribution. The model has two parameter matrices with the same size as the observation matrix which we factorize into a product of excitation and basis matrices. Entries of the corresponding basis and excitation matrices follow a Gamma prior. To estimate the parameters of the model, variational Bayesian inference is used. A lower bound approximation for the objective function is used to find an analytically tractable solution for the model. An online extension of the algorithm is also proposed for more scalability and to adapt to streaming data. The model is evaluated on five different applications: part-based decomposition, collaborative filtering, market basket analysis, transactions prediction and items classification, topic mining, and graph embedding on biomedical networks.

Probabilistic topic models are considered as an effective framework for text analysis that uncovers the main topics in an unlabeled set of documents. However, the inferred topics by traditional topic models are often unclear and not easy to interpret because they do not account for semantic structures in language. Recently, a number of topic modeling approaches tend to leverage domain knowledge to enhance the quality of the learned topics, but they still assume a multinomial or Gaussian document likelihood in the Euclidean space, which often results in information loss and poor performance. In this article, we propose a Bayesian embedded spherical topic model (ESTM) that combines both knowledge graph and word embeddings in a non-Euclidean curved space, the hypersphere, for better topic interpretability and discriminative text representations. Extensive experimental results show that our proposed model successfully uncovers interpretable topics and learns high-quality text representations useful for common natural language processing (NLP) tasks across multiple benchmark datasets.

In recent years, clustering methods based on deep generative models have received great attention in various unsupervised applications, due to their capabilities for learning promising latent embeddings from original data. This article proposes a novel clustering method based on variational autoencoder (VAE) with spherical latent embeddings. The merits of our clustering method can be summarized as follows. First, instead of considering the Gaussian mixture model (GMM) as the prior over latent space as in a variety of existing VAE-based deep clustering methods, the von Mises-Fisher mixture model prior is deployed in our method, leading to spherical latent embeddings that can explicitly control the balance between the capacity of decoder and the utilization of latent embedding in a principled way. Second, a dual VAE structure is leveraged to impose the reconstruction constraint for the latent embedding and its corresponding noise counterpart, which embeds the input data into a hyperspherical latent space for clustering. Third, an augmented loss function is proposed to enhance the robustness of our model, which results in a self-supervised manner through the mutual guidance between the original data and the augmented ones. The effectiveness of the proposed deep generative clustering method is validated through comparisons with state-of-the-art deep clustering methods on benchmark datasets. The source code of the proposed model is available at https://github.com/fwt-team/DSVAE.

Recent years have witnessed the emergence of new and more sophisticated malware targeting the Internet of Things. Moreover, the public release of the source code of popular malware families such as Mirai has spawned diverse variants, making it harder to disambiguate their ownership, lineage, and correct label. Such a rapidly evolving landscape makes it also harder to deploy and generalize effective learning models against retired, updated, and/or new threat campaigns. In this paper, we present EVOLIoT, a novel approach aiming at combating "concept drift" and the limitations of inter-family IoT malware classification by detecting drifting IoT malware families and understanding their diverse evolutionary trajectories. We introduce a robust and effective contrastive method that learns and compares semantically meaningful representations of IoT malware binaries and codes without the need for expensive target labels. We find that the evolution of IoT binaries can be used as an augmentation strategy to learn effective representations to contrast (dis)similar variant pairs. We discuss the impact and findings of our analysis and present several evaluation studies to highlight the tangled relationships of IoT malware, as well as the efficiency of our contrastively learned feature vectors in preserving semantics and reducing out-of-vocabulary size in cross-architecture IoT malware binaries.

Recently, video compression gained a large focus among computer vision problems in media technologies. Using state of the art video compression methods, videos can be transmitted in a better quality requiring less bandwidth and memory. The advent of neural network-based video compression methods remarkably promoted video coding performance. In this paper, a video compression method is presented based on Recurrent Neural Network (RNN). The method includes an encoder, a middle module, and a decoder. Binarizer is utilized in the middle module to achieve better quantization performance. In encoder and decoder modules, long short-term memory (LSTM) units are used to keep the valuable information and eliminate unnecessary ones to iteratively reduce the quality loss of reconstructed video. This method reduces the complexity of neural network-based compression schemes and encodes the videos with less quality loss. The proposed method is evaluated using peak signal-to-noise ratio (PSNR), video multimethod assessment fusion (VMAF), and structural similarity index measure (SSIM) quality metrics. The proposed method is applied to two different public video compression datasets and the results show that the method outperforms existing standard video encoding schemes such as H.264 and H.265.

Site preparation by mounding is a commonly used silvicultural treatment that improves tree growth conditions by mechanically creating planting microsites called mounds. Following site preparation, the next critical step is to count the number of mounds, which provides forest managers with a precise estimate of the number of seedlings required for a given plantation block. Counting the number of mounds is generally conducted through manual field surveys by forestry workers, which is costly and prone to errors, especially for large areas. To address this issue, we present a novel framework exploiting advances in Unmanned Aerial Vehicle (UAV) imaging and computer vision to estimate the number of mounds on a planting block accurately. The proposed framework comprises two main components. First, we exploit a visual recognition method based on a deep learning algorithm for multiple object detection by segmentation. This enables a preliminary counting of visible mounds, as well as other frequently seen objects (e.g., trees, debris, accumulation of water), to be used to characterize the planting block. Second, since visual recognition could be limited by several perturbation factors (e.g., mound erosion, occlusion), we employ a machine learning estimation function to predict the final number of mounds based on the local block properties extracted in the first stage. We evaluate the proposed framework on a new UAV dataset representing numerous planting blocks with varying features. The proposed method outperformed manual counting methods in terms of relative counting precision, indicating that it has the potential to be advantageous and efficient under challenging situations.

We present CNN-n-GRU, a new end-to-end (E2E) architecture built of an n-layer convolutional neural network (CNN) followed sequentially by an n-layer Gated Recurrent Unit (GRU) for speech emotion recognition. CNNs and RNNs both exhibited promising outcomes when fed raw waveform voice inputs. This inspired our idea to combine them into a single model to maximise their potential. Instead of using handcrafted features or spectrograms, we train CNNs to recognise low-level speech representations from raw waveform, which allows the network to capture relevant narrow-band emotion characteristics. On the other hand, RNNs (GRUs in our case) can learn temporal characteristics, allowing the network to better capture the signal’s time-distributed features. Because a CNN can generate multiple levels of representation abstraction, we exploit early layers to extract high-level features, then to supply the appropriate input to subsequent RNN layers in order to aggregate long-term dependencies. By taking advantage of both CNNs and GRUs in a single model, the proposed architecture has important advantages over other models from the literature. The proposed model was evaluated using the TESS dataset and compared to state-of-the-art methods. Our experimental results demonstrate that the proposed model is more accurate than traditional classification approaches for speech emotion recognition.

Probabilistic topic models are powerful techniques for analyzing and understanding large collections of text documents to learn meaningful patterns of words. Their supervised extensions also capture topics conditioned on the response metadata associated with each document such as user rating. However, inferring such information from data often comes at the detriment of topics quality, leading to uninterpreted and meaningless topics. In this paper, we propose a novel Supervised-Embedded Spherical Topic Model (S-ESTM) that balances two goals: interpretable and coherent topics explaining the data and accurate prediction of the associated response values. Our model combines word embeddings and knowledge graph embeddings to effectively encode the semantic information of text and the related background knowledge to guide the inference of supervised topics. In S-ESTM, document constituents are drawn as points on spherical manifolds along with topics using the von Mises-Fisher distribution. Efficient variational inference methods for posterior approximation and latent parameter estimation are derived and various empirical studies on real-world datasets are also provided. Our experiments demonstrate that our model can discover discriminative and coherent topical patterns associated with regression tasks, while achieving improved prediction quality.

We introduce Knowledge-enhanced Spherical Representation Learning (K-SRL), a generative probabilistic model of text documents that combines word embeddings and knowledge graph embeddings to effectively encode the semantic information of text and the related background knowledge into a low-dimensional representation. More specifically, the proposed model represents each text document as a combination of both words and entities linked to an external large knowledge graph and models them as points on the unit hypersphere using the von Mises-Fisher distribution. Furthermore, we develop an efficient variational Bayesian inference algorithm to learn unsupervised text embeddings in the spherical space. Experimental results on multiple benchmark datasets demonstrate that our model outperforms existing probabilistic models on common text classification tasks, including text categorization and sentiment analysis.

Abstract In this article, first, we propose a novel unsupervised learning method based on a hierarchical Dirichlet process mixture of shifted‐scaled Dirichlet (SSD) distributions. Second, we extend it to a hierarchical Pitman–Yor process mixture of SSD distributions. The goal is to find a model that properly fits complex real‐world data. Our models are based on SSD distributions that are more flexible than Dirichlet distribution in fitting proportional data. Simultaneous data fitting (parameter estimate) and model selection (model complexity determination) are possible with the suggested methods. We applied batch and online variational inference for learning the models. The online setting allows us to feed our models with large‐scale streaming data. The effectiveness of our proposed models is evaluated by four realistic and challenging applications, namely, spam email detection, texture clustering, traffic sign detection, and vehicle detection. Experimental results demonstrate the potential of our models to fit proportional data.

This paper aims at proposing an unsupervised hierarchical nonparametric Bayesian framework for modeling axial data (i.e., observations are axes of direction) that can be partitioned into multiple groups, where each observation within a group is sampled from a mixture of Watson distributions with an infinite number of components that are allowed to be shared across different groups. First, we propose a hierarchical nonparametric Bayesian model for modeling grouped axial data based on the hierarchical Pitman-Yor process mixture model of Watson distributions. Then, we demonstrate that by setting the discount parameters of the proposed model to 0, another hierarchical nonparametric Bayesian model based on hierarchical Dirichlet process can be derived for modeling axial data. To learn the proposed models, we systematically develop a closed-form optimization algorithm based on the collapsed variational Bayes (CVB) inference. Furthermore, to ensure the convergence of the proposed learning algorithm, an annealing mechanism is introduced to the framework of CVB inference, leading to an averaged collapsed variational Bayes inference strategy. The merits of the proposed models for modeling grouped axial data are demonstrated through experiments on both synthetic data and real-world applications involving gene expression data clustering and depth image analysis.

In topic models, collections are organized as documents where they arise as mixtures over latent clusters called topics. A topic is a distribution over the vocabulary. In large-scale applications, parametric or finite topic mixture models such as LDA (latent Dirichlet allocation) and its variants are very restrictive in performance due to their reduced hypothesis space. In this article, we address the problem related to model selection and sharing ability of topics across multiple documents in standard parametric topic models. We propose as an alternative a BNP (Bayesian nonparametric) topic model where the HDP (hierarchical Dirichlet process) prior models documents topic mixtures through their multinomials on infinite simplex. We, therefore, propose asymmetric BL (Beta-Liouville) as a diffuse base measure at the corpus level DP (Dirichlet process) over a measurable space. This step illustrates the highly heterogeneous structure in the set of all topics that describes the corpus probability measure. For consistency in posterior inference and predictive distributions, we efficiently characterize random probability measures whose limits are the global and local DPs to approximate the HDP from the stick-breaking formulation with the GEM (Griffiths-Engen-McCloskey) random variables. Due to the diffuse measure with the BL prior as conjugate to the count data distribution, we obtain an improved version of the standard HDP that is usually based on symmetric Dirichlet (Dir). In addition, to improve coordinate ascent framework while taking advantage of its deterministic nature, our model implements an online optimization method based on stochastic, at document level, variational inference to accommodate fast topic learning when processing large collections of text documents with natural gradient. The high value in the predictive likelihood per document obtained when compared to the performance of its competitors is also consistent with the robustness of our fully asymmetric BL-based HDP. While insuring the predictive accuracy of the model using the probability of the held-out documents, we also added a combination of metrics such as the topic coherence and topic diversity to improve the quality and interpretability of the topics discovered. We also compared the performance of our model using these metrics against the standard symmetric LDA. We show that online HDP-LBLA (Latent BL Allocation)’s performance is the asymptote for parametric topic models. The accuracy in the results (improved predictive distributions of the held out) is a product of the model’s ability to efficiently characterize dependency between documents (topic correlation) as now they can easily share topics, resulting in a much robust and realistic compression algorithm for information modeling.

One of the pillar generative machine learning approaches in time series data study and analysis is the hidden Markov model (HMM). Early research focused on the speech recognition application of the model with later expansion into numerous fields, including video classification, action recognition, and text translation. The recently developed generalized Dirichlet HMMs have proven efficient in proportional sequential data modeling. As such, we focus on investigating a maximum a posteriori (MAP) framework for the inference of its parameters. The proposed approach differs from the widely deployed Baum-Welch through the placement of priors that regularizes the estimation process. A feature selection paradigm is also integrated simultaneously in the algorithm. For validation, we apply our proposed approach in the classification of dynamic textures and the recognition of infrared actions.

Clustering is a fundamental problem that frequently arises in many fields, such as pattern recognition, data mining, and machine learning. Although various clustering algorithms have been developed in the past, traditional clustering algorithms with shallow structures cannot excavate the interdependence of complex data features in latent space. Recently, deep generative models, such as autoencoder (AE), variational AE (VAE), and generative adversarial network (GAN), have achieved remarkable success in many unsupervised applications thanks to their capabilities for learning promising latent representations from original data. In this work, first we propose a novel clustering approach based on both Wasserstein GAN with gradient penalty (WGAN-GP) and VAE with a Gaussian mixture prior. By combining the WGAN-GP with VAE, the generator of WGAN-GP is formulated by drawing samples from the probabilistic decoder of VAE. Moreover, to provide more robust clustering and generation performance when outliers are encountered in data, a variant of the proposed deep generative model is developed based on a Student's-t mixture prior. The effectiveness of our deep generative models is validated though experiments on both clustering analysis and samples generation. Through the comparison with other state-of-art clustering approaches based on deep generative models, the proposed approach can provide more stable training of the model, improve the accuracy of clustering, and generate realistic samples.

Clustering frequency vectors is a challenging task on large data sets considering its high dimensionality and sparsity nature. Generalized Dirichlet multinomial (GDM) distribution is a competitive generative model for count data in terms of accuracy, yet its parameters estimation process is slow. The exponential-family approximation of the multivariate Polya distribution has shown to be efficient to train and cluster data directly, without dimensionality reduction. In this article, we derive an exponential-family approximation to the GDM distributions, and we call it (EGDM). A mixture model is developed based on the new member of the exponential-family of distributions, and its parameters are learned through the deterministic annealing expectation-maximization (DAEM) approach as a new clustering algorithm for count data. Moreover, we propose to estimate the optimal number of EGDM mixture components based on the minimum message length (MML) criterion. We have conducted a set of empirical experiments, concerning text, image, and video clustering, to evaluate the proposed approach performance. Results show that the new model attains a superior performance, and it is considerably faster than the corresponding method for GDM distributions.

Ransomware attacks are among the most disruptive cyber threats, causing significant financial losses while impacting productivity, accessibility, and reputation. Despite their end goals (encryption/locking), ransomware are often designed to evade detection by executing a series of pre-attack API calls, namely “paranoia” activities, for determining a suitable execution environment. In this work, we present a first-of-a-kind effort to utilize such paranoia activities for characterizing ransomware distinguishable behaviors. To this end, we draw-upon more than 3K samples from recent/prominent ransomware families to fingerprint their uniquely leveraged paranoia activities. Specifically, by leveraging techniques rooted in Natural Language Processing (NLP) such as Occurrence of Words (OoW), we model ransomware-generated evasion API calls while tailoring various machine and deep learning algorithms to perform ransomware classification. The thoroughly conducted evaluations demonstrate the effectiveness of the implemented approach, with the Random Forest (RF) and OoW techniques producing an optimal classification accuracy (94.92%). The insights/findings from this work not only shed light on contemporary ransomware-specific evasion methods, but also (i) indicates that such tactics could be employed effectively as features for ransomware family attribution while (ii) laying the foundation for implementing proactive and portable countermeasures for further ransomware attack detection/mitigation by solely utilizing ransomware-generated paranoia activities.

In this paper, we propose hierarchical Pitman-Yor process mixtures of multivariate Beta distributions and learn this novel clustering method by online variational inference. The flexibility of this mixture model and its non-parametric hierarchical structure help in fitting our data. Also, the model complexity and model parameters are estimated simultaneously. We apply our proposed model to real medical applications. Our motivation is that labelling healthcare data is sensitive and expensive. Also, interpretability and evidence-based decision-making are some basic needs of medicine. These conditions led us to focus on clustering as it doesn’t need labelling. Another driving reason is that the amount of publicly available data in medicine is less compared to other fields due to the confidential regulations. To evaluate our proposed model, we compare its performance with other similar alternatives. The experimental results indicate the potential of our proposed model.

Nowadays, there is considerable and growing interest in applying accurate analysis tools to obtain meaningful information and extract knowledge from a huge amount of data. In this sense, unsupervised algorithms and clustering techniques have gained an increasing interest. These methods are helpful specifically when data annotation is time-consuming and costly. In this paper, we propose a new clustering method based on a Dirichlet process mixture of multivariate Beta distributions. To learn this novel Bayesian nonparametric model, we applied stochastic expectation propagation inference framework. This framework is able to define the model complexity and estimate the model’s parameters simultaneously. To demonstrate the efficiency of our model, we perform an experimental analysis using three real applications, breast, lung and colon histopathological tissue analysis. Our goal is to show that our algorithm could be considered as a machine learning framework in computer-assisted diagnosis and play the role of a complementary opinion to help the pathologists in making decisions with more accuracy.

Traditional object tracking approaches solve the problem as a supervised learning task. Due to variations in video frames over time, the appearance model of the object is updated for each video frame. Using density or mixture based model to represent target object is a common technique in Bayesian approaches. However, in histogram based representation where the data is proportional, Gaussian distribution is not the proper choice. In this paper, we propose a framework for feature representation on the simplex manifold for proportional data utilizing the histogram representation of the target object in initial frame. Extracted density features are concatenated with histogram based features to get a better representation of the target object. A set of parameter vectors determine the appearance features of the target object in the subsequent frames. To precisely model proportional data, we employ five appropriate distributions; namely, Dirichlet, Generalized Dirichlet, Scaled Dirichlet, Beta-Liouville and Inverted Dirichlet distributions. We present experimental results to show that the proposed method, which incorporates both generative and discriminative features, has an improved tracking performance.

Variational inference approaches are gaining more and more success and are applied in various data mining applications. In this work, we present a variational learning framework for the Shifted Scaled Dirichlet finite mixture model. Furthermore, we introduce component splitting based on Entropy approximation to estimate the number of components in our model along with the parameters of the distribution. We test the performance of this model on two real-life datasets, both related to human activity recognition.

In this paper, we propose a clustering-based online news topic detection and tracking (TDT) approach based on hierarchical Bayesian nonparametric framework that allows topics to be shared across different news stories in a corpus. Our approach is formulated using the hierarchical Pitman-Yor process mixture model with the inverted Beta-Liouville (IBL) distribution as its component density, which has shown superior performance in modeling text data than the widely used Gaussian distribution. Moreover, we theoretically develop a convergence-guaranteed online learning algorithm that can effectively learn the proposed TDT model from a stream of news stories based on varational Bayes. The merits of our TDT approach are illustrated by comparing it with other well-defined clustering-based TDT approaches on different news data sets.

Abstract The multinomial distribution has been widely used to model count data. To increase clustering efficiency, we use an approximation to the Fisher scoring algorithm, which is more robust regarding the choice of initial parameter values. Then, we use a novel approach to estimate the optimal number of components, based on minimum message length criterion. Moreover, we consider a generalization of the multinomial model obtained by introducing the Dirichlet as prior, yielding the Dirichlet Compound Multinomial (DCM). Even though DCM can address the burstiness phenomenon of count data, the presence of Gamma function in its density function usually leads to undesired complications. In this article, we use two alternative representations of DCM distribution to perform clustering based on finite mixture models, where the mixture parameters are estimated using the minorization–maximization framework. To evaluate and compare the performance of our proposed models, we have considered three challenging real‐world applications that involve high‐dimensional count vectors, namely, sentiment analysis, facial expression recognition, and human action recognition. The results show that the proposed algorithms increase the clustering efficiency of their respective models remarkably, and the best results are achieved by the second parametrization of DCM, which can accommodate over‐dispersed count data.

Abstract Bounded support Gaussian mixture model (BGMM) has been proposed for data modelling as an alternative to unbounded support mixture models for the cases when the data lies in bounded support. In this paper, we propose applications of multivariate BGMM in data clustering for more insightful analysis of the model. We also propose minimum message length (MML) criterion for model selection in data clustering using multivariate BGMM. The presented model is applied to data clustering in several speech (TSP and Spoken Digits) and image databases (MNIST and Fashion MNIST). We also propose the application of BGMM in code‐book generation at feature extraction phase. Inspired by the success of bag of visual words approach in computer vision, it is also introduced in speech data representation and validated through experiments presented in this paper. For validation of model selection criterion, MML is applied to different medical, speech and image datasets. Experimental results obtained during the model selection through MML are further compared with seven different model selection criteria. The results presented in the paper demonstrate the effectiveness of BGMM for clustering speech and image databases, code‐book generation through clustering for feature representation and model selection.

Abstract Technological advances led to the generation of large scale complex data. Thus, extraction and retrieval of information to automatically discover latent pattern have been largely studied in the various domains of science and technology. Consequently, machine learning experienced tremendous development and various statistical approaches have been suggested. In particular, data clustering has received a lot of attention. Finite mixture models have been revealed to be one of the flexible and popular approaches in data clustering. Considering mixture models, three crucial aspects should be addressed. The first issue is choosing a distribution which is flexible enough to fit the data. In this paper, a model based on multivariate Beta distributions is proposed. The two other challenges in mixture models are estimation of model's parameters and model complexity. To tackle these challenges, variational inference techniques demonstrated considerable robustness. In this paper, two methods are studied, namely, batch and online variational inferences and the models are evaluated on four medical applications including image segmentation of colorectal cancer, multi‐class colon tissue analysis, digital imaging in skin lesion diagnosis and computer aid detection of Malaria.

Challenges in realtime installation of surveillance systems is an active area of research, especially with the use of adaptable machine learning techniques. In this paper, we propose the use of variational learning of Beta-Liouville (BL) hidden Markov models (HMM) for AR in an online setup. This proposed incremental framework enables continuous adjustment of the system for better modelling. We evaluate the proposed model on the visible IOSB dataset to validate the framework.

In this paper, we propose a novel framework for replacing advertisement contents in soccer videos with an automatic way by using deep learning strategies. We begin by applying UNET (an image segmentation convolutional neural network technique) for content segmentation and detection. Subsequently, after reconstructing the segmented content in the video frames (considering the apparent loss in detection), we will replace the unwanted content by new one using a homography mapping procedure. Furthermore, the replacement key points in each frame will be tracked into the next frames considering the camera zoom-in and zoom-out controlling. Since the movement of objects in video can disrupt the alignment between frames and correspondingly make the homography matrix calculation erroneous, we use Mask R-CNN to mask and remove the moving objects from the scene. Such framework is denominated as REP-Model which stands for a replacing model.

Data mining techniques have been successfully utilized in different applications of significant fields, including medical research. With the wealth of data available within health-care systems, there is a lack of practical analysis tools to discover hidden relationships and trends in data. Among all statistical frameworks, regression has been proven to be one of the most potent tools in prediction. The complexity of medical data that is unfavorable for most models is a considerable challenge in prediction. The ability of a model to perform accurately and efficiently in disease diagnosis is essential. Thus, a model must be selected to fit the data well, such that the learning from previous data is most efficient, and the diagnosis of the disease is highly accurate. In this work, a bivariate Beta regression model has been proposed, which is based on a flexible bivariate Beta distribution with three shape parameters. Then, the performance of this algorithm is evaluated in terms of the accuracy of the prediction and compared to a similar regression model. The accuracy of model performance depends on the nature and complexity of the dataset. The results in this paper show the merits of our work.

In this paper, we propose a nonparametric Bayesian approach based on a mixture of the exponential-approximation to the Dirichlet Compound Multinomial (EDCM), which has been shown to be very flexible and efficient for high-dimensional sparse count data modeling. Our approach can be viewed as the first attempt to extend the finite EDCM mixture model to the infinite case, which allows the elicitation of prior belief about the parameters and the number of clusters through Markov Chain Monte Carlo sampling within Metropolis-Hastings. This extension provides a natural representation of uncertainty regarding the challenging problem of model selection, where the problems of overfitting and underfitting are prevented due to the nature of the nonparametric Bayesian approach. The resulting statistical model is applied to a challenging application, namely human interaction recognition.

Computer network technology is developing quickly, and the advancement of internet techniques is growing faster. Furthermore, people and companies have became more aware of the importance of network security. To protect the network from different attacks, it is necessary to instantly detect intrusions with significantly low False-Positive Rates (FPRs). Many Anomaly-based Detection Systems (ADS) have been proposed in the past. The performance of these systems depends on many factors such as features selection or extraction, missing or inaccurate records in data, over-fitting, under-fitting, high bias, and variance in data. Thus, it is important to take all these factors into account. Recently, a novel Geometric Area Analysis (GAA) technique based on Trapezoidal Area Estimation (TAE) has been proposed based on the Beta Mixture Model (BMM). In this work, we evaluate GAA and TAE techniques using other flexible mixture models based on inverted Beta, generalized Dirichlet, and generalized inverted Dirichlet distributions. The evaluation of this work is performed on two datasets, namely the NSL-KDD and UNSW-NB15. The results have shown the efficiency of the proposed ADS demonstrated by obtaining high accuracy and low false-positive rates in all attack types.

Detecting a wide variety of malicious activities in the cyber domain has become increasingly crucial, and thus numerous intrusion detection systems (IDS) have been developed over the past years. Recently, lightweight ADS has been proposed based on a Dirichlet mixture model which is integrated with a decision-making method based on a lower-upper Interquartile range. However, the high-dimensional network data presents a challenge in such systems, as irrelevant features may decline the model performance. This work proposes a hybrid anomaly detection system that uses combinations of different feature selection techniques with a suitable mixture model to provide high accuracy in detecting each attack type. The performance of the proposed framework is evaluated on two well-known datasets, namely, the NSL-KDD and UNSW-NB15. The results have shown the efficiency of the proposed ADS demonstrated by archiving high accuracy and low false-positive rates in all attack types.

In this paper, we propose a probabilistic feature mapping technique for proportional data where the features are learned in an unsupervised manner. In computer vision, count data are very relevant and we can represent that as proportion which can be modeled with distributions for proportional data. We are interested in computing probabilistic features using scaled Dirichlet and shifted scaled Dirichlet distributions and map with discriminative features to improve classification accuracy. The parameters of these distributions is updated using maximum likelihood approach. Our experimental results show that the proposed feature mapping technique improves the accuracy of baseline kernels of support vector machines (SVM) for image classification tasks.

Hidden Markov models (HMM) have recently risen as a key generative machine learning approach for time series data study and analysis. While early works focused only on applying HMMs for speech recognition, HMMs are now prominent in various fields such as video classification and genomics. In this paper, we develop a Maximum A Posteriori framework for learning the Generalized Dirichlet HMMs that have been proposed recently as an efficient way for modeling sequential proportional data. In contrast to the conventional Baum Welch algorithm, commonly used for learning HMMs, the proposed algorithm places priors for the learning of the desired parameters; hence, regularizing the estimation process. We validate our proposed approach on a challenging video processing application; namely, dynamic texture classification.

In the last decades, the development of generative/discriminative approaches for classifying different kinds of data has attracted scholars' attention. Considering the strengths and weaknesses of both approaches, several hybrid learning approaches which combined the desirable properties of both have been developed. Our goal in this paper is to combine Support Vector Machines (SVMs), as a powerful classification tool, and Bernoulli mixture model in order to classify binary data. We propose using Bernoulli mixture model for generating probabilistic kernels for SVM based on information divergence. These kernels make intelligent use of unlabeled binary data to achieve good data discrimination. We demonstrate the merits of the proposed hybrid learning approach for the problem of classifying binary and texture images.

Hidden Markov models (HMM) have recently risen as a key generative machine learning approach for time series data study and analysis. While early works focused only on applying HMMs for speech recognition, HMMs are now prominent in various fields such as stock market forecasting, video classification, and genomics. In this paper, we develop a Maximum A Posteriori (MAP) framework for learning the Dirichlet and Beta-Liouville HMMs that have been proposed recently as an efficient way for modeling sequential proportional data. In contrast to the conventional Baum Welch algorithm, commonly used for learning HMMs, the proposed algorithm places priors for the learning of the desired parameters; hence, regularizing the estimation process. We validate our proposed approach on two challenging real applications; namely, dynamic texture classification and infrared action recognition.

This paper presents a variational learning framework for the infinite generalized Gaussian mixture (IGGM) model. The generalized Gaussian distribution (GGD) has a proven capability in modeling complex multidimensional data due to the flexibility of its shape parameter. Infinite model addresses the model selection problem; i.e., determination of the number of clusters without recourse to the classical selection criteria such that the number of mixture components increases automatically to best model available data accordingly. We also incorporate feature selection to consider the features that are most appropriate in constructing an approximate model in terms of clustering accuracy. Experimental results on a medical application and image categorization show the effectiveness of the proposed algorithm.

Along with the exponential growth of online video creation platforms such as Tik Tok and Instagram, state of the art research involving quick and effective action/gesture recognition applications remains crucial. This work addresses the challenge of classifying such short video clips, using a domain-specific feature design approach, capable of performing significantly well using little training data. The method is based on Gunner Farneback dense optical flow (GF-OF) estimation strategy, Gaussian mixture models, and information divergence. We first aim to obtain accurate 3D representations of the human movements/actions through clustering the results given by GF-OF using K-means method of vector quantization. We then proceed by representing the result of one instance of each action by a Gaussian mixture model. Furthermore, using Kullback-Leibler divergence (KL-divergence), we attempt to find similarities between the trained actions and the ones in the test videos. Classification is done by matching each testing video to the trained action with the highest similarity (lowest KL-divergence). We have performed experiments on the KTH and Weizmann Human Action datasets, and the results reveal the discriminative nature of our proposed methodology in comparison with other state of the art techniques.

Positive vectors appear naturally in many applications, yet heir modeling has not been adequately addressed in the past. Inverted Dirichlet offers a good representation and modeling of positive non-Gaussian vectors, and its generalizations offer even more practical and flexible alternatives. In this paper, we propose three regression models based on flexible distributions for semi-bounded data, namely, inverted Dirichlet, inverted generalize Dirichlet, and inverted Beta-Liouville. The efficiency of these models is tested via real-world applications, including software defects prediction, age estimation, spam filtering, and disease diagnosis. Our results show that the three proposed regression models outperform other commonly used regression approaches.

In this paper, we propose a distribution based feature mapping technique to improve the baseline accuracy of SVM kernels in different computer vision tasks. The proposed technique is based on learning parameters from the data and use that parameters to make inference from new data. The learned parameters can be thought of as prior knowledge about the data representation. Utilizing such prior knowledge about the data distribution increases the discriminative power of the classifier. Our proposed feature mapping technique is based on inverted Dirichlet, generalized inverted Dirichlet and inverted Beta Liouville distributions. These distributions are efficient in modelling semi-bounded data which are prevalent in computer vision problems. Our experimental results demonstrate the effectiveness of the proposed method in texture recognition, natural scene recognition and human action recognition in videos.

This issue of New Methods and Approaches in Decision and Control of Intelligent and Cyber Systems contains a collection of extended versions of best presented papers in CoDIT 2019 conference and ad...

Data mining techniques have been successfully utilized in different applications of significant fields, including medical research. With the wealth of data available within the health-care systems, there is a lack of practical analysis tools to discover hidden relationships and trends in data. The complexity of medical data that is unfavorable for most models is a considerable challenge in prediction. The ability of a model to perform accurately and efficiently in disease diagnosis is extremely significant. Thus, the model must be selected to fit the data better, such that the learning from previous data is most efficient, and the diagnosis of the disease is highly accurate. This work is motivated by the limited number of regression analysis tools for multivariate counts in the literature. We propose two regression models for count data based on flexible distributions, namely, the multinomial Beta-Liouville and multinomial scaled Dirichlet, and evaluated the proposed models in the problem of disease diagnosis. The performance is evaluated based on the accuracy of the prediction which depends on the nature and complexity of the dataset. Our results show the efficiency of the two proposed regression models where the prediction performance of both models is competitive to other previously used regression models for count data and to the best results in the literature.

Abstract This paper proposes an unsupervised algorithm for learning a finite mixture of scaled Dirichlet distributions. Parameters estimation is based on the maximum likelihood approach, and the minimum message length (MML) criterion is proposed for selecting the optimal number of components. This research work is motivated by the flexibility issues of the Dirichlet distribution, the widely used model for multivariate proportional data, which has prompted a number of scholars to search for generalizations of the Dirichlet. By introducing the extra parameters of the scaled Dirichlet, several useful statistical models could be obtained. Experimental results are presented using both synthetic and real datasets. Moreover, challenging real‐world applications are empirically investigated to evaluate the efficiency of our proposed statistical framework.

The visible spectrum is the most widely used modality for video media. Nonetheless, it is highly dependent on the lighting conditions. Hence, infrared (IR) imaging lower light sensitivity characterisation presents the untapped potential for robust automatic recognition systems. This is applicable to many applications including IR action recognition (AR), which is a relatively young field in IR. As such, in this study, the authors tackle IR and multimodal AR with the proposed utilisation of variational learning of Beta‐Liouville (BL) hidden Markov models (HMMs). Furthermore, to the best of the authors' knowledge, this is the first evaluation of the BL HMM in visible AR and in multimodal fusion for AR. They present the results of the proposed model on the infrared action recognition and the IOSB datasets. Experimental results demonstrate promising outcomes. The importance of using IR and multispectral fusion in AR is also highlighted by the results.

Mixture models are broadly applied in image processing domains. Related existing challenges include failure to approximate exact data shapes, estimate correct number of components, and ignore irrelevant features. In this study, the authors develop a statistical self‐refinement framework for the background subtraction task by using Dirichlet Process‐based asymmetric Gaussian mixture model. The parameters of this model are learned using variational inference methods. They also incorporate feature selection simultaneously within the framework to avoid noisy influence from uninformative features. To validate the proposed framework, they report their results on background subtraction tasks on 8 different datasets for infrared and visible videos.

The wireless industry is moving towards developing smart cellular architectures that dynamically adjust the use of the network elements according to the service demand, and automating their operations in order to minimize both capital expenditure (CAPEX) and operation expenditure (OPEX). This involves developing efficient and unsupervised radio access network (RAN) planning, which has a direct impact on the system performance and CAPEX. This intelligent cellular planning aims at providing the base stations (BSs) configurations (e.g., coverage, user associations and antenna radiation pattern) that minimize the number of deployed BSs and meet the requirements in terms of coverage and capacity. The cellular planning optimization problem has been shown to be complex and non-scalable. Moreover, most of the existing cellular planning techniques result in an over or under provisioning architecture. Motivated by the above, we propose in this paper a novel and efficient unsupervised planning process. We make use of statistical machine learning (SML) to solve the problem at hand. The core idea of SML is that the planning parameters are treated as random variables. The parameters that maximize the corresponding joint probability distribution, conditioned on observations of users' positions, are learned or inferred using Gibbs sampling theory and Bayes' theory. To apply this theory to the planning problem, we make significant efforts to properly formulate the problem to be able to incorporate the constraints into the inference process and extract the planning parameters from the inferred model. Through several numerical examples, we compare the performance of the proposed approach to clustering-based and optimization-based existing planning approaches, and demonstrate the efficacy of our approach. We also demonstrate how our approach can leverage existing cellular infrastructures into the new design.

Burstiness and overdispersion phenomena of count vectors pose significant challenges in modeling such data accurately. While the dependency assumption of the multinomial distribution causes its failure to model frequency vectors in several machine learning and data mining applications, researchers found that by extending the multinomial distribution to the Dirichlet Compound multinomial (DCM), both phenomena modeling can be addressed. However, Dirichlet distribution is not the best choice, as a prior, given its negative-correlation and equal-confidence requirements. Thus, we propose to use a flexible generalization of the Dirichlet distribution, namely, the shifted-scaled Dirichlet, as a prior to the multinomial, which grants the model a capability to better fit real data, and we call the new model the Multinomial Shifted-Scaled Dirichlet (MSSD). Given that the likelihood function plays a key role in statistical inference, e.g., in maximum likelihood estimation and Fisher information matrix investigation, we propose to improve the efficiency of computing the MSSD log-likelihood by approximating its function based on Bernoulli polynomials where the log-likelihood function is computed using the proposed mesh algorithm. Moreover, given the sparsity and high-dimensionality nature of count vectors, we propose to improve its computation efficiency by approximating the novel MSSD as a member of the exponential family of distribution, which we call EMSSD. The clustering is based on mixture models, and for learning a model, selection approach is seamlessly integrated with the estimation of the parameters. The merits of the proposed approach are validated via challenging real-world applications such as hate speech detection in Twitter, real-time recognition of criminal action, and anomaly detection in crowded scenes. Results reveal that the proposed clustering frameworks offer a good compromise between other state-of-the-art techniques and outperform other approaches previously used for frequency vectors modeling. Besides, comparing to the MSSD, the approximation EMSSD has reduced the computational complexity in high-dimensional feature spaces.

The light field information would be captured through light field cameras and in different directions regarding 3D image view recordings. In this paper, in order to increase the spatio-angular super-resolution quality and to decrease the reconstruction error regarding the light field information, we use a graph-based light field super-resolution strategy. Accordingly, in order to apply the complementary data in the light field views, we use a graph regularizer for the total recovery of the information and an edge-preserving technique that represents an isometry between curves in the 2D manifold and 5D space of the RGB image views. Moreover, the reconstruction of the light field information is based on applying the alternating direction method of multipliers (ADMM) algorithm. Accordingly, a recent enhanced ADMM model has been used in this paper which is denominated as “Plug-and-Play” and permits the user to plug an image reconstruction technique and a denoising methodology as the first and second sub-problems respectively. On that account, we would be able to resolve the light field super-resolution problem considering the graph-based light field structure as the first sub-problem and the edge-preserving technique as the denoising methodology. Consequently, by applying the proposed super-resolution strategy, the super-resolved light field outcome would be more favorable in terms of visual quality and reconstruction errors in comparison with other state-of-the-art methodologies.

In this article, we propose an effective mixture model-based approach to modeling and clustering positive data vectors. Our mixture model is based on the inverted Beta-Liouville (IBL) distribution which is extracted from the family of Liouville distributions. To cope with the problem of determining the appropriate number of clusters in our approach, a nonparametric Bayesian framework is used to extend the IBL mixture to an infinite mixture model in which the number of clusters is assumed to be infinite initially and will be inferred automatically during the learning process. To optimize the proposed model, we propose a convergence-guaranteed learning algorithm based on the averaged collapsed variational Bayes inference that can effectively learn model parameters with closed-form solutions. The effectiveness of the proposed infinite IBL mixture model for modeling and clustering positive vectors is validated through both synthetic and real-world data sets.

Nowadays a massive amount of data is generated as the development of technology and services has accelerated. Therefore, the demand for data clustering in order to gain knowledge has increased in many sectors such as medical sciences, risk assessment and product sales. Moreover, binary data has been widely used in various applications including market basket data and text documents. While applying classic widely used k-means method is inappropriate to cluster binary data, this work proposes an improvement of K-medoids algorithm using binary similarity measures instead of Euclidean distance which is generally deployed in clustering algorithms. We have considered two challenging applications, namely; text clustering and binary images categorization to show the merits of the proposed framework.

A novel regression algorithm has been proposed, Beta-Liouville regression, to solve regression of compositional data, where the prediction is multi-dimensional and sums to unity. Applications include market-share data mining in relation to its score in Google-trends and smart building occupancy estimation. The study of Google-trends in relation to market-shares gives a good estimate of whether the company's investment in advertisements have yielded any results in improving their market-shares. Secondly, occupant behaviour in buildings gives useful insights on the required levels of air conditioning, lighting and even initiating help during emergency. Sensors to estimate the occupancy of a smart building include microphone, door/window positions, motion detection, power consumption. The Beta-Liouville regression algorithm is compared to ordinary least squares regression with compositional transformations and Dirichlet regression.

Recent years have witnessed tremendous advancement in data modelling. In this paper, we focus on Dynamic textures (DT). We propose the use of Fisher kernels with Generalized Dirichlet based hidden Markov models (HMM) for DT recognition. Experiments demonstrate promising results on the DynTeX Alpha dataset using the proposed generative discriminative approach. This is the first application of Generalized Dirichlet HMMs to DT recognition, to the best of our knowledge.

Learning a hidden Markov model (HMM) is typically based on the computation of a likelihood which is intractable due to a summation over all possible combinations of states and mixture components. This estimation is often tackled by a maximization strategy, which is known as the Baum-Welch algorithm. However, some drawbacks of this approach have led to the consideration of Bayesian methods that add a prior over the parameters in order to work with the posterior probability and the marginal likelihood. These approaches can lead to good models but to the cost of extremely long computations (e.g., Markov Chain Monte Carlo). More recently, variational Bayesian frameworks have been proposed as a Bayesian alternative that keeps the computation tractable and the approximation tight. It relies on the introduction of a prior over the parameters to be learned and on an approximation of the true posterior distribution. After proving good standing in the case of finite mixture models and discrete and Gaussian HMMs, we propose here to derive the equations of the variational learning of the Dirichlet mixture-based HMM, and to extend it to the generalized Dirichlet. The latter case presents several properties that make the estimation more accurate. We prove the validity of this approach within the context of unusual event detection in public areas using the University of California San Diego data sets. HMMs are trained over normal video sequences using the typical Baum-Welch approach versus the variational one. The variational learning leads to more accurate models for the detection and localization of anomaly, and the general HMM approach is shown to be versatile enough to handle the detection of various synthetically generated tampering events.

This paper proposes a Bayesian nonparametric framework for clustering axially symmetric data. Our approach is based on a Dirichlet processes mixture model with Watson distributions, which can also be considered as the infinite Watson mixture model. In this paper, first, we extend the finite Watson mixture model into its infinite counterpart based on the framework of truncated Dirichlet process mixture model with a stick-breaking representation. Second, we propose a coordinate ascent mean-field variational inference algorithm that can effectively learn the parameters of our model with closed-form solutions; Third, to cope with a massive data set, we develop a stochastic variational inference algorithm to learn the proposed model through the method of stochastic gradient ascent; Finally, the proposed nonparametric Bayesian model is evaluated through simulated axially symmetric data sets and a real-world application, namely, gene expression data clustering.

A fast exact nearest neighbor search algorithm over large scale data is proposed based on semi-convex hull tree, where each node represents a semi-convex hull, which is made of a set of hyper planes. When performing the task of nearest neighbor queries, unnecessary distance computations can be greatly reduced by quadratic programming. GPUs are also used to accelerate the query process. Experiments conducted on both Intel(R) HD Graphics 4400 and Nvidia Geforce GTX1050 TI, as well as theoretical analysis show that the proposed algorithm yields significant improvements and outperforms current k-d tree based nearest neighbor query algorithms and others.

In this paper, an unsupervised learning algorithm of a finite multi-dimensional generalized Gamma mixture model (GGMM) is presented to tackle the issue of simultaneously clustering positive vectors and determining the number of clusters. The parameters of the model are estimated using maximum likelihood (ML) which is conducted via expectation maximization (EM) [1]. Newton Raphson's optimization algorithm is also utilized to solve the issue of the non-existence of closed form. Moreover, we developed a method based on Minimum message length (MML) criterion, in order to select the optimal number of clusters which best describes the data. Furthermore, Experiments are conducted using both synthetic data and real data sets of images representing shapes and textures to test the performance of the proposed model.

This paper proposes an unsupervised learning algorithm for a finite mixture model of shifted-scaled Dirichlet distributions. Maximum likelihood and Newton raphson approaches are used for parameters estimation. In this research work, we address the flexibility challenge of the Dirichlet distribution by having another set of parameters for the location (beside the Scale parameter) that add functional probability models. This paper evaluates the capability of the discussed model to perform the categorization using both synthetic and real data related to the medical science to help in selecting wart treatment method, in the business field to detect the reasons behind employees' absenteeism, and the writer identification application to define the author of off-line handwritten documents. We also compare the model performance against scaled Dirichlet, the classic Dirichlet, and Gaussian mixture models. Finally, experimental results are presented on the selected datasets. Besides, we apply the minimum message length to determine the optimal number of the components found within each dataset.

In this paper, we propose the bounded Laplace mixture model (BLMM). We also propose a new modeling scheme for wavelet coefficients based on BLMM and we apply it to image clustering and content based image retrieval (CBIR). The clustering stage is also performed by BLMM. In the proposed applications, BLMM is applied for feature extraction where each image is decomposed into a set of wavelet subspaces and a two component BLMM is adopted to illustrate the statistical characteristics of the wavelet coefficients for each wavelet subspace. The model parameters adapted from proposed model, reflect the image features of wavelet domain for each subspace and selected to formulate the feature space which is further used in clustering and CBIR. UIUC, KTH-TIPS and DTD databases are considered to demonstrate the viability and effectiveness of proposed algorithm in image clustering and CBIR. From set of experiments, BLMM has demonstrated its effectiveness in modeling the wavelet coefficients in feature extraction, image clustering and CBIR.

A novel unsupervised Bayesian image categorization framework based on asymmetric Gaussian mixture (AGM) model is proposed and the mixture parameter estimation is achieved by sampling-based reversible jump Markov chain Monte Carlo (RJMCMC) method. Previous researches have reveled that AGM outperforms classic symmetric mixture models (i.e Gaussian mixture model (GMM)) since the model adapts both symmetric and asymmetric datasets yielding better fitting accuracy. Moreover, the introduction of RJMCMC, a hybrid self-adapted sampling-based MCMC implementation, enables model transfer throughout parameter learning process, therefore, automatically converges to the optimal number of categories. In order to better identify visual features from the challenging UIUC sport events dataset, the image representative data is generated by adopting scale-invariant feature transform (SIFT), bag-of-visual-words (BOVW) and probabilistic latent semantic analysis (pLSA) techniques. Eventually, irrelevant and unneeded information will be filtered by feature selection. A comparison between AGM and other popular classifiers is given to discover its merits and the direction of future work is suggested.

A novel framework is developed for the modelling and clustering of proportional data (i.e. normalised histograms) based on the Beta‐Liouville mixture model. This framework is based on incremental model selection, by testing if a given component was truly Beta‐Liouville distributed. Specifically, the authors compare the theoretical maximum entropy of the given component with the estimated entropy obtained by the MeanNN estimator. If a significant difference was gained from this comparison, this component is considered as not well fitted and is then splitted into two new components with a proper initialisation. Our approach is tested through synthetic data sets and real‐world applications which involve human gesture recognition and vehicle tracking for traffic monitoring purposes, which demonstrate that the authors' approach is superior to comparable techniques.

We propose a Bayesian approach to learn finite generalized inverted Dirichlet mixture models. The developed approach performs simultaneous parameters estimation, model complexity determination, and feature selection via a reversible jump Markov Chain Monte Carlo (RJMCMC) algorithm. A challenging application that concerns video forgery detection is deployed to validate our statistical framework and to show its merits.

In this paper, we address the problem of data clustering into homogeneous components in an unsupervised way. Data clustering is one of the major topics in computer vision which has widespread potential applications from various domains such as pattern recognition, data mining, remote sensing, and bioinformatics. In pattern recognition, statistical methods have been widely used and proved effective in generating accurate models. In particular, the popular finite Gaussian mixture models which are able to provide superior performance for data clustering and classification. In this work, we present and evaluate the performance of four well-known Gaussian-based mixture models for data clustering namely: Gaussian mixture model (GMM), Generalized Gaussian mixture model (GGMM), Bounded Gaussian mixture model (BGMM) and Bounded Generalized Gaussian mixture model (BGGMM). The aim of this work is to show that the choice of the component model is very critical in mixture decomposition. Experimental results show close clustering accuracy between different models. However, the bounded generalized Gaussian mixture model provides the best performance in the case of multidimensional data.

In this paper, a hierarchical Dirichlet process (HDP) mixture model of generalized inverted Dirichlet (GID) distributions with an unsupervised feature selection scheme is developed. The proposed model is learned via a principled variational framework and then deployed for video modeling and segmentation. Experimental results show the merits of our developed statistical framework.

An effective novel algorithm to reduce the dimensionality of labeled proportional data is presented which uses an optimal linear projection to project the data into a low-dimensional space. Assuming that each class of the projected data is generated by a mixture of Dirichlet distributions, KL-divergence is used as a dissimilarity measure to maximize the mutual information of projected classes, thus improving separability. Finally, genetic algorithm is used to find such optimal projection. The proposed algorithm is designed as a preprocessing step for binary classification of proportional data, however, it can project multimodal data as well due to use of mixtures and, therefore, can be used for multiclass classification. Experiments show that the proposed technique is effective, and constantly produces better results compared to well-known algorithms from the same category.

In this paper, a novel statistical generative model based on hierarchical Pitman-Yor process and generalized Dirichlet distributions (GDs) is presented. The proposed model allows us to perform joint clustering and feature selection thanks to the interesting properties of the GD distribution. We develop an online variational inference algorithm, formulated in terms of the minimization of a Kullback-Leibler divergence, of our resulting model that tackles the problem of learning from high-dimensional examples. This variational Bayes formulation allows simultaneously estimating the parameters, determining the model's complexity, and selecting the appropriate relevant features for the clustering structure. Moreover, the proposed online learning algorithm allows data instances to be processed in a sequential manner, which is critical for large-scale and real-time applications. Experiments conducted using challenging applications, namely, scene recognition and video segmentation, where our approach is viewed as an unsupervised technique for visual learning in high-dimensional spaces, showed that the proposed approach is suitable and promising.

In this paper, we propose a clustering and feature selection method when dealing with L2-normalized vectors using Von Mises (vM) distribution. In particular, we propose a Bayesian framework to learn vM mixture models using reversible jump Markov chain Monte Carlo (RJMCMC) approach. Experimental results reported on real-world challenging applications involving image categorization and topic detection and tracking prove the efficiency of proposed approach.

Beta-Liouville mixture models have achieved measurable success in many computer vision and pattern recognition applications. In this paper, we develop a novel algorithm to learn this particular kind of models that have been shown to be very efficient for the clustering of proportional data. Our algorithm is based on an accelerated version of the variational Bayes approach. Experiments show that the developed algorithm work very well for the categorization of 3D shapes.

In this paper, we introduce data clustering based on infinite mixture models for spherical patterns. This particular clustering is based on Langevin distribution which has been shown to be effective to model this kind of data. The proposed learning algorithm is tackled using a fully Bayesian approach. In contrast to classical Bayesian approaches, which suppose an unknown finite number of mixture components, proposed approach assumes an infinite number of components and have witnessed considerable theoretical and computational advances in recent years. In particular, we have developed a Markov Chain Monte Carlo (MCMC) algorithm to sample from the posterior distributions associated with the selected priors for the different model parameters. Moreover, we propose an infinite framework that allows simultaneous feature selection selection and parameter estimation. The usefulness of the developed framework has been shown via topic novelty detection application.

In this paper, we formulate 3D model recognition as a statistical inference problem using a Pitman-Yor process mixture of Beta-Liouville Distributions. The proposed model is learned via a collapsed variational inference approach. Unlike classic variational Bayes, the collapsed approach does not make the non-realistic assumption that the model's parameters are independent from the assignment variables, which leads to better modelling and generalization capabilities. The merits and advantages of the proposed approach are shown via extensive experiments.

In this paper, a novel effective method to reduce the dimensionality of labeled proportional data is introduced. Most well-known existing linear dimensionality reduction methods rely on solving the generalized eigen value problem which fails in certain cases such as sparse data. The proposed algorithm is a linear method and uses a novel approach to the problem of dimensionality reduction to solve this problem while resulting higher classification rates. Data is assumed to be from two different classes where each class is matched to a mixture of generalized Dirichlet distributions after projection. Jeffrey divergence is then used as a dissimilarity measure between the projected classes to increase the inter-class variance. To find the optimal projection that yields the largest mutual information, genetic algorithm is used. The method is especially designed as a preprocessing step for binary classification, however, it can handle multi-modal data effectively due to the use of mixture models and therefore can be used for multi-class problems as well.

In modern distributed systems, service consumers are faced with pools of service providers that offer similar functionalities. This reality renders the selection of web services a challenging task. One popular solution is to base the selection decisions on the web services’ non-functional requirements depicted by a variety of QoS metrics. In this paper, we present a new approach for solving the web service selection problem; a QoS-aware trust model that leverages the correlation information among various QoS metrics. This model, based on the probability theory, estimates the trustworthiness of web services by exploiting two statistical distributions, namely, Dirichlet and generalized Dirichlet. These distributions represent the outcomes of multiple correlated QoS metrics. The former distribution is employed when the QoS metrics are positively correlated while the latter handles negatively correlated metrics. We also propose an algorithm to aggregate reputation feedback that propagate among the interacting web services. This algorithm deals with malicious feedback and various strategic behavior commonly performed by web services. Experimental results endorse the advantageous capability of our trust model and reputation algorithm compared to the state-of-the-art.

This paper proposes a hierarchical Pitman-Yor (HPY) process mixture of Dirichlet distributions. It can be viewed as an extension of our previous works that have considered Dirich-let process mixtures of Dirichlet distributions (i.e. infinite Dirichlet mixture models). The proposed model is learned via a variational Bayes approach and applied to the challenging problem of dynamic textures clustering.

It is well-known that classification tasks can be approached using either generative models or discriminative ones. While the goal of generative approaches is to learn class-conditional densities, the main goal of discriminative techniques is to learn decision boundaries directly without taking into account class-conditional densities. In classic supervised learning, we would usually represent a given object (an image, for instance) by a vector of D real-valued features and then select a given generative or discriminative approach to perform classification. In many applications, however, the object can be represented by a set (or) bag of vectors. Recent developments in machine learning, along with powerful computational tools, have enabled researchers to develop more sophisticated models to handle such applications using the so-called hybrid generative discriminative models. The main idea is based on exploiting the advantages of both families of models. Thus, the success of such an approach depends on the choice of an appropriate discriminative technique and a suitable generative one. The goal of this paper is to develop a hybrid generative discriminative framework based on support vector machine and Gamma mixture. In particular, we focus on the generation of kernels when examples (images, for instance) are structured data (i.e. described by sets of vectors) modeled by Gamma mixtures. Experimental results on real-world challenging applications, namely 3D shape class recognition, object categorization, and video event analysis, show the effectiveness of the proposed framework.

We propose model-based inference for topic novelty detection using a non-parametric Bayesian probability model. The probability model is a Dirichlet process mixture of inverted Dirichlet distributions which can be viewed as an infinite mixture model. The inference is based on variational Bayes deployed using approximate conjugate priors to the inverted Dirichlet. Detailed experimental study demonstrates the merits of our approach and shows that it gives good description of the data.

Exploiting Dirichlet process mixture models (also known as infinite mixture models) to model visual and textual data is now standard weapon in the arsenal of machine learning. This paper proposes a new accelerated variational inference approach to learn Dirichlet process mixture models with Dirichlet distributions. The choice of using Dirichlet distribution as the basic distribution is mainly due to its flexibility for modeling proportional data. Indeed, this kind of data is naturally generated by several applications involving the representation of texts, images and videos using the bag-of-words (or "visual words" in the case of images and videos) approach. The potential of the developed learning framework is shown using a challenging real application namely human action recognition in videos.

We propose a hybrid hidden Markov model (HMM) for mixed continuous/continuous and discrete/continuous outcomes. The Expectation-Maximization (EM) procedure used for the parameters estimation relies on a combination of local and global quantities that relate to outcomes of a specific type and to all outcomes, respectively. The new model is implemented for discrete outcomes and continuous outcomes that follow Gaussian, Dirichlet, and Gamma distributions. Experiments led with synthetic data show same or better modeling accuracy compared to a fully Gaussian HMM, with less parameters and thus a shorter convergence time for high dimensional data. Finally, the approach is validated with real data in a change detection scenario between a pair of images, one of which has been captured by an optic sensor and the other by a synthesis aperture radar (SAR) sensor. Based on the properties of the noises corrupting these images, a hybrid Gamma-Gaussian HMM is trained and the likelihood of the data with respect to the model is used to detect changes. The obtained results are in line with the most recent approaches for this task with the advantage of providing a very compact representation of the data.

Human activity recognition is a crucial area of computer vision research and applications. The goal of human activity recognition aims to automatically analyze and interpret ongoing events and their context from video data. Recently, the bag of visual words (BoVW) approach has been widely applied for human action recognition. Generally, a representative corpus of videos is used to build the Visual Words dictionary or codebook using a simple k-means clustering approach. This visual dictionary is then used to quantize the extracted features by simply assigning the label of the closest cluster centroid using Euclidean distance between the cluster centers and the input descriptor. Thus, each video can be represented as a frequency histogram over visual words. However, the BoVW approach has several limitations such as its need for a predefined codebook size, dependence on the chosen set of visual words, and the use of hard assignment clustering for histogram creation. In this paper, we are trying to overcome these issues by using a mixture of Asymmetric Gaussians to build the codebook. Our method is able to identify the best size for our dictionary in an unsupervised manner, to represent the set of input feature vectors by an estimate of their density distribution, and to allow soft assignments. Furthermore, we validate the efficiency of the proposed algorithm for human action recognition.

Foreground segmentation of moving regions in image sequences is a fundamental step in many vision systems including automated video surveillance, human-machine interface, and optical motion capture. Many models have been introduced to deal with the problems of modeling the background and detecting the moving objects in the scene. One of the successful solutions to these problems is the use of the well-known adaptive Gaussian mixture model. However, this method suffers from some drawbacks. Modeling the background using the Gaussian mixture implies the assumption that the background and foreground distributions are Gaussians which is not always the case for most environments. In addition, it is unable to distinguish between moving shadows and moving objects. In this paper, we try to overcome these problem using a mixture of asymmetric Gaussians to enhance the robustness and flexibility of mixture modeling, and a shadow detection scheme to remove unwanted shadows from the scene. Furthermore, we apply this method to real image sequences of both indoor and outdoor scenes. The results of comparing our method to different state of the art background subtraction methods show the efficiency of our model for real-time segmentation.

Breast cancer is a major cause of deaths among women and the leading cause of death among all cancers for middle-aged women in most developed countries. Presently there are no methods to prevent breast cancer thus early detection of this disease represents a very important factor in its treatment and plays a major role in reducing mortality. Mammography is one of the most reliable methods in early detection of breast cancer. In this paper, we present a novel algorithm for medical mammogram image classification, based on the Dirichlet mixture model. Our method can be divided into three main steps: Preprocessing, feature extraction, and image classification. First, histogram equalization is used to remove the noise and to enhance the quality of the image. Later, we extract texture information from mammographic images using the Local Binary Pattern (LBP) and Haralick texture descriptor (HTD). Then, we use the Birth and Death Markov Chain Monte Carlo to estimate the parameters of the Dirichlet mixture representing each class from our training set. Finally, in the classification stage, each mammogram image is assigned to the class increasing more its likelihood. Extensive simulations are used to show the merits of our approach.

We introduce a novel hierarchical mixture model where each component is composed of a set of finite probability densities forming a super class mixture. Our proposed model can be viewed as a mixture of mixtures to support multi-level hierarchies where the structure of the hierarchy can be altered according to users' ontological models within costless computational time. The proposed approach is generalized to adopt any probability density function and an algorithm to learn the model is proposed. In this paper, we adopt the inverted Dirichlet distribution to build the model, and a simulation study is performed to validate the proposed approach using synthetic and a real world challenging application concerning visual scenes categorization.

Service oriented computing is being increasingly exploited in the architecture of current web applications. The interactions among the deployed web services are becoming vital to accomplish heterogeneous and compound business goals. Service selection and composition are two common tasks which are highly influenced by the quality of these interactions. Thus, assigning web services QoS-based trust scores that are incrementally updated, provides a means to assist both tasks. Then, services with higher trust scores are more likely to be selected than those with smaller ones. They are, additionally, more prone to be incorporated as part of composite services. In this paper, we propose a probabilistic approach based on Bayesian networks (BN) to learn the composition structure of composite services and compute QoS-based trust scores in an online setting. The learning of the BN structure and parameters is based on modeling the QoS, which is represented by the BN's variables, using a multinomial generalized Dirichlet distribution (MGDD). The effectiveness of our approaches is empirically assessed using real and synthetic data. Our experimental results show that MGDD provides a flexible and accurate representation of the QoS. They also prove the capability of the BN approach to learn the composition structure, and further the responsibility of the constituent services in the quality of the composite service even when their QoS is partially observed.

The detection of topic trends has increasingly attracted interest over the past decades, fueled in particular by the revolution of internet and the emergence of social media. However, manual topic detection and tracking (TDT) is not efficient, this has become possible thanks to the development of modern data mining techniques and their flexibility to model potential issues. A critical challenge in this context is the representation choices of news stories along with adequate detection of new topics. To this end, we propose a unified statistical framework that allows simultaneous topic clustering and feature (word) selection in online settings based on spherical mixtures. Through empirical experiments, the proposed framework demonstrates the ability to learn new topics incrementally and improve detection quality within a reasonable time framework on diverse high-dimensional datasets.

Recently, finite Beta-Liouville mixture models have proved to be an effective and powerful knowledge representation and inference engine in several machine learning and data mining applications. In this paper, we propose a component splitting and local model selection method to address the problem of learning and selecting finite Beta-Liouville mixture models in an incremental variational way. Within the proposed principled variational learning framework, all the involved parameters and model complexity (i.e. the number of mixture components) can be estimated simultaneously in a closed-form. We demonstrate the effectiveness of the proposed approach through both synthetic data as well as two challenging real-world applications namely human activities modeling and recognition, and facial expressions recognition.

In this paper, an expectation propagation (EP) inference framework for unsupervised feature selection is proposed for modeling proportional data which naturally appear in many applications such as text and image modeling, in the context of finite mixture-based clustering. Within our framework, simultaneous clustering and feature selection is formalized using finite mixtures of generalizing Dirichlet (GD) distributions. The proposed EP-based inference framework allows to obtain a full posterior distribution on all our unsupervised feature selection model's parameters. Moreover, the complexity of the deployed mixture models and all the involved model parameters can be evaluated simultaneously. The effectiveness and efficiency of the proposed algorithm are evaluated on both synthetic data and two challenging applications namely human action videos categorization and facial expression recognition.

We present a two stage framework for automatic video text removal to detect and remove embedded video texts and fill-in their remaining regions by appropriate data. In the video text detection stage, text locations in each frame are found via an unsupervised clustering performed on the connected components produced by the stroke width transform (SWT). Since SWT needs an accurate edge map, we develop a novel edge detector which benefits from the geometric features revealed by the bandlet transform. Next, the motion patterns of the text objects of each frame are analyzed to localize video texts. The detected video text regions are removed, then the video is restored by an inpainting scheme. The proposed video inpainting approach applies spatio-temporal geometric flows extracted by bandlets to reconstruct the missing data. A 3D volume regularization algorithm, which takes advantage of bandlet bases in exploiting the anisotropic regularities, is introduced to carry out the inpainting task. The method does not need extra processes to satisfy visual consistency. The experimental results demonstrate the effectiveness of both our proposed video text detection approach and the video completion technique, and consequently the entire automatic video text removal and restoration process.

Clustering has been a subject of extensive research in data mining, pattern recognition, and other areas for several decades. The main goal is to assign samples, which are typically non-Gaussian and expressed as points in high-dimensional feature spaces, to one of a number of clusters. It is well known that in such high-dimensional settings, the existence of irrelevant features generally compromises modeling capabilities. In this paper, we propose a variational inference framework for unsupervised non-Gaussian feature selection, in the context of finite generalized Dirichlet (GD) mixture-based clustering. Under the proposed principled variational framework, we simultaneously estimate, in a closed form, all the involved parameters and determine the complexity (i.e., both model an feature selection) of the GD mixture. Extensive simulations using synthetic data along with an analysis of real-world data and human action videos demonstrate that our variational approach achieves better results than comparable techniques.

A large class of problems can be formulated in terms of the clustering process. Mixture models are an increasingly important tool in statistical pattern recognition and for analyzing and clustering complex data. Two challenging aspects that should be addressed when considering mixture models are how to choose between a set of plausible models and how to estimate the model's parameters. In this paper, we address both problems simultaneously within a unified online nonparametric Bayesian framework that we develop to learn a Dirichlet process mixture of Beta-Liouville distributions (i.e., an infinite Beta-Liouville mixture model). The proposed infinite model is used for the online modeling and clustering of proportional data for which the Beta-Liouville mixture has been shown to be effective. We propose a principled approach for approximating the intractable model's posterior distribution by a tractable one-which we develop-such that all the involved mixture's parameters can be estimated simultaneously and effectively in a closed form. This is done through variational inference that enjoys important advantages, such as handling of unobserved attributes and preventing under or overfitting; we explain that in detail. The effectiveness of the proposed work is evaluated on three challenging real applications, namely facial expression recognition, behavior modeling and recognition, and dynamic textures clustering.

In this paper, we propose a text detection method based on a feature vector generated from connected components produced via the stroke width transform. Several properties, such as variant directionality of gradient of text edges, high contrast with background, and geometric properties of text components jointly with the properties found by the stroke width transform are considered in the formation of feature vectors. Then, k-means clustering is performed by employing the feature vectors in a bid to distinguish text and non-text components. Finally, the obtained text components are grouped and the remaining components are discarded. Since the stroke width transform relies on a precise edge detection scheme, we introduce a novel bandlet-based edge detector which is quite effective at obtaining text edges in images while dismissing noisy and foliage edges. Our experimental results indicate a high performance for the proposed method and the effectiveness of our proposed edge detector for text localization purposes.

Online algorithms allow data points to be processed sequentially, which is important for real-time applications. In this paper, we propose a novel online clustering approach based on a mixture of Dirichlet processes with Dirichlet distributions, which can be viewed as an extension of the finite Dirichlet mixture model to the infinite case. Our approach is based on nonparametric Bayesian analysis, the determination of the number of clusters is sidestepped by assuming an infinite number of clusters. By learning the proposed model in an online manner with a variational learning framework, all the involved parameters can be estimated effectively and efficiently in a closed form without introducing the problem of over fitting. The proposed online infinite mixture model is validated through both synthetic data sets and a challenging real-world application namely unsupervised image categorization.

Software architectures of large-scale systems are perceptibly shifting towards employing open and distributed computing. Service Oriented Computing (SOC) is a typical example of such environment in which the quality of interactions amongst software agents is a critical concern. Agent-based web services in open and distributed architectures need to interact with each other to achieve their goals and fulfill complex user requests. Two common tasks are influenced by the quality of interactions among web services: the selection and composition. Thus, to ensure the maximum gain in both tasks, it is essential for each agent-based web service to maintain a model of its environment. This model then provides a means for a web service to predict the quality of future interactions with its peers. In this paper, we formulate this model as a machine learning problem which we analyze by modeling the trustworthiness of web services using probabilistic models. We propose two approaches for trust learning of single and composed services; Bayesian Networks and Mixture of Multinomial Dirichlet Distributions (MMDD). The effectiveness of our approaches is empirically assessed using a simulation study. Our results show that representing the quality of a web service by Multinomial Dirichlet Distribution (MDD) provides high flexibility and accuracy in modeling trust. They also show that using our approaches to estimate trust enhances web services selection and composition.

In this paper, we propose a hybrid generative discriminative framework for the challenging problem of spam emails filtering using both textual and visual features. Our framework is based on building probabilistic Support Vector Machines (SVMs) kernels from mixture of Langevin distributions. Through empirical experiments, we demonstrate the effectiveness and the merits of the proposed learning framework.

Breast cancer is one of the main causes of death among American women. The use of screening mammography is widely recommended for early diagnosis of breast cancer. In this paper, we propose a highly efficient algorithm for medical mammogram image classification, based on the generalized Beta mixture model. The proposed method, first extracts texture information from mammographic images then model it using the generalized Beta mixture models. For classification, we use the Earth Mover Distance (EMD) metric. Our work is motivated by the fact that mammographic images contain non-Gaussian texture characteristics, impossible to model using rigid distributions like the Gaussian. Experimental results are provided to show the merits of the proposed approach.

The work proposed in this paper is motivated by the need to develop powerful models and approaches to classify and learn proportional data. Indeed, an abundance of interesting data in several applications occur naturally in this form. Our goal is to discover and capture the intrinsic nature of the data by proposing some approaches that combine the major advantages of generative models namely finite mixtures and discriminative techniques namely support vector machines (SVMs). Indeed, SVMs often rely on classic kernels which are not generally meaningful for proportional data. One serious limitation of these kernels is that they do not take into account the nature of data to classify and choosing a suitable kernel continues to be a formidable challenge for data mining and machine learning researchers. Our approach builds on selecting accurate kernels generated from finite mixtures of Dirichlet, generalized Dirichlet and Beta-Liouville distributions which chief advantage is their flexibility and explanatory capabilities in the case of heterogenous proportional data. Using extensive simulations and a number of experiments involving scene modeling and classification, and automatic image orientation detection, we show the merits of the proposed mixture models and the accuracy of the generated kernels.

In this paper, we address the video completion problem for two general cases: 1) filling-in the missing regions of videos captured by a non-stationary camera, and 2) filling-in the missing part of video sequences recorded by a stationary camera. For each case, a novel video completion technique based on the bandlet transform is presented. In the first case, a priority-based exemplar algorithm, which applies the bandlet transform and its generated coefficients along with motion information, is used to fill-in the occluded moving object or the removed region. In the second case, our proposed method is followed by a foreground/background segmentation preprocessing step to generate moving objects and background frames in order to facilitate the video completion task. The technique fills-in the background frames after removing objects by means of a precise optimization in the bandlet transform domain. Then, the occluded part of a moving object is completed by a priority-based algorithm which applies frames' geometry properties using the bandlet transform. Our experimental results indicate that the proposed video completion technique maintains both the spatial and temporal consistency and also demonstrate the effectiveness of the bandlet transform in video completion.

In this paper, we focus on the variational learning of finite Dirichlet mixture models. Compared to other algorithms that are commonly used for mixture models (such as expectation-maximization), our approach has several advantages: first, the problem of over-fitting is prevented; furthermore, the complexity of the mixture model (i.e., the number of components) can be determined automatically and simultaneously with the parameters estimation as part of the Bayesian inference procedure; finally, since the whole inference process is analytically tractable with closed-form solutions, it may scale well to large applications. Both synthetic and real data, generated from real-life challenging applications namely image databases categorization and anomaly intrusion detection, are experimented to verify the effectiveness of the proposed approach.

In recent years, an increasing number of security threats have brought a serious risk to the internet and computer networks. Intrusion Detection System (IDS) plays a vital role in detecting various kinds of attacks. Developing adaptive and flexible oriented IDSs remains a challenging and demanding task due to the incessantly appearance of new types of attacks and sabotaging approaches. In this paper, we propose a novel unsupervised statistical approach for detecting network based attacks. In our approach, patterns of normal and intrusive activities are learned through finite generalized Dirichlet mixture models, in the context of Bayesian variational inference. Under the proposed variational framework, the parameters, the complexity of the mixture model, and the features saliency can be estimated simultaneously, in a closed-form. We evaluate the proposed approach using the popular KDD CUP 1999 data set. Experimental results show that this approach is able to detect many different types of intrusions accurately with a low false positive rate.

We present a bandlet transform-based technique to complete missing parts of a video sequence captured with a static camera. The method is followed by a preprocessing step to separate the foreground which consists of moving objects and background frames in order to facilitate the video completion task. The technique considers two cases: 1) filling-in the background frames after removing objects, and 2) fillingin the occluded parts of a moving object. In the first case, a precise optimization in the bandlet domain is performed to complete the background video. In the second case, a priority based exemplar algorithm, which applies bandlet geometry properties, is used to fill-in the occluded moving object. Our experimental results indicate the effectiveness of the proposed video completion technique.

The problem that we address in this paper is that of learning hierarchical object categories. Indeed, Digital media technology generates huge amount of non-textual information. Categorizing this information is a challenging task which has served important applications. An important part of this nontextual information is composed of images and videos which consists of various objects each of which may be used to effectively classify the images or videos. Object classification in computer vision can be looked upon from several different perspectives. From the structural perspective object classification models can be divided into flat and hierarchical models. Many of the well-known hierarchical structures proposed so far are based on the Dirichlet distribution. In this work, however, we present a generative hierarchical statistical model based on generalized Dirichlet distribution for the categorization of visual objects modeled as a set of local features describing patches detected using interest points detector. We demonstrate the effectiveness of the proposed model through extensive experiments.

This paper presents an unsupervised statistical model for simultaneous clustering, feature selection and outlier rejection in the case of count data. The proposed model is based on a finite discrete mixture to which a uniform component is added to ensure robustness to outliers and noise. The consideration of a finite mixture model is justified by its flexibility, its solid grounding in the theory of statistics and its competitive results. We derive a complete maximum a posteriori learning approach that does not require a priori knowledge about the number of outliers and the number of clusters. A rigorous expectation maximization (EM) algorithm, based on the formulation of a maximum a posteriori (MAP) estimation, is also provided. We report experimental results of applying our model to the challenging problems of visual scenes categorization and texture discrimination.

In this paper, we consider the problem of constructing accurate and flexible statistical representations for count data, which we often confront in many areas such as data mining, computer vision, and information retrieval. In particular, we analyze and compare several generative approaches widely used for count data clustering, namely multinomial, multinomial Dirichlet, and multinomial generalized Dirichlet mixture models. Moreover, we propose a clustering approach via a mixture model based on a composition of the Liouville family of distributions, from which we select the Beta-Liouville distribution, and the multinomial. The novel proposed model, which we call multinomial Beta-Liouville mixture, is optimized by deterministic annealing expectation-maximization and minimum description length, and strives to achieve a high accuracy of count data clustering and model selection. An important feature of the multinomial Beta-Liouville mixture is that it has fewer parameters than the recently proposed multinomial generalized Dirichlet mixture. The performance evaluation is conducted through a set of extensive empirical experiments, which concern text and image texture modeling and classification and shape modeling, and highlights the merits of the proposed models and approaches.

In many applications it is necessary to be able to classify images in a database accurately and with acceptable speed. The main problem is to assign different images to right categories. The later problem becomes more challenging while dealing with large databases with many categories and subcategories. In this paper we propose a novel classification method based on an adopted hierarchical Dirichlet generative model, previously proposed for corpora document classification. In order to adopt the model to work with image data we use the bag of visual words model. We show that if properly applied the model can achieve adequate results for hierarchical image classification. Experimental results are presented and discussed to show the merits of the proposed approach.

In this letter, we propose a clustering model that efficiently mitigates image and video under/over-segmentation by combining generalized Gaussian mixture modeling and feature selection. The model has flexibility to accurately represent heavy-tailed image/video histograms, while automatically discarding uninformative features, leading to better discrimination and localization of regions in high-dimensional spaces. Experimental results on a database of real-world images and videos showed us the effectiveness of the proposed approach.

In this paper, we propose a clustering algorithm based on both Dirichlet processes and generalized Dirichlet distribution which has been shown to be very flexible for proportional data modeling. Our approach can be viewed as an extension of the finite generalized Dirichlet mixture model to the infinite case. The extension is based on nonparametric Bayesian analysis. This clustering algorithm does not require the specification of the number of mixture components to be given in advance and estimates it in a principled manner. Our approach is Bayesian and relies on the estimation of the posterior distribution of clusterings using Gibbs sampler. Through some applications involving real-data classification and image databases categorization using visual words, we show that clustering via infinite mixture models offers a more powerful and robust performance than classic finite mixtures.

The majority of used kernels in SVMs concern continuous data, and neglect the structure of the text. In contrast to classical kernels, we propose the use of various string kernels for spam filtering. On the other hand, data preprocessing is a vital part of text classification where the objective is to generate feature vectors usable by SVM kernels. We detail a feature mapping variant in text classification (TC) that yields improved performance for the standard SVM in filtering task. Furthermore, we propose an online active framework for spam filtering.

This paper analyzes the use of visual words, as low-level image features, for learning and categorizing images. We show that this problem can be reduced to a simultaneous weighting of appropriate features and detection of clusters in a binary feature space. A probabilistic model is then proposed to quantify the effectiveness of visual words when treated as binary features. In order to learn the model, we consider a maximum a posteriori (MAP) approach. Experimental results are presented to illustrate the feasibility and merits of our approach.

Video texture is a new type of medium which can provide a new video with a continuously varying stream of images from a recorded video. It is created by reordering the input video frames in a way which can be played without any visual discontinuity. Recently, a new method of generating video textures has been proposed. It first apply principal components analysis (PCA) to extract signatures or patterns from the original video sequence, and then implement an autoregressive process (AR) model to synthesize new video textures. In this paper, we extend this video texture generation method by comparing PCA with other dimensionality reduction techniques such as probabilistic principal components analysis, kernel principal components analysis, independent component analysis, local linear embedding and Isomap. According to our experiments, these approaches prevail the original approach by providing us video textures with better quality.

This paper presents an unsupervised approach for feature selection and extraction in mixtures of generalized Dirichlet (GD) distributions. Our method defines a new mixture model that is able to extract independent and non-Gaussian features without loss of accuracy. The proposed model is learned using the Expectation-Maximization algorithm by minimizing the message length of the data set. Experimental results show the merits of the proposed methodology in the categorization of object images.

In this paper, we consider the problem of unsupervised discrete feature selection/weighting. Indeed, discrete data are an important component in many data mining, machine learning, image processing, and computer vision applications. However, much of the published work on unsupervised feature selection has concentrated on continuous data. We propose a probabilistic approach that assigns relevance weights to discrete features that are considered as random variables modeled by finite discrete mixtures. The choice of finite mixture models is justified by its flexibility which has led to its widespread application in different domains. For the learning of the model, we consider both Bayesian and information-theoretic approaches through stochastic complexity. Experimental results are presented to illustrate the feasibility and merits of our approach on a difficult problem which is clustering and recognizing visual concepts in different image data. The proposed approach is successfully applied also for text clustering.

In this paper we analyze the problem of image databases categorization using a statistical generative model. Our model is based on the multinomial generalized Dirichlet distribution recently introduced to model discrete data. The model integrates also the spatial information with color histograms. We designed experiments to show the merits of our model.

In this paper we analyze the problem of prediction using generalized Dirichlet mixtures which have been shown to be effective for approximating a wide varieties of probability distributions. The generalized Dirichlet mixture-based predictor is nonlinear and takes into account the fact that images clutter and texture are generally non-Gaussian. Experimental results involve objects detection in images and image restoration.

In this paper, we introduce an image alignment method by maximizing a Tsallis entopy-based divergence using a modified simultaneous perturbation stochastic approximation algorithm. Due to its convexity property, this divergence measure attains its maximum value when the conditional intensity probabilities between the reference image and the transformed target image are degenerate distributions. Experimental results are provided to show the registration accuracy of the proposed approach in comparison with existing entropic image alignment techniques.

In this paper, we examine the problem of count data clustering. We analyze this problem using finite mixtures of distributions. The multinomial distribution and the multinomial Dirichlet distribution (MDD) are widely accepted to model count data. We show that these two distributions cannot be the best choice in all the applications, and we propose another model called the multinomial generalized Dirichlet distribution (MGDD) that is the composition of the generalized Dirichlet distribution and the multinomial, in the same way that the MDD is the composition of the Dirichlet and the multinomial. The estimation of the parameters and the determination of the number of components in our model are based on the deterministic annealing expectation-maximization (DAEM) approach and the minimum description length (MDL) criterion, respectively. We compare our method to standard approaches such as multinomial and multinomial Dirichlet mixtures to show its merits. The comparison involves different applications such as spatial color image databases indexing, handwritten digit recognition, and text document clustering.

Many computer vision and pattern recognition problems involve the use of finite Gaussian mixture models. Finite mixture model using generalized Dirichlet distribution has been shown as a robust alternative of normal mixtures. In this paper, we adopt a Bayesian approach for generalized Dirichlet mixture estimation and selection. This approach, offers a solid theoretical framework for combining both the statistical model learning and the knowledge acquisition. The estimation of the parameters is based on the Monte Carlo simulation technique of Gibbs sampling mixed with a Metropolis-Hastings step. For the selection of the number of clusters, we used Bayes factors. We have successfully applied the proposed Bayesian framework to model IR eyes. Experimental results are shown to demonstrate the robustness, efficiency, and accuracy of the algorithm.

This paper presents a finite discrete mixture model based on both the Dirichlet and the multinomial distributions to add spatial information to color histograms. The estimation of the parameters and the determination of the number of components in our model are based on the classification expectation-maximization approach and the integrated complete likelihood criterion, respectively. The developed model is applied with success for color images databases summarization.

Content-based image suggestion (CBIS) targets the recommendation of products based on user preferences on the visual content of images. In this paper, we mo- tivate both feature selection and model order identification as two key issues for a successful CBIS. We propose a generative model in which the visual features and users are clustered into separate classes. We identify the number of both user and image classes with the simultaneous selection of relevant visual features us- ing the message length approach. The goal is to ensure an accurate prediction of ratings for multidimensional non-Gaussian and continuous image descriptors. Experiments on a collected data have demonstrated the merits of our approach.

We consider the problem of determining the structure of high-dimensional data, without prior knowledge of the number of clusters. Data are represented by a finite mixture model based on the generalized Dirichlet distribution. The generalized Dirichlet distribution has a more general covariance structure than the Dirichlet distribution and offers high flexibility and ease of use for the approximation of both symmetric and asymmetric distributions. This makes the generalized Dirichlet distribution more practical and useful. An important problem in mixture modeling is the determination of the number of clusters. Indeed, a mixture with too many or too few components may not be appropriate to approximate the true model. Here, we consider the application of the minimum message length (MML) principle to determine the number of clusters. The MML is derived so as to choose the number of clusters in the mixture model which best describes the data. A comparison with other selection criteria is performed. The validation involves synthetic data, real data clustering, and two interesting real applications: classification of web pages, and texture database summarization for efficient retrieval.

This paper applies a robust statistical scheme to the problem of unsupervised learning of high-dimensional data. We develop, analyze, and apply a new finite mixture model based on a generalization of the Dirichlet distribution. The generalized Dirichlet distribution has a more general covariance structure than the Dirichlet distribution and offers high flexibility and ease of use for the approximation of both symmetric and asymmetric distributions. We show that the mathematical properties of this distribution allow high-dimensional modeling without requiring dimensionality reduction and, thus, without a loss of information. This makes the generalized Dirichlet distribution more practical and useful. We propose a hybrid stochastic expectation maximization algorithm (HSEM) to estimate the parameters of the generalized Dirichlet mixture. The algorithm is called stochastic because it contains a step in which the data elements are assigned randomly to components in order to avoid convergence to a saddle point. The adjective "hybrid" is justified by the introduction of a Newton-Raphson step. Moreover, the HSEM algorithm autonomously selects the number of components by the introduction of an agglomerative term. The performance of our method is tested by the classification of several pattern-recognition data sets. The generalized Dirichlet mixture is also applied to the problems of image restoration, image object recognition and texture image database summarization for efficient retrieval. For the texture image summarization problem, results are reported for the Vistex texture image database from the MIT Media Lab.

This paper proposes an unsupervised algorithm for learning a finite Dirichlet mixture model. An important part of the unsupervised learning problem is determining the number of clusters which best describe the data. We extend the minimum message length (MML) principle to determine the number of clusters in the case of Dirichlet mixtures. Parameter estimation is done by the expectation-maximization algorithm. The resulting method is validated for one-dimensional and multidimensional data. For the one-dimensional data, the experiments concern artificial and real SAP image histograms. The validation for multidimensional data involves synthetic data and two real applications: shadow detection in images and summarization of texture image databases for efficient retrieval. A comparison with results obtained for other selection criteria is provided

The performance of a statistical image processing system depends in large part on the accuracy of the probabilistic model used. This paper presents a robust probabilistic mixture model based on the Dirichlet distribution. An unsupervised algorithm based on MML for learning this mixture is given, too. Experimental results involve shadows modeling and its application to shadows detection in images.

The performance of a statistical signal processing system depends in large part on the accuracy of the probabilistic model used. This paper presents a robust probabilistic mixture model based on a generalization of the Dirichlet distribution. An unsupervised algorithm for learning this mixture is given, too. The proposed approach for estimating the parameters of a Dirichlet mixture is based on the maximum likelihood (ML) and Fisher scoring methods. Experimental results involve human skin color modeling and its application to skin detection in images.

This paper presents an unsupervised algorithm for learning a finite mixture model from multivariate data. This mixture model is based on the Dirichlet distribution, which offers high flexibility for modeling data. The proposed approach for estimating the parameters of a Dirichlet mixture is based on the maximum likelihood (ML) and Fisher scoring methods. Experimental results are presented for the following applications: estimation of artificial histograms, summarization of image databases for efficient retrieval, and human skin color modeling and its application to skin detection in multimedia databases.