In the domain of model-based diagnosis (MBD), the identification of the most probable faulty components entails the initial computation of candidate diagnoses across all system elements, followed by the application of Bayesian inference to derive their posterior failure probabilities. However, this conventional approach necessitates the extraction of minimal conflict sets (MCSs) for all components—a prerequisite for generating candidate diagnoses—and subsequently solving for minimal hitting sets (MHSs) of the MCSs. Both tasks are inherently NP-hard, imposing prohibitive computational complexity as system scale increases. Even most advanced diagnostic algorithms encounter significant challenges in enumerating all diagnoses, or even a cardinality-minimal solution, within tractable time constraints for large-scale systems. To address these limitations, this work introduces a novel incremental methodology for efficiently approximating posterior component failure probabilities. A foundational framework is first proposed, leveraging structural relationships inherent to hitting sets to probabilistically characterize component fault likelihoods. Building upon this foundation, two minimization theorems are formally established, accompanied by closed-form parameterizations to optimize computational efficiency. Crucially, the proposed method bypasses the explicit enumeration of diagnoses by directly inferring the most probable faulty components from conflict set analyses. Empirical evaluations demonstrate that the approach not only sustains diagnostic accuracy exceeding 95% but also achieves a substantial computational acceleration—surpassing contemporary state-of-the-art algorithms by multiple orders of magnitude.

Emotion Recognition in Conversations (ERC) involves automatically identifying the emotion of each utterance in conversations. The emotion of an utterance is contingent to the conversation context, and thus, annotating each utterance in ERC entails repetitive screening the whole conversation from annotators. Such a requirement leads to prohibitive cost in fine-grained labeling on utterance. In this paper, we propose an efficient coarse-grained labeling strategy for ERC, which assigns a set of emotions for each conversation. In specific, we reformulate the ERC predictors with conversation-level emotion sets as weakly-supervised learning to optimise a potential candidate for ERC, which is termed as Dataless ERC (DERC). To validate this, we propose a simple-yet-flexible DERC framework with Progressive Learning (DERC-PL). We jointly update pseudo-utterance-level emotions and the ERC predictor in a self-training manner, where we progressively update the ERC predictor from training subsets with lower noise densities to the ones with higher noise densities. We implemented several versions of \baby by incorporating various off-the-shelf ERC methods. Extensive experimental results demonstrate that the proposed \baby can be on par with existing weakly-supervised learning baselines and supervised learning ERC methods.

ABSTRACT Active domain adaptation (active DA) provides an effective solution by selectively labelling a limited number of target samples to significantly enhance adaptation performance. However, existing active DA methods often struggle in real‐world scenarios where, due to data privacy concerns, only a pre‐trained source model is available, rather than the source samples. To address this issue, we propose a novel method called the structure‐based uncertainty estimation model (SUEM) for source‐free active domain adaptation (SFADA). To be specific, we introduce an innovative active sample selection strategy that combines both uncertainty and diversity sampling to identify the most informative samples. We assess the uncertainty in target samples using structure‐wise probabilities and implement a diversity selection method to minimise redundancy. For the selected samples, we not only apply standard‐supervised loss but also conduct interpolation consistency training to further explore the structural information of the target domain. Extensive experiments across four widely used datasets demonstrate that our method is comparable to or outperforms current UDA and active DA methods.

Variational Inference (VI) has gained popularity as a flexible approximate inference scheme for computing posterior distributions in Bayesian models. Original VI methods use Kullback-Leibler (KL) divergence to construct variational objectives. However, KL divergence has zero-forcing behavior and is completely agnostic to the metric of the underlying data distribution, resulting in bad approximations. To alleviate this issue, we propose a new variational objective by using Optimal Transport (OT) distance, which is a metric-aware divergence, to measure the difference between approximate posteriors and priors. The superior performance of OT distance enables us to learn more accurate approximations. We further enhance the objective by gradually including the OT term using a hyperparameter λ for over-parameterized models. We develop a Variational inference method with OT (VOT) which presents a gradient-based black-box framework for solving Bayesian models, even when the density function of approximate distribution is not available. We provide the consistency analysis of approximate posteriors and demonstrate the practical effectiveness on Bayesian neural networks and variational autoencoders.

Wasserstein barycenter, built on the theory of Optimal Transport (OT), provides a powerful framework to aggregate probability distributions, and it has increasingly attracted great attention within the machine learning community. However, it is often intractable to precisely compute, especially for high dimensional and continuous settings. To alleviate this problem, we develop a novel regularization by using the fact that c-cyclical monotonicity is often necessary and sufficient conditions for optimality in OT problems, and incorporate it into the dual formulation of Wasserstein barycenters. For efficient computations, we adopt a variational distribution as the approximation of the true continuous barycenter, so as to frame the Wasserstein barycenters problem as an optimization problem with respect to variational parameters. Upon those ideas, we propose a novel end-to-end continuous approximation method, namely Variational Wasserstein Barycenters with c-Cyclical Monotonicity Regularization (VWB-CMR), given sample access to the input distributions. We show theoretical convergence analysis and demonstrate the superior performance of VWB-CMR on synthetic data and real applications of subset posterior aggregation.

Optimal Transport (OT) has become a powerful tool to compare probability distributions. However, it suffers from a severe computational burden for high dimensional and continuous distributions. To this end, we develop two novel methods for the Kantorovich and Monge formulations, which are the fundamental problems in OT. First, we learn the optimal joint distribution in the Kantorovich formulation and propose an algorithm, namely Cop-OT, which transforms the primal objective of the Kantorovich problem into a tractable objective with respect to the copula parameter. Second, based on the copula formulation of the joint distribution, we learn the optimal map in the Monge problem and propose an algorithm, namely Map-OT, which describes the optimal map using a parameterized function estimated by approximating the barycentric projection of the optimal joint distribution and then obtains a tractable objective with respect to parameters of interest. Both of them can be solved by stochastic optimization with a stable optimizing process. Empirical results demonstrate that Cop-OT and Map-OT can gain more accurate approximations of the Kantorovich and Monge problems compared with the baseline methods.

Mean-field variational inference, built on fully factorisations, can be efficiently solved; however, it ignores the dependencies between latent variables, resulting in lower performance. To address this, the copula variational inference (CVI) method is proposed by using the well-established copulas to effectively capture posterior dependencies, leading to better approximations. However, it suffers from a computational issue, where the optimisation for big models with massive latent variables is quite time-consuming. This is mainly caused by the expensive sampling when forming noisy Monte Carlo gradients in CVI. For CVI speedup, in this paper we propose a novel fast CVI (abbr. FCVI). In FCVI, we derive the gradient of CVI objective by an expectation of the mean-field factorisation. Therefore, we can achieve a much efficient sampling from the D-dimensional mean-field factorisation, enabling to reduce the sampling complexity from O(D2) to O(D). To evaluate FCVI, we compare it against baseline methods on modelling performance and runtime. Experimental results demonstrate that FCVI is on a par with CVI, but runs much faster.

Optimal Transport (OT) formulates a powerful framework by comparing probability distributions, and it has increasingly attracted great attention within the machine learning community. However, it suffers from severe computational burden, due to the intractable objective with respect to the distributions of interest. Especially, there still exist very few attempts for continuous OT, i.e., OT for comparing continuous densities. To this end, we develop a novel continuous OT method, namely Copula OT (Cop-OT). The basic idea is to transform the primal objective of continuous OT into a tractable form with respect to the copula parameter, which can be efficiently solved by stochastic optimization with less time and memory requirements. Empirical results on real applications of image retrieval and synthetic data demonstrate that our Cop-OT can gain more accurate approximations to continuous OT values than the state-of-the-art baselines.

Recently, dataless text classification has attracted increasing attention. It trains a classifier using seed words of categories, rather than labeled documents that are expensive to obtain. However, a small set of seed words may provide very limited and noisy supervision information, because many documents contain no seed words or only irrelevant seed words. In this paper, we address these issues using document manifold, assuming that neighboring documents tend to be assigned to a same category label. Following this idea, we propose a novel Laplacian seed word topic model (LapSWTM). In LapSWTM, we model each document as a mixture of hidden category topics, each of which corresponds to a distinctive category. Also, we assume that neighboring documents tend to have similar category topic distributions. This is achieved by incorporating a manifold regularizer into the log-likelihood function of the model, and then maximizing this regularized objective. Experimental results show that our LapSWTM significantly outperforms the existing dataless text classification algorithms and is even competitive with supervised algorithms to some extent. More importantly, it performs extremely well when the seed words are scarce.

Overdispersed black-box variational inference employs importance sampling to reduce the variance of the Monte Carlo gradient in black-box variational inference. A simple overdispersed proposal distribution is used. This paper aims to investigate how to adaptively obtain better proposal distribution for lower variance. To this end, we directly approximate the optimal proposal in theory using a Monte Carlo moment matching step at each variational iteration. We call this adaptive proposal moment matching proposal (MMP). Experimental results on two Bayesian models show that the MMP can effectively reduce variance in black-box learning, and perform better than baseline inference algorithms.

In this paper, we develop a generic black-box expectation propagation (BBEP) algorithm that can be directly applied to Bayesian models without model-specific derivations. BBEP is built on the spirit of using Monte Carlo estimates, where the moment matching step in EP is replaced with Monte Carlo approximations. To avoid high variance, we employ importance sampling for variance reduction and analyze how to find an optimal proposal distribution. We compare BBEP against the state-of-the-art black-box algorithms on both synthetic and real-world data sets. The experimental results indicate that BBEP can reach better predictive performance than baseline algorithms, and even can be on a par with analytical solutions in some settings.

In topic modelling, the main computational problem is to approximate the posterior distribution given an observed collection. Commonly, we must resort to variational methods for approximations; however, we do not know which variational variant is the best choice under certain settings. In this paper, we focus on four topic modelling inference methods, including mean-field variation Bayesian, collapsed variational Bayesian, hybrid variational-Gibbs and expectation propagation, and aim to systematically compare them. We analyse them from two perspectives, i.e. the approximate posterior distribution and the type of -divergence; and then empirically compare them on various data-sets by two popular metrics. The empirical results are almost matching our analysis, where they indicate that CVB0 may be the best variational variant for topic models.

Gaussian LDA integrates topic modeling with word embeddings by replacing discrete topic distribution over word types with multivariate Gaussian distribution on the embedding space. This can take semantic information of words into account. However, the Euclidean similarity used in Gaussian topics is not an optimal semantic measure for word embeddings. Acknowledgedly, the cosine similarity better describes the semantic relatedness between word embeddings. To employ the cosine measure and capture complex topic structure, we use von Mises-Fisher (vMF) mixture models to represent topics, and then develop a novel mix-vMF topic model (MvTM). Using public pre-trained word embeddings, we evaluate MvTM on three real-world data sets. Experimental results show that our model can discover more coherent topics than the state-of-the-art baseline models, and achieve competitive classification performance.