Variational autoencoder (VAE)-based frameworks possess a natural advantage in modeling the shared and private information inherent in multimodal data. However, current models focus on improving the quality of shared representations from the reconstruction perspective, lacking explicit mechanisms to model their underlying semantic structure. In this paper, we propose the multimodal Gaussian mixture variational autoencoder with consistency regularizations, which introduces a Gaussian mixture prior over the shared latent space to enhance its semantic structure and encourage the formation of cluster-aware latent representations. To address the cross-modal inconsistency problem under missing modality conditions, we propose a cluster-guided regularization strategy that enforces the cross-modal consistency using the pseudo-category labels from unsupervised clustering. Additionally, we design a self-supervised contrastive regularization strategy to align semantically similar representations across modalities. Extensive experiments on MNIST-SVHN and MNIST-CDCB datasets demonstrate that our method significantly outperforms prior state-of-the-art models in generation, classification, and retrieval tasks.

Despite the significant advances of Convolutional neural networks (CNNs) and Transformers in image deraining, they either suffer from limited receptive fields or incur quadratic complexity, leading to an imbalance between performance and efficiency. Recently, state space models (SSMs) have demonstrated significant potential in modeling long-range dependencies while maintaining linear complexity. However, existing Mamba-based approaches lack the exploration of useful complementary information from multiple image scales, which could be beneficial for facilitating rain removal. In this paper, we propose an effective multi-scale state-space model-based framework (MS-RainMamba) to explore richer scale-space information for better image deraining. Specifically, we design a local-enhanced state space module to better aggregate rich local and global information. In contrast to existing methods that adopt fixed-scale scanning for feature extraction, we develop a multi-scale hierarchical 2D scanning technique to better help image restoration. Experimental results on six benchmarks show that the proposed method performs favorably against state-of-the-art models.

Due to the presence of rain, the visibility of images captured outdoors on rainy days will be severely degraded. Rain removal using image processing technology can reduce the influence of rain to estimate rain-free images. However, existing traditional rain removal methods are very time-consuming, and newly emerging deep learning-based methods require a large amount of data and computational resources, resulting in long time consumption and poor visual effect. To solve the problems of existing methods, a new method is proposed to remove oblique rain streaks in windy conditions better. Firstly, the directional gradient constraint is proposed to locate oblique rain streaks in the rain layer effectively. Then, a sparse prior for oblique rain streaks is presented to enhance oblique rain streaks removal. After that, an optimization problem combining the directional gradient, sparse priors, and non-negativity constraints is presented. Finally, the alternating direction method of multipliers is exploited to solve the optimization problem effectively. Experiment results show that our method outperforms other methods in removing oblique rain streaks and requires less time.

Text matching is a critical task in natural language processing to measure semantic similarity between two texts. A significant portion of online texts are labeled with a variety of coarse topic responses. These supervised topic indicators can provide prior structured and explicable semantics for textual similarity modeling. However, most existing state-of-the-art neural network methods cannot benefit from such complementary topic signals. Therefore, we propose a novel Topic Supervision BERT-based model (TSB) for text matching. TSB provides a reference multi-task joint training framework involving two types of topic supervision, including explicit and implicit topic supervision. To constrain consistent topic correspondences between texts, we introduce a supervised auxiliary learning task to incorporate explicit pre-defined topic supervision. Furthermore, to adapt to latent topic structures for mutual benefit between text representations and multiple tasks, we integrate a topic model into a contextual text representation model BERT to mine and incorporate implicit self-learnable topic supervision. Experimental results show that TSB supplements explicit and implicit topic information through a multi-task learning approach, which significantly improves the performance of text matching on two public datasets, especially on challenging short text matches.

The visualization of complex network traffic involving a large number of communication devices is a common yet challenging task. Traditional layout methods create the network graph with overwhelming visual clutter, which hinders the network understanding and traffic analysis tasks. The existing graph simplification algorithms (e.g. community-based clustering) can effectively reduce the visual complexity, but lead to less meaningful traffic representations. In this paper, we introduce a new method to the traffic monitoring and anomaly analysis of large networks, namely Structural Equivalence Grouping (SEG). Based on the intrinsic nature of the computer network traffic, SEG condenses the graph by more than 20 times while preserving the critical connectivity information. Computationally, SEG has a linear time complexity and supports undirected, directed and weighted traffic graphs up to a million nodes. We have built a Network Security and Anomaly Visualization (NSAV) tool based on SEG and conducted case studies in several real-world scenarios to show the effectiveness of our technique.

Ising mean field is a basic variational inference method for Ising model, which can provide an effective approximate solution for large-scale inference problem. The main idea is to transform a probabilistic inference problem into a functional extremum problem by variational calculus, and solve the functional extremum problem to obtain approximate marginal distributions. The process of solving the functional extremum is an important step and a computational core for variational inference. But the traditional full variational iteration methods make the variable information intercross with each other deeply. From the view of incomplete variational iterations, we propose a message family propagation method for Ising mean field to compute a marginal distribution family of object variable.