Fault diagnosis plays a significant role in the intelligent maintenance of industrial processes. Although there are numerous deep learning methods designed for fault diagnosis, certain challenges remain difficult to address in real-world industrial scenarios. These challenges include ensuring high- precision fault detection while maintaining industrial privacy in the presence of sensitive data, as well as achieving accurate detection when fault samples are limited. This paper proposes an industrial data generation framework using diffusion and flow- based models. The proposed approach learns the distribution of the original data to generate synthetic samples with similar characteristics, effectively resolving the dual challenges of data sensitivity and scarcity. Experiments on the Tennessee Eastman dataset demonstrate that our approach outperforms existing methods in fault diagnosis performance.

Attention-based arbitrary style transfer methods, including CNN-based, Transformer-based, and Diffusion-based, have flourished and produced high-quality stylized images. However, they perform poorly on the content and style images with the same semantics, i.e., the style of the corresponding semantic region of the generated stylized image is inconsistent with that of the style image. We argue that the root cause lies in their failure to consider the relationship between local regions and semantic regions. To address this issue, we propose a plug-and-play semantic continuous-sparse attention, dubbed SCSA, for arbitrary semantic style transfer—each query point considers certain key points in the corresponding semantic region. Specifically, semantic continuous attention ensures each query point fully attends to all the continuous key points in the same semantic region that reflect the overall style characteristics of that region; Semantic sparse attention allows each query point to focus on the most similar sparse key point in the same semantic region that exhibits the specific stylistic texture of that region. By combining the two modules, the resulting SCSA aligns the overall style of the corresponding semantic regions while transferring the vivid textures of these regions. Qualitative and quantitative results demonstrate that SCSA enables attention-based arbitrary style transfer methods to produce high-quality semantic stylized images. The code can be found in https://github.com/scn-00/SCSA.

Point cloud videos capture dynamic 3D motion while reducing the effects of lighting and viewpoint variations, making them highly effective for recognizing subtle and continuous human actions. Although Selective State Space Models (SSMs) have shown good performance in sequence modeling with linear complexity, the spatio-temporal disorder of point cloud videos hinders their unidirectional modeling when directly unfolding the point cloud video into a 1D sequence through temporally sequential scanning. To address this challenge, we propose the Unified Spatio-Temporal State Space Model (UST-SSM), which extends the latest advancements in SSMs to point cloud videos. Specifically, we introduce Spatial-Temporal Selection Scanning (STSS), which reorganizes unordered points into semantic-aware sequences through prompt-guided clustering, thereby enabling the effective utilization of points that are spatially and temporally distant yet similar within the sequence. For missing 4D geometric and motion details, Spatio-Temporal Structure Aggregation (STSA) aggregates spatio-temporal features and compensates. To improve temporal interaction within the sampled sequence, Temporal Interaction Sampling (TIS) enhances fine-grained temporal dependencies through non-anchor frame utilization and expanded receptive fields. Experimental results on the MSR-Action3D, NTU RGB+D, and Synthia 4D datasets validate the effectiveness of our method. Our code is available at https://github.com/wangzy01/UST-SSM.

Diffusion and flow matching models have recently been used to solve various linear inverse problems in image restoration, such as super-resolution and inpainting. Using a pre-trained diffusion or flow-matching model as a prior, most existing methods modify the reverse-time sampling process by incorporating the likelihood information from the measurement. However, they struggle in challenging scenarios, such as high measurement noise or severe ill-posedness. In this paper, we propose Flow with Interpolant Guidance (FIG), an algorithm where reverse-time sampling is efficiently guided with measurement interpolants through theoretically justified schemes. Experimentally, we demonstrate that FIG efficiently produces highly competitive results on a variety of linear image reconstruction tasks on natural image datasets, especially for challenging tasks. Our code is available at: https://riccizz.github.io/FIG/.

In practical compressed sensing (CS), the obtained measurements typically necessitate quantization to a limited number of bits prior to transmission or storage. This nonlinear quantization process poses significant recovery challenges, particularly with extreme coarse quantization such as 1-bit. Recently, an efficient algorithm called QCS-SGM was proposed for quantized CS (QCS) which utilizes score-based generative models (SGM) as an implicit prior. Due to the adeptness of SGM in capturing the intricate structures of natural signals, QCS-SGM substantially outperforms previous QCS methods. However, QCS-SGM is constrained to (approximately) row-orthogonal sensing matrices as the computation of the likelihood score becomes intractable otherwise. To address this limitation, we introduce an advanced variant of QCS-SGM, termed QCS-SGM+, capable of handling general matrices effectively. The key idea is a Bayesian inference perspective on the likelihood score computation, wherein expectation propagation is employed for its approximate computation. Extensive experiments are conducted, demonstrating the substantial superiority of QCS-SGM+ over QCS-SGM for general sensing matrices beyond mere row-orthogonality.

We theoretically analyze the model selection consistency of least absolute shrinkage and selection operator (Lasso), both with and without post-thresholding, for high-dimensional Ising models. For random regular (RR) graphs of size $p$ with regular node degree $d$ and uniform couplings $\theta_0$, it is rigorously proved that Lasso without post-thresholding is model selection consistent in the whole paramagnetic phase with the same order of sample complexity $n=\Omega{(d^3\log{p})}$ as that of $\ell_1$-regularized logistic regression ($\ell_1$-LogR). This result is consistent with the conjecture in Meng, Obuchi, and Kabashima 2021 using the non-rigorous replica method from statistical physics and thus complements it with a rigorous proof. For general tree-like graphs, it is demonstrated that the same result as RR graphs can be obtained under mild assumptions of the dependency condition and incoherence condition. Moreover, we provide a rigorous proof of the model selection consistency of Lasso with post-thresholding for general tree-like graphs in the paramagnetic phase without further assumptions on the dependency and incoherence conditions. Experimental results agree well with our theoretical analysis.

We analyze the performance of the least absolute shrinkage and selection operator (Lasso) for the linear model when the number of regressors $N$ grows larger keeping the true support size $d$ finite, i.e., the ultra-sparse case. The result is based on a novel treatment of the non-rigorous replica method in statistical physics, which has been applied only to problem settings where $N$, $d$ and the number of observations $M$ tend to infinity at the same rate. Our analysis makes it possible to assess the average performance of Lasso with Gaussian sensing matrices without assumptions on the scaling of $N$ and $M$, the noise distribution, and the profile of the true signal. Under mild conditions on the noise distribution, the analysis also offers a lower bound on the sample complexity necessary for partial and perfect support recovery when $M$ diverges as $M = O(\log N)$. The obtained bound for perfect support recovery is a generalization of that given in previous literature, which only considers the case of Gaussian noise and diverging $d$. Extensive numerical experiments strongly support our analysis.

We consider the problem of recovering an unknown signal from general nonlinear measurements obtained through a generalized linear model (GLM). Based on the unitary transform approximate message passing (UAMP) and expectation propagation, a unitary transform based generalized AMP (GUAMP) algorithm is proposed for general measurement matrices, in particular highly correlated matrices. Experimental results on quantized compressed sensing demonstrate that the proposed GUAMP significantly outperforms state-of-the-art Generalized AMP (AMP) and generalized vector AMP (GVAMP) under correlated matrices.

This work finds the analytical expression of the global minima of a deep linear network with weight decay and stochastic neurons, a fundamental model for understanding the landscape of neural networks. Our result implies that zero is a special point in deep neural network architecture. We show that weight decay strongly interacts with the model architecture and can create bad minima at zero in a network with more than $1$ hidden layer, qualitatively different from a network with only $1$ hidden layer. Practically, our result implies that common deep learning initialization methods are insufficient to ease the optimization of neural networks in general.

We theoretically analyze the typical learning performance of $\ell_{1}$-regularized linear regression ($\ell_1$-LinR) for Ising model selection using the replica method from statistical mechanics. For typical random regular graphs in the paramagnetic phase, an accurate estimate of the typical sample complexity of $\ell_1$-LinR is obtained. Remarkably, despite the model misspecification, $\ell_1$-LinR is model selection consistent with the same order of sample complexity as $\ell_{1}$-regularized logistic regression ($\ell_1$-LogR), i.e., $M=\mathcal{O}\left(\log N\right)$, where $N$ is the number of variables of the Ising model. Moreover, we provide an efficient method to accurately predict the non-asymptotic behavior of $\ell_1$-LinR for moderate $M, N$, such as precision and recall. Simulations show a fairly good agreement between theoretical predictions and experimental results, even for graphs with many loops, which supports our findings. Although this paper mainly focuses on $\ell_1$-LinR, our method is readily applicable for precisely characterizing the typical learning performances of a wide class of $\ell_{1}$-regularized $M$-estimators including $\ell_1$-LogR and interaction screening.

Non-orthogonal multiple access (NOMA) on shared resources has been identified as a promising technology in 5G to improve resource efficiency and support massive access in all kinds of transmission modes. Power domain and code domain NOMA have been extensively studied and evaluated in both literatures and 3GPP standardization, especially for the uplink where large number of users would like to send their messages to the base station. Though different in the transmitter side design, power domain NOMA and code domain NOMA share the same need of the advanced multi-user detection (MUD) design at the receiver side. Various multi-user detection algorithms have been proposed, balancing performance and complexity in different ways, which is important for the implementation of NOMA in practical networks. In this paper, we introduce a unified variational inference (VI) perspective on various universal NOMA MUD algorithms such as belief propagation (BP), expectation propagation (EP), vector EP (VEP), approximate message passing (AMP) and vector AMP (VAMP), demonstrating how they could be derived from and adapted to each other within the VI framework. Moreover, we unveil and prove that conventional elementary signal estimator (ESE) and linear minimum mean square error (LMMSE) receivers are special cases of EP and VEP, respectively, thus bridging the gap between classic linear receivers and message passing based nonlinear receivers. Such a unified perspective would not only help the design and adaptation of NOMA receivers, but also open a door for the systematic design of joint active user detection and multi-user decoding for sporadic grant-free transmission.

Neural networks with binary weights are computation-efficient and hardware-friendly, but their training is challenging because it involves a discrete optimization problem. Surprisingly, ignoring the discrete nature of the problem and using gradient-based methods, such as the Straight-Through Estimator, still works well in practice. This raises the question: are there principled approaches which justify such methods? In this paper, we propose such an approach using the Bayesian learning rule. The rule, when applied to estimate a Bernoulli distribution over the binary weights, results in an algorithm which justifies some of the algorithmic choices made by the previous approaches. The algorithm not only obtains state-of-the-art performance, but also enables uncertainty estimation and continual learning to avoid catastrophic forgetting. Our work provides a principled approach for training binary neural networks which also justifies and extends existing approaches.

Non-orthogonal multiple access (NoMA) as an efficient way of radio resource sharing has been identified as a promising technology in 5G to help improving system capacity, user connectivity, and service latency in 5G communications. This paper provides a brief overview of the progress of NoMA transceiver study in 3GPP, with special focus on the design of turbo-like iterative multi-user (MU) receivers. There are various types of MU receivers depending on the combinations of MU detectors and interference cancellation (IC) schemes. Link-level simulations show that expectation propagation algorithm (EPA) with hybrid parallel interference cancellation (PIC) is a promising MU receiver, which can achieve fast convergence and similar performance as message passing algorithm (MPA) with much lower complexity.

In this letter, we present a unified Bayesian inference framework for generalized linear models (GLM), which iteratively reduces the GLM problem to a sequence of standard linear model (SLM) problems. This framework provides new perspectives on some established GLM algorithms derived from SLM ones and also suggests novel extensions for some other SLM algorithms. Specific instances elucidated under such framework are the GLM versions of approximate message passing (AMP), vector AMP, and sparse Bayesian learning. It is proved that the resultant GLM version of AMP is equivalent to the well-known generalized approximate message passing. Numerical results for one-bit quantized compressed sensing demonstrate the effectiveness of this unified framework.

Sparse code multiple access (SCMA) scheme is considered to be one promising non-orthogonal multiple access technology for the future fifth generation (5G) communications. Due to the sparse nature, message passing algorithm (MPA) has been used at the receiver to achieve close to maximum likelihood (ML) detection performance with much lower complexity. However, the complexity order of MPA is still exponential with the size of codebook and the degree of signal superposition on a given resource element. In this paper, we propose a novel low complexity iterative receiver based on expectation propagation algorithm (EPA), which reduces the complexity order from exponential to linear. Simulation results demonstrate that the proposed EPA receiver achieves nearly the same block error rate (BLER) performance as the conventional message passing algorithm (MPA) receiver with orders less complexity.

Multibeam satellites allow for significant boost in capacity by reusing the available spectrum and regularized zero-forcing (RZF) precoding promises to be one efficient technique to manage the inter-beam interference in the forward link. In this paper, the RZF precoding problem is first cast within an equivalent Bayesian inference framework. Then, we propose two kinds of message passing based precoding approaches, namely variational message passing based RZF (VMP-RZF) and approximate message passing based RZF (AMP-RZF). Compared with evaluating RZF directly, our proposed methods circumvent the matrix inversion operation and thus enjoy a much lower complexity. Simulation results demonstrate that the normalized mean square errors of both VMP-RZF and AMP-RZF with respect to the true RZF are negligible while AMP-RZF converges faster than VMP-RZF.

Iterative decoding of orthogonal convolutional code is widely used for its excellent performance. However, the iterations lead to high complexity and long decoding delay, which is unsuitable for low- rate voice service in mobile satellite communications with on-board processing. In this paper, a novel scheme of orthogonal convolutional coding as well as an associated non-iterative joint decoding algorithm based on factor graph are proposed. Due to its non-iterative nature, this novel scheme has low decoding complexity and short latency, which indicates its potential on-board use in the low-rate satellite voice service, or other kinds of services with a high bit error rate (BER) tolerance and a tight delay requirement. Simulation results demonstrate the efficacy of the proposed novel coding scheme and non-iterative decoding algorithm in both AWGN and Rician channels.

An alternative derivation for the well-known approximate message passing (AMP) algorithm proposed by Donoho is presented in this letter. Compared with the original derivation, which exploits central limit theorem and Taylor expansion to simplify belief propagation (BP), our derivation resorts to expectation propagation (EP) and the neglect of high-order terms in large system limit. This alternative derivation leads to a different yet provably equivalent form of message passing, which explicitly establishes the intrinsic connection between AMP and EP, thereby offering some new insights in the understanding and improvement of AMP.

In this paper, we propose an expectation propagation based iterative multi-user detection algorithm for multiple input multiple output interleave-division multiple access (MIMO-IDMA) systems with high-order modulation. The proposed detector can be well integrated into the traditional structure of turbo receivers for MIMO-IDMA systems. By formulating a scalar factor graph representation of the multi-user detector and choosing Gaussian distribution as the projection set for the symbol belief, the overall detection complexity can be reduced to scaling linearly with the number of users, the number of receive antennas and transmit antennas. Numerical results for coded MIMO-IDMA systems with 16-QAM modulation show that our proposed algorithm outperforms the factor graph based detection with Gaussian approximation in terms of the bit error rate (BER) performance with lower complexity.