Cross-modal alignment is a crucial task in multimodal learning aimed at achieving semantic consistency between vision and language. This requires that image-text pairs exhibit similar semantics. Traditional algorithms pursue embedding consistency to achieve semantic consistency, ignoring the non-semantic information present in the embedding. An intuitive approach is to decouple the embeddings into semantic and modality components, aligning only the semantic component. However, this introduces two main challenges: (1) There is no established standard for distinguishing semantic and modal information. (2) The modality gap can cause semantic alignment deviation or information loss. To align the true semantics, we propose a novel cross-modal alignment algorithm via Constrained Decoupling and Distribution Sampling (CDDS). Specifically, (1) A dual-path UNet is introduced to adaptively decouple the embeddings, applying multiple constraints to ensure effective separation. (2) A distribution sampling method is proposed to bridge the modality gap, ensuring the rationality of the alignment process. Extensive experiments on various benchmarks and model backbones demonstrate the superiority of CDDS, outperforming state-of-the-art methods by 6.6% to 14.2%.

Image super-resolution aims to enhance the detail representation and visual clarity of low-resolution images. Although latent diffusion model (LDM)-based super-resolution approaches have achieved remarkable progress, their use of aggressive downsampling to compress degraded images into the latent space often results in loss of high-frequency details, thereby limiting the structural fidelity of the reconstructed images. To address this, we propose a plug-and-play Edge Reconstruction Network (ERN) that recovers high-resolution edge prior from low-resolution inputs. By explicitly including this high-resolution edge prior as structural priors in the diffusion process, the model’s ability to reconstruct high-frequency details can be greatly improved. In ERN, new edges of images are calculated using edge pixels. Since edge pixels are treated as discrete sampling points on the edge curve, the neural network interpolates edge pixels only, effectively eliminating non-edge information interference, thereby reducing jagged edges and block artifacts. In addition, the network’s reconstruction accuracy and robustness in complex structure scenes improve significantly as it learns the mapping between low-resolution images and their corresponding high-resolution edge images. Another key contribution is an automatic label generation method based on surface fitting, which extracts edge labels with quadratic polynomial accuracy from GT images, providing reliable supervision for edge reconstruction and alleviating the scarcity of high-quality edge labels. Extensive experiments demonstrate ERN’s effectiveness on SR task. It can be seamlessly integrated into existing LDM-based methods, with only ∼ 15M additional parameters yielding a 0.1 dB ∼ 1.2 dB PSNR gain, achieving a favorable balance between performance and computational cost. The code will be released at https://github.com/YunyangXu/ERN .

Semi-supervised learning (SSL) has emerged as a promising paradigm for medical image segmentation, aiming to alleviate the scarcity of high-quality annotations by combining limited labeled and abundant unlabeled data. However, existing SSL methods suffer from inherent limitations: 1) consistency regularization overly relies on enforcing prediction consistency under different perturbations, neglecting deep exploration of semantic and discriminative features; 2) pseudo-labeling methods are prone to introducing noise, which in turn undermines the stability of model training. To enable high-quality and more stable model learning, we propose a common pattern prior-driven network (CPP-Net) for semi-supervised medical image segmentation. To improve training quality, CPP-Net proposes a pattern learning mechanism that extracts each class's core semantic information for high-quality feature learning. At its core, it is a dynamically updated common pattern bank (CP-Bank), which stores class-specific patterns learned throughout training and serves as high-quality prior knowledge for the model. By reusing CP-Bank patterns, CPP-Net reconstructs current-stage features, reduces redundant learning of shared patterns, and boosts feature robustness and discriminability. Furthermore, an information gain-driven update strategy is proposed to ensure that the CP-Bank is aligned with the historical mean of pattern distributions, preventing excessive bias toward transient local patterns. To enhance training stability, a dynamic regulation function is designed to adaptively modulate the impact of pseudo-labels according to their confidence, thereby mitigating the adverse effects of low-confidence data. Through extensive experiments on various 2D/3D medical image segmentation datasets, CPP-Net demonstrates its effectiveness and generalizability, and achieves 7.5% mean Dice improvement over SOTA.

Deep learning has achieved significant advancements in medical image segmentation, but existing models still face challenges in accurately segmenting lesion regions. The main reason is that some lesion regions in medical images have unclear boundaries, irregular shapes, and small tissue density differences, leading to label ambiguity. However, the existing model treats all data equally without taking quality differences into account in the training process, resulting in noisy labels negatively impacting model training and unstable feature representations. In this paper, a data-driven alternating learning (DALE) paradigm is proposed to optimize the model’s training process, achieving stable and high-precision segmentation. The paradigm focuses on two key points: (1) reducing the impact of noisy labels, and (2) calibrating unstable representations. To mitigate the negative impact of noisy labels, a loss consistency-based collaborative optimization method is proposed, and its effectiveness is theoretically demonstrated. Specifically, the label confidence parameters are introduced to dynamically adjust the influence of labels of different confidence levels during model training, thus reducing the influence of noise labels. To calibrate the learning bias of unstable representations, a distribution alignment method is proposed. This method restores the underlying distribution of unstable representations, thereby enhancing the discriminative capability of fuzzy region representations. Extensive experiments on various benchmarks and model backbones demonstrate the superiority of the DALE paradigm, achieving an average performance improvement of up to 7.16%.

Cross-modal alignment is an important multi-modal task, aiming to bridge the semantic gap between different modalities. The most reliable fundamention for achieving this objective lies in the semantic consistency between matched pairs. Conventional methods implicitly assume embeddings contain solely semantic information, ignoring the impact of non-semantic information during alignment, which inevitably leads to information bias or even loss. These non-semantic information primarily manifest as stylistic variations in the data, which we formally define as style information. An intuitive approach is to separate style from semantics, aligning only the semantic information. However, most existing methods distinguish them based on feature columns, which cannot represent the complex coupling relationship between semantic and style information. In this paper, we propose PICO, a novel framework for suppressing style interference during embedding interaction. Specifically, we quantify the probability of each feature column representing semantic information, and regard it as the weight during the embedding interaction. To ensure the reliability of the semantic probability, we propose a prototype iterative construction method. The key operation of this method is a performance feedback-based weighting function, and we have theoretically proven that the function can assign higher weight to prototypes that bring higher performance improvements. Extensive experiments on various benchmarks and model backbones demonstrate the superiority of PICO, outperforming state-of-the-art methods by 5.2%-14.1%.

Multivariate time series (MTS) forecasting is considered as a challenging task due to complex and nonlinear interdependencies between time steps and series. With the advance of deep learning, significant efforts have been made to model long-term and short-term temporal patterns hidden in historical information by recurrent neural networks (RNNs) with a temporal attention mechanism. Although various forecasting models have been developed, most of them are single-scale oriented, resulting in scale information loss. In this article, we seamlessly integrate multiscale analysis into deep learning frameworks to build scale-aware recurrent networks and propose two multiscale recurrent network (MRN) models for MTS forecasting. The first model called MRN-SA adopts a scale attention mechanism to dynamically select the most relevant information from different scales and simultaneously employs input attention and temporal attention to make predictions. The second one named as MRN-CSG introduces a novel cross-scale guidance mechanism to exploit the information from coarse scale to guide the decoding process at fine scale, which results in a lightweight and more easily trained model without obvious loss of accuracy. Extensive experimental results demonstrate that both MRN-SA and MRN-CSG can achieve state-of-the-art performance on five typical MTS datasets in different domains. The source codes will be publicly available at https://github.com/qguo2010/MRN.

Time series forecasting is a crucial task in various domains. Caused by factors such as trends, seasonality, or irregular fluctuations, time series often exhibits non-stationary. It obstructs stable feature propagation through deep layers, disrupts feature distributions, and complicates learning data distribution changes. As a result, many existing models struggle to capture the underlying patterns, leading to degraded forecasting performance. In this study, we tackle the challenge of non-stationarity in time series forecasting with our proposed framework called U-Mixer. By combining Unet and Mixer, U-Mixer effectively captures local temporal dependencies between different patches and channels separately to avoid the influence of distribution variations among channels, and merge low- and high-levels features to obtain comprehensive data representations. The key contribution is a novel stationarity correction method, explicitly restoring data distribution by constraining the difference in stationarity between the data before and after model processing to restore the non-stationarity information, while ensuring the temporal dependencies are preserved. Through extensive experiments on various real-world time series datasets, U-Mixer demonstrates its effectiveness and robustness, and achieves 14.5% and 7.7% improvements over state-of-the-art (SOTA) methods.

Many contrastive learning based models have achieved advanced performance in image-text matching tasks. The key of these models lies in analyzing the correlation between image-text pairs, which involves cross-modal interaction of embeddings in corresponding dimensions. However, the embeddings of different modalities are from different models or modules, and there is a significant modality gap. Directly interacting such embeddings lacks rationality and may capture inaccurate correlation. Therefore, we propose a novel method called DIAS to bridge the modality gap from two aspects: (1) We align the information representation of embeddings from different modalities in corresponding dimension to ensure the correlation calculation is based on interactions of similar information. (2) The spatial constraints of inter- and intra-modalities unmatched pairs are introduced to ensure the effectiveness of semantic alignment of the model. Besides, a sparse correlation algorithm is proposed to select strong correlated spatial relationships, enabling the model to learn more significant features and avoid being misled by weak correlation. Extensive experiments demonstrate the superiority of DIAS, achieving 4.3%-10.2% rSum improvements on Flickr30k and MSCOCO benchmarks.