Image super-resolution aims to enhance the detail representation and visual clarity of low-resolution images. Although latent diffusion model (LDM)-based super-resolution approaches have achieved remarkable progress, their use of aggressive downsampling to compress degraded images into the latent space often results in loss of high-frequency details, thereby limiting the structural fidelity of the reconstructed images. To address this, we propose a plug-and-play Edge Reconstruction Network (ERN) that recovers high-resolution edge prior from low-resolution inputs. By explicitly including this high-resolution edge prior as structural priors in the diffusion process, the model’s ability to reconstruct high-frequency details can be greatly improved. In ERN, new edges of images are calculated using edge pixels. Since edge pixels are treated as discrete sampling points on the edge curve, the neural network interpolates edge pixels only, effectively eliminating non-edge information interference, thereby reducing jagged edges and block artifacts. In addition, the network’s reconstruction accuracy and robustness in complex structure scenes improve significantly as it learns the mapping between low-resolution images and their corresponding high-resolution edge images. Another key contribution is an automatic label generation method based on surface fitting, which extracts edge labels with quadratic polynomial accuracy from GT images, providing reliable supervision for edge reconstruction and alleviating the scarcity of high-quality edge labels. Extensive experiments demonstrate ERN’s effectiveness on SR task. It can be seamlessly integrated into existing LDM-based methods, with only ∼ 15M additional parameters yielding a 0.1 dB ∼ 1.2 dB PSNR gain, achieving a favorable balance between performance and computational cost. The code will be released at https://github.com/YunyangXu/ERN .

Semi-supervised learning (SSL) has emerged as a promising paradigm for medical image segmentation, aiming to alleviate the scarcity of high-quality annotations by combining limited labeled and abundant unlabeled data. However, existing SSL methods suffer from inherent limitations: 1) consistency regularization overly relies on enforcing prediction consistency under different perturbations, neglecting deep exploration of semantic and discriminative features; 2) pseudo-labeling methods are prone to introducing noise, which in turn undermines the stability of model training. To enable high-quality and more stable model learning, we propose a common pattern prior-driven network (CPP-Net) for semi-supervised medical image segmentation. To improve training quality, CPP-Net proposes a pattern learning mechanism that extracts each class's core semantic information for high-quality feature learning. At its core, it is a dynamically updated common pattern bank (CP-Bank), which stores class-specific patterns learned throughout training and serves as high-quality prior knowledge for the model. By reusing CP-Bank patterns, CPP-Net reconstructs current-stage features, reduces redundant learning of shared patterns, and boosts feature robustness and discriminability. Furthermore, an information gain-driven update strategy is proposed to ensure that the CP-Bank is aligned with the historical mean of pattern distributions, preventing excessive bias toward transient local patterns. To enhance training stability, a dynamic regulation function is designed to adaptively modulate the impact of pseudo-labels according to their confidence, thereby mitigating the adverse effects of low-confidence data. Through extensive experiments on various 2D/3D medical image segmentation datasets, CPP-Net demonstrates its effectiveness and generalizability, and achieves 7.5% mean Dice improvement over SOTA.

Time series forecasting is a crucial task in various domains. Caused by factors such as trends, seasonality, or irregular fluctuations, time series often exhibits non-stationary. It obstructs stable feature propagation through deep layers, disrupts feature distributions, and complicates learning data distribution changes. As a result, many existing models struggle to capture the underlying patterns, leading to degraded forecasting performance. In this study, we tackle the challenge of non-stationarity in time series forecasting with our proposed framework called U-Mixer. By combining Unet and Mixer, U-Mixer effectively captures local temporal dependencies between different patches and channels separately to avoid the influence of distribution variations among channels, and merge low- and high-levels features to obtain comprehensive data representations. The key contribution is a novel stationarity correction method, explicitly restoring data distribution by constraining the difference in stationarity between the data before and after model processing to restore the non-stationarity information, while ensuring the temporal dependencies are preserved. Through extensive experiments on various real-world time series datasets, U-Mixer demonstrates its effectiveness and robustness, and achieves 14.5% and 7.7% improvements over state-of-the-art (SOTA) methods.

Many contrastive learning based models have achieved advanced performance in image-text matching tasks. The key of these models lies in analyzing the correlation between image-text pairs, which involves cross-modal interaction of embeddings in corresponding dimensions. However, the embeddings of different modalities are from different models or modules, and there is a significant modality gap. Directly interacting such embeddings lacks rationality and may capture inaccurate correlation. Therefore, we propose a novel method called DIAS to bridge the modality gap from two aspects: (1) We align the information representation of embeddings from different modalities in corresponding dimension to ensure the correlation calculation is based on interactions of similar information. (2) The spatial constraints of inter- and intra-modalities unmatched pairs are introduced to ensure the effectiveness of semantic alignment of the model. Besides, a sparse correlation algorithm is proposed to select strong correlated spatial relationships, enabling the model to learn more significant features and avoid being misled by weak correlation. Extensive experiments demonstrate the superiority of DIAS, achieving 4.3%-10.2% rSum improvements on Flickr30k and MSCOCO benchmarks.

Superpixel generation is increasingly an important area for computer vision tasks. While superpixels with highly regular shapes are preferred to make the subsequent processing easier, the accuracy of the superpixel boundaries is also necessary. Previous methods usually depend on a distance function considering both spatial and color coherency regularization on the whole image, which however is hard to balance between shape regularity and boundary adherence, especially when the desired number of superpixels is small. In addition, non-adaptive parameters and insufficient contour information also affect the performance of segmentation. To mitigate these problems, we propose a robust divide-and-conquer superpixel segmentation method, of which the core idea is that we apply a new contour information extraction and a pixel clustering to separate the input image into flat and non-flat regions, where the former targets shape regularity and the latter emphasizes boundary adherence, followed by an efficient hierarchical merging to clean up tiny and dangling superpixels. Our algorithm requires no additional parameter tuning except the desired number of superpixels since our internal parameters are self-adaptive to the image contents. Experimental results demonstrate that for public benchmark datasets, our algorithm consistently generates more regular superpixels with stronger boundary adherence than state-of-the-art methods while maintaining a competitive efficiency. The source code is available athttps://github.com/YunyangXu/HQSGRD.

Smoothing images, especially with rich texture, is an important problem in computer vision. Obtaining an ideal result is difficult due to complexity, irregularity, and anisotropicity of the texture. Besides, some properties are shared by the texture and the structure in an image. It is a hard compromise to retain structure and simultaneously remove texture. To create an ideal algorithm for image smoothing, we face three problems. For images with rich textures, the smoothing effect should be enhanced. We should overcome inconsistency of smoothing results in different parts of the image. It is necessary to create a method to evaluate the smoothing effect. We apply texture pre-removal based on global sparse decomposition with a variable smoothing parameter to solve the first two problems. A parametric surface constructed by an improved Bessel method is used to determine the smoothing parameter. Three evaluation measures: edge integrity rate, texture removal rate, and gradient value distribution are proposed to cope with the third problem. We use the alternating direction method of multipliers to complete the whole algorithm and obtain the results. Experiments show that our algorithm is better than existing algorithms both visually and quantitatively. We also demonstrate our method's ability in other applications such as clip-art compression artifact removal and content-aware image manipulation.

Abstract Self‐similarity, a prior of natural images, has attracted much attention. The attribute means that low‐rank group matrices can be constructed from similar image patches. For low‐rank approximation denoising methods based on singular value decomposition (SVD) the ability to accurately construct group matrices with noise and handle singular values are keys. Here, combining image priors, a two‐stage clustering method to adaptively construct group matrices is designed. The method is anti‐noise, that is, when noise levels are high, these matrices are more accurate than that constructed by other algorithms. Then, according to the significance of singular values and singular vectors, singular vectors of the low‐rank estimations are corrected so that the residual noise in the low‐rank estimations is further suppressed. For back projection , the authors use the original noise level and the residual image to adaptively determine projection parameters and new noise levels . So, authors' back projection can provide a good foundation for authors' two‐stage denoising methods, better remove noise and preserve image details. Experimental results show that compared with the existing state‐of‐the‐art denoising algorithms, the proposed algorithm achieves competitive denoising performances in terms of quantitative metrics and preserving details. Especially with the increase of noise, the competitiveness of authors' algorithms is gradually enhanced.

Abstract Coronary artery calcium is a strong and independent marker of atherosclerosis and cardiovascular disease. Typically, the accurate segmentation of computed tomography images of the chest is an important prerequisite and basis for coronary artery calcium identification and analysis. However, this is very challenging in practice because the boundaries of coronary artery calcium, the small lesions with large shape variation, are very blurry, resulting in poor performance in existing studies. To tackle this challenge, we present a novel Attention‐based Multi‐Scale Network called AMSN, which can process information through both the main and boundary branches in parallel. Key to our AMSN is a new non‐local multi‐scale context encoder module, which is mainly composed of the multi‐scale attention mechanism and local global long short‐term memory module. By aggregating the multi‐scale context information, i.e. high‐resolution low‐level and low‐resolution high‐level features, the model's feature representative capability and deployment ability are improved effectively. Besides, we introduce a new boundary preserving loss, which can consider the boundary information of all coronary artery calcium together and establish links for the segmentation of different coronary artery calcium simultaneously. Extensive experiments demonstrate our AMSN enables reliable accurate coronary artery calcium segmentation for assisted cardiovascular disease diagnosis clinically.

Based on the image self‐similarity and singular value decomposition (SVD) techniques, the authors propose an iterative adaptive global denoising method. For the structural differences between image patches, they adaptively determine the size of the search window. In each window, a similar image patch matrix is constructed based on the multi‐scale similarity measure. In order to ensure the speed of the method, the adaptive step size and the number of image patches are introduced, and all image patches are denoised in different iterations. This not only ensures the speed of the method, suppresses residual noise, but also reduces the artefacts caused by the fixed step size and the number of image patches. Therefore, the problem of image denoising is converted to the estimation of low‐rank matrix. New singular values are estimated according to the noise level, and similar image patch matrices without noise are estimated using them and corresponding singular vectors. Experimental results show that compared with the state‐of‐the‐art denoising algorithms, this method has a higher PSNR and FSIM, and has a good visual effect. The new method can be applied to image and video restoration, target recognition and image classification.

Part of important structural edges in the image is smoothed due to the small gradients, while the others are preserved with greater gradients. Therefore, the authors propose a two‐stage image smoothing method based on edge‐patch histogram equalisation and patch decomposition. The authors' purpose is to increase the gradient of important structural edges while reducing the gradient of the texture region. Therefore, they divide the image into edge‐patches where the structural edges are concentrated or non‐edge‐patches where the texture details are concentrated by image segmentation. The edge‐patch needs to be equalised by the histograms for increasing the gradient of the edge pixels. All patches are decomposed to extract the smooth component for reducing the gradient of pixels. The smooth component of each patch is smoothed via gradient minimisation. In order to ensure the continuity of the patch boundaries, the edge‐patch is inversely equalised. Finally, the whole image is smoothed via gradient minimisation for removing residual textures and seams. Experimental results demonstrate that the proposed method is more competitive in maintaining important structural edges and removing texture details than the state‐of‐the‐art approaches. The proposed method can be applied to many areas of image processing.

Natural image priors play an important role in image denoising, and various prior-based methods have been widely proposed for noise removal. However, these methods tend to smooth the fine image textures while suppressing noise, degrading the image visual quality. To address this problem, in this paper, we propose an adaptive texture-preserving denoising method. In contrast to most existing prior-based denoising methods, two types of priors [gradient histogram matching priors and nonlocal self-similarity (NSS) priors] are proposed, and their combination is used for image denoising. We introduce a hyper-Laplacian distribution of the gradient histogram matching prior, which enforces the gradient histogram of the denoised image to be as close as possible to the estimated reference histogram from the original image. Meanwhile, the proposed model obtained by introducing the NSS priors effectively preserves fine image details and generates sharp image edges. To improve the accuracy of the method, a content-adaptive parameter selection scheme based on edge detection filters is proposed. Moreover, the optimization problem with two types of priors and the content-adaptive parameter added into the objective function becomes a challenging non-convex optimization problem. To effectively solve this problem, we have developed a new numerical solution based on augmented Lagrangian multipliers and alternating minimization scheme. The experimental results demonstrate that the proposed method effectively preserves the texture features of the denoised images and outperforms several variational methods and other state-of-the-art methods in terms of various evaluation indices and visual quality, especially at medium and high noise levels.

Example-based super-resolution algorithms, which predict unknown high-resolution image information using a relationship model learnt from known high- and low-resolution image pairs, have attracted considerable interest in the field of image processing. In this paper, we propose a multi-example feature-constrained back-projection method for image super-resolution. Firstly, we take advantage of a feature-constrained polynomial interpolation method to enlarge the low-resolution image. Next, we consider low-frequency images of different resolutions to provide an example pair. Then, we use adaptive k NN search to find similar patches in the low-resolution image for every image patch in the high-resolution low-frequency image, leading to a regression model between similar patches to be learnt. The learnt model is applied to the low-resolution high-frequency image to produce high-resolution high-frequency information. An iterative back-projection algorithm is used as the final step to determine the final high-resolution image. Experimental results demonstrate that our method improves the visual quality of the high-resolution image.

As one of the most popular image oversegmentations, superpixel has been commonly used as supporting regions for primitives to reduce computations in various computer vision tasks. In this paper, we propose a novel superpixel segmentation approach based on a distance function that is designed to balance among boundary adherence, intensity homogeneity, and compactness (COM) characteristics of the resulting superpixels. Given an expected number of superpixels, our method begins with initializing the superpixel seed positions to obtain the initial labels of pixels. Then, we optimize the superpixels iteratively based on the defined distance measurement. We update the positions and intensities of superpixel seeds based on the three-sigma rule. The experimental results demonstrate that our algorithm is more effective and accurate than previous superpixel methods and achieves a comparable tradeoff between superpixel COM and adherence to object boundaries.

Image super‐resolution (SR) for a single low‐resolution image is an important and challenging task in image processing. In this study, the authors propose a novel non‐local feature back‐projection method for image SR, which can effectively reduce jaggy and ringing artefacts common, in general, iterative back‐projection (IBP) method. In their method, the objective high‐resolution (HR) image is obtained by projecting reconstructed errors back to HR image iteratively. To optimise the initial HR image and constrain anisotropic errors propagation during IBP process, an efficient non‐local feature interpolation algorithm is designed. Specially, edge information is used as constraints to make the interpolation surface preserve better shape. Furthermore, as post‐processing, non‐local similarities are utilised to remove noise and irregularities induced by errors propagation. Experimental results show that their method achieves better performance than state‐of‐the‐art methods in terms of both quantitative metrics and visual qualities.

This paper proposes a new algorithm based on low-rank matrix recovery to remove salt & pepper noise from surveillance video. Unlike single image denoising techniques, noise removal from video sequences aims to utilize both temporal and spatial information. By grouping neighboring frames based on similarities of the whole images in the temporal domain, we formulate the problem of removing salt & pepper noise from a video tracking sequence as a low-rank matrix recovery problem. The resulting nuclear norm and L 1-norm related minimization problems can be efficiently solved by many recently developed methods. To determine the low-rank matrix, we use an averaging method based on other similar images. Our method can not only remove noise but also preserve edges and details. The performance of our proposed approach compares favorably to that of existing algorithms and gives better PSNR and SSIM results.

Based on the assumption that low-resolution image is sampled from an original scene approximated by piecewise polynomial surface, this paper proposes two different constraints to make the fitting surface of the original scene preserve the image's edge characteristics. We first construct a cubic parametric curve to approximate the edge in each local area and obtain an auxiliary pixel set by re-sampling the curve, then introduce a weight function to accord pixels different degrees of effects on surface reconstruction. By re-sampling the constructed surface, the enlarged image can be easily obtained. Extensive experimental results on various types of low-resolution images demonstrate that our method produces generally better results both in terms of quantitative evaluation and subjective visual quality.

ABSTRACT This paper presents a new robust multi‐level partition of unity (MPU) method, which constructs an implicit surface from a triangular mesh via the new error metric between the mesh and the implicit surface. The new error metric employs a weighted function of inner points and vertices of a triangle to fit an implicit surface, which can control the approximation error between the surface and vertices of the triangle. Furthermore, it is applied to the MPU method by utilizing the dual graph of a triangular mesh, and the general quadric implicit surface is used for surface representation. Compared with the MPU method, the new method generates fewer subdivision cells with the same approximation error and performs more steadily especially when given triangular mesh with fewer vertices. Copyright © 2013 John Wiley & Sons, Ltd.

Conventional polynomial interpolation methods produce images with blurred edges, while edge-directed interpolation methods make enlarged images with good quality edges but with detail distortion in the non-edge portion. A new method for constructing a fitting surface to image data is presented. Unlike existing methods which produce enlarged images using image data as interpolation data, the new method constructs the fitting surface using the image data as constraints to reverse the sampling process for improving the fitting precision. To remove the zigzagging artifact, for each pixel and its nearby region, the edge information is used to determine the quadratic polynomial which approximates the original scene with a quadratic polynomial precision. Comparison results of the new method with other methods are included.

There has been a significant improvement in stereo matching with the introduction of adaptive support weights. Existing local methods mainly focus on the computation of support weight which is critical in cost aggregation and usually get excellent results. However, the negative effects of occluded regions are often ignored, which results in the problem of foreground fattening and blurred depth borders. This paper proposes a novel support aggregation strategy by utilizing the occlusion information obtained from left-right consistency check. The weights of invalid points are noticeably reduced at each disparity estimation stage. Experimental results on the Middlebury images show that our method is highly effective in improving the disparities of points around occluded areas and depth discontinuities. According to the Middlebury benchmark, the proposed method achieves the best performance among all the local methods. Moreover, our approach can be easily integrated into nearly all the existing support weights strategies.

The problem of constructing surface to fit image data is discussed. The data points of an image can be regarded being sampled from an original surface which can be approximated by piecewise quadratic polynomials. On each local region, a quadratic polynomial surface is constructed with the image data as constraint. The combination of all the quadratic polynomial surfaces forms the fitting surface which approximates the original surface with a quadratic polynomial precision. The experiments for comparing the new method with the existing ones are included.