Time series forecasting is crucial for applications in various domains. Conventional methods often rely on global decomposition into trend, seasonal, and residual components, which become ineffective for real-world series dominated by local, complex, and highly dynamic patterns. Moreover, the high model complexity of such approaches limits their applicability in real-time or resource-constrained environments. In this work, we propose a novel reliability-aware codebook-assisted time series forecasting framework (ReCast) that enables lightweight and robust prediction by exploiting recurring local shapes. ReCast encodes local patterns into discrete embeddings through patch-wise quantization using a learnable codebook, thereby compactly capturing stable regular structures. To compensate for residual variations not preserved by quantization, ReCast employs a dual-path architecture comprising a quantization path for efficient modeling of regular structures and a residual path for reconstructing irregular fluctuations. A central contribution of ReCast is a reliability-aware codebook update strategy, which incrementally refines the codebook via weighted corrections. These correction weights are derived by fusing multiple reliability factors from complementary perspectives by a distributionally robust optimization (DRO) scheme, ensuring adaptability to non-stationarity and robustness to distribution shifts. Extensive experiments demonstrate that ReCast outperforms state-of-the-art (SOTA) models in accuracy, efficiency, and adaptability to distribution shifts.

Cross-modal alignment is a crucial task in multimodal learning aimed at achieving semantic consistency between vision and language. This requires that image-text pairs exhibit similar semantics. Traditional algorithms pursue embedding consistency to achieve semantic consistency, ignoring the non-semantic information present in the embedding. An intuitive approach is to decouple the embeddings into semantic and modality components, aligning only the semantic component. However, this introduces two main challenges: (1) There is no established standard for distinguishing semantic and modal information. (2) The modality gap can cause semantic alignment deviation or information loss. To align the true semantics, we propose a novel cross-modal alignment algorithm via Constrained Decoupling and Distribution Sampling (CDDS). Specifically, (1) A dual-path UNet is introduced to adaptively decouple the embeddings, applying multiple constraints to ensure effective separation. (2) A distribution sampling method is proposed to bridge the modality gap, ensuring the rationality of the alignment process. Extensive experiments on various benchmarks and model backbones demonstrate the superiority of CDDS, outperforming state-of-the-art methods by 6.6% to 14.2%.

Financial time series forecasting is a long‐standing challenge due to the high complexity, randomness, and nonstationary of data. While predictions are typically made based on historical data with multiple features, existing models often fail to effectively integrate and utilize these diverse inputs, limiting forecasting accuracy. To address this, we propose MSA‐xLSTM, a multimodal stock price prediction model incorporating a sentiment attention mechanism. This model is designed to enhance the integration of sentiment features and technical indicators through a multiscale processing approach and sentiment attention module. Specifically, sentiment index sequences are extracted from textual sources such as financial news and stock comments. For technical indicators, both basic and derived metrics are analyzed for correlation with the prediction target, and only highly relevant ones are selected. We validate the effectiveness of each model component through comparative and ablation experiments. Furthermore, backtesting experiments demonstrate the model’s practical value in real‐world trading scenarios.

Image super-resolution aims to enhance the detail representation and visual clarity of low-resolution images. Although latent diffusion model (LDM)-based super-resolution approaches have achieved remarkable progress, their use of aggressive downsampling to compress degraded images into the latent space often results in loss of high-frequency details, thereby limiting the structural fidelity of the reconstructed images. To address this, we propose a plug-and-play Edge Reconstruction Network (ERN) that recovers high-resolution edge prior from low-resolution inputs. By explicitly including this high-resolution edge prior as structural priors in the diffusion process, the model’s ability to reconstruct high-frequency details can be greatly improved. In ERN, new edges of images are calculated using edge pixels. Since edge pixels are treated as discrete sampling points on the edge curve, the neural network interpolates edge pixels only, effectively eliminating non-edge information interference, thereby reducing jagged edges and block artifacts. In addition, the network’s reconstruction accuracy and robustness in complex structure scenes improve significantly as it learns the mapping between low-resolution images and their corresponding high-resolution edge images. Another key contribution is an automatic label generation method based on surface fitting, which extracts edge labels with quadratic polynomial accuracy from GT images, providing reliable supervision for edge reconstruction and alleviating the scarcity of high-quality edge labels. Extensive experiments demonstrate ERN’s effectiveness on SR task. It can be seamlessly integrated into existing LDM-based methods, with only ∼ 15M additional parameters yielding a 0.1 dB ∼ 1.2 dB PSNR gain, achieving a favorable balance between performance and computational cost. The code will be released at https://github.com/YunyangXu/ERN .

Semi-supervised learning (SSL) has emerged as a promising paradigm for medical image segmentation, aiming to alleviate the scarcity of high-quality annotations by combining limited labeled and abundant unlabeled data. However, existing SSL methods suffer from inherent limitations: 1) consistency regularization overly relies on enforcing prediction consistency under different perturbations, neglecting deep exploration of semantic and discriminative features; 2) pseudo-labeling methods are prone to introducing noise, which in turn undermines the stability of model training. To enable high-quality and more stable model learning, we propose a common pattern prior-driven network (CPP-Net) for semi-supervised medical image segmentation. To improve training quality, CPP-Net proposes a pattern learning mechanism that extracts each class's core semantic information for high-quality feature learning. At its core, it is a dynamically updated common pattern bank (CP-Bank), which stores class-specific patterns learned throughout training and serves as high-quality prior knowledge for the model. By reusing CP-Bank patterns, CPP-Net reconstructs current-stage features, reduces redundant learning of shared patterns, and boosts feature robustness and discriminability. Furthermore, an information gain-driven update strategy is proposed to ensure that the CP-Bank is aligned with the historical mean of pattern distributions, preventing excessive bias toward transient local patterns. To enhance training stability, a dynamic regulation function is designed to adaptively modulate the impact of pseudo-labels according to their confidence, thereby mitigating the adverse effects of low-confidence data. Through extensive experiments on various 2D/3D medical image segmentation datasets, CPP-Net demonstrates its effectiveness and generalizability, and achieves 7.5% mean Dice improvement over SOTA.

This paper presents MonoRelief V2, an end-to-end model designed for directly recovering 2.5D reliefs from single images under complex material and illumination variations. In contrast to its predecessor, MonoRelief V1 (Gao et al. 2025), which was solely trained on synthetic data, MonoRelief V2 incorporates real data to achieve improved robustness, accuracy and efficiency. To overcome the challenge of acquiring large-scale real-world dataset, we generate approximately 15,000 pseudo-real images using a text-to-image generative model, and derive corresponding depth pseudo-labels through fusion of depth and normal predictions. Furthermore, we construct a small-scale real-world dataset (800 samples) via multi-view reconstruction and detail refinement. MonoRelief V2 is then progressively trained on the pseudo-real and real-world datasets. Comprehensive experiments demonstrate its state-of-the-art performance both in depth and normal predictions, highlighting its strong potential for a range of downstream applications.

Coronary artery segmentation in X-ray angiography is crucial for cardiovascular diagnosis and treatment planning, yet remains challenging due to the inherently low signal-to-noise ratio (SNR) and low contrast of the images, which degrade obscure vascular structures and fine details. Although the Segment Anything Model (SAM) has demonstrated strong potential in various vision tasks, its capabilities for coronary artery segmentation in X-ray angiography have yet to be fully explored. To overcome these challenges, we propose CoroSAM, a novel promptfree SAM fine-tuning framework tailored for coronary artery segmentation in X-ray angiography. CoroSAM introduces two key innovations: (1) Gated Spatial-Frequency Adapter (GSFA) designed to fine-tune SAM's image encoder by dynamically integrating both spatial- and frequency-domain information to better capture vascular structures under noisy conditions; and (2) Orientation-Guided Adapter (OGA), the first to our knowledge to incorporate orientation maps for refining structural details in SAM predictions, enhancing structural awareness and improving robustness under low-contrast conditions. Extensive experiments on four public datasets demonstrate that CoroSAM consistently outperforms state-of-the-art methods. The code is available at https://github.com/HITZhengGW/CoroSAM

Deep learning has achieved significant advancements in medical image segmentation, but existing models still face challenges in accurately segmenting lesion regions. The main reason is that some lesion regions in medical images have unclear boundaries, irregular shapes, and small tissue density differences, leading to label ambiguity. However, the existing model treats all data equally without taking quality differences into account in the training process, resulting in noisy labels negatively impacting model training and unstable feature representations. In this paper, a data-driven alternating learning (DALE) paradigm is proposed to optimize the model’s training process, achieving stable and high-precision segmentation. The paradigm focuses on two key points: (1) reducing the impact of noisy labels, and (2) calibrating unstable representations. To mitigate the negative impact of noisy labels, a loss consistency-based collaborative optimization method is proposed, and its effectiveness is theoretically demonstrated. Specifically, the label confidence parameters are introduced to dynamically adjust the influence of labels of different confidence levels during model training, thus reducing the influence of noise labels. To calibrate the learning bias of unstable representations, a distribution alignment method is proposed. This method restores the underlying distribution of unstable representations, thereby enhancing the discriminative capability of fuzzy region representations. Extensive experiments on various benchmarks and model backbones demonstrate the superiority of the DALE paradigm, achieving an average performance improvement of up to 7.16%.

Cross-modal alignment is an important multi-modal task, aiming to bridge the semantic gap between different modalities. The most reliable fundamention for achieving this objective lies in the semantic consistency between matched pairs. Conventional methods implicitly assume embeddings contain solely semantic information, ignoring the impact of non-semantic information during alignment, which inevitably leads to information bias or even loss. These non-semantic information primarily manifest as stylistic variations in the data, which we formally define as style information. An intuitive approach is to separate style from semantics, aligning only the semantic information. However, most existing methods distinguish them based on feature columns, which cannot represent the complex coupling relationship between semantic and style information. In this paper, we propose PICO, a novel framework for suppressing style interference during embedding interaction. Specifically, we quantify the probability of each feature column representing semantic information, and regard it as the weight during the embedding interaction. To ensure the reliability of the semantic probability, we propose a prototype iterative construction method. The key operation of this method is a performance feedback-based weighting function, and we have theoretically proven that the function can assign higher weight to prototypes that bring higher performance improvements. Extensive experiments on various benchmarks and model backbones demonstrate the superiority of PICO, outperforming state-of-the-art methods by 5.2%-14.1%.

Tool shape selection and path planning are critical for 5-axis CNC flank milling of freeform surfaces, typically addressed using optimization algorithms where initialization plays a pivotal role. Existing approaches rely on user-specified initialization of either tool shapes or motion paths, often resulting in suboptimal outcomes. This paper introduces a fully automated method that simultaneously initializes both tool shapes and motion paths, achieving high-precision machining with efficient surface coverage. Our approach explores a solution space of potential tool axes represented by line segments near the design surface. To efficiently manage the vast number of lines, we integrate space voxelization with a discrete distance field for effective line sampling. A graph-based algorithm generates feasible line sequences for motion paths, while path optimization refines a single tool shape across multiple paths simultaneously. The method identifies optimal tool shapes of various sizes, each paired with corresponding motion paths for multi-pass machining. Experiments on industrial benchmark models and freeform surfaces validate the effectiveness and practicality of the proposed approach.

In medical imaging detection of oral Cone Beam Computed Tomography (CBCT), there exist tiny lesions that are challenging to detect with low accuracy. The existing detection models are relatively complex. To address this, this paper presents a dual-stage YOLO detection method improved based on YOLOv8. Specifically, we first reconstruct the backbone network based on MobileNetV3 to enhance computational speed and efficiency. Second, we improve detection accuracy from three aspects: we design a composite feature fusion network to enhance the model’s feature extraction capability, addressing the issue of decreased detection accuracy for small lesions due to the loss of shallow information during the fusion process; we further combine spatial and channel information to design the C2f-SCSA module, which delves deeper into the lesion information. To tackle the problem of limited types and insufficient samples of lesions in existing CBCT images, our team collaborated with a professional dental hospital to establish a high-quality dataset, which includes 15 types of lesions and over 2000 accurately labeled oral CBCT images, providing solid data support for model training. Experimental results indicate that the improved method enhances the accuracy of the original algorithm by 3.5 percentage points, increases the recall rate by 4.7 percentage points, and raises the mean Average Precision (mAP) by 3.3 percentage points, a computational load of only 7.6 GFLOPs. This demonstrates a significant advantage in intelligent diagnosis of full-mouth lesions while improving accuracy and reducing computational load.

This study focuses on multi-human relief modeling using a single photograph. Although previous studies successfully modeled 3D humans from single photographs, they were limited to reconstructing 3D individuals and could not be applied to multi-human scenes with complex inter-body and outer-body occlusions. In this study, we introduce MMRelief, a novel solution that takes a significant step toward high-quality and generalized multi-human relief modeling. MMRelief uses a three-step approach to achieve its objectives. First, it predicts an occlusion-aware depth map based on ZoeDepth [12]. Subsequently, it predicts a detailed normal map using a photo-to-normal network. Finally, MMRelief combines the strengths of both maps and constructs human relief using depth-constrained normal integration. Experimental results demonstrate that MMRelief has achieved state-of-the-art performance in normal human estimation. It can handle different styles of human photos with varying poses and dresses while producing reliefs with accurate body occlusions, reasonable depth ordering, and faithful geometrical details. The project page is at https://github.com/yanqingliu3856/MMRelief.

With the application of multisource information in different fields, the combination of different types of information, such as numerical data and text information, has become a favourable choice for performing stock market analyses. Despite the rich information provided by multisource data, building structured relationships remains challenging. In addition, some market relationship-based analysis methods use a predefined graph structure as a stock relationship graph, which makes it impossible to sensitively aggregate attribute features, and these methods cannot dynamically update market relationships or relationship strengths. In this paper, we propose a novel dynamic attribute-driven graph attention network incorporating sentiment (AGATS) information, transaction data, and text data. Inspired by behavioural finance , we separately extract sentiment information as a factor of technical indicators, and further realize the early fusion of technical indicators and textual data through tensor fusion. In particular, real-time intramarket dependencies and key attribute information are captured with graph networks, enabling dynamic relationship and relationship strength updates. Experiments conducted on real datasets show that our model is capable of ourperforming previously developed methods in prediction and trading.

Incomplete multi-view clustering has gained considerable attention in recent years due to the prevalence of incomplete multi-view data in real-world applications. However, existing methods often struggle to effectively deal with large-scale datasets, particularly those with a significant number of missing instances. To address these issues, we propose a novel method called Hubness-Enabled Clustering and Recovery for Large-Scale Incomplete Multi-View Data (HENRI). HENRI utilizes the consensus hubs of all views to identify informative anchors to handle large-scale incomplete datasets. Furthermore, it incorporates a novel sample-level fusion strategy that effectively integrates information from all views, leading to remarkable outcomes in both cluster formation and missing data reconstruction. HENRI demonstrates exceptional capability in capturing the underlying structures of the data and recovering missing information, even when faced with a significant number of instances with incomplete data in partial views. To validate its effectiveness, we conducted experiments on 6 complete datasets and 31 incomplete datasets, comparing against 11 baseline methods. The results are impressive, demonstrating the superior performance of HENRI over the state-of-the-art methods.

Deep implicit functions (DIFs) effectively represent shapes by using a neural network to map 3D spatial coordinates to scalar values that encode the shape's geometry, but it is difficult to establish correspondences between shapes directly, limiting their use in medical image registration. The recently presented deformation field-based methods achieve implicit templates learning via template field learning with DIFs and deformation field learning, establishing shape correspondence through deformation fields. Although these approaches enable joint learning of shape representation and shape correspondence, the decoupled optimization for template field and deformation field, caused by the absence of deformation annotations lead to a relatively accurate template field but an underoptimized deformation field. In this paper, we propose a novel implicit template learning framework via a shared hybrid diffeomorphic flow (SHDF), which enables shared optimization for deformation and template, contributing to better deformations and shape representation. Specifically, we formulate the signed distance function (SDF, a type of DIFs) as a one-dimensional (1D) integral, unifying dimensions to match the form used in solving ordinary differential equation (ODE) for deformation field learning. Then, SDF in 1D integral form is integrated seamlessly into the deformation field learning. Using a recurrent learning strategy, we frame shape representations and deformations as solving different initial value problems of the same ODE. We also introduce a global smoothness regularization to handle local optima due to limited outside-of-shape data. Experiments on medical datasets show that SHDF outperforms state-of-the-art methods in shape representation and registration.

Deep multi-view subspace clustering aims to reveal a common subspace structure by exploiting rich multi-view information. Despite promising progress, current methods focus only on multi-view consistency and complementarity, often overlooking the adverse influence of entangled superfluous information in features. Moreover, most existing works lack scalability and are inefficient for large-scale scenarios. To this end, we innovatively propose a deep subspace clustering method via Multi-view Feature Decoupling (MvFD). First, MvFD incorporates well-designed multi-type auto-encoders with self-supervised learning, explicitly decoupling consistent, complementary, and superfluous features for every view. The disentangled and interpretable feature space can then better serve unified representation learning. By integrating these three types of information within a unified framework, we employ information theory to obtain a minimal and sufficient representation with high discriminability. Besides, we introduce a deep metric network to model self-expression correlation more efficiently, where network parameters remain unaffected by changes in sample numbers. Extensive experiments show that MvFD yields State-of-the-Art performance in various types of multi-view datasets.

Multivariate time series (MTS) forecasting is considered as a challenging task due to complex and nonlinear interdependencies between time steps and series. With the advance of deep learning, significant efforts have been made to model long-term and short-term temporal patterns hidden in historical information by recurrent neural networks (RNNs) with a temporal attention mechanism. Although various forecasting models have been developed, most of them are single-scale oriented, resulting in scale information loss. In this article, we seamlessly integrate multiscale analysis into deep learning frameworks to build scale-aware recurrent networks and propose two multiscale recurrent network (MRN) models for MTS forecasting. The first model called MRN-SA adopts a scale attention mechanism to dynamically select the most relevant information from different scales and simultaneously employs input attention and temporal attention to make predictions. The second one named as MRN-CSG introduces a novel cross-scale guidance mechanism to exploit the information from coarse scale to guide the decoding process at fine scale, which results in a lightweight and more easily trained model without obvious loss of accuracy. Extensive experimental results demonstrate that both MRN-SA and MRN-CSG can achieve state-of-the-art performance on five typical MTS datasets in different domains. The source codes will be publicly available at https://github.com/qguo2010/MRN.

Neural implicit shape representation has drawn significant attention in recent years due to its smoothness, differentiability, and topological flexibility. However, directly modeling the shape of a neural implicit surface, especially as the zero-level set of a neural signed distance function (SDF), with sparse geometric control is still a challenging task. Sparse input shape control typically includes 3D curve networks or, more generally, 3D curve sketches, which are unstructured and cannot be connected to form a curve network, and therefore more difficult to deal with. While 3D curve networks or curve sketches provide intuitive shape control, their sparsity and varied topology pose challenges in generating high-quality surfaces to meet such curve constraints. In this paper, we propose NeuVAS, a variational approach to shape modeling using neural implicit surfaces constrained under sparse input shape control, including unstructured 3D curve sketches as well as connected 3D curve networks. Specifically, we introduce a smoothness term based on a functional of surface curvatures to minimize shape variation of the zero-level set surface of a neural SDF. We also develop a new technique to faithfully model G 0 sharp feature curves as specified in the input curve sketches. Comprehensive comparisons with the state-of-the-art methods demonstrate the significant advantages of our method.

In this article, we introduce MonoRelief, a novel method that combines the strengths of a depth map and a normal map to achieve high-quality relief recovery from a single image. By constructing a large-scale relief dataset that encompasses a diverse range of relief shapes, materials, and lighting conditions, we enable the training of a robust normal estimation network capable of handling various types of relief images. Furthermore, we leverage the state-of-the-art method, DepthAnything v2 (Yang et al. 2024), to generate depth maps from the input images. By integrating the strengths of both maps, MonoRelief recovers 2.5D reliefs with reasonable depth structures and intricate geometrical details. We validate the effectiveness and robustness of MonoRelief through comprehensive experiments, and showcase its potential in a variety of downstream applications, including Image-to-Relief, Text-to-Relief, Lines-to-Relief and relief reproduction.

We propose a new method for computing smooth and integrable cross fields on 2D and 3D surfaces. our approach first computes smooth cross fields by minimizing the Dirichlet energy. Unlike existing optimization-based methods, our technique determines the singularity configuration-i.e., the number, locations, and indices of singularities-by iteratively adjusting them. Singularities can move, merge and split, akin to the behavior of like charges repelling and unlike charges attracting. Once all singularities stop moving, we obtain a cross field with (locally) the lowest Dirichlet energy. In simply connected domains, this cross field is guaranteed to be integrable. However, this property does not hold in multiply connected domains. To make a smooth cross field integrable, we construct a vector field $\bf c$c that characterizes the deviation of the cross field from a curl-free field. We then optimize the locations of singularities by moving them along the field lines of $\bf c$c. Our method is fundamentally different from existing integer programming-based approaches, as it avoids combinatorial optimization. It is fully automatic and includes a parameter to control the number of singularities. Our method is well suited for smooth models where exact boundary alignment and sparse hard directional constraints are desired, and can guide seamless conformal parameterization and T-junction-free quadrangulation.

Time series forecasting is a crucial task in various domains. Caused by factors such as trends, seasonality, or irregular fluctuations, time series often exhibits non-stationary. It obstructs stable feature propagation through deep layers, disrupts feature distributions, and complicates learning data distribution changes. As a result, many existing models struggle to capture the underlying patterns, leading to degraded forecasting performance. In this study, we tackle the challenge of non-stationarity in time series forecasting with our proposed framework called U-Mixer. By combining Unet and Mixer, U-Mixer effectively captures local temporal dependencies between different patches and channels separately to avoid the influence of distribution variations among channels, and merge low- and high-levels features to obtain comprehensive data representations. The key contribution is a novel stationarity correction method, explicitly restoring data distribution by constraining the difference in stationarity between the data before and after model processing to restore the non-stationarity information, while ensuring the temporal dependencies are preserved. Through extensive experiments on various real-world time series datasets, U-Mixer demonstrates its effectiveness and robustness, and achieves 14.5% and 7.7% improvements over state-of-the-art (SOTA) methods.

Many contrastive learning based models have achieved advanced performance in image-text matching tasks. The key of these models lies in analyzing the correlation between image-text pairs, which involves cross-modal interaction of embeddings in corresponding dimensions. However, the embeddings of different modalities are from different models or modules, and there is a significant modality gap. Directly interacting such embeddings lacks rationality and may capture inaccurate correlation. Therefore, we propose a novel method called DIAS to bridge the modality gap from two aspects: (1) We align the information representation of embeddings from different modalities in corresponding dimension to ensure the correlation calculation is based on interactions of similar information. (2) The spatial constraints of inter- and intra-modalities unmatched pairs are introduced to ensure the effectiveness of semantic alignment of the model. Besides, a sparse correlation algorithm is proposed to select strong correlated spatial relationships, enabling the model to learn more significant features and avoid being misled by weak correlation. Extensive experiments demonstrate the superiority of DIAS, achieving 4.3%-10.2% rSum improvements on Flickr30k and MSCOCO benchmarks.

Tensor robust principal component analysis (TRPCA) is a classical way for low-rank tensor recovery, which minimizes the convex surrogate of tensor rank by shrinking each tensor singular value equally. However, for real-world visual data, large singular values represent more significant information than small singular values. In this paper, we propose a nonconvex TRPCA (N-TRPCA) model based on the tensor adjustable logarithmic norm. Unlike TRPCA, our N-TRPCA can adaptively shrink small singular values more and shrink large singular values less. In addition, TRPCA assumes that the whole data tensor is of low rank. This assumption is hardly satisfied in practice for natural visual data, restricting the capability of TRPCA to recover the edges and texture details from noisy images and videos. To this end, we integrate nonlocal self-similarity into N-TRPCA, and further develop a nonconvex and nonlocal TRPCA (NN-TRPCA) model. Specifically, similar nonlocal patches are grouped as a tensor and then each group tensor is recovered by our N-TRPCA. Since the patches in one group are highly correlated, all group tensors have strong low-rank property, leading to an improvement of recovery performance. Experimental results demonstrate that the proposed NN-TRPCA outperforms existing TRPCA methods in visual data recovery. The demo code is available at https://github.com/qguo2010/NN-TRPCA.

Recently, facial-expression recognition (FER) has primarily focused on images in the wild, including factors such as face occlusion and image blurring, rather than laboratory images. Complex field environments have introduced new challenges to FER. To address these challenges, this study proposes a cross-fusion dual-attention network. The network comprises three parts: (1) a cross-fusion grouped dual-attention mechanism to refine local features and obtain global information; (2) a proposed C2 activation function construction method, which is a piecewise cubic polynomial with three degrees of freedom, requiring less computation with improved flexibility and recognition abilities, which can better address slow running speeds and neuron inactivation problems; and (3) a closed-loop operation between the self-attention distillation process and residual connections to suppress redundant information and improve the generalization ability of the model. The recognition accuracies on the RAF-DB, FERPlus, and AffectNet datasets were 92.78%, 92.02%, and 63.58%, respectively. Experiments show that this model can provide more effective solutions for FER tasks.

Currently, the task of remote sensing image segmentation still faces some challenges, such as variations in illumination, shadows, and occlusions present in remote sensing images. Additionally, there may be similarities and confusions between different types of terrain features. In this paper, we aim to explore how to utilize information exchange between multiple modalities to reduce the impact of interfering factors. To fully exploit the complementary information between different modalities, we establish an information exchange mechanism between optical images (visible light + infrared) features and Digital Surface Model (DSM) features. This allows them to interact and express themselves in a shared feature space, facilitating the acquisition of complementary information from different modalities. Furthermore, through a multimodal fusion encoder and decoder based on Transformer design, the optical features and DSM features are integrated, enabling the learning of high-level semantic representations in different dimensions. Extensive subjective, objective comparative experiments, and ablation experiments are conducted on the ISPRS Vaihingen and Potsdam datasets to evaluate the proposed method. The mIoU on the Vaihingen and Potsdam datasets reached 85.06% and 87.6% respectively, while the OA reached 92.01% and 91.92% respectively. The source code will be available at https://github.com/JunyuFan/MIEFNet.

Recently, self-supervised denoising methods have attracted significant attention due to the considerable challenge posed by constructing a large-scale real noise dataset for supervised training. The most representative self-supervised denoisers are based on blind-spot networks (BSNs), which exclude the central pixel of receptive field. However, excluding any input pixel potentially leads to the loss of vital information required for accurate predictions, especially when the excluded pixel corresponds to the output position. In addition, a standard BSN has struggled to effectively reduce real-world noise due to the spatial correlation of noise, though it makes the significant results with independently distributed synthetic noise. In this paper, we propose a novel self-supervised real-world image denoising framework called Complementary-BSN based on two reciprocal branches (Mask-Map branch and Enhanced-PD-BSN branch) with an efficient loss function to employ the pixels information ignored by masked convolution and provide additional optimization target for self-supervised output. Specifically, we exploit a block-wise random-placing (BRP) scheme for further weaken the noisy correlation to avoid the illusion of image structure recovery due to existing complex noise and make Complementary-BSN more suitable for real noise. Additionally, we develop an efficient strategy (multi-stride PD (MPD)) to fuse multiple PD strides for inference, narrowing the restoration gap between textural and flat regions. Extensive experiments on real-world datasets demonstrate that our method achieves superior performance to other state-of-the-art (SOTA) self-supervised denoising methods. The code is available athttps://github.com/cuijin7382/Complementary-BSN.

The emergence of high-resolution (HR) remote sensing imagery showcases the continual advancements in remote sensing technology but also sets higher demands for related tasks in the field, including remote sensing image change detection. Due to their outstanding performance in extracting salient features, convolutional neural networks (CNNs) have played a significant role and become widely utilized in many computer vision tasks. The encoder–decoder structure has confirmed the effectiveness of integrating multilevel feature information, as it allows for the synthesis of both local and global information of features. The exploration of the potential relationships between multilevel features and their efficient integration remains of significant importance. Furthermore, thanks to the advent of the transformer, many modern approaches have seen great improvements in high-level semantic understanding of images. In this article, we propose an attention-filtering network based on a branch transformer for effective change detection in remote sensing images. A hybrid attention fusion module (HAFM) is used to efficiently fuse features of different granularities and perform progressive information filtering on the extracted multilevel features to obtain an effective change feature. We also propose a branch transformer block (BTB) to efficiently aggregate global long-range dependencies and spatial details from the change feature. Extensive comparative experiments conducted on three different HR remote sensing datasets have verified the effectiveness of our method.

Stock prediction is a challenging task due to multiple influencing factors and complex market dependencies. Traditional solutions are based on a single type of information. With the success of multi-source information in different fields, the combination of different types of information such as numerical and textual information has become a promising option. Although multi-source information provides rich multi-view information, how to mine and construct structured relationships from them is a difficult problem. Specifically, most existing methods usually extract features from commonly used multi-source information as predictive information sources, without further pre-constructing stock relationship graphs with dependencies using broader information. More importantly, they typically treat each stock as an isolated forecasting, or employ stock market correlations based on a fixed predefined graph structure, but current methods are not sensitive enough to aggregate the attribute features extracted from multi-source information and stock relationship graph, to obtain the dynamic update of market relations and relationship strength. The stock market is highly temporally, and the attributes of nodes are affected by the time perception of other attributes, which is not fully considered. To address these problems, we propose a novel dynamic attributes-driven graph attention networks incorporating sentiment (DGATS) information, transaction data, and text data. Inspired by behavioral finance, we separately extract sentiment information as a factor of technical indicators, and further realize the early fusion of technical indicators and textual data through Kronecker product-based tensor fusion. In particular, by LSTM and temporal attention network, the short-term and long-term transition features are gradually grasped from the local composition of the fused stock trading sequence. Furthermore, real-time intra-market dependencies and key attributes information are captured with graph networks, enabling dynamic updates of relationships and relationship strengths in predefined graphs. Experiments on the real datasets show that the architecture can outperform the previous methods in prediction performance.

Deep learning models applied to medical image segmentation have achieved remarkable performance across various tasks. Despite their high accuracy, these models may produce predictions that clinicians deem anatomically implausible. This limitation arises due to the inherent anatomical variability and indistinct boundaries present in clinical segmentation targets, impeding the effectiveness of existing research efforts. To overcome these challenges, we propose an innovative architectural framework known as Densely Cross-scale Attention (DCA) network. This framework efficiently incorporates multi-scale local and global attention dependencies using dense connections, resulting in anatomically-plausible segmentations. DCA comprises three new modules: 1) a multi-level feature aggregation module that merges multi-scale local features to acquire feature representations with diverse granularities, thereby compensating for information loss caused by gradient dispersion; 2) a cross-scale attention module that facilitates the modeling of long-range dependencies across scales while effectively diminishing task-irrelevant noise; and 3) a multi-scale self-attention module that captures the multi-scale global spatial relationships of single-scale local features, further enhancing the model’s resilience in addressing complex lesions. The proposed efficient dense connections offer a versatile and potent methodology for modeling long-range dependencies across scales in any segmentation network. The efficacy of our approach is demonstrated through experiments conducted on two clinically relevant datasets, substantiating its ability to achieve superior segmentation accuracy and anatomical plausibility.

This paper aims at extending the method of Zhang et al. (2023) to produce not only portrait bas-reliefs from single photographs, but also high-depth reliefs with reasonable depth ordering. We cast this task as a problem of style-aware photo-to-depth translation, where the input is a photograph conditioned by a style vector and the output is a portrait relief with desired depth style. To construct ground-truth data for network training, we first propose an optimization-based method to synthesize high-depth reliefs from 3D portraits. Then, we train a normal-to-depth network to learn the mapping from normal maps to relief depths. After that, we use the trained network to generate high-depth relief samples using the provided normal maps from Zhang et al. (2023). As each normal map has pixel-wise photograph, we are able to establish correspondences between photographs and high-depth reliefs. By taking the bas-reliefs of Zhang et al. (2023), the new high-depth reliefs and their mixtures as target ground-truths, we finally train a encoder-to-decoder network to achieve style-aware relief modeling. Specially, the network is based on a U-shaped architecture, consisting of Swin Transformer blocks to process hierarchical deep features. Extensive experiments have demonstrated the effectiveness of the proposed method. Comparisons with previous works have verified its flexibility and state-of-the-art performance.

Superpixel generation is increasingly an important area for computer vision tasks. While superpixels with highly regular shapes are preferred to make the subsequent processing easier, the accuracy of the superpixel boundaries is also necessary. Previous methods usually depend on a distance function considering both spatial and color coherency regularization on the whole image, which however is hard to balance between shape regularity and boundary adherence, especially when the desired number of superpixels is small. In addition, non-adaptive parameters and insufficient contour information also affect the performance of segmentation. To mitigate these problems, we propose a robust divide-and-conquer superpixel segmentation method, of which the core idea is that we apply a new contour information extraction and a pixel clustering to separate the input image into flat and non-flat regions, where the former targets shape regularity and the latter emphasizes boundary adherence, followed by an efficient hierarchical merging to clean up tiny and dangling superpixels. Our algorithm requires no additional parameter tuning except the desired number of superpixels since our internal parameters are self-adaptive to the image contents. Experimental results demonstrate that for public benchmark datasets, our algorithm consistently generates more regular superpixels with stronger boundary adherence than state-of-the-art methods while maintaining a competitive efficiency. The source code is available athttps://github.com/YunyangXu/HQSGRD.

Video super-resolution techniques aim to obtain high-resolution equivalents of existing low-resolution videos through a series of operations. In recent research, transformers have been increasingly popular because of their remarkable abilities in parallel computing and efficient extraction of space-time sequence features from videos. Moreover, combining self-attention and multi-scale methods has yielded excellent results. However, the combination of the two methods has limitations, current up-sampling methods struggle to match the global modeling capacity of self-attention mechanisms. Therefore, this paper proposes three strategies to combine the two methods. Based on the approximation strategy, we first construct a new bilinear up-sampling method for multi-scale acquisition. Convolution and cross-attention techniques are then used to correct and align features at different scales to prevent large deviations in feature extraction at a specific scale, which can affect subsequent feature extraction. Finally, to effectively solve the common computational complexity,$ C^{0} $continuity, and neuron death problems of existing activation functions, a new method to construct the activation function is proposed. The cubic spline function is used to construct a new activation function approximating tanh. The new activation function is$ C^{2} $continuous, which is piecewise defined by cubic polynomial curves. In this study, better results were achieved on three public video super-resolution test sets: REDS4, Vid4, and Vimeo-90K-T. Experiments demonstrated that the proposed method could provide a new solution for video super-resolution tasks.

Deep learning models for semi-supervised medical image segmentation have achieved unprecedented performance for a wide range of tasks. Despite their high accuracy, these models may however yield predictions that are considered anatomically impossible by clinicians. Moreover, incorporating complex anatomical constraints into standard deep learning frameworks remains challenging due to their non-differentiable nature. To address these limitations, we propose a Constrained Adversarial Training (CAT) method that learns how to produce anatomically plausible segmentations. Unlike approaches focusing solely on accuracy measures like Dice, our method considers complex anatomical constraints like connectivity, convexity, and symmetry which cannot be easily modeled in a loss function. The problem of non-differentiable constraints is solved using a Reinforce algorithm which enables to obtain a gradient for violated constraints. To generate constraint-violating examples on the fly, and thereby obtain useful gradients, our method adopts an adversarial training strategy which modifies training images to maximize the constraint loss, and then updates the network to be robust to these adversarial examples. The proposed method offers a generic and efficient way to add complex segmentation constraints on top of any segmentation network. Experiments on synthetic data and four clinically-relevant datasets demonstrate the effectiveness of our method in terms of segmentation accuracy and anatomical plausibility.

We present an unsupervised domain adaptation method for image segmentation which aligns high-order statistics, computed for the source and target domains, encoding domain-invariant spatial relationships between segmentation classes. Our method first estimates the joint distribution of predictions for pairs of pixels whose relative position corresponds to a given spatial displacement. Domain adaptation is then achieved by aligning the joint distributions of source and target images, computed for a set of displacements. Two enhancements of this method are proposed. The first one uses an efficient multi-scale strategy that enables capturing long-range relationships in the statistics. The second one extends the joint distribution alignment loss to features in intermediate layers of the network by computing their cross-correlation. We test our method on the task of unpaired multi-modal cardiac segmentation using the Multi-Modality Whole Heart Segmentation Challenge dataset and prostate segmentation task where images from two datasets are taken as data in different domains. Our results show the advantages of our method compared to recent approaches for cross-domain image segmentation. Code is available at https://github.com/WangPing521/Domain_adaptation_shape_prior.

Feature refinement and feature fusion are two key steps in convolutional neural networks–based salient object detection (SOD). In this article, we investigate how to utilize multiple guidance mechanisms to better refine and fuse extracted multi-level features and propose a novel multi-guidance SOD model dubbed as MGuid-Net. Since boundary information is beneficial for locating and sharpening salient objects, edge features are utilized in our network together with saliency features for SOD. Specifically, a self-guidance module is applied to multi-level saliency features and edge features, respectively, which aims to gradually guide the refinement of lower-level features by higher-level features. After that, a cross-guidance module is devised to mutually refine saliency features and edge features via the complementarity between them. Moreover, to better integrate refined multi-level features, we also present an accumulative guidance module, which exploits multiple high-level features to guide the fusion of different features in a hierarchical manner. Finally, a pixelwise contrast loss function is adopted as an implicit guidance to help our network retain more details in salient objects. Extensive experiments on five benchmark datasets demonstrate our model can identify salient regions of an image more effectively compared to most of state-of-the-art models.

In this paper, we present an end-to-end neural solution to model portrait bas-relief from a single photograph, which is cast as a problem of image-to-depth translation. The main challenge is the lack of bas-relief data for network training. To solve this problem, we propose a semi-automatic pipeline to synthesize bas-relief samples. The main idea is to first construct normal maps from photos, and then generate bas-relief samples by reconstructing pixel-wise depths. In total, our synthetic dataset contains 23 k pixel-wise photo/bas-relief pairs. Since the process of bas-relief synthesis requires a certain amount of user interactions, we propose end-to-end solutions with various network architectures, and train them on the synthetic data. We select the one that gave the best results through qualitative and quantitative comparisons. Experiments on numerous portrait photos, comparisons with state-of-the-art methods and evaluations by artists have proven the effectiveness and efficiency of the selected network.

Transformer architecture has emerged to be successful in many natural language processing tasks. However, its applications to clinical practice remain largely unexplored. In this study, we propose a Robust Cross-Scale Hybrid Transformer (RCSHT) architecture for medical image segmentation, which can effectively enhance the multi-scale feature representations while integrating local features with global dependencies. Specifically, we propose two new modules based on the self-attention mechanism: PSCM and PCCM, which perform matrix low-rank transformation on cross-scale images in spatial and channel space, respectively, to effectively enhance the discriminant ability of long-range dependencies of multi-scale images and effectively reduce the computational complexity. Meanwhile, we use spatial positional encoding to perform spatial regularization on the two attention modules, so that the model has more opportunities to obtain valuable spatial details and further enhance the spatial retention ability. Besides, by fusing outputs of the two modules, the robustness and discriminant ability of the feature representations are further improved. Note that our hybrid architecture allows transformers to be initialized as convolutional networks without pre-training. Extensive experiments conducted on our newly proposed dataset have demonstrated our RCSHT notably outperforms the state-of-the-art methods by a large margin, holding the promise to generalize well on other medical image segmentation.

In recent years, underwater image enhancement methods based on deep learning have achieved remarkable results. Since the images obtained in complex underwater scenarios lack a ground truth, these algorithms mainly train models on underwater images synthesized from in-air images. Synthesized underwater images are different from real-world underwater images; this difference leads to the limited generalizability of the training model when enhancing real-world underwater images. In this work, we present an underwater image enhancement method that does not require training on synthetic underwater images and eliminates the dependence on underwater ground-truth images. Specifically, a novel domain adaptation framework for real-world underwater image enhancement inspired by transfer learning is presented; it transfers in-air image dehazing to real-world underwater image enhancement. The experimental results on different real-world underwater scenes indicate that the proposed method produces visually satisfactory results.

Serving as an essential step for many applications of image processing, superpixel generation has attracted a lot of attentions. Most existing superpixel generation algorithms focus on the boundary adherence and compactness of the superpixels, but ignore the topological consistency between the superpixels, which severely limites their applications in the subsequent tasks, especially in the CNN based image processing tasks. In this paper, we present a fast lattice superpixel generation algorithm, which can generate superpixels with lattice topology like the original pixels. We also propose a local similarity loss function to improve the segmentation accuracy of the generated lattice superpixels. The whole algorithm is parallelly implemented on GPU. We perform extensive experiments on three datasets (i.e., BSDS500, NYUv2 and VOC) to verify the efficacy of our algorithm. The experimental results show that our method achieves competitive results compared to the state-of-the-art methods.

In this paper, we propose to compute Voronoi diagrams over mesh surfaces driven by an arbitrary geodesic distance solver, assuming that the input is a triangle mesh as well as a collection of sites P = { Pi } m i =1 on the surface. We propose two key techniques to solve this problem. First, as the partition is determined by minimizing the m distance fields, each of which rooted at a source site, we suggest keeping one or more distance triples, for each triangle, that may help determine the Voronoi bisectors when one uses a mark-and-sweep geodesic algorithm to predict the multi-source distance field. Second, rather than keep the distance itself at a mesh vertex, we use the squared distance to characterize the linear change of distance field restricted in a triangle, which is proved to induce an exact VD when the base surface reduces to a planar triangle mesh. Specially, our algorithm also supports the Euclidean distance, which can handle thin-sheet models (e.g. leaf) and runs faster than the traditional restricted Voronoi diagram (RVD) algorithm. It is very extensible to deal with various variants of surface-based Voronoi diagrams including (1) surface-based power diagram, (2) constrained Voronoi diagram with curve-type breaklines, and (3) curve-type generators. We conduct extensive experimental results to validate the ability to approximate the exact VD in different distance-driven scenarios.

Many real-life tasks such as military reconnaissance and traffic monitoring require high-quality images. However, images acquired in foggy or hazy weather pose obstacles to the implementation of these real-life tasks; consequently, image dehazing is an important research problem. To meet the requirements of practical applications, a single image dehazing algorithm has to be able to effectively process real-world hazy images with high computational efficiency. In this article, we present a fast and robust semi-supervised dehazing algorithm named SADnet for practical applications. SADnet utilizes both synthetic datasets and natural hazy images for training, so it has good generalizability for real-world hazy images. Furthermore, considering the uneven distribution of haze in the atmospheric environment, a Channel-Spatial Self-Attention (CSSA) mechanism is presented to enhance the representational power of the proposed SADnet. Extensive experimental results demonstrate that the presented approach achieves good dehazing performances and competitive running times compared with other state-of-the-art image dehazing algorithms.

In clinical practice, as a powerful and independent risk indicator of cardiovascular disease (CVD), accurate coronary artery calcium (CAC) segmentation can provide important information for the early diagnosis of CVD. However, due to the small and inconsistent CAC usually has fuzzy boundaries, which leads existing segmentation methods to suffer from unsatisfactory performance. To tackle this challenge, we propose a novel Efficient Multi-scale Vision Transformers for CAC segmentation (CAC-EMVT), which uses both the local and global branches to jointly model short- and long-range dependencies. CAC-EMVT is mainly composed of three modules: 1) a key factor sampling (KFS) module, which is used to mine the key factors of the image to perform low-rank reconstruction of highly structured features; 2) a non-local sparse context fusion (NSCF) module, which is used to efficiently model the global context information of shallow texture features; and 3) a non-local multi-scale context aggregation (NMCA) module, which can be applied to cross-level features to collect long-range dependencies from multiple scales. Undeniably, the newly proposed decomposable positional encoding plays a vital role in the performance improvement of the above modules. Extensive experiments are conducted on the CT scans of 130 CVD patients under 4-fold cross-validation and have demonstrated our CAC-EMVT notably outperforms the state-of-the-art methods in terms of both the mean Dice similarity coefficient (mDice) of 75.39%± 3.17 and mean surface distance (MSD) of 1.93%± 0.46. This reveals the effectiveness and the potential of our model in the clinical setting.

Tensor robust principal component analysis (TRPCA) is an important algorithm for color image denoising by treating the whole image as a tensor and shrinking all singular values equally. In this paper, to improve the denoising performance of TRPCA, we propose a variant of TRPCA model. Specifically, we first introduce a nonconvex TRPCA (N-TRPCA) model which can shrink large singular values more and shrink small singular values less, so that the physical meanings of different singular values can be preserved. To take advantage of the structural redundancy of an image, we further group similar patches as a tensor according to nonlocal prior, and then apply the N-TRPCA model on this tensor. The denoised image can be obtained by aggregating all processed tensors. Experimental results demonstrate the superiority of the proposed denoising method beyond state-of-the-arts.

Abstract Different from other types of bas‐reliefs, a flower bas‐relief contains a large number of depth‐discontinuity edges. Most existing line‐based methods reconstruct free‐form surfaces by ignoring the depth‐discontinuities, thus are less efficient in modeling flower bas‐reliefs. This paper presents a neural‐based solution which benefits from the recent advances in CNN. Specially, we use line gradients to encode the depth orderings at leaf edges. Given a line drawing, a heuristic method is first proposed to compute 2D gradients at lines. Line gradients and dense curvatures interpolated from sparse user inputs are then fed into a neural network, which outputs depths and normals of the final bas‐relief. In addition, we introduce an object‐based method to generate flower bas‐reliefs and line drawings for network training. Extensive experiments show that our method is effective in modelling bas‐reliefs with depth‐discontinuity edges. User evaluation also shows that our method is intuitive and accessible to common users.

Smoothing images, especially with rich texture, is an important problem in computer vision. Obtaining an ideal result is difficult due to complexity, irregularity, and anisotropicity of the texture. Besides, some properties are shared by the texture and the structure in an image. It is a hard compromise to retain structure and simultaneously remove texture. To create an ideal algorithm for image smoothing, we face three problems. For images with rich textures, the smoothing effect should be enhanced. We should overcome inconsistency of smoothing results in different parts of the image. It is necessary to create a method to evaluate the smoothing effect. We apply texture pre-removal based on global sparse decomposition with a variable smoothing parameter to solve the first two problems. A parametric surface constructed by an improved Bessel method is used to determine the smoothing parameter. Three evaluation measures: edge integrity rate, texture removal rate, and gradient value distribution are proposed to cope with the third problem. We use the alternating direction method of multipliers to complete the whole algorithm and obtain the results. Experiments show that our algorithm is better than existing algorithms both visually and quantitatively. We also demonstrate our method's ability in other applications such as clip-art compression artifact removal and content-aware image manipulation.

Image segmentation is a basic problem in medical image analysis and useful for disease diagnosis. However, the complexity of medical images makes image segmentation difficult. In recent decades, fuzzy clustering algorithms have been preferred due to their simplicity and efficiency. However, they are sensitive to noise. To solve this problem, many algorithms using non-local information have been proposed, which perform well but are inefficient. This paper proposes an improved fuzzy clustering algorithm utilizing nonlocal self-similarity and a low-rank prior for image segmentation. Firstly, cluster centers are initialized based on peak detection. Then, a pixel correlation model between corresponding pixels is constructed, and similar pixel sets are retrieved. To improve efficiency and robustness, the proposed algorithm uses a novel objective function combining non-local information and a low-rank prior. Experiments on synthetic images and medical images illustrate that the algorithm can improve efficiency greatly while achieving satisfactory results.

Abstract Self‐similarity, a prior of natural images, has attracted much attention. The attribute means that low‐rank group matrices can be constructed from similar image patches. For low‐rank approximation denoising methods based on singular value decomposition (SVD) the ability to accurately construct group matrices with noise and handle singular values are keys. Here, combining image priors, a two‐stage clustering method to adaptively construct group matrices is designed. The method is anti‐noise, that is, when noise levels are high, these matrices are more accurate than that constructed by other algorithms. Then, according to the significance of singular values and singular vectors, singular vectors of the low‐rank estimations are corrected so that the residual noise in the low‐rank estimations is further suppressed. For back projection , the authors use the original noise level and the residual image to adaptively determine projection parameters and new noise levels . So, authors' back projection can provide a good foundation for authors' two‐stage denoising methods, better remove noise and preserve image details. Experimental results show that compared with the existing state‐of‐the‐art denoising algorithms, the proposed algorithm achieves competitive denoising performances in terms of quantitative metrics and preserving details. Especially with the increase of noise, the competitiveness of authors' algorithms is gradually enhanced.

Abstract Coronary artery calcium is a strong and independent marker of atherosclerosis and cardiovascular disease. Typically, the accurate segmentation of computed tomography images of the chest is an important prerequisite and basis for coronary artery calcium identification and analysis. However, this is very challenging in practice because the boundaries of coronary artery calcium, the small lesions with large shape variation, are very blurry, resulting in poor performance in existing studies. To tackle this challenge, we present a novel Attention‐based Multi‐Scale Network called AMSN, which can process information through both the main and boundary branches in parallel. Key to our AMSN is a new non‐local multi‐scale context encoder module, which is mainly composed of the multi‐scale attention mechanism and local global long short‐term memory module. By aggregating the multi‐scale context information, i.e. high‐resolution low‐level and low‐resolution high‐level features, the model's feature representative capability and deployment ability are improved effectively. Besides, we introduce a new boundary preserving loss, which can consider the boundary information of all coronary artery calcium together and establish links for the segmentation of different coronary artery calcium simultaneously. Extensive experiments demonstrate our AMSN enables reliable accurate coronary artery calcium segmentation for assisted cardiovascular disease diagnosis clinically.

BACKGROUND: Chromatin 3D conformation plays important roles in regulating gene or protein functions. High-throughout chromosome conformation capture (3C)-based technologies, such as Hi-C, have been exploited to acquire the contact frequencies among genomic loci at genome-scale. Various computational tools have been proposed to recover the underlying chromatin 3D structures from in situ Hi-C contact map data. As connected residuals in a polymer, neighboring genomic loci have intrinsic mutual dependencies in building a 3D conformation. However, current methods seldom take this feature into account. RESULTS: We present a method called ShNeigh, which combines the classical MDS technique with local dependence of neighboring loci modeled by a Gaussian formula, to infer the best 3D structure from noisy and incomplete contact frequency matrices. We validated ShNeigh by comparing it to two typical distance-based algorithms, ShRec3D and ChromSDE. The comparison results on simulated Hi-C dataset showed that, while keeping the high-speed nature of classical MDS, ShNeigh can recover the true structure better than ShRec3D and ChromSDE. Meanwhile, ShNeigh is more robust to data noise. On the publicly available human GM06990 Hi-C data, we demonstrated that the structures reconstructed by ShNeigh are more reproducible between different restriction enzymes than by ShRec3D and ChromSDE, especially at high resolutions manifested by sparse contact maps, which means ShNeigh is more robust to signal coverage. CONCLUSIONS: Our method can recover stable structures in high noise and sparse signal settings. It can also reconstruct similar structures from Hi-C data obtained using different restriction enzymes. Therefore, our method provides a new direction for enhancing the reconstruction quality of chromatin 3D structures.

Abstract In contrast to 3D model that can be freely observed, p ortrait bas‐relief projects slightly from the background and is limited by fixed viewpoint. In this paper, we propose a novel method to reconstruct the underlying 3D shape from a single 2.5D bas‐relief, providing observers wider viewing perspectives. Our target is to make the reconstructed portrait has natural depth ordering and similar appearance to the input. To achieve this, we first use a 3D template face to fit the portrait. Then, we optimize the face shape by normal transfer and Poisson surface reconstruction. The hair and body regions are finally reconstructed and combined with the 3D face. From the resulting 3D shape, one can generate new reliefs with varying poses and thickness, freeing the input one from fixed view. A number of experimental results verify the effectiveness of our method.

This paper proposes a kernel-blending connection approximated by a neural network (KBNN) for image classification. A kernel mapping connection structure, guaranteed by the function approximation theorem, is devised to blend feature extraction and feature classification through neural network learning. First, a feature extractor learns features from the raw images. Next, an automatically constructed kernel mapping connection maps the feature vectors into a feature space. Finally, a linear classifier is used as an output layer of the neural network to provide classification results. Furthermore, a novel loss function involving a cross-entropy loss and a hinge loss is proposed to improve the generalizability of the neural network. Experimental results on three well-known image datasets illustrate that the proposed method has good classification accuracy and generalizability.

The detection of 3D interest points is a central problem in computer graphics, computer vision, and pattern recognition. It is also an important preprocessing step in the analysis of 3D model matching. Although studied for decades, detecting 3D interest points remains a challenge. In this study, a novel multiscale bilateral filtering method is presented to detect 3D interest points. This method first simplifies repeatedly the input 3D mesh to form k multiresolution meshes. For each mesh, on the basis of the computed saliency of the mesh vertex, the bilateral filtering is used to remove the noise of the mesh saliencies and the global contrast to normalise the saliencies, and then the interest points are extracted on the basis of the normalised saliency. The proposed method then gathers and clusters all interest points detected on the k multiresolution meshes, and the centres of these clusters are treated as the final interest points. In this method, both the spatial closeness and the geometric similarities of the mesh vertices are considered during the bilateral filtering process. The experimental results validate the effectiveness of the proposed method to detect 3D interest points. This method is also tested the potential to distinguish 3D models.

Based on the image self‐similarity and singular value decomposition (SVD) techniques, the authors propose an iterative adaptive global denoising method. For the structural differences between image patches, they adaptively determine the size of the search window. In each window, a similar image patch matrix is constructed based on the multi‐scale similarity measure. In order to ensure the speed of the method, the adaptive step size and the number of image patches are introduced, and all image patches are denoised in different iterations. This not only ensures the speed of the method, suppresses residual noise, but also reduces the artefacts caused by the fixed step size and the number of image patches. Therefore, the problem of image denoising is converted to the estimation of low‐rank matrix. New singular values are estimated according to the noise level, and similar image patch matrices without noise are estimated using them and corresponding singular vectors. Experimental results show that compared with the state‐of‐the‐art denoising algorithms, this method has a higher PSNR and FSIM, and has a good visual effect. The new method can be applied to image and video restoration, target recognition and image classification.

Part of important structural edges in the image is smoothed due to the small gradients, while the others are preserved with greater gradients. Therefore, the authors propose a two‐stage image smoothing method based on edge‐patch histogram equalisation and patch decomposition. The authors' purpose is to increase the gradient of important structural edges while reducing the gradient of the texture region. Therefore, they divide the image into edge‐patches where the structural edges are concentrated or non‐edge‐patches where the texture details are concentrated by image segmentation. The edge‐patch needs to be equalised by the histograms for increasing the gradient of the edge pixels. All patches are decomposed to extract the smooth component for reducing the gradient of pixels. The smooth component of each patch is smoothed via gradient minimisation. In order to ensure the continuity of the patch boundaries, the edge‐patch is inversely equalised. Finally, the whole image is smoothed via gradient minimisation for removing residual textures and seams. Experimental results demonstrate that the proposed method is more competitive in maintaining important structural edges and removing texture details than the state‐of‐the‐art approaches. The proposed method can be applied to many areas of image processing.

Automated pancreas segmentation in abdominal computed tomography (CT) scans is of high clinical relevance (i.e. pancreas cancer diagnosis and prognosis), but extremely difficult because the pancreas is a soft, small, and flexible abdominal organ with high anatomical variability, which causes the previous segmentation methods to result in low precision. In this study, the authors present a new deep recurrent adversarial network (DRAN) to tackle this challenge. DRAN contains three steps: (i) preserving global resolution of CT scans and modifying the receptive field of kernel adaptively through a dilated convolution autoencoder module; (ii) modelling contextual spatial correlation between neighbouring CT scan patches benefits from a specially designed local long short‐term memory module; and (iii) improving the performance and generalisation by leveraging an adversarial module, which can constrain the spatial smoothness consistency between continuous CT scans based on the long‐range spatial interaction. The system is evaluated on a dataset of 80 manually segmented CT volumes, using four‐fold cross‐validation. Its performance surpasses other state‐of‐the‐art methods, with the Dice similarity coefficient of and pixel‐wise accuracy of . Also, they perform a qualitative evaluation by an expert further revealing the effectiveness and potential of their DRAN as a clinical segmentation tool.

Superpixel segmentation approach, as a preprocessing step in computer vision tasks, groups pixels into perceptually coherence atomic regions to replace the pixel grid in images and reduce the primitives and redundancy of subsequent works. In this study, the authors proposed a fast and robust superpixel generation method based on non‐iterative framework with the constraint of linear path. They collected neighbouring pixels as the initial superpixels one by one in the conventional order at first. To make the superpixels attach to the most object boundaries well and robust to noise, they defined a new distance measurement between pixels and superpixel seeds by considering the colour difference of pixels in the neighbourhood and along the linear path from the pixel to the seed. Meanwhile, they proposed a new way to set parameters adaptively based on the intrinsic quality of images. Then, they refined the initial superpixels by merging the smallest ones until the number of superpixel meets the expectation. The experimental results on clean and noisy images demonstrate that the proposed method is effective and presents a competitive performance in computational efficiency with the state‐of‐the‐art real‐time method.

To address the hazy image degradation problem, we introduce a single image numerical iterative dehazing method based on local physical features. The method involves three components: region division based on haze density, local atmospheric light estimation and transmission map estimation, and recovery of hazy image scene radiance by using an iterative algorithm. Because of the nonuniform haze density within an image, we first employ the affinity propagation (AP) clustering algorithm to divide a hazy image into different haze density regions. Second, to reflect the difference in atmospheric light among regions and avoid the generation of halo artifacts in recovered images, we estimate the local atmospheric light in each region to replace the global atmospheric light and then estimate the transmission via a dark channel prior. Finally, an iterative dehazing algorithm, which can be used to not only further optimize local atmospheric light and transmission but also remove haze completely, is developed based on a physical model. Experimental results illustrate that our method can effectively improve the quality of a foggy image without sacrificing color fidelity and can retain image details sufficiently.

In this paper, a novel single image super-resolution (SR) method based on progressive-iterative approximation is proposed. To preserve textures and clear edges, the image SR reconstruction is treated as an image progressive-iterative fitting procedure and achieved by iterative interpolation. Due to different features in different regions, we first employ the nonsubsampled contourlet transform (NSCT) to divide the image into smooth regions, texture regions, and edges. Then, a hybrid interpolation scheme based on curves and surfaces is proposed, which differs from the traditional surface interpolation methods. Specifically, smooth regions are interpolated by the non-uniform rational basis spline (NURBS) surface geometric iteration. To retain textures, control points are increased, and the progressive-iterative approximation of the NURBS surface is employed to interpolate the texture regions. By considering edges in an image as curve segments that are connected by pixels with dramatic changes, we use NURBS curve progressive-iterative approximation to interpolate the edges, which sharpens the edges and can maintain the image edge structure without jaggy and block artifacts. The experimental results demonstrate that the proposed method significantly outperforms the state-of-the-art methods in terms of both subjective and objective measures.

We present a novel solution to enable portrait relief modeling from a single image. The main challenges are geometry reconstruction, facial details recovery and depth structure preservation. Previous image-based methods are developed for portrait bas-relief modeling in 2.5D form, but not adequate for 3D-like high relief modeling with undercut features. In this paper, we propose a template-based framework to generate portrait reliefs of various forms. Our method benefits from Shape-from-Shading (SFS). Specifically, we use bi-Laplacian mesh deformation to guide the relief modeling. Given a portrait image, we first use a template face to fit the portrait. We then apply bi-Laplacian mesh deformation to align the facial features. Afterwards, SFS-based reconstruction with a few user interactions is used to optimize the face depth, and create a relief with similar appearance to the input. Both depth structures and geometric details can be well constructed in the final relief. Experiments and comparisons to other methods demonstrate the effectiveness of the proposed method.

Abstract As an art form between drawing and sculpture, relief has been widely used in a variety of media for signs, narratives, decorations and other purposes. Traditional relief creation relies on both professional skills and artistic expertise, which is extremely time‐consuming. Recently, automatic or semi‐automatic relief modelling from a 3D object or a 2D image has been a subject of interest in computer graphics. Various methods have been proposed to generate reliefs with few user interactions or minor human efforts, while preserving or enhancing the appearance of the input. This survey provides a comprehensive review of the advances in computer‐assisted relief modelling during the past decade. First, we provide an overview of relief types and their art characteristics. Then, we introduce the key techniques of object‐space methods and image‐space methods respectively. Advantages and limitations of each category are discussed in details. We conclude the report by discussing directions for possible future research.

Low‐rank approximation has shown great potential in various image tasks. It is found that there is a specific functional relationship about singular values between the original image and a series of noisy images, which can be used to construct the singular values of a noise‐free image. In this study, the authors propose a novel denoising method based on the above facts and low‐rank approximation theory. Firstly, they estimate the noise energy distribution of the group matrix in the singular value decomposition (SVD) domain using the energy characteristics of the image with different noise levels. The energy distribution of the noise is shrunk to obtain the energy distribution of the true signal. Then, based on the optimal energy compaction property of SVD, the low‐rank property of matrix is constrained in the SVD domain to obtain the low‐rank approximation of the matrix. Moreover, an iterative back projection method is adopted in this study to suppress residual noise. A new noise standard deviation estimation approach, targeted at the back projection process, is proposed to effectively optimise the denoising results during the iteration. Experimental results show that the authors’ method efficiently decreases the noise and achieves comparable denoising performance to the state‐of‐the‐art methods regarding both quantitative measurement and visual effect.

In the traditional active contour models, global region‐based methods fail to segment images with intensity inhomogeneity, and local region‐based methods are sensitive to initial contour. In this study, a novel fuzzy energy‐based active contour model is proposed to segment medical images, which are always corrupted by intensity inhomogeneity. In order to deal with intensity inhomogeneity, a local energy term is first constructed by substituting a non‐local weight for Gaussian kernel widely used in traditional local region‐based models. Second, the defined adaptive force can drive the level set function to adaptively increase or decrease according to image intensity information. Therefore, the initial contour can be initialised as a constant function, which eliminates the problem caused by contour initialisation. Moreover, the proposed active contour model is a convex function. Thus, the problem, resulting from optimising a non‐convex functional in the traditional active contour models, can be avoided. Experimental results validate the superiorities and effectiveness of the proposed model for image segmentation with comparisons of those yielded by several state‐of‐the‐art techniques.

Natural image priors play an important role in image denoising, and various prior-based methods have been widely proposed for noise removal. However, these methods tend to smooth the fine image textures while suppressing noise, degrading the image visual quality. To address this problem, in this paper, we propose an adaptive texture-preserving denoising method. In contrast to most existing prior-based denoising methods, two types of priors [gradient histogram matching priors and nonlocal self-similarity (NSS) priors] are proposed, and their combination is used for image denoising. We introduce a hyper-Laplacian distribution of the gradient histogram matching prior, which enforces the gradient histogram of the denoised image to be as close as possible to the estimated reference histogram from the original image. Meanwhile, the proposed model obtained by introducing the NSS priors effectively preserves fine image details and generates sharp image edges. To improve the accuracy of the method, a content-adaptive parameter selection scheme based on edge detection filters is proposed. Moreover, the optimization problem with two types of priors and the content-adaptive parameter added into the objective function becomes a challenging non-convex optimization problem. To effectively solve this problem, we have developed a new numerical solution based on augmented Lagrangian multipliers and alternating minimization scheme. The experimental results demonstrate that the proposed method effectively preserves the texture features of the denoised images and outperforms several variational methods and other state-of-the-art methods in terms of various evaluation indices and visual quality, especially at medium and high noise levels.

Superpixel generation, which is an essential step in many image processing applications, has attracted increasing attention from researchers. In this paper, we present an efficient flooding-based superpixel generation algorithm that generates compact and highly boundary adherent superpixels. In particular, by considering various superpixel properties, we measure the similarities between image pixels by proposing a new distance metric that combines various image features (e.g., colors, spatial locations, neighbor information, and texture features). To control the relative significance of these image features, we acquire the weights of image features (e.g., colors, texture features, and neighbor information of pixels) through a neural network. Then, the final superpixels are obtained through a greedy optimization that considers both the current superpixel and its neighboring superpixels. We perform extensive experiments on two datasets to verify the efficacy of our algorithm. The results show that our algorithm has considerable advantages over existing state-of-the-art methods, particularly regarding the compactness of the resulting superpixels.

This article presents a new computational framework for constructing 3D self-supporting surfaces with isotropic stress. Inspired by the self-supporting property of catenary and the fact that catenoid (the surface of revolution of the catenary curve) is a minimal surface, we discover the relation between 3D self-supporting surfaces and 4D minimal hypersurfaces (which are 3-manifolds). Lifting the problem into 4D allows us to convert gravitational forces into tensions and reformulate the equilibrium problem to total potential energy minimization, which can be solved using a variational method. We prove that the hyper-generatrix of a 4D minimal hyper-surface of revolution is a 3D self-supporting surface, implying that constructing a 3D self-supporting surface is equivalent to volume minimization. We show that the energy functional is simply the surface’s gravitational potential energy, which in turn can be converted into a surface reconstruction problem with mean curvature constraint. Armed with our theoretical findings, we develop an iterative algorithm to construct 3D self-supporting surfaces from triangle meshes. Our method guarantees convergence and can produce near-regular triangle meshes, thanks to a local mesh refinement strategy similar to centroidal Voronoi tessellation. It also allows users to tune the geometry via specifying either the zero potential surface or its desired volume. We also develop a finite element method to verify the equilibrium condition on 3D triangle meshes. The existing thrust network analysis methods discretize both geometry and material by approximating the continuous stress field through uniaxial singular stresses, making them an ideal tool for analysis and design of beam structures. In contrast, our method works on piecewise linear surfaces with continuous material. Moreover, our method does not require the 3D-to-2D projection, therefore it also works for both height and non-height fields.

In image denoising, high‐frequency components are more notable to the human eyes than low‐frequency components. While high‐frequency components contain more variations and represent the detailed textures, the reconstructions of these components are much harder and it is a remaining challenge in image denoising. In this study, a novel edge‐preserving image denoising algorithm is proposed, it treats the low‐ and high‐frequency components of the image separately. For restoration of high‐frequency components, a neighbourhood regression method is proposed. An energy minimisation function is developed to combine the low‐ and high‐frequency components into one model. Experiments show that the proposed method outperforms the state‐of‐the‐art methods in peak signal‐to‐noise ratio, edges preservation and visual performance.

This paper presents a novel single-image super-resolution (SR) procedure, which upscales a given low-resolution (LR) input image to a high-resolution image while preserving the textural and structural information. First, we construct a new type of bivariate rational fractal interpolation model and investigate its analytical properties. This model has different forms of expression with various values of the scaling factors and shape parameters; thus, it can be employed to better describe image features than current interpolation schemes. Furthermore, this model combines the advantages of rational interpolation and fractal interpolation, and its effectiveness is validated through theoretical analysis. Second, we develop a single-image SR algorithm based on the proposed model. The LR input image is divided into texture and non-texture regions, and then, the image is interpolated according to the characteristics of the local structure. Specifically, in the texture region, the scaling factor calculation is the critical step. We present a method to accurately calculate scaling factors based on local fractal analysis. Extensive experiments and comparisons with the other state-of-the-art methods show that our algorithm achieves competitive performance, with finer details and sharper edges.

To recover the corrupted pixels, traditional inpainting methods based on low-rank priors generally need to solve a convex optimization problem by an iterative singular value shrinkage algorithm. In this paper, we propose a simple method for image inpainting using low rank approximation, which avoids the time-consuming iterative shrinkage. Specifically, if similar patches of a corrupted image are identified and reshaped as vectors, then a patch matrix can be constructed by collecting these similar patch-vectors. Due to its columns being highly linearly correlated, this patch matrix is low-rank. Instead of using an iterative singular value shrinkage scheme, the proposed method utilizes low rank approximation with truncated singular values to derive a closed-form estimate for each patch matrix. Depending upon an observation that there exists a distinct gap in the singular spectrum of patch matrix, the rank of each patch matrix is empirically determined by a heuristic procedure. Inspired by the inpainting algorithms with component decomposition, a two-stage low rank approximation (TSLRA) scheme is designed to recover image structures and refine texture details of corrupted images. Experimental results on various inpainting tasks demonstrate that the proposed method is comparable and even superior to some state-of-the-art inpainting algorithms.

Total variation (TV) and non-local patch similarity have been used successfully to enhance the performance of compressive sensing (CS) approaches. However, such techniques can often remove important details in the image or introduce reconstruction artifacts. This paper presents a novel CS method, which uses an adaptive reweighted TV strategy to better preserve image edges. Our method also leverages the redundancy of non-local image patches through the use of weighted low rank regularization. An optimization strategy based on the ADMM algorithm is used to reconstruct images efficiently. Experimental results show our method to outperform state-of-the-art CS approaches, for various sampling ratios.

We introduce a new approach for performing image inpainting, by devising an integrative method based on the superpixel segmentation technique and considering minimal user input. Image inpainting methods are concerned with filling in missing or replacing undesired regions in an image. Typically, inpainting methods consider and extrapolate known image data. Superpixels in the immediate neighborhood of the in-painting region are computed and used as source image data to fill in (or replace) the inpainting area. A user provides additional information by specifying line segments in the image to assist the otherwise automatic inpainting process, to ensure that only desirable superpixels are utilized when copying them into the inpainting region. User interaction is minimal, as it is merely necessary to specify a small number of line segments that define image parts to be used as source data in distinct inpainting regions. We provide experimental results demonstrating that our method performs well when compared against other methods, especially concerning the preservation of edges and texture in the inpainted regions.

As one of the most popular image oversegmentations, superpixel has been commonly used as supporting regions for primitives to reduce computations in various computer vision tasks. In this paper, we propose a novel superpixel segmentation approach based on a distance function that is designed to balance among boundary adherence, intensity homogeneity, and compactness (COM) characteristics of the resulting superpixels. Given an expected number of superpixels, our method begins with initializing the superpixel seed positions to obtain the initial labels of pixels. Then, we optimize the superpixels iteratively based on the defined distance measurement. We update the positions and intensities of superpixel seeds based on the three-sigma rule. The experimental results demonstrate that our algorithm is more effective and accurate than previous superpixel methods and achieves a comparable tradeoff between superpixel COM and adherence to object boundaries.

We present a novel superpixel generation algorithm based on a new definition of geodesic distance, called bilateral geodesic distance. In contrast to the traditional geodesic distance, the new bilateral geodesic distance of two pixels considers the distance between their positions as well as their color difference. Superpixel generation is essentially a problem of clustering image pixels with respect to a set of properly selected seeds. We first use an adaptive hexagonal subdivision method to determine the initial seed-based image gradient. Then, we use the bilateral geodesic distance to measure the similarity between the pixels and the seeds. We apply an improved fast marching method to generate superpixels’ contour regions with the expansion velocities dependent on a new gradient formulation that depends on the seeds’ properties. The experimental results indicate that our algorithm is not only much faster than the structure-based method, which uses conventional geodesic distance, but also outperforms the existing methods in terms of region compactness and region boundary regularity.

Serving as a key step for applications of image processing, superpixel generation has been attracting increasing attention. RGB-D images are used pervasively in scenes reconstruction and representation, benefiting from their contained depth data. In this paper, we present a novel framework for generating superpixels focus on RGB-D images of indoor scenes, based on a weighted geodesic driven metric that combines both color and geometric information. In particular, taking into account the unique structures of indoor scenarios, we first denoise the given RGB-D image, and construct the corresponding triangular mesh. A new weighted geodesic driven metric is defined by introducing a weight function constrained with normal vectors and colors. Under this metric, an energy function is defined to measure our over-segmentation of the triangular mesh, by optimizing which, we can acquire an optimal over-segmentation of the triangular mesh with object boundaries respected, such that vertices in each sub-region have similar geometric structures and color intensities. Re-mapping the over-segmentation of the triangular mesh to the RGB-D image results in desired superpixels. We perform extensive experiments on a large-scale database of RGB-D images to verify the efficacy of our algorithm. The results show that our algorithm has considerable advantages over the existing state-of-the-art methods.

We present a novel approach for interactively and intuitively editing 3D facial animation in this paper. It determines a new expression by combining the user-specified constraints with the priors contained in a pre-recorded facial expression set, which effectively overcomes the generation of an unnatural expression caused by only user-constraints. The approach is based on the framework of example-based linear interpolation. It adaptively segments the face model into soft regions based on user-interaction. In dependently modeling each region, we propose a new function to estimate the blending weight of each example that matches the user-constraints as well as the spatio-temporal properties of the face set. In blending the regions into a single expression, we present a new criterion that fully exploits the spatial proximity and the spatio-temporal motion consistency over the face set to measure the coherency between vertices and use the coherency to reasonably propagate the influence of each region to the entire face model. Experiments show that our approach, even with inappropriate user’s edits, can create a natural expression that optimally satisfies the user-desired goal.

We present a new approach for constructing the 3D panorama for an indoor environment by joining together aligned submaps. For each submap, the trajectory of the moving camera is estimated based on the Kanade-Lucas-Tomasi (KLT) features. Our method can update the feature set status by adding new features and removing expiring ones adaptively to accommodate scene changes. The accuracy of the estimated poses is further improved through sparse bundle adjustment. Furthermore, we utilize a linear optimization framework to align all submaps to obtain a consistently extended 3D panorama and to refine the visual odometry at the same time. We evaluated our approach on publicly available benchmark datasets. The experiments demonstrate that the proposed method achieves low translational drift and is robust even when the camera moves very fast.

This paper proposes a novel super-resolution method that exploits the sparse representation and non-local similarity of patches for the effective reconstruction of images. Highresolution images are reconstructed from low resolution observations with an efficient technique based on the alternating direction method of multipliers (ADMM). A robust iterative back-projection approach is used in a post-processing step to remove residual noise and artifacts in the reconstructed image. Experiments on benchmark medical images illustrate the advantage of our method, in terms of PSNR and SSIM, compared to state of the art approaches.

Abstract Bas‐relief is designed to provide 3D perception for the viewers under illumination. For the problem of bas‐relief generation from 3D object, most existing methods ignore the influence of illumination on bas‐relief appearance. In this paper, we propose a novel method that adaptively generate bas‐reliefs with respect to illumination conditions. Given a 3D object and its target appearance, our method finds an adaptive surface that preserves the appearance of the input. We validate our approach through a variety of applications. Experimental results indicate that the proposed approach is effective in producing bas‐reliefs with desired appearance under illumination.

This paper proposes a novel method for image magnification by exploiting the property that the intensity of an image varies along the direction of the gradient very quickly. It aims to maintain sharp edges and clear details. The proposed method first calculates the gradient of the low-resolution image by fitting a surface with quadratic polynomial precision. Then, bicubic interpolation is used to obtain initial gradients of the high-resolution (HR) image. The initial gradients are readjusted to find the constrained gradients of the HR image, according to spatial correlations between gradients within a local window. To generate an HR image with high precision, a linear surface weighted by the projection length in the gradient direction is constructed. Each pixel in the HR image is determined by the linear surface. Experimental results demonstrate that our method visually improves the quality of the magnified image. It particularly avoids making jagged edges and bluring during magnification.

Image super‐resolution (SR) for a single low‐resolution image is an important and challenging task in image processing. In this study, the authors propose a novel non‐local feature back‐projection method for image SR, which can effectively reduce jaggy and ringing artefacts common, in general, iterative back‐projection (IBP) method. In their method, the objective high‐resolution (HR) image is obtained by projecting reconstructed errors back to HR image iteratively. To optimise the initial HR image and constrain anisotropic errors propagation during IBP process, an efficient non‐local feature interpolation algorithm is designed. Specially, edge information is used as constraints to make the interpolation surface preserve better shape. Furthermore, as post‐processing, non‐local similarities are utilised to remove noise and irregularities induced by errors propagation. Experimental results show that their method achieves better performance than state‐of‐the‐art methods in terms of both quantitative metrics and visual qualities.

Nonlocal self-similarity of images has attracted considerable interest in the field of image processing and has led to several state-of-the-art image denoising algorithms, such as block matching and 3-D, principal component analysis with local pixel grouping, patch-based locally optimal wiener, and spatially adaptive iterative singular-value thresholding. In this paper, we propose a computationally simple denoising algorithm using the nonlocal self-similarity and the low-rank approximation (LRA). The proposed method consists of three basic steps. First, our method classifies similar image patches by the block-matching technique to form the similar patch groups, which results in the similar patch groups to be low rank. Next, each group of similar patches is factorized by singular value decomposition (SVD) and estimated by taking only a few largest singular values and corresponding singular vectors. Finally, an initial denoised image is generated by aggregating all processed patches. For low-rank matrices, SVD can provide the optimal energy compaction in the least square sense. The proposed method exploits the optimal energy compaction property of SVD to lead an LRA of similar patch groups. Unlike other SVD-based methods, the LRA in SVD domain avoids learning the local basis for representing image patches, which usually is computationally expensive. The experimental results demonstrate that the proposed method can effectively reduce noise and be competitive with the current state-of-the-art denoising algorithms in terms of both quantitative metrics and subjective visual quality.

Traditional deblurring algorithms are often focused on natural-scaled images, which are not adapted for document texts and images without having some negative impacts on the accuracy of the OCR and the visual quality. In this paper, we propose a gradient histogram preservation method. An effective optimization method was developed and achieves satisfying results for kernel estimation. By combining the gradient histogram preservation prior with conventional image deblurring methods, it significantly improves the simulations and experimental results on document images and a high SSIM is achieved with the proposed method.

It is challenging to consistently smooth natural images, yet smoothing results determine the quality of a broad range of applications in computer vision. To achieve consistent smoothing, we propose a novel optimization model making use of the redundancy of natural images, by defining a nonlocal concentration regularization term on the gradient. This nonlocal constraint is carefully combined with a gradient-sparsity constraint, allowing details throughout the whole image to be removed automatically in a data-driven manner. As variations in gradient between similar patches can be suppressed effectively, the new model has excellent edge preserving, detail removal, and visual consistency properties. Comparisons with state-of-the-art smoothing methods demonstrate the effectiveness of the new method. Several applications, including edge manipulation, image abstraction, detail magnification, and image resizing, show the applicability of the new method.

This paper proposes a new algorithm based on low-rank matrix recovery to remove salt & pepper noise from surveillance video. Unlike single image denoising techniques, noise removal from video sequences aims to utilize both temporal and spatial information. By grouping neighboring frames based on similarities of the whole images in the temporal domain, we formulate the problem of removing salt & pepper noise from a video tracking sequence as a low-rank matrix recovery problem. The resulting nuclear norm and L 1-norm related minimization problems can be efficiently solved by many recently developed methods. To determine the low-rank matrix, we use an averaging method based on other similar images. Our method can not only remove noise but also preserve edges and details. The performance of our proposed approach compares favorably to that of existing algorithms and gives better PSNR and SSIM results.

Abstract We present a heuristic approach to tetrahedral mesh generation for implicit closed surfaces. It consists of a surface sampling step and a volume sampling step that both work in a unified optimization framework. First, high‐quality isotropic samplings as well as a triangular mesh on the surface are generated. Then uniform volume samplings are determined by optimizing the point distribution inside the closed surface domain. Finally, the tetrahedral mesh is easily obtained by constrained Delaunay triangulation. Experimental results show that the new method can generate ideal tetrahedral meshes for closed implicit surfaces efficiently that are Delaunay based. Our method has the advantage of high efficiency and nice performance at surface boundaries. Copyright © 2015 John Wiley & Sons, Ltd.

The medial axis transform (MAT) is an important shape representation for shape approximation, shape recognition, and shape retrieval. Despite years of research, there is still a lack of effective methods for efficient, robust and accurate computation of the MAT. We present an efficient method, called Q-MAT , that uses quadratic error minimization to compute a structurally simple, geometrically accurate, and compact representation of the MAT. We introduce a new error metric for approximation and a new quantitative characterization of unstable branches of the MAT, and integrate them in an extension of the well-known quadric error metric (QEM) framework for mesh decimation. Q-MAT is fast, removes insignificant unstable branches effectively, and produces a simple and accurate piecewise linear approximation of the MAT. The method is thoroughly validated and compared with existing methods for MAT computation.

Through semantically grouping pixels in local neighborhoods, superpixels can capture image redundancy and significantly improve the performance of post-processing algorithms. In this paper, we investigate the application of superpixels in FCM framework, and propose a modified FCM algorithm SPFCM which utilizes superpixels as clustering objects instead of pixels. Superpixel and its neighborhood increase the clustering granularity and allow us to compute the objective function on a naturally adaptive domain rather than on a fixed window, so our algorithm can make full use of the spatial information and is more robust to noise. Due to the compact image representation based on superpixels, the computational complexity of our method is also drastically reduced. Experimental results on both synthetic and real images demonstrate the effectiveness and efficiency of our algorithm.

Superpixel generation is widely used in image segmentation. In this paper, we present an efficient flooding based superpixel generation algorithm. A new distance metric is defined for estimating pixels and seeds' similarity with COLOR, COMPACTNESS and SMOOTHNESS constraints. A seeds update strategy based on Lloyd's algorithm is adopted for optimizing seeds and superpixels' contour regions. Experimental results show our algorithm outperforms the existing methods. Boundaries of superpixels generated by our algorithm can fit the original image boundaries better.

This paper presents a novel hybrid strategy for depth map enhancement. Given a low resolution and noisy depth map, we improve its spatial resolution and quality with a registered high resolution color image. The unreliable pixels are removed firstly based on the local structure of center pixel. Then the initial depth map is mapped to color image's coordinate. And the holes created by up sampling and unreliable invalidation are filled by nonlocal filter-based method. Different from the previous local filter-based methods, nonlocal filter does not require the optimal support window and more natural pixel similarity measure metric is given. By quantitative and qualitative experiments on publicly available test sequences, we demonstrate the effectiveness and efficiency of our proposed method on the depth enhancement task, compared with the existing methods.

Based on the assumption that low-resolution image is sampled from an original scene approximated by piecewise polynomial surface, this paper proposes two different constraints to make the fitting surface of the original scene preserve the image's edge characteristics. We first construct a cubic parametric curve to approximate the edge in each local area and obtain an auxiliary pixel set by re-sampling the curve, then introduce a weight function to accord pixels different degrees of effects on surface reconstruction. By re-sampling the constructed surface, the enlarged image can be easily obtained. Extensive experimental results on various types of low-resolution images demonstrate that our method produces generally better results both in terms of quantitative evaluation and subjective visual quality.

ABSTRACT This paper presents a new robust multi‐level partition of unity (MPU) method, which constructs an implicit surface from a triangular mesh via the new error metric between the mesh and the implicit surface. The new error metric employs a weighted function of inner points and vertices of a triangle to fit an implicit surface, which can control the approximation error between the surface and vertices of the triangle. Furthermore, it is applied to the MPU method by utilizing the dual graph of a triangular mesh, and the general quadric implicit surface is used for surface representation. Compared with the MPU method, the new method generates fewer subdivision cells with the same approximation error and performs more steadily especially when given triangular mesh with fewer vertices. Copyright © 2013 John Wiley & Sons, Ltd.

We present an automatic method for fitting multiple B-spline curves to unorganized planar points. The method works on point clouds which have complicated topological structures and a single curve is insufficient for fitting the shape. A divide-and-merge algorithm is developed for dividing the unorganized data points into several groups while each group represents a smooth curve. Each point group is then fitted with a B-spline curve by the SDM method. Our algorithm also sets up automatically the control polygon of initial B-spline curves. Experiments demonstrate the capability of the presented algorithm in accurate reconstruction of topological structures of point clouds.

Conventional polynomial interpolation methods produce images with blurred edges, while edge-directed interpolation methods make enlarged images with good quality edges but with detail distortion in the non-edge portion. A new method for constructing a fitting surface to image data is presented. Unlike existing methods which produce enlarged images using image data as interpolation data, the new method constructs the fitting surface using the image data as constraints to reverse the sampling process for improving the fitting precision. To remove the zigzagging artifact, for each pixel and its nearby region, the edge information is used to determine the quadratic polynomial which approximates the original scene with a quadratic polynomial precision. Comparison results of the new method with other methods are included.

Conventional fuzzy C-means (FCM) algorithm does not consider spatial information in the clustering, which makes it sensitive to noise and inefficient. In order to overcome these problems, we propose a fast anti-noise FCM algorithm for image segmentation, which constructs a new spatial function by combining pixel gray value similarity and membership. This spatial function is used to update the membership which in turn is used to obtain the cluster centers iteratively. The proposed algorithm can achieve desirable segmentation results in less iterations and reduce the effect of noise effectively. Experimental results show that the proposed algorithm outperforms conventional FCM and other extended FCM algorithms.

There has been a significant improvement in stereo matching with the introduction of adaptive support weights. Existing local methods mainly focus on the computation of support weight which is critical in cost aggregation and usually get excellent results. However, the negative effects of occluded regions are often ignored, which results in the problem of foreground fattening and blurred depth borders. This paper proposes a novel support aggregation strategy by utilizing the occlusion information obtained from left-right consistency check. The weights of invalid points are noticeably reduced at each disparity estimation stage. Experimental results on the Middlebury images show that our method is highly effective in improving the disparities of points around occluded areas and depth discontinuities. According to the Middlebury benchmark, the proposed method achieves the best performance among all the local methods. Moreover, our approach can be easily integrated into nearly all the existing support weights strategies.

We present a new method for modeling discrete constant mean curvature (CMC) surfaces, which arise frequently in nature and are highly demanded in architecture and other engineering applications. Our method is based on a novel use of the CVT ( centroidal Voronoi tessellation ) optimization framework. We devise a CVT-CMC energy function defined as a combination of an extended CVT energy and a volume functional. We show that minimizing the CVT-CMC energy is asymptotically equivalent to minimizing mesh surface area with a fixed volume, thus defining a discrete CMC surface. The CVT term in the energy function ensures high mesh quality throughout the evolution of a CMC surface in an interactive design process for form finding. Our method is capable of modeling CMC surfaces with fixed or free boundaries and is robust with respect to input mesh quality and topology changes. Experiments show that the new method generates discrete CMC surfaces of improved mesh quality over existing methods.

In this article we propose a novel, robust, and high-capacity watermarking method for 3D meshes with arbitrary connectivities in the spatial domain based on affine invariants. Given a 3D mesh model, a watermark is embedded as affine-invariant length ratios of one diagonal segment to the residing diagonal intersected by the other one in a coplanar convex quadrilateral. In the extraction process, a watermark is recovered by combining all the watermark pieces embedded in length ratios through majority voting. Extensive experimental results demonstrate the robustness, high computational efficiency, high capacity, and affine-transformation-invariant characteristics of the proposed approach.

Abstract Shape deformation is a useful tool for shape modeling and animation in computer graphics. In this paper, we propose a novel surface deformation method based on a feature sensitive (FS) metric. Firstly, taking unit normal vectors into account, we derive a FS Laplacian operator, which is more sensitive to featured regions of mesh models than existing operators. Secondly, we use the 1‐ring tetrahedron in the dual mesh, a volumetric structure, to encode geometric details. To preserve the shape of the tetrahedron, we introduce linear tetrahedron constraints minimizing both the distortion of the base triangle and the change of the corresponding height. These ensure that geometric details are accurately preserved during deformation. The time complexity of our new method is similar to that of existing linear Laplacian methods. Examples are included to show that our FS deformation method better preserves mesh details, especially features, than existing Laplacian methods. Copyright © 2010 John Wiley & Sons, Ltd.

Recently, local stereo matching has experienced large progress by the introduction of adaptive support-weights. In this paper, we aim at eliminating negative effects of occlusions by proposing an occlusion-based method to improve traditional support weights. Weights of occluded points are greatly reduced while computing matching costs, initial disparities and final disparities. Experimental results on the Middlebury images demonstrate that our method is very effective in improving disparities of points around occluded areas and depth discontinuities. According to the Middlebury benchmark, the proposed algorithm is now the top performer among local stereo methods. Moreover, this approach can be easily integrated into nearly all existing support weights strategies.

Abstract Updating a Delaunay triangulation when data points are slightly moved is the bottleneck of computation time in variational methods for mesh generation and remeshing. Utilizing the connectivity coherence between two consecutive Delaunay triangulations for computation speedup is the key to solving this problem. Our contribution is an effective filtering technique that confirms most bi‐cells whose Delaunay connectivities remain unchanged after the points are perturbed. Based on bi‐cell flipping, we present an efficient algorithm for updating two‐dimensional and three‐dimensional Delaunay triangulations of dynamic point sets. Experimental results show that our algorithm outperforms previous methods.

The problem of constructing surface to fit image data is discussed. The data points of an image can be regarded being sampled from an original surface which can be approximated by piecewise quadratic polynomials. On each local region, a quadratic polynomial surface is constructed with the image data as constraint. The combination of all the quadratic polynomial surfaces forms the fitting surface which approximates the original surface with a quadratic polynomial precision. The experiments for comparing the new method with the existing ones are included.

A new method for constructing triangular patches to pass the C1interpolation conditions (boundary curves and cross-boundary slopes), on the boundary of triangles is presented. The triangular patch is constructed by a basic triangular operator and an error triangular operator. The basic operator is a polynomial of degree six, which approximates the interpolation conditions with a higher approximation precision, while the error operator is constructed by the side-vertex method, which passes the C1error boundary conditions. The C1error boundary conditions are formed by the C1interpolation conditions minus the boundary curves and cross-boundary slopes taken from the basic operator. The basic operator and the error operator are put together to form the triangular patch. Comparison results of the new method with other two methods are included.

A new geometric method is presented for sub-pixel boundary extraction. Many existing methods can obtain boundary pixels, so this paper tries to locate accurately the object edge inside a pixel. Assuming that the boundary is straight line at a pixel size, the new method analyzes the neighboring pixels blurred by the boundary, and finds the boundary by a geometric method. It is proved that the result is accurate for the straight boundary edge. In addition, the new method is proved not to be sensitive to noise. Examples are given to compare the new method and interpolation method at last.

Given a triangulated mesh with the positions and outer surface normal values at the vertexes, this paper presents a scheme to construct a G1 continuous triangular interpolation surface. Most of local interpolation methods suffer from similar defect that the shape of the constructed surface is flat, the shape of the constructed surfaces by scheme of this paper is not flat, and some surfaces can be reconstructed accurately, such as spheres, columns and cones. Firstly, the scheme estimate if the local parameters related to an edge approximately belong to a simple kind of surfaces which has center point or center axis, similar to a sphere, a column or a cone; if so then reconstruct the sub-patch with the normal extension method; else construct the sub-patch with side-vertex method, where cubic Hermite curve is used, and the lengths of endpoint tangent vectors are determined by shape interpolation method; the patch of a triangular domain is the convex combination of the three sub- patches.

A new method for smoothing triangular meshes is presented. Mean curvature normal is used to define a Quasi-Laplacian for smoothing inner vertices at a local region. Vertices are moved along the normal direction in a more appropriate velocity which can make mesh smoothing and shape preserving harmonizing well. For the boundary vertices, a new method for estimating the mean curvature normal is presented, so that for an arbitrary triangular mesh, the inner and the boundary vertices can be smoothed by the same smoothing process. Features of the original mesh can be preserved by the weighted mean curvature normal restriction of the neighbors of one vertex effectively. Experiments of comparison between the new method and previous methods are included in this paper.

Current systems for real-time distributed CSCW are largely rooted in traditional GUI-based groupware. These groupware applications generally lag behind in features and compatibility with single-user applications. As a result, having to learn new GUIs for a sporadic task discourages many users. Many researchers agree that TUIs (tangible user interfaces) are especially suited for collocated collaboration. We conclude that sketching with pencil and eraser together for early design is suited for remote collaboration and communication. Current sketching systems are largely for single-user and sketching with pencil. In this paper, we present a method supporting sketching with pencil and eraser together for remote collaboration.

Based on the smoothness criterion of minimum curvature variation of the curve, tangent angle constraints guaranteeing an optimized geometric Hermite (OGH) curve both mathematically and geometrically smooths is given, and new methods for constructing composite optimized geometric Hermite (COH) curves are presented in this paper. The comparison of the new methods with Yong and Cheng's methods based on strain energy minimization is included.

A new method to obtain explicit re-parameterization using piecewise rational linear functions is presented in this paper. Based on the outstanding performance of Mobius transformation on modifying pieces with monotonic parametric speed, we create a partition of the original curve, in which the parametric speed of each segment is of monotonic variation. The values of new parameters corresponding to the subdivision points are specified a priori as the ratio of its cumulative arc length and its total arc length. C/sup 1/ continuity conditions are imposed to each segment, thus, with respect to the new parameters, the objective function is linear and admits a closed-form optimization. Analysis of examples shows that, our method brings a curve very close to the arc length parameterization under L/sub 2/ norm but with fewer segments.

The current cubic spline curve interpolation scheme is derived based on the implicit assumption that the magnitude of the first derivative of the curve is close to a constant. However, the assumption is not realized by the interpolation scheme. A knot choosing technique for parametric cubic spline interpolating curve construction that accommodates this assumption is presented. A comparison of the new method with several existing methods is performed and test results are included.

This paper discusses the problem of constructing C 2 quartic spline surface interpolation. Decreasing the continuity of the quartic spline to C 2 offers additional freedom degrees that can be used to adjust the precision and the shape of the interpolation surface. An approach to determining the freedom degrees is given, the continuity equations for constructing C 2 quartic spline curve are discussed, and a new method for constructing C 2 quartic spline surface is presented. The advantages of the new method are that the equations that the surface has to satisfy are strictly row diagonally dominant, and the discontinuous points of the surface are at the given data points. The constructed surface has the precision of quartic polynomial. The comparison of the interpolation precision of the new method with cubic and quartic spline methods is included.

The construction of a C/sup 1/ piecewise bicubic Bezier surface that interpolates a set of data points and lies on the same side of a given plane as the data points is presented. The main tasks in this construction process are the adjustment of the tangent vectors and the determination of the twist vectors at the data points. The tangent vectors are modified by decreasing their magnitudes. The twist vectors are first set by an energy minimization process and then filtered through a modification process to ensure the constructed surface lies on one side of the given plane. The technique for determining the twist vectors has an interpolation precision of cubic polynomials.

A method to scale a multi-surface object while holding the shape and size of specific features (trimming curves) unchanged is presented. The new method includes an earlier version for a one-NURBS-surface object as a special case by considering more general objects and more general features. The constrained scaling process is attach-and-deform based. The new surface is constructed by attaching the original features to a scaled version of the given object. The attaching process requires several transformations and a deformation of the scaled object. The resulting object has the same features as the original object while reflecting the shape and curvature distribution of the scaled object. The presented approach maintains a NURBS representation for each component surface of the resulting object and hence, is compatible with most of the current data-exchange standards. Test results on several car body surfaces with trimming curves are included. The quality of the resulting surfaces is examined using the highlight line model.