Recent restoration methods for handling real old photos have achieved significant improvements using generative networks. However, the restoration quality under the usual generative architectures is greatly affected by the encoded properties of latent space, which reflect pivotal semantic information in the recovery process. Therefore, how to find the suitable latent space and identify its semantic factors is an important issue in this challenging task. To this end, we propose a novel generative network with hyperbolic embeddings to restore old photos that suffer from multiple degradations. Specifically, we transform high-dimensional Euclidean features into a compact latent space via the hyperbolic operations. In order to enhance the hierarchical representative capability, we perform the channel mixing and group convolutions for the intermediate hyperbolic features. By using attention-based aggregation mechanism in a hyperbolic space, we can further obtain the resulting latent vectors, which are more effective in encoding the important semantic factors and improving the restoration quality. Besides, we design a diversity loss to guide each latent vector to disentangle different semantics. Extensive experiments have shown that our method is able to generate visually pleasing photos and outperforms state-of-the-art restoration methods.

The 2D virtual try-on task aims to transfer a target clothing image to the corresponding region of a person image. Although an extensive amount of research has been conducted due to its immense applications, this task still remains a great challenge to handle some complicated issues (e.g., non-rigid shapes, large occlusions and arbitrary poses). To this end, we propose a novel network with structural and textural consistency-preserving mechanism for producing high-fidelity try-on images. Specifically, we first generate the semantic layout of a clothing-agnostic person to obtain the segmentation map, which is used as the transforming conditions of the target clothes. Based on a recurrent network structure, the transform lookup is performed to iteratively update a dense flow. Then, we adopt a thin-plate-spline-based warping method to estimate the coarse offset flow for all key-point positions. Guided by this sparse flow, a multi-scale deformable convolution module is designed to further iteratively predict the fine offsets for densely sampled positions, by which the clothing item and person shape can be accurately aligned. Finally, we develop a refinement module to effectively fuse the global and local features, which can render accurate geometric structures of the body parts and maintain texture sharpness of the clothes. Extensive experiments on benchmark datasets demonstrate that our method outperforms other state-of-the-art methods in terms of quantitative and qualitative try-on results. The code is available on: https://github.com/TJU-WEIHAO/MLCN .

Change detection (CD) in remote-sensing images is one of the most crucial topics in the computer vision community. Most recent CD pipelines focus on introducing attention mechanism to enhance the discriminative ability of network, but their crude model architectures lead to inaccurate predictions and irregular boundaries. In this letter, we present a boundary-aware Siamese network (BASNet) for accurate remote-sensing CD. Based on the encoder–decoder architecture, we first propose a novel multiscale paired fusion module (MPFM) to effectively fuse the same-level feature pairs from the Siamese encoding stream. In addition, we design a location guidance module (LGM) to accurately locate the changed regions. Based on the observation that hierarchical features show different level information, we propose a multilevel feature aggregation module (MFAM) to merge the bottom-up features. Finally, we introduce a hybrid loss that fuses structural similarity (SSIM) loss and binary cross entropy (BCE) loss to focus on the structural integrity and boundary quality of changed regions. Experimental results on two public datasets demonstrate that our proposed method significantly improves the performance and outperforms the other state-of-the-art methods.

Attention mechanisms have achieved great success for image super-resolution as they can effectively improve the feature representation ability. However, most attention-based methods produce the static attention weights, which are applied identically for all input samples. This popular attention strategy is difficult to automatically adapt the content variations of each individual input, hence hindering further improvements of the magnification performance. To explore towards resolving this challenge, we propose a variational hybrid network with newly dynamic attention mechanisms for image super-resolution tasks. Specifically, we design a multi-scale variational encoder network to transform the curvature map of an input image into the latent space. This is made possible for randomly generated latent variables to reflect the valuable high-frequency information and recalibrate the main network. We utilize these latent variables to further generate controllable attention weights, which modulate not only frequency parameters of convolutional kernels but also spatial characteristics of feature maps for boosting representation power. Moreover, a curvature-domain loss is designed to help the main network to concentrate more on high-frequency geometric structures. Experimental results have revealed that our method can generate more realistic and visually pleasing high-resolution images in comparison to state-of-the-art methods.

Old photos are an important carrier to preserve the past. Usually, the degradation of old photos is rather diverse and complex. Therefore, the existing methods to solve conventional restoration tasks are difficult to generalize. To solve this problem, we propose a novel method based on generative adversarial network. Our method utilizes the class-attributes of old photos to complete restoration in latent space. Specifically, we divide the process of restoring old photos into two stages, one is global defect restoration stage and the other is local detail restoration stage. In global defect restoration stage, we extract the latent representations of four classes of high-level attributes that are smoothness, clarity, connectivity and completeness. We use latent class-attribute information to restore global defects in latent space and we obtain conditional control vector through a condition network to guide the subsequent local detail restoration stage. In local detail restoration stage, we propose a dynamic condition-guided restoration module that selects the most suitable combination of features to further restore local details through a dynamic network. In addition, we propose a dual discriminator to pay more attention to style and defect restoration. We ignore the complex degradation of old photos to directly restore advanced class-attributes. Therefore, our method has better generalization performance. Experiments show that our method is superior to other existing methods of image restoration in terms of visual quality and numerical metrics.

Point cloud is often noisy and incomplete. Existing completion methods usually generate the complete shapes for missing regions of 3D objects based on the deterministic learning frameworks, which only predict a single reconstruction output. However, these methods ignore the ill-posed nature of the completion problem and do not fully account for multiple possible completion predictions corresponding to one incomplete input. To address this problem, we propose a flow-based network together with a multi-modal mapping strategy for 3D point cloud completion. Specially, an encoder is first introduced to encode the input point cloud data into a rich latent representation suitable for conditioning in all flow-layers. Then we design a conditional normalizing flow architecture to learn the exact distribution of the plausible completion shapes over the multi-modal latent space. Finally, in order to fully utilize additional shape information, we propose a tree-structured decoder to perform the inverse mapping for complete shape generation with high fidelity. The proposed flow network is trained using a single loss named the negative log-likelihood to capture the distribution variations between input and output, without complex reconstruction loss and adversarial loss. Extensive experiments on ShapeNet dataset, KITTI dataset and measured data demonstrate that our method outperforms the state-of-the-art point cloud completion methods through qualitative and quantitative analysis.

In this paper, we propose a novel dictionary-driven image denoising method based on correlation preserving on graphs. To overcome the drawbacks of the instable and unreliable correlations among a set of learned basis vectors, two effective regularized strategies are employed in our coding process. Specifically, a graph-based regularizer is built to preserve the global similarity, which can adaptively capture both geometric structures and discriminative features of textured patches. In particular, edge weights in the graph are obtained by seeking a nonnegative low-rank construction. Besides, a locality constraint is designed to automatically preserve not only spatial neighborhood information but also internal consistency present in noisy patches while learning an overcomplete dictionary. Experimental results show that our method achieves state-of-the-art denoising results in terms of both PSNR and subjective visual quality.

In this paper, we present a novel structure preserving method for single image super-resolution to well construct edge structures and small detail structures. In our approach, the sharp edges are recovered via a novel edge preserving interpolation technique based on a well estimated gradient field and the edge preserving method, which incorporate the local and non-local structure information. The gradient of interpolated high-resolution(HR) image is then regarded as an edge preserving constraint to reconstruct the detail structures. Experimental results demonstrate that the new approach can reconstruct faithfully the HR images with sharp edges and texture structures, and annoying artifacts (blurring, jaggies, ringing, etc.) are greatly suppressed. It outperforms the state-of-the-art approaches, based on subjective and objective evaluations.

In this paper, we present a new image restoration framework based on two high-level regularizations that can predict and preserve the better informative structures in the image. The sparse representation of a blurred image is first obtained to globally encode the salient structures by applying a group of coupled framelet filters. Then a physical meaning regularizer is derived to estimate the point spread function based on the frequency response characteristics of the image. Moreover, based on the operator of structure tensor, a novel nonlocal total variation as the regularizer is established to measure the image variation and non-local self-similarity. Finally, these two high-level regularizers are integrated into an objective function to constrain the ill-posedness. Compared with the state-of-the-art restoration methods, our algorithm can not only suppress strong noises effectively but also recover the sharp structures from the severe and complex blurred images.

Super-resolution enhancement is a kind of promising approach to enhance the spatial resolution of images. To super-resolve a satisfying result, regularization term design and blur kernel estimation are two important aspects which need to be carefully considered. In this paper, we propose a robust regularized super-resolution reconstruction approach based on two sparsity properties to deal with these two aspects. Firstly, we design a sparse reweighted TV L1 prior to restrict the first derivative of the upsampled image. Then, noticing that only deblurring sparse high gradient areas can sharpen the super-resolution result, we design an over-deblurring control method to decrease the artifacts caused by inaccurate blur kernel estimation. We also design a fast optimization algorithm to solve our model. The experimental results show that the proposed approach achieves a remarkable performance both in visual quality and run time.

In this paper, we propose a novel interpolation algorithm for adapting the human visual system (HVS) and applying in real-time image upscaling. The defined statistical features are first computed in a local window of the low-resolution (LR) counterpart. Then the most correlated neighbours of a missing pixel in high-resolution (HR) image are adaptively selected based on local structural analysis for the prominent edges and fine textures. Finally, the unknown pixel values are estimated through a designed directional clustering model (DCM) which incorporates HVS information into the weighted coefficients. The extensive experimental results show that the proposed image interpolation method can accurately reconstruct the structures of HR image in term of arbitrary magnification factors and effectively suppress the jaggy/ringing artifacts with low computation complexity.

This paper describes a new method to reconstruct high-resolution video sequences from several observed low-resolution images based on an adaptive Mumford-shah model which is extended by using nonlocal information and low-rank representation. In our regularization framework, joint image restoration and motion estimation are first implemented and then detailed information can be recovered by incorporating new model as a prior term.