Low-light image enhancement is a computer vision task that aims to improve the visual perceptual quality of images captured in poorly illuminated scenes. At present, deep learning-based low-light enhancement methods can obtain high-quality enhanced images. However, it does not consider the statistical characteristics of different regions, such as edge, structure, and texture. The uncertainty of image regions is not well characterized and utilized. To address this problem, we propose a novel UnCertainty-driven Cycle-Consistent Generative Adversarial Network (UTrCGAN) to improve the performance of low-light enhancement. UTrCGAN first decomposes the unpaired low/normal-light images into reflectance and illumination components based on the Retinex theory. Then a generative adversarial network guided by uncertainty constraint is proposed to enhance the illumination component, in which the quality of the enhanced image is further improved by the guidance of variance estimation. Experimental results on the widely-used LOL dataset show that UTrCGAN outperforms the state-of-the-art methods in terms of visual quality and quantitative metrics.

The continuous emission of spike stream offers more significant advantages over traditional fixed low sampling rate cameras. Although many reconstruction methods from spike streams have been proposed, the quality of recovered images remains suboptimal. Coarse-to-fine high-speed motion scene reconstruction reconstructs the spike sequence by dividing the dynamic and static regions. However, issues of limited texture richness and low contrasts are unsolved during the reconstruction of static spike. To address these issues, we propose a Coarse-to-Fine spatio-temporal Luminance-Aware Reconstruction (CFLAR) framework. Specially, we propose an adaptive luminance-aware reconstruction in spatio-temporal domain. To be specific, in the spatial domain, we perform region division and region merging of spike sequences based on luminance information, while non-uniform quantization of luminance information is mainly achieved through binary division and adaptive parameters division. In the temporal domain, we integrate alterable window length into the texture from playback to propose adaptive time shift window reconstruction, which enables to obtain reconstructed images with richer textures and higher contrasts. Experimental results demonstrate that our CFLAR method outperforms state-of-the-art approaches in terms of objective and subjective quality.

Single-view 3D reconstruction is a fundamental operation in computer vision. Although significant progress has been made by learning-based approaches, it remains a challenge that the reconstructed mesh is usually coarse since the geometric prior is ignored. In this paper, we propose a geometric prior based graph convolution neural network model (named G2CNN) for single-view 3D reconstruction with Loop subdivision. G2CNN is a data-driven deep neural network (DNN) with the geometry knowledge. To make the reconstructed results with abundant geometric details, we generate shapes with a coarse-to-fine strategy and utilize the Gaussian curvature loss as a geometric supervision. Furthermore, to produce the physically accurate 3D geometry, the mesh subdivision module is designed with Loop subdivision to exploit the vertex localizations and connectivity, which can refine and smooth the mesh surface. Experimental results on both synthesized data and real data demonstrate the effectiveness of our method in terms of both subjective and objective quality.

Depth map captured by 3D cameras usually suffers from low resolution and insufficient quality, which limits its applications in real world. Thus, it is an essential task to develop efficient and effective techniques to handle various depth degradations. In this paper, we propose a color guided depth map super-resolution method with nonlocal autoregressive modeling. Considering that textures in depth map demonstrate distinct geometry direction, we exploit the multi-directional dictionary which is effective in recovering subtle structures of depth patches. We further introduce two regularization terms into the sparse representation framework. Firstly, a patch based autoregressive model is introduced to represent the local patterns in a small area. Secondly, inspired by the structure consistence between depth map and color image, we propose a color guided nonlocal similarity to provide nonlocal constraint to the local structures, which is very helpful in preserving local structures and suppressing noise. Experimental results demonstrate the superior of our method compared with state-of-the-art methods.

Multispectral images (MSIs) contain more spectral information than traditional 2D images, which can provide a more accurate representation of objects. MSIs are easily affected by various noises when captured by sensors. In recent years, many MSI denoising methods, especially the Kronecker-basis-representation (KBR) method, have achieved great success. KBR uses tensor representation and decomposition to achieve good MSI denoising performance. However, each full band patch (FBP) group is decomposed in this method so that too many dictionary atoms are generated. In this paper, we propose a structural tensor sparsity promoting (STSP) model for MSI denoising. In order to decrease the number of dictionary atoms, we cluster FBP groups and learn orthogonal dictionaries for each class rather than each FBP group. To improve the denoising performance, the structural similarity among FBP groups are utilized in the STSP model by enforcing nonlocal centralized sparse constraint, where the compromise parameter is statistically and adaptively determined. Experimental results on the the CAVE dataset demonstrate that our model outperforms the state-of-art methods in terms of both objective and subjective quality.

Depth maps have been widely used in many real world applications, such as human-computer interaction and virtual reality. However, due to the limitation of current depth sensing technology, the captured depth maps usually suffer from low resolution and insufficient quality. In this paper, we propose a depth map super-resolution method via joint local gradient and nonlocal structural regularizations. Depth maps contain mainly smooth areas separated by textures which demonstrate distinct geometry direction characteristic. Motivated by this, we classify depth map patches according to their geometrical directions and learn a compact online dictionary in each class. We further introduce two regularization terms into the sparse representation framework. Firstly, a multi-directional total variation model is proposed to characterize the local patterns in the gradient domain. Secondly, a nonlocal autoregressive model is introduced to provide nonlocal constraint to the local structures, which can effectively restore image details and suppress noise. Quantitative and qualitative evaluations compared with state-of-the-art methods demonstrate that the proposed method achieves superior performance for various configurations of magnification factors and datasets.

As a powerful statistical signal modeling technique, sparse representation has been widely used in various image restoration (IR) applications. The sparsity-based methods have achieved leading performance in the past few decades. However, in recent years it has been surpassed by other methods, especially the recent deep learning based methods. In this paper, we address the question that whether sparse representation can be competitive again. The way we answer this question is to redesign it with a deep architecture. To be specific, we propose an end-to-end deep architecture that follows the process of the sparse representation based IR. In particular, we learn a sparse convolutional dictionary to replace the traditional dictionary, and a convolutional neural network (CNN) denoising prior to replace the image prior. Through end-to-end training, the parameters in convolutional dictionary and CNN denoiser can be jointly optimized. Experimental results on several representative IR tasks, including image denoising, deblurring and super-resolution, demonstrate that the proposed deep network can achieve superior performance against state-of-the-art model-based and learning-based methods.

Under the extreme conditions such as excessive light, insufficient light or high-speed motion, the detection of vehicles by frame-based cameras still has challenges. Event cameras can capture the frame and event data asynchronously, which is of great help to address the object detection under the aforementioned extreme condition. We propose a fusion network with Attention Fusion module for vehicle object detection by jointly utilizing the features of both frame and event data. The frame and event data are separately fed into the symmetric framework based on Gaussian YOLOv3 to model the bounding box (bbox) coordinates of YOLOv3 as the Gaussian parameters and predict the localization uncertainty of bbox with a redesigned cross-entropy loss function of bbox. The feature maps of these Gaussian parameter and confidence map in each layer are deeply fused in the Attention Fusion module. Finally, the feature maps of the frame and event data are concatenated to the detection layer to improve the detection accuracy. The experimental results show that the method presented in this paper outperforms the state-of-the-art methods only using the traditional frame-based network and the joint network combining the event and frame information.

Light field is a kind of 4D signal that contains rich information about position and angle of light, which can express the scene more accurately. Light field is easily affected by noise for the hardware sensitivity. This paper utilizes the intrinsic tensor sparsity model and integrates super-resolution(SR) into a unified light field denoising method based on tensor operation. Avoiding vectorization, we make full use of correlation of light field. By exploiting SR method, we avoid sub-pixel mis-alignment in the searching process of similar patch. Experimental results validate that our proposed method outperforms the state-of-art methods in terms of both objective and subjective quality on the HCI light field old dataset.

Texture transfer has been successfully applied in computer vision and computer graphics. Since non-stationary textures are usually complex and anisotropic, it is challenging to transfer these textures by simple supervised method. In this paper, we propose a general solution for non-stationary texture transfer, which can preserve the local structure and visual richness of textures. The inputs of our framework are source texture and semantic annotation pair. We record different semantics as different regions and obtain the color and distribution information from different regions, which is used to guide the the low-level texture transfer algorithm. Specifically, we exploit these local distributions to regularize the texture transfer objective function, which is minimized by iterative search and voting steps. In the search step, we search the nearest neighbor fields of source image to target image through Generalized PatchMatch (GPM) algorithm. In the voting step, we calculate histogram weights and coherence weights for different semantic regions to ensure color accuracy and texture continuity, and to further transfer the textures from the source to the target. By comparing with state-of-the-art algorithms, we demonstrate the effectiveness and superiority of our technique in various non-stationary textures.

Due to the fact that sparse coding in redundant sparse dictionary learning model is NP-hard, interest has turned to the non-redundant sparsifying transform as its sparse coding is computationally cheap. However, natural images typically contain diverse textures that cannot be sparsified well by a non-redundant system. In this paper we propose a new approach for learning redundant sparsifying transform based on equi-angular frame, where the frame and its dual frame are corresponding to applying the forward and the backward transforms. The uniform mutual coherence in the sparsifying transform is enforced by the equi-angular constraint, which better sparsifies diverse textures. In addition, an efficient algorithm is proposed for learning the redundant transform. Experimental results for image representation illustrate the superiority of our proposed method over non-redundant sparsifying transforms. The image denoising results show that our proposed method achieves superior denoising performance, in terms of subjective and objective quality, compared to the K-SVD, the data-driven tight frame method, the learning based sparsifying transform and the overcomplete transform model with block cosparsity (OCTOBOS).

Abdominal computed tomography (CT) is a common modality to detect liver lesions. Liver segmentation in CT scan is important for diagnosis and analysis of liver lesions. However, the accuracy of existing liver segmentation methods is slightly insufficient. In this paper, we propose a liver segmentation architecture named CR-U-Net, which is composed of cascade U-Net combined with residual mapping. We make use of the MDice loss function for training in CR-U-Net, and the second-level of cascade network is deeper than the first-level to extract more detailed image features. Morphological algorithms are utilized as an intermediate-processing step to improve the segmentation accuracy. In addition, we evaluate our proposed CR-U-Net on liver segmentation task under the dataset provided by the 2017 ISBI LiTS Challenge. The experimental result demonstrates that our proposed CR-U-Net can outperform the state-of-the-art methods in term of the performance measures, such as Dice score, VOE, and so on.

Light field, a promising representation to describe the scene appearance, is susceptible to various noise due to the current sensor design. This paper proposes a novel tensor-based denoising method for the 4D light field that consists of two main steps. First, we generalize the intrinsic tensor sparsity measure to light field images by exploiting the nonlocal similarity across the spatial and angular dimensions. Second, we further exploit the spatial-angular correlation by integrating light field super-resolution into the denoising process to eliminate the sub-pixel misalignment of different views. After a back-projection from the refined high-resolution central view under an intensity consistency criteria, the denoising performance for the light field can be boosted. Experimental results validate the superior performance of the proposed method in terms of both PSNR and visual quality on the HCI light field dataset.

Traditional synthesis/analysis sparse representation models signals in a one dimensional (1D) way, in which a multidimensional (MD) signal is converted into a 1D vector. 1D modeling cannot sufficiently handle MD signals of high dimensionality in limited computational resources and memory usage, as breaking the data structure and inherently ignores the diversity of MD signals (tensors). We utilize the multilinearity of tensors to establish the redundant basis of the space of multi linear maps with the sparsity constraint, and further propose MD synthesis/analysis sparse models to effectively and efficiently represent MD signals in their original form. The dimensional features of MD signals are captured by a series of dictionaries simultaneously and collaboratively. The corresponding dictionary learning algorithms and unified MD signal restoration formulations are proposed. The effectiveness of the proposed models and dictionary learning algorithms is demonstrated through experiments on MD signals denoising, image super-resolution and texture classification. Experiments show that the proposed MD models outperform state-of-the-art 1D models in terms of signal representation quality, computational overhead, and memory storage. Moreover, our proposed MD sparse models generalize the 1D sparse models and are flexible and adaptive to both homogeneous and inhomogeneous properties of MD signals.

The conventional sparse model relies on data representation in the form of vectors. It represents the vector-valued or vectorized one dimensional (1D) version of an signal as a highly sparse linear combination of basis atoms from a large dictionary. The 1D modeling, though simple, ignores the inherent structure and breaks the local correlation inside multidimensional (MD) signals. It also dramatically increases the demand of memory as well as computational resources especially when dealing with high dimensional signals. In this paper, we propose a new sparse model TenSR based on tensor for MD data representation along with the corresponding MD sparse coding and MD dictionary learning algorithms. The proposed TenSR model is able to well approximate the structure in each mode inherent in MD signals with a series of adaptive separable structure dictionaries via dictionary learning. The proposed MD sparse coding algorithm by proximal method further reduces the computational cost significantly. Experimental results with real world MD signals, i.e. 3D Multi-spectral images, show the proposed TenSR greatly reduces both the computational and memory costs with competitive performance in comparison with the state-of-the-art sparse representation methods. We believe our proposed TenSR model is a promising way to empower the sparse representation especially for large scale high order signals.

Image super-resolution with sparsity prior provides promising performance. However, traditional sparse-based super resolution methods transform a two dimensional (2D) image into a one dimensional (1D) vector, which ignores the intrinsic 2D structure as well as spatial correlation inherent in images. In this paper, we propose the first image super-resolution method which reconstructs a high resolution image from its low resolution counterpart via a two dimensional sparse model. Correspondingly, we present a new dictionary learning algorithm to fully make use of the corresponding relationship of two pairs of 2D dictionaries of low and high resolution images, respectively. Experimental results demonstrate that our proposed image super-resolution with 2D sparse model outperforms state-of-the-art 1D sparse model based super resolution methods in terms of both reconstruction ability and memory usage.

Image super-resolution based on sparse model with patch clustering and nonlocal similarity provides promising performance. However, the traditional one dimensional (1D) sparse model enforces a 1D dictionary for every cluster of patches to capture complex structures and different features in images. The total dictionary will take expensive memory, which can be alleviated at cost of representation power. Recently, two dimensional (2D) sparse model has been proved to efficiently represent images and save memory usage. In this paper, we propose to integrate 2D sparse model with patch clustering and nonlocal similarity into a variational framework as 2D nonlocal sparse representation (2DNSR) for image SR to save memory cost and ensure SR performances. We also present a 2DNSR algorithm for image SR where each group of similar patches decompose on the respective 2D dictionaries. Experimental results on image SR demonstrate our proposed 2D nonlocal representation outperforms 2D sparse model and achieves competitive performance to state-of-the-art 1D nonlocal sparse models whereas with much less memory costs.

Two dimensional (2D) sparse representation provides promising performance in image denoising by cooperatively exploiting horizontal and vertical features inherent in images by two dictionaries. In this paper, we first propose integrating the 2D sparse model with clustering and nonlocal regularization into a unified variational framework, defined as 2D nonlocal sparse representation (2DNSR), for optimization. Within this framework, we then present a dictionary learning method for image denoising which jointly decomposes groups of similar noisy patches on subsets of 2D dictionaries. We finally present a 2DNSR-based algorithm for image denoising. Experimental results on image denoising show our proposed 2D nonlocal sparse representation outperforms the 2D sparse model and achieves competitive performance to state-of-the-art nonlocal sparse models whereas with much less memory costs.

It is well known that acupuncture treatment has an effect on patients with ischemie cerebrovascular disease. This study was aimed to summarize the basic laws of acupoints of ischemie cerebrovascular disease in acupuncture treatment using text mining techniques. First, we proposed the text-mining-based method to collect related literatures about acupuncture treatment of ischemie cerebrovascular disease from Chinese Biomedicai Literature Database, and then the ACCESS database was constructed. Second, structured query language was applied to data processing as well as data stratification algorithm was adopted to analyze the basic laws of acupoints of ischemie cerebrovascular disease in acupuncture treatment. Final, 62,903 documents were retrieved and the results showed that the higher frequency of single point in the treatment of ischemie cerebrovascular disease were sorted in descending order as the following: Renzhong(570), Burong(205), Fufen(186), Youmen(84), Shuifen(58), Baihui(57), Zusanli(55). Moreover, the higher frequency of paired points in the treatment of ischemie cerebrovascular disease were sorted in descending order as the following: Baihui with Dazhui(21), Neiguan with Zusanli(21), Quchi with Zusanli(21), Baihui with Qubin(10), Neiguan with Renzhong(9), Neiguan with Sanyinjiao(9), Neiguan with Yinjiao(9). These findings suggest that the method of text mining can be used to analyze basic acupoint laws through statistical frequency of acupuncture treatment of ischemie cerebrovascular disease and to provide a reference for clinical acupoints.

An analysis sparse model represents an image signal by multiplying it using an analysis dictionary, leading to a sparse outcome. It transforms an image (two dimensional signal) into a one-dimensional (1D) vector. However, this 1D model ignores the two dimensional property and breaks the local spatial correlation inside images. In this paper, we propose a two dimensional (2D) analysis sparse model. Our 2D model uses two analysis dictionaries to efficiently exploit the horizontal and vertical features simultaneously. The corresponding sparse coding and dictionary learning algorithm are also presented in this paper. The 2D sparse model is further evaluated for image denoising. Experimental results demonstrate our 2D analysis sparse model outperforms a state-of-the-art 1D analysis model in terms of both denoising ability and memory usage.

Sparse representation has been proved to be very efficient in machine learning and image processing. Traditional image sparse representation formulates an image into a one dimensional (1D) vector which is then represented by a sparse linear combination of the basis atoms from a dictionary. This 1D representation ignores the local spatial correlation inside one image. In this paper, we propose a two dimensional (2D) sparse model to much efficiently exploit the horizontal and vertical features which are represented by two dictionaries simultaneously. The corresponding sparse coding and dictionary learning algorithm are also presented in this paper. The 2D synthesis model is further evaluated in image denoising. Experimental results demonstrate our 2D synthesis sparse model outperforms the state-of-the-art 1D model in terms of both objective and subjective qualities.

In order to show the realistic 3D mesh in geometry image-based 3D mesh compression, in addition to coding geometry image, normal-map image is usually required to code. But normal-map image are difficult to compress because it captures more details of the original mesh, and it has less spatial correlation between pixels than geometry image. This paper proposes a novel coding framework to solve this problem, we effectively predict the normal-map image based on the correlation between geometry image and normal-map image, and we also utilize the strong correlation among three components of normal-map image to improve the predicting accuracy. In this framework we only need to code geometry image and residual image which generated from normal-map image and its prediction. Experimental results show that comparing with the method which coding geometry image and normal-map image using JPEG2000 directly, our coding framework not only improves the coding efficiency of geometry images and normal-map images, but also enhances the realistic effect of 3D mesh significantly.