In the interactive segmentation, users initially click on the target object to segment the main body and then provide corrections on mislabeled regions to iteratively refine the segmentation masks. Most existing methods transform these user-provided clicks into interaction maps and concatenate them with image as the input tensor. Typically, the interaction maps are determined by measuring the distance of each pixel to the clicked points, ignoring the relation between clicks and mislabeled regions. We propose a Dynamic Click Transform Network (DCT-Net), consisting of Spatial-DCT and Feature-DCT, to better represent user interactions. Spatial-DCT transforms each user-provided click with individual diffusion distance according to the target scale, and Feature-DCT normalizes the extracted feature map to a specific distribution predicted from the clicked points. We demonstrate the effectiveness of our proposed method and achieve favorable performance compared to the state-of-the-art on three standard benchmark datasets.

We present a new way of recursive image filtering called the two-way recursive filtering. Different from most recursive image filters that pose the 2D image filtering as iterative 1D row/column filtering, we consider the propagation on a 2D image plane directly. To compute the filtered value of a pixel, our method takes feedback from two neighboring pixels, one connected horizontally and the other connected vertically. In this way, the proposed filter has a dense receptive field, leading to fewer iterations needed to filter an image. We demonstrate its ability in high-quality edge-aware image smoothing without compromising the efficiency compared to 1D filters. We show that the proposed filter can also be formulated as a layer in a deep neural network. The feedback coefficients of the recursive filter are learned via backpropagating through the proposed filter. Our experiments on segmentation refinement show that our method produces favorable results against the three-way connection scheme in a previous segmentation refinement algorithm. To the best of our knowledge, this is the first work that explores the two-way pixel connectivity for recursive filtering.

Person re-identification (Re-ID) aims to match images of the same person across distinct camera views. In this paper, we propose the Space-Time Guided Association Learning (STGAL) for unsupervised Re-ID without ground truth identity nor image correspondence observed during training. By exploiting the spatial-temporal information presented in pedestrian data, our STGAL is able to identify positive and negative image pairs for learning Re-ID feature representations. Experiments on a variety of datasets confirm the effectiveness of our approach, which achieves promising performance when comparing to the state-of-the-art methods.

In this paper, we propose a deep CNN to tackle the image restoration problem by learning formatted information. Previous deep learning based methods directly learn the mapping from corrupted images to clean images, and may suffer from the gradient exploding/vanishing problems of deep neural networks. We propose to address the image restoration problem by learning the structured details and recovering the latent clean image together, from the shared information between the corrupted image and the latent image. In addition, instead of learning the pure difference (corruption), we propose to add a residual formatting layer and an adversarial block to format the information to structured one, which allows the network to converge faster and boosts the performance. Furthermore, we propose a cross-level loss net to ensure both pixel-level accuracy and semantic-level visual quality. Evaluations on public datasets show that the proposed method performs favorably against existing approaches quantitatively and qualitatively.

Superpixel segmentation has been widely used in many computer vision tasks. Existing superpixel algorithms are mainly based on hand-crafted features, which often fail to preserve weak object boundaries. In this work, we leverage deep neural networks to facilitate extracting superpixels from images. We show a simple integration of deep features with existing superpixel algorithms does not result in better performance as these features do not model segmentation. Instead, we propose a segmentation-aware affinity learning approach for superpixel segmentation. Specifically, we propose a new loss function that takes the segmentation error into account for affinity learning. We also develop the Pixel Affinity Net for affinity prediction. Extensive experimental results show that the proposed algorithm based on the learned segmentation-aware loss performs favorably against the state-of-the-art methods. We also demonstrate the use of the learned superpixels in numerous vision applications with consistent improvements.

In this paper, we review the design of Minimum Barrier Distance and propose a new path-wise distance metric called Minimum Spanning Distance (MSD). Unlike most existing distance metrics, which only define distance between two pixels on gray-scale images, the proposed distance metric conceptually estimates the color space spanned by the colors on the path of interest. Therefore, the MSD takes into consideration the three channels on color images at the same time to compute distance. Compared with other distance metrics, MSD can not only achieve the highest numerical scores but also produce visually good segmentation maps in our experiment of interactive segmentation on the Gulshan dataset.

In the field of approximate nearest neighbor (ANN) search, rare of the existing approaches are tailored for video applications. The Ring Intersection Approximate Nearest Neighbor (RIANN) is the first ANN search algorithm for videos. It achieves real-time by performing the ANN search on the sparse grid and interpolating others. For some applications, the dense ANN search is needed to ensure the searching accuracy. To achieve dense ANN search in real-time, we consider the parallel computing as a solution. However, the RIANN algorithm is not suitable for parallel computing as the algorithm itself suffers from bad thread coherency. In this paper, we propose the Sphere Ring Intersection Approximate Nearest Neighbor (SRIANN), which solves the problem of bad thread coherency and improves the accuracy of ANN search compared to the original RIANN method. The experimental results show that the proposed method is the only one able to perform dense ANN search for CIF videos in real-time.

Bilateral filtering (BLF) and median filtering (MF) are key components in many applications. As the image resolution grows rapidly, implementation of efficient filtering is highly demanded. In this paper, we present a unified VLSI architecture that is able to compute both kinds of filters for 4k2k videos at 30fps. One feature of this design is that we leverage an emerging layer-based algorithm for both BLF and MF, which turns the non-linear BLF or MF into a set of linear filtering followed by output interpolation. It allows more accurate approximation than previous works. Moreover, to address the introduced design challenges of filter units and comparison units, we propose an architecture with moving sum method (MSM) of box filtering and a parallel component selector (PCS), respectively. The MSM requires only one line buffer, and the PCS architecture allows us to complete the comparison in one cycle. Furthermore, a shared pre-computation unit is also proposed to support both kinds of filters. As a result, our design costs only 6.33KB on-chip memory and 161.5K logic gates to support both BLF and MF. It is much efficient than existing architectures which support BLF only.

In this paper, we present a real-time salient object detection system based on the minimum spanning tree. Due to the fact that background regions are typically connected to the image boundaries, salient objects can be extracted by computing the distances to the boundaries. However, measuring the image boundary connectivity efficiently is a challenging problem. Existing methods either rely on superpixel representation to reduce the processing units or approximate the distance transform. Instead, we propose an exact and iteration free solution on a minimum spanning tree. The minimum spanning tree representation of an image inherently reveals the object geometry information in a scene. Meanwhile, it largely reduces the search space of shortest paths, resulting an efficient and high quality distance transform algorithm. We further introduce a boundary dissimilarity measure to compliment the shortage of distance transform for salient object detection. Extensive evaluations show that the proposed algorithm achieves the leading performance compared to the state-of-the-art methods in terms of efficiency and accuracy.

Image super-resolution has gained much attention in these years, while video super-resolution remains almost unchanged. In this paper, we propose a fast super-resolution method for video. We exploit recent development of learning-based technique that achieves state-of-the-art in accuracy and efficiency for image super-resolution. We leverage the temporal coherency of video contents to approximate the nearest neighbor search in learning-based SR. Experimental results show that our method is able to produce visually similar or better results while being 20 times faster than baseline frame-by-frame fast image SR and being orders of magnitude faster than complex optimization-based video SR.

Bilateral filtering is a commonly used technique in image processing. However, being nonlinear, it is computationally expensive. The situation gets worse while the filter radius grows up. Several works have been proposed to accelerate the computation. Nevertheless, most techniques are tailored for grayscale image bilateral filtering or confined to specific kernel functions. In this paper, we propose a constant time bilateral filter for color images. Specifically, we extend an existing constant time bilateral filtering technique, which has been demonstrated the state-of-the-art for gray-scale images. We generalize the original approach as a three-stage algorithm. Based on the generalization, we explain the drawbacks of its naïve extension for color images and propose a solution that adapts to the image content. Experimental results demonstrate the effectiveness of our solution.

Image denoising is always an active problem in image processing over the years. Though the principles to gray-scale image denoising have been extensively explored in recent advances. Relatively few works focus on the problem of color image denoising. In this paper, we address the denoising problem for natural color images. We propose a simple yet effective denoising algorithm based on the local color line assumption for natural color images. We further show that the proposed denoising strategy can be easily realized by slight modification to the guided filter. Moreover, the intermediate data in guided filter computation is utilized to make the denoising algorithm highly adaptable to local noise level. Experimental results show that the proposed method is effective to natural color image denoising and comparable to other complex methods with low complexity.

Recently, more and more natural image statistics are found useful for image restoration problems. In this paper, we propose a noise reduction technique by use of color-line assumption for natural color images. Based on the color-line model, we propose an algorithm to analyse local color statistics and recover the original image by promoting color linearity of a local patch. Moreover, the proposed method is employed on superpixels to alleviate the boundary effect of the denoising operation. The experimental results show that the proposed method can collaborate with existing noise reduction methods to successfully further boost the quality in both perceptual and objective evaluations.