Image deblurring and super-resolution are very important in image processing such as face verification. However, when in the outdoors, we often get blurry and low resolution images. To solve the problem, we propose a deep gated fusion attention network (DGFAN) to generate a high resolution image without blurring artifacts. We extract features from two task-independent structures for deburring and super-resolution to avoid the error propagation in the cascade structure of deblurring and super-resolution. We also add an attention module in our network by using channel-wise and spatial-wise features for better features and propose an edge loss function to make the model focus on facial features like eyes and nose. DGFAN performs favorably against the state-of-arts methods in terms of PSNR and SSIM. Also, using the clear images generated by DGFAN can improve the accuracy on face verification.

Interactive image segmentation algorithms rely on the user to provide annotations as the guidance. When the task of interactive segmentation is performed on a small touchscreen device, the requirement of providing precise annotations could be cumbersome to the user. We design a new interaction mechanism that actively queries seeds for guiding the user to label. Our method enforces sparsity and diversity criteria on the selection of query seeds, and at each round of interaction, the user is only presented with a small number of informative query seeds that are far apart from each other. Therefore, the user merely has to swipe through on the ROI-relevant query seeds for checking which ones of the query seeds are inside the region of interest (ROI). This kind of interaction should be easy since those gestures are commonly used on a touchscreen. As a result, we are able to derive a user-friendly interaction mechanism for annotation on small touchscreen devices. The performance of our algorithm is evaluated on six publicly available datasets. The evaluation results show that our algorithm achieves high segmentation accuracy, with short computational time and less user feedback.

Deblurring from a motion blurred image has been studied for some times. Recently, convolution neural network(CNN) has been used widely and it can be used on finding the blur kernel or the latent sharp edge of a blurred image. In recent years, the generative adversarial network (GAN) performs well on style transformation. We consider that a deblurring problem as a style transformation problem. We focus on improving the DeblurGAN's generator, which is the state-of-the-art of the deblurring method and present a new kind of block which combined inception block, residual block and dense block to do deblurring from motion blur. By using the conception of DenseNet which can avoid overfitting. The improved DeblurGAN presents better in both structural similarity measure and by visual effect.

We present a new segmentation method that leverages latent photographic information available at the moment of taking pictures. Photography on a portable device is often done by tapping to focus before shooting the picture. This tap-and-shoot interaction for photography not only specifies the region of interest but also yields useful focus/defocus cues for image segmentation. However, most of the previous interactive segmentation methods address the problem of image segmentation in a post-processing scenario without considering the action of taking pictures. We propose a learning-based approach to this new tap-and-shoot scenario of interactive segmentation. The experimental results on various datasets show that, by training a deep convolutional network to integrate the selection and focus/defocus cues, our method can achieve higher segmentation accuracy in comparison with existing interactive segmentation methods.

We present a new algorithm for unsupervised video segmentation based on boundary-aware optical flow. Existing video segmentation methods usually tweak their segmentation model to tolerate the inaccuracy in the estimation of optical flow around object boundaries. In contrast, we directly manipulate the optical flow for better quality. We smooth the optical flow via transductive inference to make the flow consistent within the object and fit to the object boundaries. We then use the boundary-aware optical flow to estimate the initial foreground object region from each frame for learning the appearance model. The learned appearance model is consequently used to refine the segmentation result. Experiments on the DAVIS dataset show that our method performs favorably against the existing ones.

This paper presents a fast algorithm for deriving the defocus map from a single image. Existing methods of defocus map estimation often include a pixel-level propagation step to spread the measured sparse defocus cues over the whole image. Since the pixel-level propagation step is time-consuming, we develop an effective method to obtain the whole-image defocus blur using oversegmentation and transductive inference. Oversegmentation produces the superpixels and hence greatly reduces the computation costs for subsequent procedures. Transductive inference provides a way to calculate the similarity between superpixels, and thus helps to infer the defocus blur of each superpixel from all other superpixels. The experimental results show that our method is efficient and able to estimate a plausible superpixel-level defocus map from a given single image.

Soft-decision adaptive interpolation (SAI) provides a powerful result in preserving edge structures for interpolation from low-resolution images to obtain high-resolution images. However, the SAI algorithm may produce artifacts in the smooth regions and texture patterns of the interpolated images. To improve the SAI algorithm, we propose an evolutionary game, where every pixel is regarded as a player. The players are divided into two roles, one for unknown high-resolution pixels and the other for known low-resolution pixels. Each role has a different strategy set. The evolutionarily stable strategy in each local image region is a mixed strategy adopted by every player. By considering the mixed strategy as weights for interpolation, we can adaptively estimate the high-resolution image. Experimental results show that the proposed algorithm improves the SAI algorithm by alleviating the artifacts both in PSNR and visual quality.

Dominant sets are a new graph partition method for pairwise data clustering proposed by Pavan and Pelillo. We address the problem of dominant sets with a coalitional game model, in which each data point is treated as a player and similar data points are encouraged to group together for cooperation. We propose betrayal and hermit rules to describe the cooperative behaviors among the players. After applying the betrayal and hermit rules, an optimal and stable graph partition emerges, and all the players in the partition will not change their groups. For computational feasibility, we design an approximate algorithm for finding a dominant set of mutually similar players and then apply the algorithm to an application such as image denoising. In image denoising, every pixel is treated as a player who seeks similar partners according to its patch appearance in its local neighborhood. By averaging the noisy effects with the similar pixels in the dominant sets, we improve nonlocal means image denoising to restore the intrinsic structure of the original images and achieve competitive denoising results with the state-of-the-art methods in visual and quantitative qualities.

We introduce and address the problem of video object cosegmentation, which concerns the task of segmenting the common object in a pair of video sequences. We present a new algorithm that works on super-voxels in videos to solve this task. The algorithm computes i the intra-video relative motion derived from dense optical flow and ii) the inter-video co-features based on Gaussian mixture models. The experimental results show that, by integrating the intra-video and inter-video information, our algorithm is able to obtain better results of segmenting video objects.

In this paper, we present a temporal-state shape context (TSSC) method that exploits space-time shape variations for human action recognition. In our method, the silhouettes of objects in a video clip are organized into three temporal states. These states are defined by fuzzy time intervals, which can lessen the degradation of recognition performance caused by time warping effects. The TSSC features capture local characteristics of the space-time shape induced by consecutive changes of silhouettes. Experimental results show that our method is effective for human action recognition, and is reliable when there are various kinds of deformations. Moreover, our method can identify spatially inconsistent parts between two shapes of the actions, which could be useful in action analysis applications.

In this paper, we propose a new reversible image data hiding algorithm with high data embedding rate and low stego-image distortion. The proposed method is called as a multilayer algorithm. It can embed data in the image repeatedly in a non-overlapping block fashion. In the first layer, the image is divided into 2 x 2 blocks in a raster-scan order and at most 3 bits can be embedded in each block. In the second layer, the resulting stego-image from the first layer is also divided into 2 x 2 blocks in a raster-scan order from the second pixel to alleviate the stego-image distortion, and so on. Each layer has its threshold to control its data embedding rate. The threshold is adaptively adjusted according to the ratio of the size of the total data bitstream to the size of the already embedded bitstream. Our experimental results show the proposed method achieves good performance of high data embedding rate and high stego-image quality.

A novel reversible visible watermarking algorithm is proposed. It can fully remove the watermark from the visible watermarked image such that the original image can be restored. Pixel values of original image beneath the watermark are mapped to a small range [α, α + 127] to generate a visible watermarked image. Since the mapping is many-to-one, taking inverse mapping can only approximate the original image. To restore the original image, the difference image of subtracting the approximated image from the original image and other side information are losslessly compressed to be embedded in the visible watermarked image by a reversible data embedding algorithm. We proposed a key-based scheme for the compromise between transparency and robustness. The key is a random variable with discrete normal distribution. In addition, only users with correct key can restore the original image. In the experimental results, we show the transparent degree of watermark can be controlled by the variance of the key. Users with wrong key can not restore the original image from the visible watermarked image.

There are many researches that have been proposed for embedding data into digital video. However, most of those schemes extending data hiding technique for still images to videos by treating each single frame as a still image and embed data in intra-frame. In this paper, we propose an effective data-hiding scheme that embeds data in digital videos using the phase angle of the motion vector of the macroblock in the inter-frame. The scheme can be applied to either compressed or uncompressed videos. Furthermore, the embedded data can be extracted directly without using the original video sequences. Our experimental results prove the feasibility of the proposed method.

In this paper a new construction algorithm of visual cryptography is presented. First, we extend SFCOD (space filling curve ordered dither - one of the techniques of halftoning) to transform a gray-level image into an image with fewer grayscale values. In addition, we extend the basic visual cryptography model to handle more than two grayscale values. Then the extended visual cryptography model can be applied to encode this image. This scheme satisfies the security and contrast conditions. It can reveal more details of original images in the decoded images than ordinary visual cryptography scheme.

A lot of digital watermarking techniques have been proposed to resolve the issues of copyright protection. However, almost proposed watermarking methods keep the watermarking algorithm private to ensure the embedded watermark secret. If the watermarking technique needs to be widespread applied to realistic multimedia environment, the algorithm used by watermarking techniques should be public. In this paper, a novel watermarking scheme which can be public is presented. The proposed watermarking technique is developed based on the following criterions: (1) the watermarking algorithm is open; (2) the embedded watermark can be extracted and embedded by the people who own the secret key. The watermarking scheme employs the advance encryption standard (AES) and the Reed-Solomon code, to make the watermarking algorithm public. Simulation shows that the proposed algorithm can be very robust to resolve the ownership of the digital image.

Digital watermarking plays an important role in copyright protection, content authentication, and annotation of digital multimedia. To embed a watermark quickly, we embed the watermark in the compressed domain instead of the frequency domain. For high imperceptibility, a new video watermarking, based on the human visual system (HVS), is proposed. It embeds the watermark in the block that is highly tolerant of noise. The proposed algorithm directly substitutes the variable length code (VLC) without inverse quantization and forward quantization. It can greatly reduce computational complexity. In the experiment, we apply MPEG-1 video sequences as our test sequence and the results demonstrate the high imperceptibility of the watermark.

We focus on the problem of comparing face images under different lighting conditions. A new and robust face image matching algorithm is developed based on an accumulated consistency measure of corresponding normalized gradients at face contour locations between two face images. The proposed new image matching approach is motivated by the characteristics of high image gradient along the face contour. To amend the matching problem due to lighting changes between face images, we define a new consistency measure to be the inner product between two normalized gradient vectors at the corresponding locations in two images. The normalized gradient is obtained by dividing the computed gradient vector by a maximal gradient magnitude in a local neighborhood centered at the pixel of computation. Then we compute the summation of the individual consistency measure from normalized gradients at all the contour pixels to be the robust matching measure between two face images. To alleviate the problem due to shadow and intensity saturation, we introduce an intensity weighting function for each individual consistency measure to form a weighted consistency measure. We test the proposed image matching algorithm on the Yale Face Database, which contains 15 persons captured under three very different lighting conditions. The proposed method can achieve satisfactory recognition results in the experiments.

Digital watermarking has been proposed for copyright protection of multimedia data. Most current researchers focus on the robust watermarking techniques to embed a perceptually invisible watermark in the image without considering the multiple ownership deadlock problem of the watermarked image. We propose a digital watermark technique to resolve the rightful ownerships of a digital image. The original image is processed by a standard hash algorithm to generate a hash value, which can be viewed as the fingerprint of the image. The hash value is used as a secret key for a standard encryption algorithm such as data encryption standard (DES) or the advanced encryption standard. (AES). A meaningful watermark is then encrypted by the standard encryption algorithm We then develop a robust technique based on wavelet transform to embed the encrypted watermark in the image. In our simulation, we use the secure hash algorithm SHA and the block cipher RC6, which was a candidate of the AES, to encrypt the watermark. Experimental results shows that the proposed method can be effective and robust to resolve the rightful ownerships of digital images.

In this paper, a quantization table based on the human visual system is designed for the baseline JPEG coder. By incorporating the human visual system with the uniform quantizer, a perceptual quantization table is derived. The quantization table is easy to adapt to the specified resolution for viewing and printing. Experimental results indicate that the HVS-based quantization table can achieve improvements in PSNR about 0.2-2.0 dB without increasing the complexity in both encoder and encoder.

We propose a variable block truncation coding with optimal quadtree segmentation (VBTC) to compress still images. Subject to a given bit budget, the best quadtree segmentation of an image is found for VBTC such that the distortion of the reconstructed image is minimal. Compared to the conventional variable BTC (vBTC) and hierarchical BTC (HBTC) at 1.425 bits per pixel (bpp), our proposed VBTC achieves 17.6% and 54.9% error reduction, respectively. A bit plane reduction scheme is also applied for obtaining lower bit rates. Our simulation results show that our proposed VBTC combined with bit plane reduction scheme is superior to VBTC, HBTC, and adaptive compression coding (ACC).

In this paper, we propose an optimal quadtree segmentation of an image for variable blocksize vector quantization (VBVQ) such that the total distortion of the reconstructed image is minimal and the total required bits don't exceed the bit budget. The above constraint problem is converted into an equivalent unconstrained problem by using Lagrange multiplier. We prune the full quadtree by comparing the Lagrangian costs of the parent and four child nodes. If the adjacent subblocks merge into a larger block reducing the Lagrangian cost, these subblocks will be merged. Otherwise, these subblocks will be vector quantized. From our simulation results, we see that the reconstructed image of our proposed algorithm has 1-3 db higher PSNR than the fixed blocksize VQ and conventional VBVQ algorithms.

Displaying a true color image in a low cost color monitor or printer is very important in many applications. The basic problem is how to display 2/sup 24/ colors with K(usually K=256 or less) colors, called a color palette. The authors propose an algorithm to design the K-color palette by an unbalanced greedy binary tree. The algorithm splits the node with largest error reduction into two children by a hyperplane perpendicular to one of r,g,b axis among available nodes in the tree. The color image is then quantized by the K-color palette.>

Motion estimation is very important in video-phone, video-conference and HDTV, which will become part of everyday life in the near future. Conventionally, it uses full search algorithm because of its computational regularity suitable for VLSI implementation. However, its search range around the search center is 16 X 16 pixels, which requires 256 processors. To increase the search range by full search for HDTV applications, the number of processors will also increase. This makes it very difficult for VLSI implementation. This paper proposes a two stage hierarchical search algorithm to overcome the difficulty. In the first stage, k best motion vectors are found and then fine tuned in the second stage. Both stages use full search and can be easily implemented by VLSI. Instead of using multi-layers hierarchical search technique, we propose a two stage hierarchical search algorithm. It does not require sampling process and can be implemented in a much simpler hardware circuit.

Median filtering is a nonlinear filtering technique which is known for preserving sharp changes in signals and being particularly effective in removing impulse noise. It is very widely used in speech and image processing since it was first described by Tukey. Recently, Neuvo first introduced the concepts of vector median. In this approach, the samples of the vector- value input signals are processed as vectors instead of componentwise scalar processing. The vector median utilizes the correlation between different components. Thus, it is good for HDTV color image processing. In this paper, we first discuss the difference between componentwise median filter and vector median filter. Then, we propose several new vector median filters to increase signal noise ratio. A fast computational architecture for these algorithms is also addressed.

The authors present a systolic algorithm and its variations for the k-nearest neighbors problem (kNNP). Multiple-shot queries with different ranges (k values) can be served in a pipelined fashion. A partitioning scheme is developed to handle large size problems. Performance of the algorithm is analyzed. Formulas for the optimal array size in terms of computation time and area-time-time product (ATT) are derived. The algorithm can solve a multiple-shot kNNP in N+2 square root N*K systolic steps using square root N*K processing elements, where N is the problem size (i.e. the number of points), and K is the sum of all k-values.>

A bit-level systolic array based on the majority of a bit string of the input samples for a one-dimensional standard median filter is developed. The majority function is implemented by a 1-b odd/even transposition network, which can be designed very easily. For a window size n, the latency time and the computation time of the array are about n+6 times the gate delay. A bit-level systolic array for 2-D separable median filter is also proposed. It is constructed using the basic cells of the 1-D standard median filter.>