Video super-resolution aims at generating a high-resolution video from its low-resolution counterpart. With the rapid rise of deep learning, many recently proposed video super-resolution methods use convolutional neural networks in conjunction with explicit motion compensation to capitalize on statistical dependencies within and across low-resolution frames. Two common issues of such methods are noteworthy. Firstly, the quality of the final reconstructed HR video is often very sensitive to the accuracy of motion estimation. Secondly, the warp grid needed for motion compensation, which is specified by the two flow maps delineating pixel displacements in horizontal and vertical directions, tends to introduce additional errors and jeopardize the temporal consistency across video frames. To address these issues, we propose a novel dynamic local filter network to perform implicit motion estimation and compensation by employing, via locally connected layers, sample-specific and position-specific dynamic local filters that are tailored to the target pixels. We also propose a global refinement network based on ResBlock and autoencoder structures to exploit non-local correlations and enhance the spatial consistency of super-resolved frames. The experimental results demonstrate that the proposed method outperforms the state-of-the-art, and validate its strength in terms of local transformation handling, temporal consistency as well as edge sharpness.

Multi-frame image super-resolution focuses on reconstructing a high-resolution image from a set of low-resolution images with high similarity. Combining image prior knowledge with fidelity model, the Bayesian-based methods have been considered as an effective technique in super-resolution. The minimization function derived from maximum a posteriori probability (MAP) is composed of a fidelity term and a regularization term. In this paper, based on the MAP estimation, we propose a novel initialization method for super-resolution imaging. For the fidelity term in our proposed method, the half-quadratic estimation is used to choose error norm adaptively instead of using fixed and norms. Besides, a spatial weight matrix is used as a confidence map to scale the estimation result. For the regularization term, we propose a novel regularization method based on adaptive bilateral total variation (ABTV). Both the fidelity term and the ABTV regularization guarantee the robustness of our framework. The fidelity term is mainly responsible for dealing with misregistration, blur, and other kinds of large errors, while the ABTV regularization aims at edge preservation and noise removal. The proposed scheme is tested on both synthetic data and real data. The experimental results illustrate the superiority of our proposed method in terms of edge preservation and noise removal over the state-of-the-art algorithms.

This study presents an automatic, human perception based chroma‐keying algorithm that extracts the objects of interest (i.e. foreground) from monochromatic background. Given an image to be chroma keyed, the global colour distribution and the local texture property are analysed in CIECAM02 colour appearance model. After the analysis, input image is automatically segmented into three parts: foreground, background, and uncertain regions. Afterwards, the background colour is propagated from known background to uncertain region by using interpolation functions; and the foreground colour is estimated based on global colour distribution and a linear cost criteria. The quantitative and perceptual comparisons on the matting results show that the proposed method can reliably remove the background region, correctly restore the intrinsic foreground colour, and accurately keep the fine details. In addition, the authors implement the proposed method on a heterogeneous parallel computing architecture which efficiently distributes the workload among different processors. The simulation results show that the foreground objects can be accurately extracted from high‐definition and/or ultra‐high‐definition videos in real time.

We present Hybrid Key, an automatic tool using chromatic information as the key signal for compositing multiple video layers together. This tool is based on our recently proposed chroma-keying mechanism which simulates human perception by utilizing a hybrid color space and reaches the minimum criteria of real-time high-definition video processing [1]. In this paper, we improve the quality of our perception-based algorithm, and optimized the implementation with a hybrid GPU/CPU architecture. Our new system introduces the task prioritizing and asynchronous processing without jeopardizing the quality. It efficiently distributes computational burdens between different processing units and therefore leaving more room for final alpha and foreground estimation as well as the newly added alpha optimization. In addition, our implementation can make use of GPU specified optimizations to further improve its efficiency. These advantages not only ensure the best overall quality in comparison to other state-of-the-art chroma keyers, but also provide a cost-efficient real-time 4K high-definition chroma keying solution and create the first iPad/iPhone application for high definition chroma keying on the go.

Image quality evaluation is very important. In applications involving signal transmission, the Reduced- or No-Reference quality metrics are generally more practical than the Full-Reference metrics. In this study, we propose a quality estimation method based on a novel semi-fragile and adaptive watermarking scheme. The proposed scheme uses the embedded watermark to estimate the degradation of cover image under different distortions. The watermarking process is implemented in DWT domain of the cover image. The correlated DWT coefficients across the DWT subbands are categorized into Set Partitioning in Hierarchical Trees (SPIHT). Those SPHIT trees are further decomposed into a set of bitplanes. The watermark is embedded into the selected bitplanes of the selected DWT coefficients of the selected tree without causing significant fidelity loss to the cover image. The accuracy of the quality estimation is made to approach that of Full-Reference metrics by referring to an "Ideal Mapping Curve" computed a priori. The experimental results show that the proposed scheme can estimate image quality in terms of PSNR, wPSNR, JND and SSIM with high accuracy under JPEG compression, JPEG2000 compression, Gaussian low-pass filtering and Gaussian noise distortion. The results also show that the proposed scheme has good computational efficiency for practical applications.

Comparison of image processing techniques is critically important in deciding which algorithm, method, or metric to use for enhanced image assessment. Image fusion is a popular choice for various image enhancement applications such as overlay of two image products, refinement of image resolutions for alignment, and image combination for feature extraction and target recognition. Since image fusion is used in many geospatial and night vision applications, it is important to understand these techniques and provide a comparative study of the methods. In this paper, we conduct a comparative study on 12 selected image fusion metrics over six multiresolution image fusion algorithms for two different fusion schemes and input images with distortion. The analysis can be applied to different image combination algorithms, image processing methods, and over a different choice of metrics that are of use to an image processing expert. The paper relates the results to an image quality measurement based on power spectrum and correlation analysis and serves as a summary of many contemporary techniques for objective assessment of image fusion algorithms.

In this paper, we present a video watermarking algorithm based on the log-polar mapping and phase-only filtering method. The log-polar mapping (LPM) domain is obtained first from the magnitude of the Fourier spectrum of the frame. Then, the watermark pattern is embedded in the LPM domain based on the feature that rotation and scaling transformations in the spatial domain result in cyclically translational shifts in the logpolar mapping domain. A matching template is cut from the LPM domain after watermark embedding and is used for correlation matching to find the RST parameters for the watermarked video undergone geometric attacks. Our new phase-only filtering method is used and it is the only filter that can provide an acceptable discrimination while rotation or scaling or both applied to the watermarked video. A square portion from I-frame of each Group of Picture (GOP) is used for watermark embedding based on our analysis that a square image has better tolerance to rotation attack than a rectangular image. The experimental results demonstrate that this algorithm is robust against rotation, scaling, translation transform, noise addition, filtering, MPEG-2 compression, etc.

The survey propagation (SP) algorithm for solving$k$-SAT problems has been shown recently as an instance of the belief propagation (BP) algorithm. In this paper, we show that for general constraint-satisfaction problems, SP may not be reducible from BP. We also establish the conditions under which such a reduction is possible. Along our development, we present a unification of the existing SP algorithms in terms of a probabilistically interpretable iterative procedure — weighted probabilistic token passing.

In this paper, we present a DCT domain watermarking approach for H.264/AVC video coding standard. This scheme is resilient to compressed-domain temporal adaptation. A cryptographic hash function is used to generate a semi-fragile watermark to provide content-based authentication. The embedded watermark can withstand frame-dropping due to temporal adaptation, yet it is able to detect malicious attacks such as content modification, transcoding etc. Simulation results demonstrate that the watermarking scheme is computationally efficient and suitable for practical use.

This paper introduces a haptic data reduction and transmission technique to reduce the packet rate in six degree-of-freedom (6-DoF) haptic-enabled telementoring systems. The presented method relies on the limitations of human haptic perception (i.e. the Just Noticeable Differences) with respect to a user's hand position and orientation in order to reduce the number of packets transmitted without compromising transparency. A haptic prediction model is exploited to further reduce the amount of haptic packets transmitted, and to improve the reconstruction of data samples on the receiver side. Several distance metrics are also discussed to evaluate the acuity of human haptic perception when data reduction is performed in 6-DoF settings. Psychophysical experiments validate the effectiveness of the suggested algorithm as great haptic data reduction is achieved (up to 96%), while preserving the overall quality of the telementoring environment.

In this paper, a new rotation and scaling invariant image watermarking scheme is proposed based on rotation invariant feature and image normalization. A mathematical model is established to approximate the image based on the mixture generalized Gaussian distribution, which can facilitate the analysis of the watermarking processes. Using maximum a posteriori probability based image segmentation, the cover image is segmented into several homogeneous areas. Each region can be represented by a generalized Gaussian distribution, which is critical for the analysis of the watermarking processes mathematically. The rotation invariant features are extracted from the segmented areas and are selected as reference points. Sub-regions centered at the feature points are used for watermark embedding and extraction. Image normalization is applied to the sub-regions to achieve scaling invariance. Meanwhile, the watermark embedding and extraction schemes are analyzed mathematically based on the established mathematical model. The watermark embedding strength is adjusted adaptively using the noise visibility function and the probability of error is analyzed mathematically. The mathematical relationship between fidelity and robustness is established. The experimental results show the effectiveness and accuracy of the proposed scheme.

The purpose of this paper is to study the performance of voice transportation in wireless ad hoc networks. First we analyze the characteristics of wireless channel and speech coding standard AMR-WB, and then we propose an adaptive source network rate control scheme to battle packet loss. The basic idea is to adapt the voice coding bit rate to the available network bandwidth in order to maximize the quality of voice. The PESQ MOS value is used as the evaluation criteria for the adaptive rate control scheme. Moreover, the discontinuous transmission operation and packet combination are utilized to improve the performance further. The experimental results show the effectiveness of this scheme.

While trivial research has been conducted in watermarking and authentication of H.264 video in recent years, most techniques require cascaded operations within a video adaptation scenario. In this paper, we propose an authentication scheme for adapted H.264 video content to detect integrity at the receiver's side without the need for cascaded operations. The proposed scheme utilizes MPEG-21 gBSD for hard authentication of H.264 video in the compressed domain and does not necessitate any cascaded decompression and recompression. The design uses content-based authentication which is derived from a hash value. The authentication data is embedded as a fragile watermark, and the marking space is selected during the adaptation process of the H.264 video by parsing the gBSD. The authentication information is embedded in already encoded videos during adaptation. Proof of concept and performance evaluation is also presented.

This paper introduces a novel generic method aimed at predicting motion and force information in haptic media. An autoregressive (AR) model is presented for the prediction of both, haptic movement and force. The conditional maximum likelihood technique is utilized in order to accurately estimate the adaptive coefficients of the AR model. Furthermore, the incorporation of concepts from haptic perceptibility, i.e. the just noticeable difference (JND), has been demonstrated to optimize the suggested algorithm, while preserving the immersiveness of the haptic-enabled environment. The proposed technique has also proved to provide accurate prediction results for non-linear haptic movement and force information while simultaneously remaining computationally efficient.

In this paper, a rotation invariant feature and image normalization based image watermarking algorithm is proposed. The rotation invariant features are extracted. Subregions centered at those feature points are used for watermark embedding and detection. Image normalization is applied to those subregions to achieve the scaling invariance. This image watermarking algorithm is rotation and scaling invariant and the experimental results show its effectiveness.

As a practical and novel application of watermarking, this paper presents a digital watermarking-based image quality evaluation method that can accurately estimate image quality in terms of the classical objective metrics, such as peak signal-to-noise ratio (PSNR), weighted PSNR (wPSNR), and Watson just noticeable difference (JND), without the need for the original image. In this method, a watermark is embedded into the discrete wavelet transform (DWT) domain of the original image using a quantization method. Considering that different images have different frequency distributions, the vulnerability of the watermark for the image is adjusted using automatic control. After the auto-adjustment, the degradation of the extracted watermark can be used to estimate image quality in terms of the classical metrics with high accuracy. We calculated PSNR, wPSNR, and Watson JND quality measures for JPEG compressed images and compared the values with those estimated using the watermarking-based approach. We found that the calculated and estimated measures of quality to be highly correlated, suggesting that the proposed method can provide accurate measures of image quality under JPEG compression. Furthermore, given the similarity between JPEG and MPEG-2, this achievement has paved the road for the practical and accurate quality evaluation of MPEG-2 compressed video. We believe that this achievement is of great importance to video broadcasting

A celebrated algorithmic discovery in solving constraint satisfaction problems, survey propagation (SP) and its generalization have recently demonstrated their power in areas of communications and data compression. It is known that under certain Markov random field (MRF) formalism of k-SAT problems, SP may be interpreted as an instance of belief propagation (BP). In this paper, we borrow the notion of generalized states from system theory and coding theory, and introduce a new MRF formalism - normal realization - for k-SAT problems. We show that when BP applies to this MRF, generalized SP is resulted. This new MRF formalism appears to be simpler than the existing one and the interpretation of SP as BP in this framework also appears more transparent

Based on log-polar mapping (LPM) and phase correlation, the paper presents a novel digital image watermarking scheme that is invariant to rotation, scaling, and translation (RST). We embed a watermark in the LPMs of the Fourier magnitude spectrum of an original image, and use the phase correlation between the LPM of the original image and the LPM of the watermarked image to calculate the displacement of watermark positions in the LPM domain. The scheme preserves the image quality by avoiding computing the inverse log-polar mapping (ILPM), and produces smaller correlation coefficients for unwatermarked images by using phase correlation to avoid exhaustive search. The evaluations demonstrate that the scheme is invariant to rotation and translation, invariant to scaling when the scale is in a reasonable range, and very robust to JPEG compression.

This paper presents a novel standard baseline JPEG compatible still image compression method that allows to flexibly control the decompression quality of regions in image.Therefore, it allows both higher quality of important parts and lower bitrate as a whole, and it reflects the sense of human beings.We use fuzzy technique to determine important regions in the same manner as human beings do, allocate higher bitrate to important regions while lower to the rest parts of image.The paper introduces the important region determination, description, arbitrarily shaped region filling, importance adaptive image compression, decompression, and finally gives experimental results.