Different from the traditional video quality enhancement, the goal of compressed video quality enhancement is to reduce the artifacts brought by the video compression. The existing multi-frame methods for compressed video quality enhancement heavily rely on optical flow, which is both inefficient and limited in performance. In this paper, a Multi-frame Residual Dense Network (MRDN) with deformable convolution is developed to improve the quality of the compressed video, by utilizing high-quality frame to compensate the low-quality frame. Specifically, the proposed network consists of the developed Motion Compensation (MC) module and Quality Enhancement (QE) module, aiming to compensate and enhance the quality of the input frame, respectively. Besides, a novel edge enhancement loss is conducted on the enhanced frame, in order to enhance edge structure during the training. Finally, the experimental results over a public benchmark show that our method outperforms the state-of-the-art methods for compressed video quality enhancement task.

In this paper, we propose an exact and efficient O(1) edge-aware filter. Starting from the generic 1D weighted average filter, by enforcing two assumptions on the filter kernel, we derive the fast O(1) algorithm to exactly implement the filter by two pass recursions. Performing the 1D filter horizontally and vertically can easily extend it to 2D case. A typical real valued geodesic kernel and a novel complex valued bilateral-like kernel are investigated in our proposed filtering framework. Experiments show that both kernels work well for the edge-aware smoothing task, and the bilateral-like kernel based filter shows the similar ability as the guided filter in the image feathering application.

In this paper, we propose a precise and fast eye center localization method. The method is based on SDM [1] algorithm, which has achieved impressive performance in face alignment area. In this work, we further improve the basic SDM algorithm in two aspects: 1) employ the coarse to fine strategy to enhance the robustness to initialization; 2) utilize multiple nonlinear features to ensure the robustness while improve the accuracy. Experimental results on BioID database shows that our improved SDM achieves preferable localization accuracy against the state-of-the-art methods at a speed as fast as 2ms to locate two eyes.

Integral imaging display (IID) is a promising technology to provide realistic 3D image without glasses. To achieve a large screen IID with a reasonable fabrication cost, a potential solution is a tiled-lens-array IID (TLA-IID). However, TLA-IIDs are subject to 3D image artifacts when there are even slight misalignments between the lens arrays. This work aims at compensating these artifacts by calibrating the lens array poses with a camera and including them in a ray model used for rendering the 3D image. Since the lens arrays are transparent, this task is challenging for traditional calibration methods. In this paper, we propose a novel calibration method based on defining a set of principle observation rays that pass lens centers of the TLA and the camera's optical center. The method is able to determine the lens array poses with only one camera at an arbitrary unknown position without using any additional markers. The principle observation rays are automatically extracted using a structured light based method from a dense correspondence map between the displayed and captured pixels. Experiments show that lens array misalignments can be estimated with a standard deviation smaller than 0.4 pixels. Based on this, 3D image artifacts are shown to be effectively removed in a test TLA-IID with challenging misalignments.

Facial expression tracking is a challenging task because the head pose may change in a large range and the expression is highly non-rigid. It can be formulized as an energy minimization problem. The two most important issues are the construction of the cost function and the selection of the initial value. In this paper, we present a fast and robust expression tracking algorithm called Composite Constraints AAM. Firstly, a novel cost function is proposed to enhance the convergence by combining multiple constraints in a unified framework. Secondly, widely used local features are strictly tested with face videos, and an efficient motion estimation method is presented to provide a good initial value to the iterative optimization process. Experimental result demonstrates that our system can track the head pose and facial expression with very high stability in real time speed.

Active Appearance Model (AAM) based face tracking has advantages of accurate alignment, high efficiency, and effectiveness for handling face deformation. However, AAM suffers from the generalization problem and has difficulties in images with cluttered backgrounds. In this paper, we introduce two novel constraints into AAM fitting to address the above problems. We first introduce a temporal matching constraint in AAM fitting. In the proposed fitting scheme, the temporal matching enforces an inter-frame local appearance constraint between frames. The resulting model takes advantage of temporal matching's good generalizability, but does not suffer from the mismatched points. To make AAM more stable for cluttered backgrounds, we introduce a color-based face segmentation as a soft constraint. Both constraints effectively improve the AAM tracker's performance, as demonstrated with experiments on various challenging real-world videos.

In this work, we address the problem of tracking three dimensional (3D) faces and facial actions in video sequences. The main contributions of the paper are as follows. First, we develop an extended 2D+3D Active Appearance Models (AAM) based 3D face and facial action tracking framework using 2D view-based AAMs and a modified 3D face model. Second, we develop a robust shape initialization method based on local feature matching to track fast face motion. Experiments evaluating the effectiveness of the proposed algorithm are reported.

Facial feature tracking is a crucial and challenging task in computer vision. Recently online-learning methods have become increasingly popular on account of their strong ability to adapt to variations and have achieved good results in tracking. However, all previous work used only raw intensity to build the model, which is very sensitive to condition changes. In this work, we present a real time, fully automatic facial feature detection and tracking approach using adaptive observation models based on edge structure, which is more reliable especially when the lighting state alters during tracking. Experimental results demonstrate that using edge map measures in observation modeling can improve the accuracy and robustness of tracking.