In this article, we introduce a new neonatal facial expression database for pain analysis. This database, called facial expression of neonatal pain (FENP), contains 11,000 neonatal facial expression images associated with 106 Chinese neonates from two children's hospitals, i.e., the Children's Hospital Affiliated to Nanjing Medical University and Second Affiliated Hospital Affiliated to Nanjing Medical University in China. The facial expression images cover four categories of facial expressions, i.e., severe pain expression, mild pain expression, crying expression and calmness expression, where each category contains 2750 neonatal facial expression images. Based on this database, we also investigate the pain facial expression recognition problem using several state-of-the-art facial expression features and expression recognition methods, such as Gabor+SVM, LBP+SVM, HOG+SVM, LBP+HOG+SVM, and several Convolutional Neural Network (CNN) methods (including AlexNet, VGGNet, GoogLeNet, ResNet and DenseNet). The experimental results indicate that the proposed neonatal pain facial expression database is very suitable for the study of both neonatal pain and facial expression recognition. Moreover, the FENP database is publicly available after signing a license agreement (the users can contact Jingjie Yan ([email protected]), Guanming Lu ([email protected])) or Xiaonan Li ([email protected]).

In this paper, we investigate a novel learning-based method for dynamic scene deblurring. Since the inference model is formulated as an encoder-decoder, the core task has turned to learning blur-invariant hidden features from the blurred images to a great degree. To achieve state-of-the-art results in terms of both deblurring accuracy and efficiency, a simple, robust and computationally efficient deep auto-encoder is developed tailored specifically for blind deblurring, which is learned in an adversarial fashion based on use of the recent Wasserstein generative adversarial networks. Thanks to the designed framework, the new model has shown comparable or superior performance both qualitatively and quantitatively to existing state-of-the-art methods. What is more important, instead of exploiting the multi-scale strategy as previous methods, our model is just single-scale capable of achieving 6 times more efficiency than the closest competitor by Nah et al. [1], which is a more complex multi-scale deep method.

This paper aims to propose a candidate solution to the challenging task of single-image blind super-resolution (SR), via extensively exploring the potentials of learning-based SR schemes in the literature. The task is formulated into an energy functional to be minimized with respect to both an intermediate super-resolved image and a nonparametric blur-kernel. The functional includes a so-called convolutional consistency term which incorporates a nonblind learning-based SR result to better guide the kernel estimation process, and a bi-[Formula: see text]-[Formula: see text]-norm regularization imposed on both the super-resolved sharp image and the nonparametric blur-kernel. A numerical algorithm is deduced via coupling the splitting augmented Lagrangian (SAL) and the conjugate gradient (CG) method. With the estimated blur-kernel, the final SR image is reconstructed using a simple TV-based nonblind SR method. The proposed blind SR approach is demonstrated to achieve better performance than [T. Michaeli and M. Irani, Nonparametric Blind Super-resolution, in Proc. IEEE Conf. Comput. Vision (IEEE Press, Washington, 2013), pp. 945–952.] in terms of both blur-kernel estimation accuracy and image ehancement quality. In the meanwhile, the experimental results demonstrate surprisingly that the local linear regression-based SR method, anchored neighbor regression (ANR) serves the proposed functional more appropriately than those harnessing the deep convolutional neural networks.

The digital health and wellbeing movement has led to development of digital mindfulness applications that aim to help people to become mindful. In this paper we suggest a broad scheme for classifying and ordering apps intended to support mindfulness. This scheme consists of four levels of what we here term digital mindfulness. One crucial aspect of the fourth level is that artifacts at this level allow for what we term as presence-with and presence-in as opposed to presence-through, which occurs at the first three levels. We articulate our four levels along with specific design qualities through concrete examples of existing mindfulness apps and through research through design (RtD) work conducted with design fiction examples. We then use a working design case prototype to further illustrate the possibilities of presence-with and presence-in. We hope our four levels of digital mindfulness framework will be found useful by other researchers in discussing and planning the design of their own mindfulness apps and digital artifacts.

Nonlocal image representation methods, including group-based sparse coding and block-matching 3-D filtering, have shown their great performance in application to low-level tasks. The nonlocal prior is extracted from each group consisting of patches with similar intensities. Grouping patches based on intensity similarity, however, gives rise to disturbance and inaccuracy in estimation of the true images. To address this problem, we propose a structure-based low-rank model with graph nuclear norm regularization. We exploit the local manifold structure inside a patch and group the patches by the distance metric of manifold structure. With the manifold structure information, a graph nuclear norm regularization is established and incorporated into a low-rank approximation model. We then prove that the graph-based regularization is equivalent to a weighted nuclear norm and the proposed model can be solved by a weighted singular-value thresholding algorithm. Extensive experiments on additive white Gaussian noise removal and mixed noise removal demonstrate that the proposed method achieves a better performance than several state-of-the-art algorithms.

Predicting facial attributes from faces in the wild is very challenging due to pose and lighting variations in the real world. The key to this problem is to build proper feature representations to cope with these unfavourable conditions. Given the success of Convolutional Neural Network (CNN) in image classification, the high-level CNN feature, as an intuitive and reasonable choice, has been widely utilized for this problem. In this paper, however, we consider the mid-level CNN features as an alternative to the high-level ones for attribute prediction. This is based on the observation that face attributes are different: some of them are locally oriented while others are globally defined. Our investigations reveal that the mid-level deep representations outperform the prediction accuracy achieved by the (fine-tuned) high-level abstractions. We empirically demonstrate that the mid-level representations achieve state-of-the-art prediction performance on CelebA and LFWA datasets. Our investigations also show that by utilizing the mid-level representations one can employ a single deep network to achieve both face recognition and attribute prediction.

This paper addresses the problem of transferring CNNs pre-trained for face recognition to a face attribute prediction task. To transfer an off-the-shelf CNN to a novel task, a typical solution is to fine-tune the network towards the novel task. As demonstrated in the state-of-the-art face attribute prediction approach, fine-tuning the high-level CNN hidden layer by using labeled attribute data leads to significant performance improvements. In this paper, however, we tackle the same problem but through a different approach. Rather than using an end-to-end network, we select face descriptors from off-the-shelf hierarchical CNN representations for recognizing different attributes. Through such an adaptive representation selection, even without any fine-tuning, our results still outperform the state-of-the-art face attribute prediction approach on the latest large-scale dataset for an error rate reduction of more than 20%. Moreover, by using intensive empirical probes, we have identified several key factors that are significant for achieving promising face attribute prediction performance. These results attempt to gain and update our understandings of the nature of CNN features and how they can be better applied to the transferred novel tasks.

A novel bimodal emotion recognition approach from facial expression and speech based on the sparse kernel reduced-rank regression (SKRRR) fusion method is proposed in this paper. In this method, we use the openSMILE feature extractor and the scale invariant feature transform feature descriptor to respectively extract effective features from speech modality and facial expression modality, and then propose the SKRRR fusion approach to fuse the emotion features of two modalities. The proposed SKRRR method is a nonlinear extension of the traditional reduced-rank regression (RRR), where both predictor and response feature vectors in RRR are kernelized by being mapped onto two high-dimensional feature space via two nonlinear mappings, respectively. To solve the SKRRR problem, we propose a sparse representation (SR)-based approach to find the optimal solution of the coefficient matrices of SKRRR, where the introduction of the SR technique aims to fully consider the different contributions of training data samples to the derivation of optimal solution of SKRRR. Finally, we utilize the eNTERFACE '05 and AFEW 4.0 bimodal emotion database to conduct the experiments of monomodal emotion recognition and bimodal emotion recognition, and the results indicate that our presented approach acquires the highest or comparable bimodal emotion recognition rate among some state-of-the-art approaches.

A touch-less interaction technology on vision based wearable device is designed and evaluated. Users interact with the application with dynamic hands/feet gestures in front of the camera. Several proof-of-concept prototypes with eleven dynamic gestures are developed based on the touch-less interaction. At last, a comparing user study evaluation is proposed to demonstrate the usability of the touch-less approach, as well as the impact on user's emotion, running on a wearable framework or Google Glass.

This paper proposes a novel swarm intelligence optimization method which integrates bacterial foraging optimization (BFO) with quantum computing, called quantum bacterial foraging optimization (QBFO) algorithm. In QBFO, a multi-qubit which can represent a linear superposition of states in search space probabilistically is used to represent a bacterium, so that the quantum bacteria representation has a better characteristic of population diversity. A quantum rotation gate is designed to simulate the chemotactic step to drive the bacteria toward better solutions. Several tests are conducted based on benchmark functions including multi-peak function to evaluate optimization performance of the proposed algorithm. The numeric results show that the proposed QBFO has more powerful properties in convergence rate, stability and the ability of searching for the global optimal solution than the original BFO and quantum genetic algorithm. In addition, we applied our proposed QBFO to solve the traveling salesman problem, which is a well-known NP-hard problem in combinatorial optimization. The results indicate that the proposed QBFO shows better convergence behavior without premature convergence, and has more powerful properties in convergence rate, stability and the ability of searching for the global optimal solution, as compared to ant colony optimization algorithm and quantum genetic algorithm.

Effective and efficient generation of keypoints from images is the first step of many computer vision applications, such as object matching. The last decade presented us with an arms race toward faster and more robust keypoint detection, feature description and matching. This resulted in several new algorithms, for example Scale Invariant Features Transform (SIFT), Speed-up Robust Feature (SURF), Oriented FAST and Rotated BRIEF (ORB) and Binary Robust Invariant Scalable Keypoints (BRISK). The keypoint detection has been improved using various techniques in most of these algorithms. However, in the search for faster computing, the accuracy of the algorithms is decreasing. In this paper, we present SICK (Scale-Invariant Corner Keypoints), which is a novel method for fast keypoint detection. Our experiment results show that SICK is faster to compute and more robust than recent state-of-the-art methods.

In this paper, we introduce the Block Matching (BM) as an alternative patch-based local matching approach for solving the face recognition problem. The Block Matching enables an image patch of the probe face image to search for its best matching from displaced positions in the gallery face image. This matching strategy is very effective for handling spatial shift between two images and it is radically different from that of the widely used LBP type patch-based local matching approaches. Our evaluations on the FERET and CMU-PIE databases show that the performance of this simple method is well comparable (superior) to that of the popular LBP approach. We argue that the Block Matching could provide face recognition a new approach with more flexible algorithm architecture. One can expect that it could lead to much higher performance when combining with other feature extraction techniques, like Gabor wavelet and deep learning.

In this work, we are presenting a technique that allows for accurate estimation of frequencies in higher dimensions than the original image content. This technique uses asymmetrical Principal Component Analysis together with Discrete Wavelet Transform (aPCA-DWT). For example, high quality content can be generated from low quality cameras since the necessary frequencies can be estimated through reliable methods. Within our research, we build models for interpreting facial images where super-resolution versions of human faces can be created. We have worked on several different experiments, extracting the frequency content in order to create models with aPCA-DWT. The results are presented along with experiments of deblurring and zooming beyond the original image resolution. For example, when an image is enlarged 16 times in decoding, the proposed technique outperforms interpolation with more than 7 dB on average.

This paper presents a novel approach for performing intuitive 3D gesture-based interaction using depth data acquired by Kinect. Unlike current depth-based systems that focus only on classical gesture recognition problem, we also consider 3D gesture pose estimation for creating immersive gestural interaction. In this paper, we formulate gesture-based interaction system as a combination of two separate problems, gesture recognition and gesture pose estimation. We focus on the second problem and propose a direct method for recovering hand motion parameters. Based on the range images, a new version of optical flow constraint equation is derived, which can be utilized to directly estimate 3D hand motion without any need of imposing other constraints. Our experiments illustrate that the proposed approach performs properly in real-time with high accuracy. As a proof of concept, we demonstrate the system performance in 3D object manipulation. This application is intended to explore the system capabilities in real-time biomedical applications. Eventually, system usability test is conducted to evaluate the learn ability, user experience and interaction quality in 3D interaction in comparison to 2D touch-screen interaction.

We present a hand-and-foot-based multimodal interaction approach for handheld devices. Our method combines input modalities (i.e., hand and foot) and provides a coordinated output to both modalities along with audio and video. Human foot gesture is detected and tracked using contour-based template detection (CTD) and Tracking-Learning-Detection (TLD) algorithm. 3D foot pose is estimated from passive homography matrix of the camera. 3D stereoscopic and vibrotactile are used to enhance the immersive feeling. We developed a multimodal football game based on the multimodal approach as a proof-of-concept. We confirm our systems user satisfaction through a user study.

We propose a design of teleconference system which express nonverbal behavior (in our case head gesture) along with audio-video communication. Previous audio-video conferencing systems are abortive in presenting nonverbal behaviors which we, as human, usually use in face to face interaction. Recently, research in teleconferencing systems has expanded to include nonverbal cues of remote person in their distance communication. The accurate representation of non-verbal gestures for such systems is still challenging because they are dependent on hand-operated devices (like mouse or keyboard). Furthermore, they still lack in presenting accurate human gestures. We believe that incorporating embodied interaction in video teleconferencing, (i.e., using the physical world as a medium for interacting with digital technology) can result in nonverbal behavior representation. The experimental platform named Tele-Embodied Agent (TEA) is introduced which incorperates remote person's head gestures to study new paradigm of embodied interaction in video teleconferencing. Our preliminary test shows accuracy (with respect to pose angles) and efficiency (with respect to time) of our proposed design. TEA can be used in medical field, factories, offices, gaming industry, music industry and for training.

In this paper we present a vision based intuitive interaction method for smart mobile devices. It is based on markerless finger gesture detection which attempts to provide a 'natural user interface'. There is no additional hardware necessary for real-time finger gesture estimation. To evaluate the strengths and effectiveness of proposed method, we design two smart phone applications, namely circle menu application - provides user with graphics and smart phone's status information, and bouncing ball game- a finger gesture based bouncing ball application. The users interact with these applications using finger gestures through the smart phone's camera view, which trigger the interaction event and generate activity sequences for interactive buffers. Our preliminary user study evaluation demonstrates effectiveness and the social acceptability of proposed interaction approach.

Number of mobile devices such as Smartphones or Tablet PCs has been dramatically increased over the recent years. New mobile devices are equipped with integrated cameras and large displays that make the interaction with the device easier and more efficient. Although most of the previous works on interaction between humans and mobile devices are based on 2D touch-screen displays, camera-based interaction opens a new way to manipulate in 3D space behind the device in the camera's field of view. In this paper, our gestural interaction relies on particular patterns from local orientation of the image called rotational symmetries. This approach is based on finding the most suitable pattern from a large set of rotational symmetries of different orders that ensures a reliable detector for fingertips and user's gesture. Consequently, gesture detection and tracking can be used as an efficient tool for 3D manipulation in various virtual/augmented reality applications.

Neural networks can easily fall into a local extremum and have slow convergence rate. Quantum Genetic Algorithm (QGA) has features of small population size and fast convergence. Based on the investigation of QGA, we propose a novel neural network model, Radial Basis Function (RBF) networks optimized by Quantum Genetic Algorithm (QGA-RBF model). Then we investigate the performance of the proposed QGA-RBF on solving MIMO-OFDM signal detection problem. A novel signal detector based on QGA-RBF for MIMO-OFDM system is also proposed. The simulation results show that the proposed detector has more powerful properties in bit error rate than QGA based detector, RBF based detector and MMSE algorithm based detector, namely a 4-6 dB gain in performance can be achieved. The performance of the proposed detector is closer to optimal, compared with the other detectors.

Number of mobile devices such as mobile phones or PDAs has been dramatically increased over the recent years. New mobile devices are equipped with integrated cameras and large displays which make the interaction with device easier and more efficient. Although most of the previous works on interaction between humans and mobile devices are based on 2D touch-screen displays, camera-based interaction opens a new way to manipulate in 3D space behind the device in the camera's field of view. This paper suggests the use of particular patterns from local orientation of the image called Rotational Symmetries to detect and localize human gesture. Relative rotation and translation of human gesture between consecutive frames are estimated by means of extracting stable features. Consequently, this information can be used to facilitate the 3D manipulation of virtual objects in various applications in mobile devices.

Huge efforts have been devoted to face recognition technology and remarkable results, noticed. Such advances will provide us the possibility to build a new generation of search engine: persons photo fetching. It is a real computing challenge to find a person from a very large or extremely large database which might hold face images of millions or hundred millions of people. A candidate solution is to use partial information (signature) about all the face images for search, making the retrieval speed approximately proportional to the size of a signature image. In this paper we will investigate a totally new way to compress the signature images based on the observation that the face signature images and the query images are highly correlated if they are from the same individual. The face signature image can be greatly compressed (one or two orders of magnitude improvement) by use of knowledge of the query images. We can expect the new compression algorithm to speed up face search 10 to 100 times. The challenge is that query images are not available when we compress their signature image. Our approach is to transfer the face search problem into the so-called “Wyner-Ziv Coding” problem, which could give the same compression efficiency even if the query images are not available until we decompress signature images. A practical compression scheme based on LDPC codes is developed to compress and retrieve face signature images.

This paper proposes an intuitive wireless sensor/actuator based communication network for human animal interaction for a digital zoo. In order to enhance effective observation and control over wild life, we have built a wireless sensor network. 25 video transmitting nodes are installed for animal behavior observation and experimental vibrotactile collars have been designed for effective control in an animal park.

In this paper we propose three tire based protocol for augmented reality application on mobile phones for animal situation awareness and tracking system in a wireless sensor network. We have developed and deployed an animal tracking system in a local zoo that can provide visitors information about individual animals. Here we present our ongoing work on an augmented information system that can provide online real-time animal information such as the current location, and feature of specific breed.

Vibration offers many potential benefits for the use of mobile phones. In this paper, we propose a new method of rendering live football game on mobile phones using vibration. A mobile phone is “synchronized” with the ball in the real field. By holding the phone, users are able to experience dynamic movements of the ball, to know attacking directions and which team is leading the attack. The usability test of our system shows that vibrotactile display is suitable for rendering live football information on mobile phones by adopting designed coding schemes with a right training process.

To enhance their social interactive ability, we study how to provide the sight-impaired with reliable, "on-line" emotion information. The technical challenge we address here is how to render rich facial expressions in an intuitive way. We demonstrate that manifold of facial expressions is a compact and natural way to characterize human emotions. To compute manifold of facial expressions the standard locally linear embedding (LLE) algorithm is extended to handle the problem of real-time coding of new videos. Experimental results show that using manifold of facial expressions for vibrotactile rendering is very encouraging.

In this paper we investigate important issue for real-time video over wireless ad-hoc networks on different layers. Many error control methods for this approach use multiple streams and multipath routing. Thus the new proactive, link-state routing protocol have been developed, where the protocol finds the available route in the network and also it will not cause any interruption in the video traffic between the source and the destination. The open source MPEG-4 is also implemented to get the efficient video quality for the picture.

In this paper we address the problem of transferring interactive video over wireless channels. We propose a distributed video coding solution based on Wyner-Ziv coding. To adapt to the highly time-varying wireless channels, the transmission of interactive video based on Wyner-Ziv coding is applied in a proactive way: only the current frame coded by Wyner-Ziv techniques is sent when the channel condition is poor; "future frames" are sent when there is redundant capacity in the channel. A multi-view camera array is adopted. A frontal looking camera is regarded as the master camera and others are auxiliary ones. Wyner-Ziv coding is implemented to compress video from the master camera while motion-JPEG types of codecs are used to compress auxiliary views. Depending on head motions, at least one of the side views acts as "future frame" for the current frame from the master camera. The "future frames" are then used to generate more faithful side-information for Wyner-Ziv frames. We built a three-camera array to test our idea. An average 2 dB gain is achieved in the experiments.

In Wyner-Ziv video coding, efficient compression is achieved by exploiting source statistics at the decoder only, which is radically different from conventional video coding. The performance of a Wyner-Ziv video codec is greatly dependent on the quality of reconstructed side information. In this paper we give an explicit motion compensation scheme. Unlike existing schemes, motion compensation is carried out between side information and the quantized version of the transmitted frame. This needs neither the encoder to transmit additional, implicit motion information to the receiver nor two I frames for interpolating the middle frames. This enables a sequential decoding architecture, which is a necessity for real-time video coding. Our experimental results are promising. 2dB gain in the quality of reconstructed frames has been achieved

This paper is addressing a challenging face recognition problem: face identification from one single face image. We present a novel approach to face identification, which is capable to identify a person from face images that are significantly different from the sample image in terms of illumination, camera view angles and expressions. The approach is based on a new measurement of dissimilarity between the two face images. A person is identified based on the smallest dissimilarity, which is the summation of the dissimilarities of all pairs of observations extracted from the face image on both vertical and horizontal directions. Our experiment results tested on both the AR face database and the CMU PIE face database shows that the proposed method outperforms the PCA, LDA, LFA based approaches.

This work presents a novel approach for recognizing faces in images taken from different illumination, expression, near frontal pose, partially occlusion and time delay. The method is based on one-dimensional discrete hidden Markov model (ID-DHMM) with new way of extracting observations and using observation sequences. All subjects in the system share only one HMM that is used as a means to weigh a pair of observations. The Haar wavelet transform is applied to face images to reduce the dimension of the observation vectors. The selection of the recognized person is based on the highest score, which is the summation of the likelihoods of all observation sequences extracted from image on both vertical and horizontal dimensions. Our experiment results tested on the AR face database and the CMU PIE face database show that the proposed method outperforms the PCA, LDA, LFA based approaches tested on the same databases.

Interactive video services such as video conferencing, distance learning, and online video games over the Internet and wireless networks are becoming increasingly prevalent. Because of limited bandwidth on networks and the bandwidth-hungry nature of video, interactive video requires extremely efficient resource management to gain real-time performance. Unlike pregenerated video whose traffic profiles can be computed in advance, efficiency and accuracy of dynamic resource allocation methods for interactive video depend critically on the performance of traffic prediction. Using either traffic data or image features, existing traffic prediction schemes can only provide a short-term traffic prediction. Based on a three-dimensional object's motion, this paper presents a new bandwidth prediction approach for video conferencing. We show that there is a strong correlation between video conferencing traffic and real motion of objects. The real motion can be predicted by the powerful technique Kalman filtering, and the estimated motion is used to make a long-term bandwidth prediction. The new traffic prediction model is tested and compared with the frame-based adaptive normalized least mean square error linear predictor and optical flow-based method with Kalman filtering. Experimental results show that our proposed traffic prediction model achieves much higher accuracy in long-term traffic prediction, which provides the possibility for networks to allocate resources efficiently for video conferencing services.

Recursive image subsampling which yields support areas approaching fractals is described and analyzed using iterated function systems. The subsampling scheme is suitable in, e.g., hierarchical image processing and image coding schemes. For hexagonally sampled images a hierarchical subsampling structure is given which yields hexagon-like regions with fractal borders.

In model-based, or semantic, coding, parameters describing the nonrigid motion of objects, e.g., the mimics of a face, are of crucial interest. The facial animation parameters (FAPs) specified in MPEG-4 compose a very rich set of such parameters, allowing a wide range of facial motion. However, the FAPs are typically correlated and also constrained in their motion due to the physiology of the human face. We seek here to utilize this spatial correlation to achieve efficient compression. As it does not introduce any interframe delay, the method is suitable for interactive applications, e.g., videophone and interactive video, where low delay is a vital issue.

Block-based motion compensation has been widely used for the removal of temporal redundancy existing in image sequences. However, the prediction image generated by this technique suffers from a perceptually annoying blocking-effect. This becomes especially serious in video transmission at very low bit rates. To overcome this problem, a transformed block-based motion compensation technique is proposed in this paper. Preliminary results show that both subjective and objective improvements can be obtained when a transformed block-based motion compensation scheme is employed instead of a direct block-based scheme.>

This paper addresses the issue of two-view facial motion estimation for model-based facial image coding. A new approach to estimate the motion of the head and the facial expressions is presented, which can be viewed as a two-step procedure as compared with our previous approach. In this new approach M-estimation technique is used to eliminate the effect of the facial expressions on the estimation of the head movement. In this way the global head motion can be more reliably recovered. Once the global motion is obtained, the facial expressions can be estimated. Some experimental results on synthesized and real image sequences demonstrate the effectiveness of the new algorithm.>

This paper presents a review of promising techniques for very low bit-rate, below 64 kb/s, image sequence coding. Image sequence coding at such low rates will be a crucial technique in forthcoming visual services, e.g., visual information transmission and storage. A typical application is to transmit moving videophone scenes through the existing analog telephone lines or via a mobile channel. Two types of potential coding techniques are addressed: waveform-based image sequence coding and model-based image sequence coding.

An approach to estimating the motion of the head and facial expressions in model-based facial image coding is presented. An affine nonrigid motion model is set up. The specific knowledge about facial shape and facial expression is formulated in this model in the form of parameters. A direct method of estimating the two-view motion parameters that is based on the affine method is discussed. Based on the reasonable assumption that the 3-D motion of the face is almost smooth in the time domain, several approaches to predicting the motion of the next frame are proposed. Using a 3-D model, the approach is characterized by a feedback loop connecting computer vision and computer graphics. Embedding the synthesis techniques into the analysis phase greatly improves the performance of motion estimation. Simulations with long image sequences of real-world scenes indicate that the method not only greatly reduces computational complexity but also substantially improves estimation accuracy.>

The issue of analysis/synthesis of human facial movements in model-based facial image coding is addressed. An affine model which is especially suitable to such a type of nonrigid motion as facial expressions is set up. Based on the affine model, a direct two-view motion estimation approach is presented. This approach is much faster than other existing approaches. To remove the error-accumulation problem associated with long sequence motion estimation, a new scheme which is equipped with a feedback loop connecting computer vision and computer graphics is used. Experimental results show that the method not only greatly reduces the computational complexity but also substantially improves the estimation accuracy. The synthesized image sequence using the estimated motion parameters, a 3-D model of the face, and a frame of textured image looks very natural.>

Presents a least-squares (LS) approach to globally interpreting the optical flow field. Using the derivatives of the optical flow field, one can obtain a linear and global constraint equation for the translation motion. Thus the translation motion can be robustly and simply recovered using a LS method. Once the translation motion is recovered, the rotation can be immediately recovered only using the optical flow field. Therefore, the recovered rotation motion is more reliable.>