In recent years, deep convolutional neural networks (CNNs) have become dominant in MRI reconstruction from undersampled k-space. However, most existing CNNs methods reconstruct the undersampled images either in the spatial domain or in the frequency domain, and neglecting the correlation between these two domains. This hinders the further reconstruction performance improvement. To tackle this issue, in this work, we propose a new multi-domain recurrent network (MDR-Net) with multi-domain learning (MDL) blocks as its basic units to reconstruct the undersampled MR image progressively. Specifically, the MDL block interactively processes the local spatial features and the global frequency information to facilitate complementary learning, leading to fine-grained features generation. Furthermore, we introduce an effective frequency-based loss to narrow the frequency spectrum gap, compensating for over-smoothness caused by the widely used spatial reconstruction loss. Extensive experiments on public fastMRI datasets demonstrate that our MDR-Net consistently outperforms other competitive methods and is able to provide more details.

In vision and learning, low computational complexity and high generalization are two important goals for video object detection. Low computational complexity here means not only fast speed but also less energy consumption. The sliding window object detection method with linear support vector machines (SVMs) is a general object detection framework. The computational cost is herein mainly paid in complex feature extraction and innerproduct-based classification. This paper first develops a distributed object detection framework (DOD) by making the best use of spatial-temporal correlation, where the process of feature extraction and classification is distributed in the current frame and several previous frames. In each framework, only subfeature vectors are extracted and the response of partial linear classifier (i.e., subdecision value) is computed. To reduce the dimension of traditional block-based histograms of oriented gradients (BHOG) feature vector, this paper proposes a cell-based HOG (CHOG) algorithm, where the features in one cell are not shared with overlapping blocks. Using CHOG as feature descriptor, we develop CHOG-DOD as an instance of DOD framework. Experimental results on detection of hand, face, and pedestrian in video show the superiority of the proposed method.

Smart phones and external large displays are a superior combination for content sharing and multiple person interaction. However, the potential of such a combination is not fully explored due to the lack of effective and portable interaction solutions. To solve this problem, we present a novel mobile solution called Plug&Touch, which is very easy to be set up and enables mobile users to directly and naturally operate on the large display surface. Plug&Touch utilizes vision-based methods to detect the user's finger tip and then control user interface. The Local Binary Pattern asymmetric texture difference is proposed to extract the foreground region, and the coarse-to-fine intersection process is adopted to precisely localize the finger tip. Plug&Touch has been implemented on the Symbian3 platform with two defined gestures - click and scroll. Experimental results and user evaluation has shown the practicability of Plug&Touch.

Many human-centered image and video management systems depend on robust human detection. To extract robust features for human detection, this paper investigates the following shortcomings of co-occurrence histograms of oriented gradients (CoHOGs) which significantly limit its advantages: 1) The magnitudes of the gradients are discarded, and only the orientations are used; 2) the gradients are not smoothed, and thus, aliasing effect exists; and 3) the dimensionality of the CoHOG feature vector is very large (e.g., 200,000). To deal with these problems, in this paper, we propose a framework that performs the following: 1) utilizes a novel gradient decomposition and combination strategy to make full use of the information of gradients; (2) adopts a two-stage gradient smoothing scheme to perform efficient gradient interpolation; and (3) employs incremental principal component analysis to reduce the large dimensionality of the CoHOG features. Experimental results on the two different human databases demonstrate the effectiveness of the proposed method.

The development of mobile platform has raised an emergent requirement for face-related multimedia applications. However, as the basis of such applications, face detection and tracking still suffers from large facial pose variation and limited computation resource. In this paper, a multi-view and real-time face tracking system is presented for mobile platform. First, three basic detectors are trained by local binary pattern (LBP) and Boosting algorithm. These detectors are then flexibly expanded for multi-view face detection through the rotation facility of LBP. To further accelerate the face tracking process, a robust facial pose estimation algorithm and the face matrix partition scheme is proposed. The experimental results on handset show the superior performance of the proposed method.

Over the past few years, research on scene text extraction has developed rapidly. Recently, condition random field (CRF) has been used to give connected components (CCs) 'text' or 'non-text' labels. However, a burning issue in CRF model comes from multiple text lines extraction. In this paper, we propose a two-step iterative CRF algorithm with a Belief Propagation inference and an OCR filtering stage. Two kinds of neighborhood relationship graph are used in the respective iterations for extracting multiple text lines. Furthermore, OCR confidence is used as an indicator for identifying the text regions, while a traditional OCR filter module only considered the recognition results. The first CRF iteration aims at finding certain text CCs, especially in multiple text lines, and sending uncertain CCs to the second iteration. The second iteration gives second chance for the uncertain CCs and filter false alarm CCs with the help of OCR. Experiments based on the public dataset of ICDAR 2005 prove that the proposed method is comparative with the existing algorithms.

In this paper, we propose an overlapped handwriting input method on handheld devices, which allows users to write continuously without breaks on a single size-restricted writing area. 2 issues have been considered during the implementation of the overlapped input method: previous characters on the background may obstruct the clear viewing of current character and the messy overlapped handwriting is difficult to be segmented and recognized. In our method, a quick segmentation method based on an artificial neural network is used to tackle the first problem and a novel system is implemented to recognize the messy handwriting based on the output of an isolated character recognition engine and a language model. The recognition rate for Chinese characters is about 92.5% for a testing database containing GB2312 Chinese characters and other frequently used symbols. The positive feedbacks from testers have also confirmed the validity of the proposed method.

This paper proposes a new feature descriptor named as Local Primitive Code (LPC), which exhibits impressively discriminative capability on various face datasets. Essentially, LPC descriptors are somewhat like filter banks of Gaussian derivatives to capture multi-scale and multi-orientation image local textures but meanwhile enables faster feature extraction. It employs a framework composed of two stages: Firstly facial images are preprocessed by the pool of differential and quotient filters to generate numerous filtered images with various textures, and then directional binary encoding (DBE) operates on filtered images for primitive code mining. With such stages, feature maps with complementary discriminative information will be generated, and the fusion of them can greatly improve face recognition performance. Experimental results verify its performance even on some challenging databases. The algorithm is transplanted into mobile platform and achieves real-time performance on Nokia N82.

We tackle the challenge of web image classification using additional tags information. Unlike traditional methods that only use the combination of several low-level features, we try to use semantic concepts to represent images and corresponding tags. At first, we extract the latent topic information by probabilistic latent semantic analysis (pLSA) algorithm, and then use multi-label multiple kernel learning to combine visual and textual features to make a better image classification. In our experiments on PASCAL VOC'07 set and MIR Flickr set, we demonstrate the benefit of using multimodal feature to improve image classification. Specifically, we discover that on the issue of image classification, utilizing latent semantic feature to represent images and associated tags can obtain better classification results than other ways that integrating several low-level features.

This paper introduces a novel gesture-based human-machine interface prototype, which consists of a wearable belt embedding with four surface electromyography (SEMG) sensors, a tri-axis accelerometer and an application program running on NOKIA 5800XM. The sensor belt captures hand gestures by acquiring SEMG and acceleration (ACC) signals from forearm, and sends the m out via Bluetooth. The application program receives the data and translates them into control commands of a given interaction application. Experimental results of two test schemes conducted on hand gesture recognition and media player operation demonstrate the validity of the proposed gesture-based interface prototype.

Different from traditional age estimation methods under classification or regression frameworks, this paper proposes a novel person-specific age estimation method under ranking framework. The basic idea is to consider the aging process as a personal age-ranked image sequences and extract the relevant information from this sequences. The estimation of age for an unknown face image is determined by first utilizing face recognition to find the persons in template sets who looks similar to the unseen person, then estimating the ranking order of the unseen person in corresponding person-specific image sequences, lastly mapping and fusing the rank order to its real age. Under this framework, our proposed system not only can estimate the correct the age orders of pairs of faces naturally but also can estimate the real age accurately. The proposed method has shown encouraging performance in the comparative experiments either as an age ranker or as an accurate age estimator and the experiment also proved the validity of the above assumption.

To facilitate review and management of home videos captured by mobile phones, we propose a novel instant summarization method which is applied while users are shooting. Segment boundaries and key frames are extracted without any delay, which means that the extracted frames strictly synchronize with the scene being captured. Partial-context is the major challenge of this method since only captured frames are available when summarization is applied. And limited calculation resource of mobile phones is another restricted condition in such a video analysis, especially when video compression is executed meanwhile. Several frame features are utilized for segmentation and key frame extraction; and an original key frame updating strategy is presented to optimize selected representative frames in such partial-context. Experimental results demonstrate that the proposed method is satisfactory in the aspects of low computation complexity, high effectiveness and good user experience.

This paper presents a framework for hand gesture recognition based on the information fusion of a three-axis accelerometer (ACC) and multichannel electromyography (EMG) sensors. In our framework, the start and end points of meaningful gesture segments are detected automatically by the intensity of the EMG signals. A decision tree and multistream hidden Markov models are utilized as decision-level fusion to get the final results. For sign language recognition (SLR), experimental results on the classification of 72 Chinese Sign Language (CSL) words demonstrate the complementary functionality of the ACC and EMG sensors and the effectiveness of our framework. Additionally, the recognition of 40 CSL sentences is implemented to evaluate our framework for continuous SLR. For gesture-based control, a real-time interactive system is built as a virtual Rubik's cube game using 18 kinds of hand gestures as control commands. While ten subjects play the game, the performance is also examined in user-specific and user-independent classification. Our proposed framework facilitates intelligent and natural control in gesture-based interaction.

In this paper, an active shape model (ASM) based facial feature localization strategy is proposed, which employs a local binary pattern (LBP) probability model. Due to the computation simplicity and illumination insensitivity of LBP texture descriptor and the learning ability of the probability model, the algorithm is robust and fast. In addition, component-based ASM is used to impose reasonable constraints on the shape. Multi-state shape and texture models with state classifier are trained to handle highly flexible components, i.e. eyes and mouth. Our database consisting of tens of persons with various expressions and illuminations is used to train and verify the proposed algorithm. The experiments demonstrate its accuracy, efficiency and robustness.

In this paper we present Stroke++, a novel hybrid Chinese input method for touch screen mobile phones that leverages hieroglyphic properties of Chinese characters to enable faster and easier input of Chinese characters on mobile phones. By using a special keypad layout, a friendly user interface and an adaptive radical selection algorithm, we achieved a competitive inputting performance compared with currently prevalent mobile Chinese input methods, while keeping a low entry barrier for Chineseinput novices. An extensive evaluation results show that Stroke++ out-performs the state-of-the-art keystroke-based or handwriting recognition-based Chinese character inputting methods, as far as the input speed and convenience are concerned.

Finger stroke detection is an important topic in hand based Human Computer Interaction (HCI) system. Few research studies have carried out effective solutions to this problem. In this paper, we present a novel approach for stroke detection based on mono vision. Via analyzing the optical flow field within the finger area, our method is able to detect finger stroke under various camera position and visual angles. We present a thorough evaluation for each component of the algorithm, and show its efficiency and effectiveness on solving difficult stroke detection problems.

Seamless image processing concerns stitching parts of images in a visually natural manner. In this paper, we present a seamless processing method which unifies Featuring, Optimal seam method and some recent gradient domain methods including Poisson image editing, Drag-and-drop pasting and GIST. To evaluate the processing quality visually, a variational energy function is proposed to compute both the similarity of the stitched image to each of the input images and the visibility of the seam between the stitched images. The minimum of the energy function gives a globally consistent composition in both geometrical and photometric structures. Based on 3 propositions, we study the energy function and compare it with other methods theoretically. The experimental results show the benefits of our stitching method.

This paper presents a robust hand gesture analysis method using 3D depth data. Our scheme focuses on accurate hand segmentation by eliminating the negative effect of the forearm part. In the general human computer interaction (HCI) tasks, such an assumption usually holds that the depth of hand is smaller than forearm. Therefore, the precise hand region can be obtained through the fusion of the hand geometric features and the 3D depth information in real-time. Moreover, a robust hand gesture recognition method, which combines the global structure information and the local texture variation, is included in our gesture analysis framework. The elaborate hand segmentation makes the succedent recognition problem much easier and gets more accurate recognition results. Experimental results convincingly show the effectiveness of the proposed gesture analysis strategy. Furthermore, a concrete application scenario, gesture controlled picture gallery browsing, is implemented successfully.

This paper describes a novel hand gesture recognition system that utilizes both multi-channel surface electromyogram (EMG) sensors and 3D accelerometer (ACC) to realize user-friendly interaction between human and computers. Signal segments of meaningful gestures are determined from the continuous EMG signal inputs. Multi-stream Hidden Markov Models consisting of EMG and ACC streams are utilized as decision fusion method to recognize hand gestures. This paper also presents a virtual Rubik's Cube game that is controlled by the hand gestures and is used for evaluating the performance of our hand gesture recognition system. For a set of 18 kinds of gestures, each trained with 10 repetitions, the average recognition accuracy was about 91.7% in real application. The proposed method facilitates intelligent and natural control based on gesture interaction.

This paper proposes Coupa, a novel pen interaction design to support operations of users on portable devices. The design arranges a plurality of labels on the interface, each of which has an identity. The user forms a coupling by linking two graphical items together, and thus performs an action dependent on the identities of the coupled items. During the course of operation, any item on the screen is ready for linking and coupling. To reduce mal-operations, two principles for linking are proposed, with their effectiveness proved in the usability tests. Compared with traditional systems with hierarchical menu structure and point-and-click interaction, the proposed design prominently improves the efficiency and accuracy of pen-based systems with enhanced usability.

Iris localization is a key component of practical iris recognition system. Previous algorithms show good localization performances for iris images captured in the ideal conditions. However, in practice, the quality of iris image is greatly influenced by luminance, eyelashes, hair or glasses frame, which will cause mislocalization. In order to improve the robustness of iris localization, this paper proposes a new localization algorithm based on the radial symmetry transform, in which the radial symmetry characteristic of the pupil is utilized to realize iris localization. Experimental results show that the proposed algorithm can efficiently avoid the interference of luminance and other bad conditions, and realize robust precise localization in a real-time system.

As an emerging human-computer interaction approach vision based hand interaction is more natural and efficient. However in order to achieve high accuracy, most of the existing hand posture recognition methods need a large number of labeled samples which is expensive or unavailable in practice. In this paper, a co-training based method is proposed to recognize different hand postures with a small quantity of labeled data. Hand postures examples are represented with different features and disparate classifiers are trained simultaneously with labeled data. Then the semi-supervised learning treats each new posture as unlabeled data and updates the classifiers in a co-training framework. Experiments show that the proposed method outperforms the traditional methods with much less labeled examples.

In this paper we present a novel framework of automatic text discovering for content-based multimedia application. For single image, the stroke-based binarization and the coarse-to-fine text extraction will collaborate to generate a clean text image for recognition. For image sequence, multi-frame text enhancement is adopted to increase the text/background contrast, and the recognition results are finally refined by the text string combination algorithm to get more precise semantic information. Two prototype demos have been successfully developed on mobile phones. The experimental results on different platforms show the superior performance of the proposed method.

Compared with the traditional interaction approaches, such as keyboard, mouse, pen, etc, vision based hand interaction is more natural and efficient. In this paper, we proposed a robust real-time hand gesture recognition method. In our method, firstly, a specific gesture is required to trigger the hand detection followed by tracking; then hand is segmented using motion and color cues; finally, in order to break the limitation of aspect ratio encountered in most of learning based hand gesture methods, the scale-space feature detection is integrated into gesture recognition. Applying the proposed method to navigation of image browsing, experimental results show that our method achieves satisfactory performance.

In this paper, we present a novel framework to customize multimedia messages for mobile users. The goal is to generate a video message from a series of pictures. The framework includes visual attention view detection, image grouping, image ranking, and slideshow generation. Considering the limitation of mobile device, we use a simple color feature based attention model to detect interesting regions of the images. We group the images, and rank them based on the attention view similarities. Finally a human perception based slideshow is designed to keep the mobile users' eye on attention regions efficiently. In addition, a short music is selected to match the video message. Extensive experiments and user studies show the promising performance of the proposed system.

This paper presents a novel virtual mouse system for mobile phones. The system is based on computer vision techniques. It first captures an image using the camera of the mobile phone, then extracts feature points of the scene and tracks them. By using this algorithm we can estimate the camera motion and give the translation vector to our virtual mouse interface. We use NOKIA 6630 camera phone as the developing platform which has a Mega Pixel camera on the back of it. The experiments show the promising results.

Because of the lack of prior knowledge about the color of characters and the disturbance caused by lighting conditions and noise, character segmentation from scene images is a very difficult issue. In this paper, a novel character segmentation method for color images from a digital camera is presented, combining edge detection, a watershed transform and clustering. Since the characters extracted from color images are output in a binary format, they can be input directly to an OCR system for recognition. Experimental results have proven the method's effectiveness.