Labanotation is a professional dance notation system widely used in dance education and choreography preservation. Automatically generatingLabanotation dance scores from motion capture data can save a huge amount of manual time and effort. Recently, the sequence-to-sequence (seq2seq) model is applied to the automatic Labanotation generation. This model is based on an encoder-decoder structure, which encodes the input motion sequence to a fixed-length vector and then decodes it to generate the target sequence. However, the encoding of spatial skeleton structure of motion data is not considered in the existing work. Besides, it is challenging to align between the input motion data and the output Laban symbol sequences due to the severe imbalance of sequence lengths. Therefore, in this paper, we present a new seq2seq model for more effective Labanotation generation. In the encoder, we propose a new gesture-sensitive graph convolutional network with learned adaptive joint weights and non-physical connections to learn both spatial and temporal patterns from motion data sequences. In the decoder, we exploit motion rhythm information and propose a novel rhythm-aware attention mechanism to learn a good alignment between motion sequences and Laban symbol sequences, so that we can focus on relevant parts of the input motion sequence without searching in the whole sequence when predicting a target Laban symbol. Extensive experiments on two real-world datasets show that the proposed method achieves a better performance compared with the state-of-the-art approaches on the task of automatic Labanotation generation.

Salient Object Detection (SOD) plays an important role in many image-related multimedia applications. Although there are many existing research works about the salient object detection in traditional RGB (visible-light spectrum) images, there are still many complex situations that regular RGB images cannot provide enough cues for the accurate SOD, such as the shadow effect, similar appearance between background and foreground, strong or insufficient illumination, etc. Because of the success of near-infrared spectrum in many computer vision tasks, we explore the multi-spectral SOD in the synchronized RGB images and near-infrared (NIR) images for the both simple and complex situations. We assume that the RGB SOD in the existing RGB image datasets could provide references for the multi-spectral SOD problem. In this paper, we mainly model this research problem as a deep learning based domain adaptation from the traditional RGB image data (source domain) to the multi-spectral data (target domain), and an adversarial deep domain adaptation model is proposed. We first collect and will publicize a large multi-spectral dataset, RGBN-SOD dataset, including 780 synchronized RGB and NIR image pairs for the multi-spectral SOD problem in the simple and complex situations. Intensive experimental results show the effectiveness and accuracy of the proposed deep domain adaptation for the multi-spectral SOD. Besides, due to the absence of research on the field of multi-spectral co-saliency detection, we also collect 200 synchronized RGB and NIR image pairs in addition to explore the multi-spectral co-saliency detection.

Although there are many existing research works about the salient object detection (SOD) in RGB images, there are still many complex situations that regular RGB images cannot provide enough cues for the accurate SOD, such as the shadow effect, similar appearance between background and foreground, strong or insufficient illumination, etc. Because of the success of near-infrared spectrum in many computer vision tasks, we explore the multi-spectral SOD in the synchronized RGB images and near-infrared (NIR) images for the both simple and complex situations. We assume that the RGB SOD in the existing RGB image datasets could provide references for the multi-spectral SOD problem. In this paper, we first collect and will publicize a large multi-spectral dataset including 780 synchronized RGB and NIR image pairs for the multi-spectral SOD problem in the simple and complex situations. We model this research problem as an adversarial domain adaptation from the existing RGB image dataset (source domain) to the collected multi-spectral dataset (target domain). Experimental results show the effectiveness and accuracy of the proposed adversarial domain adaptation for the multi-spectral SOD.

Labanotation is an important notation system for recording dances. Automatically generating Labanotation scores from motion capture data has attracted more interest in recent years. Current methods usually focus on individual movement segments and generate Labanotation symbols one by one. This requires segmenting the captured data sequence in advance. Manual segmentation will consume a lot of time and effort, while automatic segmentation may not be reliable enough. In this paper, we propose a sequence-to-sequence approach that can generate Labanotation scores from unsegmented motion data sequences. First, we extract effective features from motion capture data based on body skeleton analysis. Then, we train a neural network under the encoder-decoder architecture to transform the motion feature sequences to corresponding Labanotation symbols. As such, the dance score is generated. Experiments show that the proposed method performs favorably against state-of-the-art algorithms in the automatic Labanotation generation task.

Labanotation is a widely used dance notation system. The topic of generating Labanotation scores from captured dance motion data has attracted more research interest in recent years. Current methods usually generate Laban symbols via recognizing motion segments that each contains a single dance movement. However, they rely on the manual segmentation of raw dance motion sequences, which can cost a lot of time and effort. In this paper, we present a fully automatic framework to generate Labanotation scores from continuous motion data. First, we split the captured dance data to motion segments based on the Laban theory of body weight support transferring. Then, based on the segmented data, we utilize a network with both 1D-convolutional and recurrent layers to recognize body movements and generate Laban symbols. As such, the Labanotation score is created. Extensive experiments show that the proposed automatic framework performs favorably against previous solutions.

Real-time object tracking has wide applications in time-critical multimedia processing areas such as motion analysis and human-computer interaction. It remains a hard problem to balance between accuracy and speed. In this paper, we present a fast real-time context-based visual tracking algorithm with a new saliency prior context (SPC) model. Based on the probability formulation, the tracking problem is solved by sequentially maximizing the computed confidence map of target location in each video frame. To handle the various cases of feature distributions generated from different targets and their contexts, we exploit low-level features as well as fast spectral analysis for saliency to build a new prior context model. Then, based on this model and a spatial context model learned online, a confidence map is computed and the target location is estimated. In addition, under this framework, the tracking procedure can be accelerated by the fast Fourier transform. Therefore, the new method generally achieves a real-time running speed. Extensive experiments show that our tracking algorithm based on the proposed SPC model achieves real-time computation efficiency with overall best performance comparing with other state-of-the-art methods.

In this paper, we propose a preprocessing strategy for refining the pedestrian images in person re-identification(re-id). The accomplishment of the person re-id task depends on the features extracted from pedestrian appearances. The best image matches are verified based on these features as identification results. The saliency information in image scenes is often exploited for feature selection in high-level vision tasks. Inspired by the pre-attentive mechanism in human visual system, we utilize the saliency information in the re-identification data to refine the person appearance in a preprocessing step. We first obtain the eye-fixation-predicting map based on the saliency analysis of image. Then this map is used to spatially weight the image features for a better appearance. Finally, we apply these processed images to the feature extraction in a standard re-identification procedure. Experiments on the widely-used VIPeR dataset show that the proposed method improves the final performance of re-identification task.

In this paper, we present a new FFT-based visual tracking algorithm based on a new model for computing prior context distribution via a spectral saliency approach. The tracking problem is formulated under a Bayesian framework where the statistical relationships between the features of the target and its spatio-temporal context are modeled. When building the context model, the prior distribution of the possible target position is an important part worth studying. To deal with various cases of distributions based on different attributes of the target and its context, we exploit low level saliency features by spectral analysis to compute prior distribution, not limited to center-surround weights. We show by extensive experiments that the performance of the new tracking algorithm based on the new saliency prior context (SPC) model achieves real-time computation efficiency with overall best location accuracy performance compared with other state-of-the-art methods.