Image paragraph captioning aims to automatically generate a paragraph from a given image. It is an extension of image captioning in terms of generating multiple sentences instead of a single one, and it is more challenging because paragraphs are longer, more informative, and more linguistically complicated. Because a paragraph consists of several sentences, an effective image paragraph captioning method should generate consistent sentences rather than contradictory ones. It is still an open question how to achieve this goal, and for it we propose a method to incorporate objects' spatial coherence into a language-generating model. For every two overlapping objects, the proposed method concatenates their raw visual features to create two directional pair features and learns weights optimizing those pair features as relation-aware object features for a language-generating model. Experimental results show that the proposed network extracts effective object features for image paragraph captioning and achieves promising performance against existing methods.

Facial Action Unit (AU) detection is an important task to enable the emotion recognition from facial movements. In this paper, we propose a novel algorithm which utilizes identity-labeled face images to tackle the identity-based intra-class variation of AU detection that the appearances of the same AU vary significantly among different subjects, which makes existing methods generate low performance under cross-domain scenarios in case that the training and test datasets are dissimilar. The proposed method is based on network cascades consisting of two sub-tasks, face clustering and AU detection. The face clustering network, trained from a large dataset containing numerous identity-annotated face images, is designed to learn a transformation to extract identity-dependent image features, which are used to predict AU labels in the second network. The cascades are jointly trained by AU- and identity-annotated datasets that contain numerous subjects to improve the method's applicability. Experimental results show that the proposed method achieves state-of-the-art AU detection performance on benchmark datasets BP4D, UNBC-McMaster, and DISFA.

We develop a system which generates summaries from seniors' indoor-activity videos captured by a social robot to help remote family members know their seniors' daily activities at home. Unlike the traditional video summarization datasets, indoor videos captured from a moving robot poses additional challenges, namely, (i) the video sequences are very long (ii) a significant number of videoframes contain no-subject or with subjects at ill-posed locations and scales (iii) most of the well-posed frames contain highly redundant information. To address this problem, we propose to exploit pose estimation for detecting people in frames. This guides the robot to follow the user and capture effective videos. We use person identification to distinguish a target senior from other people. We also make use of action recognition to analyze seniors' major activities at different moments, and develop a video summarization method to select diverse and representative keyframes as summaries.

Manually labeling the large number of samples of Android APKs into benign or different malicious families requires tremendous human effort, while it is comparably easy and cheap to obtain a large amount of unlabeled APKs from various sources. Moreover, the fast-paced evolution of Android malware continuously generates derivative and new malware families. These families often contain new signatures, which can escape detection that uses static analysis. These practical challenges can also cause classical supervised machine learning algorithms to degrade in performance.

In this paper, we revisit the frequency allocation scheme (FAS) for the homogeneous cellular networks by taking the non-uniform distribution of users into account. Owing to the non-uniformly distributed user density from sector to sector, the traffic loads become highly unbalanced across whole network. Thus, to satisfy users' requests, the dynamic user-oriented FAS (Duo-FAS) is proposed to allocate different amount of subcarriers to sectors according to the variant user densities. However, the Duo-FAS technique can partially remove the innate benefit of interference avoidance via frequency planning, i.e. the conventional three-sectored cell architecture. That means some extra inter-cell interference (ICI) can be produced by using Duo- FAS. Fortunately, the network multiple-input and multiple-output (MIMO) technique can effectively alleviate the impact of ICI. Thanks to Duo-FAS, the number and positions of the cooperative cells when using the network MIMO technique can be significantly different from the conventional schemes. Thus, properly selecting the cooperative cells becomes a pivotal issue. The key is that for any particular user, the neighboring cells to form a cooperative network MIMO group should be adaptively selected so that the strong ICI can be eliminated. The simulation results demonstrated the higher achievable cell capacity.

The goal of face hallucination is to generate high-resolution images with fidelity from low-resolution ones. In contrast to existing methods based on patch similarity or holistic constraints in the image space, we propose to exploit local image structures for face hallucination. Each face image is represented in terms of facial components, contours and smooth regions. The image structure is maintained via matching gradients in the reconstructed high-resolution output. For facial components, we align input images to generate accurate exemplars and transfer the high-frequency details for preserving structural consistency. For contours, we learn statistical priors to generate salient structures in the high-resolution images. A patch matching method is utilized on the smooth regions where the image gradients are preserved. Experimental results demonstrate that the proposed algorithm generates hallucinated face images with favorable quality and adaptability.

The goal of single-image super-resolution is to generate a high-quality high-resolution image based on a given low-resolution input. It is an ill-posed problem which requires exemplars or priors to better reconstruct the missing high-resolution image details. In this paper, we propose to split the feature space into numerous subspaces and collect exemplars to learn priors for each subspace, thereby creating effective mapping functions. The use of split input space facilitates both feasibility of using simple functions for super-resolution, and efficiency of generating high-resolution results. High-quality high-resolution images are reconstructed based on the effective learned priors. Experimental results demonstrate that the proposed algorithm performs efficiently and effectively over state-of-the-art methods.