Facial expression recognition (FER) has always been one of the key task in affective computing. Over the years, researchers have worked to improve the performance of FER by designing models with more powerful feature extraction, embedding attention mechanism, and reconstructing missing information, etc. Different from the paradigms above, we attempt to improve FER performance by using multimodal sentiment data, such as audio and text, as privileged information (PI) for facial images. To this end, a multimodal privileged information embedded facial expression recognition network (MPI-FER) is proposed in this paper. During the training phase, this model achieves the PI embedding of multimodal data for FER by developing cross-modality translation between multimodal sentiment data. During the test phase, input images alone are sufficient for the model inference to accomplish the FER task input. The MPI-FER is a large-scale, heterogeneous deep neural network. To achieve effective training of this model with limited training samples, we design a multi-stage training strategy of module-wise pre-training followed by end-to-end fine-tuning. In addition, a strategy of filling the multimodal sentiment quaternion is proposed for implementing our method on a facial expression database consisting only of face images. We conducted extensive experiments to evaluate the proposed method on two databases of multimodal sentiment analysis (CH-SIMS and CMU-MOSI) and two databases of FER in the wild (RAF-DB and AffectNet). The results show that embedding multimodal sentiment data as privileged information into the FER task based on face images can significantly improve the accuracy of FER. Furthermore, by only using image in the test phase, the proposed method can achieve better results of multimodal sentiment analysis than those methods achieved by using multimodal sentimental data fusion.

We propose an unsupervised cross-view facial expression adaptation network (UCFEAN) to simultaneously generate and recognize cross-view facial expressions in images in an unsupervised manner. The main idea of UCFEAN is to convert the unsupervised domain adaptation between two image spaces with different appearance into semi-supervised learning (SSL) in feature spaces with the same semantic content. The cyclic image generation of cross-view facial expressions based on the generative adversarial network (GAN) is carried out to project unlabelled target images and labelled source images to the corresponding feature spaces with the same semantic content. This helps realize the unsupervised feature learning of the target image. Labels of facial expressions represented in the projected target features can then be learned using the projected source features, because the distributions of the projected features in the two domains are close enough for knowledge transfer by using SSL. Three techniques are developed to train UCFEAN in an effective and stable manner. Extensive experiments are conducted to evaluate the UCFEAN on two multi-view facial expression image databases including RaFD and Multi-PIE. The results show that the proposed method can generate realistic target images of the facial expression and recognize cross-view facial expressions with high precision.

Falls are the main cause of accidental injuries, and even death among elderly people, especially those who live alone in their homes. The absence of a reliable fall detection system has long been a serious problem for home health monitoring. A video surveillance system can be used to monitor elderly people at home to detect falls, but the traditional implementation of such intelligent detection falls short of personal privacy-related considerations; additionally, many people do not want to be watched in their homes. To solve this problem, we propose a fall detection system with visual shielding that can ensure the safety of elderly people in their homes while preserving their personal privacy. Multilayer compressed sensing is first used to achieve visually shielded video frames. By combining low-rank sparse decomposition theory with the improved local binary pattern on the three orthogonal planes, the object features are extracted from the shielded video frames. Finally, to compensate for the information lost in the compressed video to a certain extent, a private information-embedded classification model is proposed to identify fall-related behavior. The experimental results on two public fall datasets show that the proposed method delivers impressive accuracy and a low error rate while effectively distinguishing between fall- and nonfall-related behaviors in videos.

Scene images generally show the characteristics of large intra-class variety and high inter-class similarity because of complicated appearances, subtle differences, and ambiguous categorization. Hence, it is difficult to achieve satisfactory accuracy by using a single representation. For solving this issue, we present a comprehensive representation for scene recognition by fusing deep features extracted from three discriminative views, including the information of object semantics, global appearance, and contextual appearance. These views show diversity and complementarity of features. The object semantics representation of the scene image, denoted by spatial-layout-maintained object semantics features, is extracted from the output of a deep-learning-based multi-classes detector by using spatial fisher vectors, which can simultaneously encode the category and layout information of objects. A multi-direction long short-term memory-based model is built to represent contextual information of the scene image, and the activation of the fully connected layer of a convolutional neural network is used to represent the global appearance of scene image. These three kinds of deep features are then fused to draw a final conclusion for scene recognition. Extensive experiments are conducted to evaluate the proposed comprehensive representation on three benchmarks scene image database. The results show that the three deep features complement to each other strongly and are effective in improving recognition performance after fusion. The proposed method can achieve scene recognition accuracy of 89.51% on the MIT67 database, 78.93% on the SUN397 database, and 57.27% on the Places365 databases, respectively, which are better percentages than the accuracies obtained by the latest reported deep-learning-based scene recognition methods.

Exact compressed sensing (CS) recovery theoretically depends on a large number of random measurements. In this study, the authors present a novel CS measurement technique based on the cellular automata chaos (CAC) model. The proposed method selects original signal thresholding (OST) as its initial seed to realise CS signal coding. The benefits of CS coding with CAC‐OST are that: (i) the signal compression ratio of this coding method can be far below the signal sparsity level and (ii) the signal can be recovered perfectly, even with slow CS measurements. This study reports some experiments that demonstrate the excellent performance of CAC‐OST in CS coding.

A novel multi‐band joint local sparse tracking algorithm via wavelet transforms is proposed in this study. The object image may contain rich information of different types; the authors use wavelet transforms to decompose the object image into some sub‐band images first. This will help extract the information in different frequency ranges for the object. Then same block operation is executed on all the sub‐band images. The l 2, 1 mixed‐norm is used to describe the multi‐band joint local sparse representation on each patch; it can effectively extract the structural information in different frequency ranges. Thus, more accurate object appearance model can be established. Second, the coefficients on the diagonal of coefficient matrix are extracted for the confidence degrees of the candidate objects in this band, and then the confidence degree results in all the bands are fused to determine the best candidate object in the current frame. This can effectively alleviate the object drifting. Finally, both qualitative and quantitative evaluation results on 15 challenging video sequences demonstrate that the proposed tracking algorithm in this study can achieve better tracking effects compared with the other state‐of‐the‐art algorithms.

In colour compressed sensing (CS) imaging, the current two bottlenecks for application are (1) high computation cost of sparse representation (SR) with over‐complete dictionary and (2) unsatisfactory imaging quality of CS recovery with l 1 ‐norm minimisation. Thus, this study proposes a novel colour CS imaging framework. In the framework, two improvements are achieved: (1) the authors present the sparse difference to reduce the computation cost of SR in RGB colour imaging; (2) the authors use fractal dimension instead of l 1 ‐norm as the object function to actualise high quality CS recovery. The feasibility of our colour CS imaging framework is proved by sseveral experiments.

This study presents a novel compressive sensing (CS) framework to solve the high dimensional mass spectrometry (MS) signal processing in Bioinformatics. As a hot research topic, CS has attracted a great deal of attention in many fields. In theory, high sparsity is one precondition for any CS framework. However, in Bioinformatics, one application bottleneck is that only a few MS data can be considered as sparse. So sparse representation (SR) become necessary. However, this will create a new problem that the SR computation cost will be too huge to MS signal because of its high data dimensionality (usually tens of thousands or more). Therefore the authors propose theconcept ofsparse difference (SD) to realise a new CS framework. Firstly, it canacquire the prior MS information through fractal and entropy recognition. Secondly, the original signal can be perfectly recovered by SD based on the previous recognition result. The feasibility and validity of this CS framework isproved by experiments.