Micro-expression (ME) recognition holds great potential for revealing true human emotions. A significant barrier to effective ME recognition is the lack of sufficient annotated ME video data because MEs are subtle and involuntary facial expressions that are very hard to capture. To address this issue, data augmentation techniques, such as ME migration based on a driven video, have been employed to enrich training samples. Considering that MEs can be complex facial movements involving multiple action unit (AU) changes, we propose a novel ME generation approach that enables the creation of more realistic facial sequences by fusing MEs from multiple videos rather than just single driven video. To enhance the effectiveness of multi-sequence ME transfer, we adapt the thin plate spline motion model and improve traditional face alignment methods to better suit the model, facilitating multi-sequence driven ME generation. In our experiments, we conduct a downstream ME recognition task using models trained on our augmented ME sequences to demonstrate the effectiveness of our approach on the SAMM, SMIC, and CASME II datasets. The results confirm that our proposed approach outperforms state-of-the-art (SOTA) augmentation and generation methods in terms of F1 score and recognition accuracy.

Source-Free Domain Adaptation (SFDA) aims to train a target model without source data, and the key is to generate pseudo-labels using a pre-trained source model. However, we observe that the source model often produces highly uncertain pseudo-labels for hard samples, particularly those heavily affected by domain shifts, leading to these noisy pseudo-labels being introduced even before adaptation and further reinforced through parameter updates. Additionally, they continuously influence neighbor samples through propagation in the feature space. To eliminate the issue of noise accumulation, we propose a novel Progressive Curriculum Labeling (ElimPCL) method, which iteratively filters trustworthy pseudo-labeled samples based on prototype consistency to exclude high-noise samples from training. Furthermore, a Dual MixUP technique is designed in the feature space to enhance the separability of hard samples, thereby mitigating the interference of noisy samples on their neighbors. Extensive experiments validate the effectiveness of ElimPCL, achieving up to a 3.4% improvement on challenging tasks compared to state-of-the-art methods.

Micro-expressions (MEs) are subtle and brief facial expressions that occur involuntarily and may reveal hidden emotions. Due to MEs' weak intensities, it is challenging to discriminate MEs from image noise through AU detection results or spatio-temporal features. To model authentic ME patterns rather than overfitting to noise, we propose a novel multiframe strategy that captures detailed motion patterns and a two-layered feature encoding scheme to model interactions across different parts of the feature maps. Furthermore, we propose a novel facial Action Unit Graph Convolutional Network (AU GCN) that can adapt to testing input data through an AU detection module and a learnable adjacent matrix with a transformer encoder. Finally, we fuse the enhanced spatiotemporal features and AU GCN results to recognize MEs. Experimental results show that our methods outperform SOTA in F1 scores on SAMM and CASME II datasets, and also achieves the highest accuracy on CASME II dataset.

The facial expression spotting task serves as the foundation for expression recognition, focusing on identifying the onset and offset frames of facial expressions. State-of-the-art methods usually rely on detecting if optical flow intensity within facial feature regions exceeds a threshold. However, facial expressions may not recover to neutral at the offset frame, leading to sustained high optical flow intensities. Additionally, non-expression motion may also cause high optical flow intensities. Both scenarios have the potential to result in false positives during spotting. To address these issues, this paper introduces a two-stage spotting strategy and a spectrum-based post-processing method. Experimental results demonstrate the effectiveness of our approach in reducing false positives caused by the two issues. Finally, we evaluated our method on MEGC2022-TestSet and achieved an overall F1 score of 0.3776, surpasses the state-of-the-art methods.

Micro-expressions (MEs) which are types of spontaneous facial movements, are difficult to recognize due to their short duration and low intensity. Recent ME recognition methods typically depend on spatio-temporal features around the eyebrow and mouth regions where MEs occur. And these features are often extracted from the onset and apex frames in which the intensities of MEs are considered to be zero and high respectively. In this paper, we improve the effectiveness of the spatio-temporal features by proposing a two-layer encoder of Transformer to model the features’ relations. In addition, the novel recognition scheme captures the more detailed motion dynamics of MEs by employing more frames rather than the onset and apex frames. The recognition scheme is further refined by developing a graph convolution network with a trainable adjacency matrix for Action Units (AUs). Extensive experiments on multiple public datasets demonstrate that our method has better or comparable performance to SOTA methods on multiple evaluation metrics.

Intelligent transportation infrastructure is essential to intelligent transportation system (ITS). With the continuous development of Internet of Things (IoT) technology, remote monitoring has become a critical part of ITS. However, due to the limitations of network transmission, production cost, and other factors, some video monitoring can obtain only low-resolution (LR) video. LR video features are seriously lost, thus affecting the performance of ITS. In this paper, based on the research of existing super-resolution algorithms, we focus on improving the reconstruction quality of video frame sequences by aiming at the insufficient utilization of inter-frame information and low reconstruction quality of existing video super-resolution algorithms. This paper proposes a video super-resolution algorithm based on inter-frame information utilization, which can effectively improve the performance of ITS. First, a novel U-shaped feature extractor is designed to fully extract the feature expression of video frame sequences. Second, a deformable inter-frame alignment module based on residual learning is constructed to make the inter-frame alignment more accurate and thus promote the mutual utilization of inter-frame information. Finally, an up and down sampling residual block is proposed to extract features that better match the upsampling reconstruction requirements. The experimental results show that the method has better reconstruction quality for monitoring video and is advanced and applicable compared to mainstream video oversampling methods.

Image hiding has received significant attention due to the need of enhanced multimedia services such as multimedia security and meta-information embedding for multimedia augmentation. Recently, deep learning-based methods have been introduced that are capable of significantly increasing the hidden capacity and supporting full-size image hiding. However, these methods suffer from the necessity to balance the errors of the modified cover image and the recovered hidden image. In this paper, we propose a novel joint compressive autoencoder (J-CAE) framework to design an image hiding algorithm that achieves full-size image hidden capacity with small reconstruction errors of the hidden image. More importantly, our approach addresses the trade-off problem of previous deep learning-based methods by mapping the image representations in the latent spaces of the joint CAE models. Thus, both visual quality of the container image and recovery quality of the hidden image can be simultaneously improved. Extensive experimental results demonstrate that our proposed method outperforms several state-of-the-art deep learning-based image hiding techniques in terms of imperceptibility and recovery quality of the hidden images while maintaining full-size image hidden capacity.

Image hiding, one of the most important data hiding techniques, is widely used to enhance cybersecurity when transmitting multimedia data. In recent years, deep learning-based image hiding algorithms have been designed to improve the embedding capacity whilst maintaining sufficient imperceptibility to malicious eavesdroppers. These methods can hide a full-size secret image into a cover image, thus allowing full-image-to-image hiding. However, these methods suffer from a trade-off challenge to balance the possibility of detection from the container image against the recovery quality of secret image. In this paper, we propose Camouflage Generative Adversarial Network (Cam-GAN), a novel two-stage coverless full-image-to-image hiding method named, to tackle this problem. Our method offers a hiding solution through image synthesis to avoid using a modified cover image as the image hiding container and thus enhancing both image hiding imperceptibility and recovery quality of secret images. Our experimental results demonstrate that Cam-GAN outperforms state-of-the-art full-image-to-image hiding algorithms on both aspects.

Micro-expression (ME) recognition is an effective method to detect lies and other subtle human emotions. Machine learning-based and deep learning-based models have achieved remarkable results recently. However, these models are vulnerable to overfitting issue due to the scarcity of ME video clips. These videos are much harder to collect and annotate than normal expression video clips, thus limiting the recognition performance improvement. To address this issue, we propose a micro-expression video clip synthesis method based on spatial-temporal statistical and motion intensity evaluation in this paper. In our proposed scheme, we establish a micro-expression spatial and temporal statistical model (MSTSM) by analyzing the dynamic characteristics of micro-expressions and deploy this model to provide the rules for micro-expressions video synthesis. In addition, we design a motion intensity evaluation function (MIEF) to ensure that the intensity of facial expression in the synthesized video clips is consistent with those in real -ME. Finally, facial video clips with MEs of new subjects can be generated by deploying the MIEF together with the widely-used 3D facial morphable model and the rules provided by the MSTSM. The experimental results have demonstrated that the accuracy of micro-expression recognition can be effectively improved by adding the synthesized video clips generated by our proposed method.

The segmentation of small blood vessels whose width is less than 2 pixels in retinal images is a challenging problem. Existed methods rarely focus on the differences between small vessels and big vessels when doing segmentation. Therefore, previous methods are not accurate enough on small blood vessel segmentation. To effectively segment small blood vessels in retinal images including big vessels, we proposed a novel multi-label classification scheme for retinal vessel segmentation. In our proposed scheme, a local de-regression model is designed for multi-labeling and a convolutional neural network is used for multi-label classification. At addition, a local regression method is utilized to transform multi-label into binary label for locating small vessels. The experimental results show that our method achieves prominent performance for automatic retinal vessel segmentation, especially for small blood vessels.

In this paper, we explore the regularized feature selection method for person specific face verification in unconstrained environments. We reformulate the generalization of the single-task sparsity-enforced feature selection method to multi-task cases as a simultaneous sparse approximation problem. We also investigate two feature selection strategies in the multi-task generalization based on the positive and negative feature correlation assumptions across different persons. Simultaneous orthogonal matching pursuit (SOMP) is adopted and modified to solve the corresponding optimization problems. We further proposed a named simultaneous subspace pursuit (SSP) methods which generalize the subspace pursuit method to solve the corresponding optimization problems. The performance of different feature selection strategies and different solvers for face verification are compared on the challenging LFW face database. Our experimental results show that 1) the selected subsets based on positive correlation assumption are more effective than those based on the negative correlation assumption; 2) the OMP-based solvers outperform SP-based solvers in terms of feature selection and 3) the regularized methods with OMP-based solvers can outperform state-of-the-art feature selection methods.

There is an increasing use of some imperceivable and redundant local features for face recognition. While only a relatively small fraction of them is relevant to the final recognition task, the feature selection is a crucial and necessary step to select the most discriminant ones to obtain a compact face representation. In this paper, we investigate the sparsity-enforced regularization-based feature selection methods and propose a multi-task feature selection method for building person specific models for face verification. We assume that the person specific models share a common subset of features and novelly reformulated the common subset selection problem as a simultaneous sparse approximation problem. The effectiveness of the proposed methods is verified with the challenging LFW face databases.

In this paper, we propose a pairwise key distribution scheme based on a novel interconnection graph termed Hierarchical Hypercube. Motivated by the fact that sensor nodes are often deployed in groups (for example, dropped from an airplane at different locations) and hence the whole network is composed of multiple such groups, we design Hierarchical Hypercube as a two layer topology, where each group is modeled by an inner hypercube and connections between the groups are modeled using an outer hypercube. we propose a topology termed Hierarchy Hypercube. While retaining the desirable properties already shown by existing pairwise schemes, by using Hierarchical Hypercube, direct communication from a node to any other nodes is not required, either within a group or among groups, and hence this topology can effectively and realistically reflect the connectivity of the physical sensor network deployed in groups. Furthermore, we propose key pre-distribution scheme based on this novel topology and show that the new scheme possesses high probability of direct key establishment, low memory overhead, and resilience in the presence of broken communication links and compromised nodes, and thus still retain the desirable properties even when the ideal logical connection are distorted in the real deployment.