Compound Facial Expressions consist of multiple basic expressions and present increased complexity in recognition tasks. Enhancing the representation of primary and secondary expressions is critical for improving the classification performance. However, when features are enhanced indiscriminately, irrelevant information may also be amplified, leading to the ‘Indiscriminate Enhancement Trap' and robustness degradation. To address these issues, we propose a novel Bi-Enhancement based Disentangled Dual-Attention (BED2A) Framework. The bi-enhancement strategy is devised to achieve signal complementarity between channel and spatial features. By transferring semantic information from channel attention to spatial attention in the latent space, the proposed method enhances the ability to localize salient regions. Subsequently, primary and secondary expression features are disentangled across tri-branch feature enhancement architecture, thereby enabling more effective dual-attention based classification. Specifically, the Differential Cross-Consistency mechanism is introduced to disentangle inter-expression features and enhance the precision of expression representations. Dual-Attention mechanism leverages attention to primary and secondary expressions to optimize the classification center and improve inter class separability. Experimental results demonstrate that BED2A effectively optimizes feature enhancement to improve robustness, achieving state-of-the-art performance.

2D heatmap-based human pose estimation has exhibited remarkable performance. However, constrained by quantization error, heatmap-based methods heavily rely on high-resolution heatmaps and intricate post-processing to enhance detection accuracy, thereby incurring substantial computational costs. To pursue more effective keypoint representation, we propose a novel scheme named Offset-based Disentangled Representation (ODR). ODR conducts coordinate classification and offset prediction simultaneously using 1D vectors and aggregates their outputs to precisely pinpoint keypoint positions, thus freeing from dependence on high-resolution heatmaps. To eliminate the impact of long-range offsets, we propose a scale-aware eraser that generates noticeable intervals based on the relative scale of different keypoints, directing the regression task to focus on short-range offsets. By doing so, upsampling layers are no longer necessary, enabling a more concise and effective architecture for human pose estimation. Extensive experiments conducted over COCO and MPII datasets validate the superiority of ODR over counterparts based on 2D or 1D heatmaps. Our source codes are available at the link.

To implement a metaverse exhibition interaction system, the instability problem of high-quality avatar facial reenactment must be considered. How to void identity limitations and eliminate artifacts are key challenges for avatar reenactment. It also lacks the support of the application system it is implemented in. We propose a metaverse system architecture oriented to emotional interaction. And we propose a novel method for avatar expression reenactment named Overall-Local Feature Warping Fusion Model based on Optical-Flow field prediction. We solve the identity limitation by overall optical-flow estimation and local optical-flow estimation and eliminate artifacts by illumination consistency. We compare with the mainstream optical flow face reenactment methods and outperform them in identity similarity, structural similarity, and facial action unit recognition ratio. We experimentally compared our method improves by an average improvement of 3.79%. And we also implement our method in the metaverse exhibition system. Although we satisfy most of the interaction scenarios, our method is still insufficient in some side-face cases.

Micro-expressions are spontaneous, rapid and subtle facial movements that can neither be forged nor suppressed. They are very important nonverbal communication clues, but are transient and of low intensity thus difficult to recognize. Recently deep learning based methods have been developed for micro-expression (ME) recognition using feature extraction and fusion techniques, however, targeted feature learning and efficient feature fusion still lack further study according to the ME characteristics. To address these issues, we propose a novel framework Feature Representation Learning with adaptive Displacement Generation and Transformer fusion (FRL-DGT), in which a convolutional Displacement Generation Module (DGM) with self-supervised learning is used to extract dynamic features from onset/apex frames targeted to the subsequent ME recognition task, and a well-designed Transformer Fusion mechanism composed of three Transformer-based fusion modules (local, global fusions based on AU regions and full-face fusion) is applied to extract the multi-level informative features after DGM for the final ME prediction. The extensive experiments with solid leave-one-subject-out (LOSO) evaluation results have demonstrated the superiority of our proposed FRL-DGT to state-of-the-art methods.

In the past decade, with the deepening of the aging of the population and the strengthening of the health consciousness of the whole society, the Internet healthcare service has grown to be an inevitable trend of current society. We propose a collaborative adaptive architecture named Trusted Healthcare Smart Brain (THSB) for cross-blockchain intelligent collaboration of multi-institution healthcare services. THSB is an interdisciplinary system with Healthcare Internet of Things (H-IoT), blockchain, Artificial Intelligence, Cloud Computing, Big Data, and Internet. The participants of THSB include patients, rehabilitation institutions, medical service institutions, healthcare content service institutions, medical regulatory institutions, scientific research institutions, and government institutions. In addition, we propose a medical resources service balance method to maximize the utilization of medical service resources to solve the contradiction between random medical events and the normal distribution of medical resources.

Facial action unit (AU) detection is a hot topic in computer vision, but it remains challenging due to individual characteristics. Facial action features are a vital informative factor to explain facial anatomical variations but are often entangled with other facial attribution information leading to representation inconsistency within one category. We propose a novel Contrastive Disentangled Representation Autoencoder (CDAE) to learn discriminative identity-invariant representation for AU detection by factorizing face images into temporally varying action parts and stationary facial attributions components. Facial image space is mapped onto the facial action subspace and action-independent identity subspace to disentangle facial action information from identity information. In addition, we design a contrastive learning scheme to obtain a semantic-aware AU manifold by mapping the facial action features onto the continuous space of the latent variables, thus minimizing the misalignment between subjects and reducing the dimension of the facial action features. Experiment results show that CDAE outperforms or is comparable to previous AU detection methods on the challenging BP4D and DISFA benchmarks, demonstrating that the learned facial action representation is discriminative for AU detection.

The emotion of human beings tends to be complex in real conditions, generating compound expressions in human faces. Compound expression recognition is an important challenge for the assessment of human complex emotion. The recognition system based on six basic expressions cannot meet the demand of compound expressions recognition. The recognition performance of models learned from the basic expressions is poor due to the small number of compound expression datasets with highly accurate labels and insufficient sample diversity. Making full use of domains outside of compound expressions in small sample datasets will help promote diversity. We propose the Multi-Domain Fusion Generative Adversarial Network (MDFGAN), which innovatively fuses the face domain, compound expression domain and basic expression domain to obtain rich expression generation capability and high accuracy recognition. Pairing the face domain and the contour-unrelated compound expression domain in the generator will expand the sample diversity. The contour-related compound expression domain and the basic expression domain will jointly improve the expression recognition accuracy of the discriminator. Finally, we conducted comprehensive experiments on CFEE-26, CFEE-7 and CK+. In the experiments, the results of MDFGAN improved 6.79% on UF1 and 8.5% on UAR.

With the rapid development of deep learning, the research and application of micro-expression recognition are more and more extensive. In the existing solutions, the methods using the difference between onset frame and apex frame have high accuracy and low computational cost. But their extraction of dynamic features is not integrated with classification network, resulting in the lack of feedback from classification loss to the extracted dynamic features. In this paper, we propose a novel Displacement Generating Module (DGM) which uses convolution module to generate the displacement feature between onset frame and apex frame instead of traditional optical flow or dynamic image. The new DGM is integrated with the existing LEARNet to form an end-to-end micro-expression recognition network, where the classification loss can feed backward to the parameters of DGM to obtain better displacement features. We also present a random selection method of apex frame to increase the amount of training data, and present a normalization operation for the displacement features with different scales. Our new approach has been tested on SAMM, SMIC, CASME II datasets with LOSO evaluation method, and achieves 0.737 on UF1 and 0.726 on UAR, which is obviously higher than existing networks with optical flow and dynamic imaging techniques.