The area of portrait image animation, propelled by audio input, has witnessed notable progress in the generation of lifelike and dynamic portraits. Conventional methods are limited to utilizing either audios or facial key points to drive images into videos, while they can yield satisfactory results, certain issues exist. For instance, methods driven solely by audios can be unstable at times due to the relatively weaker audio signal, while methods driven exclusively by facial key points, although more stable in driving, can result in unnatural outcomes due to the excessive control of key point information. In addressing the previously mentioned challenges, in this paper, we introduce a novel approach which we named EchoMimic. EchoMimic is concurrently trained using both audios and facial landmarks. Through the implementation of a novel training strategy, EchoMimic is capable of generating portrait videos not only by audios and facial landmarks individually, but also by a combination of both audios and selected facial landmarks. EchoMimic has been comprehensively compared with alternative algorithms across various public datasets and our collected dataset, showcasing superior performance in both quantitative and qualitative evaluations. The code and models are available on the project page.

In this paper, a novel method that utilizes feature-level fusion of finger vein (FV) and finger dorsal texture (FDT) images is proposed for human identification. Motivated by Weber's law, we present α-trimmed Weber representation (α-TWR) to enhance the foreground lines (FLs), i.e., vessels underneath skin and line-like texture on skin. The proposed α-TWR is robust to illumination variation, as validated by a basic reflective and transmitted imaging model. Cross section asymmetrical coding (CSAC) is performed to extract features for each pixel. The coding value contains discriminative information on the orientation and internal point location of the FLs. The CSAC values of FV and FDT in each point are abreast in terms of binary representation. Local density weighted matching is developed to obtain the matching score between two feature maps. We experimentally show that the proposed method outperforms other unimodal and multimodal identification methods in terms of equal-error-rate.

In finger vein verification, the most important and challenging part is to robustly extract finger vein patterns from low-contrast infrared finger images with limited a priori knowledge. Although recent convolutional neural network (CNN)-based methods for finger vein verification have shown powerful capacity for feature representation and promising perspective in this area, they still have two critical issues to address. First, these CNN-based methods unexceptionally utilize fully connected layers, which restrict the size of finger vein images to process and increase the processing time. Second, the capacity of CNN for feature representation generally suffers from the low quality of finger vein ground-truth pattern maps for training, particularly due to outliers and vessel breaks. To address these issues, in this paper, we propose a novel approach termed FV-GAN to finger vein extraction and verification, based on generative adversarial network (GAN), as the first attempt in this area. Unlike the CNN-based methods, FV-GAN learns from the joint distribution of finger vein images and pattern maps rather than the direct mapping between them, with the aim at achieving stronger robustness against outliers and vessel breaks. Moreover, FV-GAN adopts fully convolutional networks as the basic architecture and discards fully connected layers, which relaxes the constraint on the input image size and reduces the computational expenditure for feature extraction. Furthermore, we design an adversarial training strategy and propose a hybrid loss function for FV-GAN. The experimental results on two public databases show significant improvement by FV-GAN in finger vein verification in terms of both verification accuracy and equal error rate.