Face texture completion plays a significant role in virtual human research, and the quality of face texture needs to be improved urgently. One of the major obstacles to single-face texture generation is that the generated textures are always incomplete for self-occlusion of the input face, and the other is that pixel details are limited by the illumination. To address this, we propose a method for complete face texture generation based on generative adversarial networks. The face parameters obtained from 3D Morphable Model are processed as conditional vectors in the encoder, and the multivariate Gaussian distribution of the latent code is used in the networks to learn the complete texture features. We established a face texture dataset CFT for training the network. Meanwhile, we show the effectiveness of the proposed approach in qualitative and quantitative experiments. The visual results under different tasks show superior performances compared with the state-of-the-art approaches.

Stereo matching CNNs suffer from performance deteriorate when evaluated under different distributions from training data. Previous domain adaptation/generalization methods are hard to maintain a robust performance in different baselines and usually require difficult adversarial optimization or intricate network structure. To solve this problem, we propose Fourier-based amplitude transform (FAT), mapping the source image to the target style without altering semantic content, which requires no training to perform the domain alignment. Specifically, we leverage the Fourier transform and its inverse to swap the low-frequency amplitude component of the source data with the target data. To effectively map style and relieve the artifacts, we introduce two factors to control the replacing area: the distance of HSV distribution between source and target images; and the difference between the source left image and its warped left image. Experiments testify FAT can significantly bridge domain gaps, making source data distribution closer to target data. Furthermore, when only training on synthetic datasets, FAT can also help different baselines achieve competitive cross-domain generalization capabilities on real datasets.

Most deep learning-based state-of-the-art stereo matching methods significantly depend on large-scale datasets. However, it is implausible to collect sufficient real-world samples with dense and clear ground-truth disparity maps in practice. Although synthetic datasets’ appearance has alleviated the demand for extensive real data, there is a domain shift between synthetic and real sets. To tackle this problem, we propose an individually trained synthetic-to-real domain adaptation (SDA) network that maps synthetic images into the real domain. Specifically, our approach translates the data style from synthetic domain to real domain while maintaining the content and the spatial information. First, edge cues are leveraged to guide domain adaptation in preserving the spatial consistency between input and the generated image. Second, we combine the spatial feature transform (SFT) layer to effectively fuse features from the edge map and the source image. Extensive experiments demonstrate that: 1) when only trained on synthetic data and generalized to real data, our model evidently outperforms many state-of-the-art domain adaptation methods; 2) our translated synthetic datasets (TSD) help to improve the generalization capability of any stereo matching CNNs. Codes and data will be available athttps://github.com/Archaic-Atom/SDA_network.

In this paper, two multi-stream asynchrony Dynamic Bayesian Network models (MS-ADBN model and MM-ADBN model) are proposed for audio-visual speech recognition (AVSR). The proposed models, with different topology structures, loose the asynchrony of audio and visual streams to word level. For MS-ADBN model, both in audio stream and in visual stream, each word is composed of its corresponding phones, and each phone is associated with observation vector. MM- ADBN model is an augmentation of MS-ADBN model, a level of hidden nodes--state level, is added between the phone level and the observation node level, to describe the dynamic process of phones. Essentially, MS-ADBN model is a word model, while MM-ADBN model is a phone model. Speech recognition experiments are done on a digit audio-visual (A-V) database, as well as on a continuous A-V database. The results demonstrate that the asynchrony description between audio and visual stream is important for AVSR system, and MM-ADBN model has the best performance for the task of continuous A-V speech recognition.