Great success has been achieved in reusing Style-GAN, but there are still challenges in reusing StyleGAN to image restoration tasks. We believe that the limited expression of StyleGAN’s latent code hinders its effective use in image restoration tasks. Therefore, we propose to extend StyleGAN’s latent code to improve its expressiveness. Specifically, we propose a new model called Extended StyleGAN Encoder (ESE). ESE is based on Feature Pyramid Network, which can encode input images both in coarse-grain and fine-grain, and generate H × W × 2C feature maps instead of 2C-dimensional style vectors. In this way, ESE can provide more information of the input images to StyleGAN, which can achieve a more accurate reconstruction. Therefore, ESE can be applied to different image restoration tasks. Experiments show that ESE can surpass some models specifically designed for regular and irregular mask inpainting tasks, while it is difficult for other GAN Inversion methods. And experiments also show that ESE achieved good performance on colorization and denoising tasks. Besides, we further analyze the impact of the extended latent code by visualizing the outputs of ESE and the intermediate outputs of StyleGAN, which demonstrates that ESE does really reuse the knowledge of StyleGAN.

In this paper, we target image-based person-to-person virtual try-on in the presence of diverse poses and large viewpoint variations. Existing methods are restricted in this setting as they estimate garment warping flows mainly based on 2D poses and appearance, which omits the geometric prior of the 3D human body shape.Moreover, current garment warping methods are confined to localized regions, which makes them ineffective in capturing long-range dependencies and results in inferior flows with artifacts.To tackle these issues, we present 3D-aware global correspondences, which are reliable flows that jointly encode global semantic correlations, local deformations, and geometric priors of 3D human bodies. Particularly, given an image pair depicting the source and target person, (a) we first obtain their pose-aware and high-level representations via two encoders, and introduce a coarse-to-fine decoder with multiple refinement modules to predict the pixel-wise global correspondence. (b) 3D parametric human models inferred from images are incorporated as priors to regularize the correspondence refinement process so that our flows can be 3D-aware and better handle variations of pose and viewpoint. (c) Finally, an adversarial generator takes the garment warped by the 3D-aware flow, and the image of the target person as inputs, to synthesize the photo-realistic try-on result. Extensive experiments on public benchmarks and our selected HardPose test set demonstrate the superiority of our method against state-of-the-art try-on approaches.

The accurate diagnosis of fundus disease can effectively reduce the disease's further deterioration and provide targeted treatment plans for patients. Fundus image classification is a multi-label classification task due to one fundus image may contain one or more diseases. For multi-label classification of fundus images, we propose two new multi-label classification networks -- MCG-Net based on graph convolutional network and MCGS-Net based on graph convolutional network and self-supervised learning. Here, the graph convolutional network is used to capture the relevant information of the multi-label fundus images, and self-supervised learning is used to enhance the generalization ability of the network by learning more unannotated data. We use the ROC curve, Precision score, Recall score, Kappa score, F-1 score, and AUC value as the evaluation metrics and test on two datasets. Compared with other methods, our methods have better classification performance and generalization ability. Our methods can significantly improve classification performance and enhance the generalization ability of multi-label fundus image classification.

Dropout is one of the most widely used methods to avoid overfitting neural networks. However, it rigidly and randomly activates neurons according to a fixed probability, which is not consistent with the activation mode of neurons in the human cerebral cortex. Inspired by gene theory and the activation mechanism of brain neurons, we propose a more intelligent adaptive dropout, in which a variational self-encoder (VAE) overlaps to an existing neural network to regularize its hidden neurons by adaptively setting activities to zero. Through alternating iterative training, the discarding probability of each hidden neuron can be learned according to the weights and thus effectively avoid the shortcomings of the standard dropout method. The experimental results in multiple data sets illustrate that this method can better suppress overfitting in various neural networks than can the standard dropout. Additionally, this adaptive dropout technique can reduce the number of neurons and improve training efficiency.