Often a face has a voice. Appearance sometimes has a strong relationship with one's voice. In this work, we study how a face can be converted to a voice, which is a face-based voice conversion. Since there is no clean dataset that contains face and speech, voice conversion faces difficult learning and low-quality problems caused by background noise or echo. Too much redundant information for face-to-voice also causes synthesis of a general style of speech. Furthermore, previous work tried to disentangle speech with bottleneck adjustment. However, it is hard to decide on the size of the bottleneck. Therefore, we propose a bottleneck-free strategy for speech disentanglement. To avoid synthesizing the general style of speech, we utilize framewise facial embedding. It applied adversarial learning with a multi-scale discriminator for the model to achieve better quality. In addition, the self-attention module is added to focus on content-related features for in-the-wild data. Quantitative experiments show that our method outperforms previous work.

Network intrusion detection is an indispensable defense in the critical era fulling of cyberattacks. However, it faces a severe class imbalanced issue, and most of the researches are conducted on simulated data. Therefore, this work introduces a hierarchical ensemble architecture with machine learning approaches. It is trained on the latest and real-world dataset to solve the above problems. The experiments show that we outperform state-of-the-art methods on real network traffic data.

Zero-shot voice conversion (VC) trained by non-parallel data has gained a lot of attention in recent years. Previous methods usually extract speaker embeddings from audios and use them for converting the voices into different voice styles. Since there is a strong relationship between human faces and voices, a promising approach would be to synthesize various voice characteristics from face representation. Therefore, we introduce a novel idea of generating different voice styles from different human face photos, which can facilitate new applications, e.g., personalized voice assistants. However, the audio-visual relationship is implicit. Moreover, the existing VCs are trained on laboratory-collected datasets without speaker photos, while the datasets with both photos and audios are in-the-wild datasets. Directly replacing the target audio with the target photo and training on the in-the-wild dataset leads to noisy results. To address these issues, we propose a novel many-to-many voice conversion network, namely Face-based Voice Conversion (FaceVC), with a 3-stage training strategy. Quantitative and qualitative experiments on the LRS3-Ted dataset show that the proposed FaceVC successfully performs voice conversion according to the target face photos. Audio samples can be found on the demo website at https://facevc.github.io/.

Automatic lesion detection from computed tomography (CT) scans is an important task in medical diagnosis. However, three frequent properties of medical data make CT lesion detection a challenging task: (1) Scale variance: Large scale variation is across lesion instances. Especially, it is extremely difficult to detect small lesions; (2) Imbalanced data: The data distributions are highly imbalanced, where few classes account for the majority of data; (3) Prediction stability: Based on our observations, an input lesion image with slightly pixel shift or translation can lead to drastic output mispredictions and this is not allowed for medical applications. To address these challenges, this paper proposes a Robust One-Stage Network (ROSNet) for robust CT lesion detection. Specifically, a novel nested structure of neural networks is developed to generate a series of feature pyramids for detecting CT lesions in various scales, an effective data sensitive class-balanced loss as well as a shift-invariant downsampling strategy are also introduced to improve the detection performance. Experiments are conducted on a large-scale and diverse dataset, DeepLesion, showing that ROSNet outperforms the best performance in MICCAI 2019 by 3.95% (2-class detection task) and 25.41% (8-class detection task) in terms of mean average precision (mAP).