Accurately predicting the pharmacological and toxicological properties of molecules is a critical step in the drug development process. Owing to the heterogeneity of molecular property prediction tasks, most of the current methods rely on building a base model and fine-tuning it to address specific properties. However, constructing a high-quality base model is a time-consuming procedure and requires a carefully designed network architecture; in addition, in certain rare molecular property prediction tasks, the base model often does not transfer well to new tasks. In this work, we adopt a meta-learning-based training framework that enables our model to adapt to diverse tasks with limited data, thereby preventing data scarcity from impacting certain molecular property predictions. Additionally, this framework leverages the correlations between different tasks, allowing the constructed model to quickly adapt to new prediction tasks. Moreover, we propose a multimodal fusion framework that combines two-dimensional molecular graphs with molecular images. In the molecular graphs, node-, motif-, and graph-level features are hierarchically guided from low to high levels, fully exploiting the molecular representation and more efficiently conducting hierarchical fusion. Experimental results indicate that our model outperforms the baseline models across various performance indicators, thereby validating the effectiveness of our approach.

Abstract Low‐light image/video enhancement is a challenging task when images or video are captured under harsh lighting conditions. Existing methods mostly formulate this task as an image‐to‐image conversion task via supervised or unsupervised learning. However, such conversion methods require an extremely large amount of data for training, whether paired or unpaired. In addition, these methods are restricted to specific training data, making it difficult for the trained model to enhance other types of images or video. In this paper, we explore a novel, fast and flexible, zero‐shot, low‐light image or video enhancement framework. Without relying on prior training or relationships among neighboring frames, we are committed to estimating the illumination of the input image/frame by a well‐designed network. The proposed zero‐shot, low‐light image/video enhancement architecture includes illumination estimation and residual correction modules. The network architecture is very concise and does not require any paired or unpaired data during training, which allows low‐light enhancement to be performed with several simple iterations. Despite its simplicity, we show that the method is fast and generalizes well to diverse lighting conditions. Many experiments on various images and videos qualitatively and quantitatively demonstrate the advantages of our method over state‐of‐the‐art methods.

Medical image super-resolution (SR) is an important medical image processing task and is often helpful for downstream medical analysis tasks. Most of the conventional SR methods tried to generate visually more convincing images whereas ignoring the following downstream tasks. In this paper, we take the Alzheimer’s disease diagnosis as the downstream task and propose a novel diagnosis-guided medical image SR network, which can make the SR and diagnosis be boosted by each other. The method contains two sub-networks, i.e., the SR network and the diagnosis network. To achieve better diagnosis performance, in the SR network, we apply the deformable convolution to capture the regions of interest (ROIs) with different and irregular sizes and shapes, which are important for diagnosis. Moreover, to integrate the two tasks, i.e., SR and diagnosis, more profoundly, we design a novel diagnosis-guided attention module, which makes the key regions for diagnosis can be reconstructed more clearly by the SR network. The extensive experiments on medical image data sets show that the proposed method often outperforms other state-of-the-art SR methods, which demonstrates its effectiveness. The codes of this paper are released in https://github.com/WJingwei/SRDA.

Brain 18F-FDG PET images are commonly-known materials for effectively predicting Alzheimer's disease (AD). However, the data volume of PET is usually insufficient, which is unfavorable to train an accurate AD prediction networks. Furthermore, the PET image is noisy with low signal-to-noise ratio, and simultaneously the feature (metabolic abnormality) used for predicting AD in PET image is not always obvious. Therefore, a contrastive-based learning method is proposed to address the challenges of PET image inherently possessed. Firstly, the slices of 3D PET image are amplified by cropping the image of anchors (i.e., an augmented version of the same image) to generate extended training data. Meanwhile, contrastive loss is adopted to enlarge inter-class feature distances and reduce intra-class feature differences using subject fuzzy labels as supervised information. Secondly, we construct a double convolutional hybrid attention module to enhance the network to learn different perceptual domains where two convolutional layers with different convolutional kernels ($7\times 7$ and $5\times 5$) are constructed. Moreover, we recommend a diagnosis mechanism by analyzing the consistency of predicted result for PET slices alone with clinical neuropsychological assessment to achieve a better AD diagnosis. The experimental results show that the proposed method outperforms the state-of-the-arts for brain 18F-FDG PET images, and hence demonstrate the advantage of the method in effectively predicting AD.

Abstract Single image superresolution (SISR) has achieved substantial progress based on deep learning. Many SISR methods acquire pairs of low‐resolution (LR) images from their corresponding high‐resolution (HR) counterparts. Being unsupervised, this kind of method also demands large‐scale training data. However, these paired images and a large amount of training data are difficult to obtain. Recently, several internal, learning‐based methods have been introduced to address this issue. Although requiring a large quantity of training data pairs is solved, the ability to improve the image resolution is limited if only the information of the LR image itself is applied. Therefore, we further expand this kind of approach by using similar HR reference images as prior knowledge to assist the single input image. In this paper, we proposed zero‐shot single image superresolution with a reference image (Ref‐ZSSR). First, we use an unconditional generative model to learn the internal distribution of the HR reference image. Second, a dual‐path architecture that contains a downsampler and an upsampler is introduced to learn the mapping between the input image and its downscaled image. Finally, we combine the reference image learning module and dual‐path architecture module to train a new generative model that can generate a superresolution (SR) image with the details of the HR reference image. Such a design encourages a simple and accurate way to transfer relevant textures from the reference high‐definition (HD) image to LR image. Compared with using only the image itself, the HD feature of the reference image improves the SR performance. In the experiment, we show that the proposed method outperforms previous image‐specific network and internal learning‐based methods.

Illumination normalization for face recognition is very important when a face is captured under harsh lighting conditions. Instead of designing hand-crafted features, in this paper we formulate face illumination normalization as an image-to-image translation task. A great challenge of face normalization is that human facial structures are particularly sensitive to image structure distortion, which frequently occurs in traditional image-to-image translation tasks. Unfortunately, sometimes even slight facial structure distortions may prohibit human eyes and machine face recognition methods from identifying face identities. To address this issue, a novel GAN- based network architecture called the asymmetric joint generative adversarial network (AJGAN) is developed to normalize face images under arbitrary illumination conditions, without known face geometry and albedo information. In addition, an illumination normalization GAN $G_1$ and an asymmetric relighting GAN $G_2$ that maps a frontal-illuminated image to images with various lighting conditions are incorporated in AJGAN to maintain personalized facial structures. To avoid image blurring caused by the under-constrained relighting mapping, we introduce a scheme of one-hot lighting labels into $G_2$ and enforce label classification loss. Furthermore, the number of training images starting from a very limited number of labels is dynamically extended by the combination of different lighting labels. Qualitative and quantitative experiments on three databases validate that AJGAN significantly outperforms the state-of-the-art methods.