We introduce LumiNet, a novel architecture that leverages generative models and latent intrinsic representations for transferring lighting from one image to another. Given a source image and a target lighting image, LumiNet generates a relit version of the source scene that captures the target’s lighting. Our approach makes two key contributions: a data curation strategy from the StyleGAN-based relighting model for our training, and a modified diffusion-based Con-trolNet that processes both latent intrinsic properties from the source image and latent extrinsic properties from the target image. We further improve lighting transfer through a learned adaptor that injects the target’s latent extrinsic properties via cross-attention and light-weight fine-tuning.Unlike traditional ControlNet, which generates images with conditional maps from a single scene, LumiNet processes latent representations from two different images -preserving geometry and albedo from the source while transferring lighting characteristics from the target. Experiments demonstrate that our method successfully transfers complex lighting phenomena including specular highlights and indirect illumination across scenes with varying spatial layouts and materials, outperforming existing approaches on challenging indoor scenes using only images as input.

Target identification is one of the crucial tasks in drug research and development, as it aids in uncovering the action mechanism of herbs/drugs and discovering new therapeutic targets. Although multiple algorithms of herb target prediction have been proposed, due to the incompleteness of clinical knowledge and the limitation of unsupervised models, accurate identification for herb targets still faces huge challenges of data and models. To address this, we proposed a deep learning-based target prediction framework termed HTINet2, which designed three key modules, namely, traditional Chinese medicine (TCM) and clinical knowledge graph embedding, residual graph representation learning, and supervised target prediction. In the first module, we constructed a large-scale knowledge graph that covers the TCM properties and clinical treatment knowledge of herbs, and designed a component of deep knowledge embedding to learn the deep knowledge embedding of herbs and targets. In the remaining two modules, we designed a residual-like graph convolution network to capture the deep interactions among herbs and targets, and a Bayesian personalized ranking loss to conduct supervised training and target prediction. Finally, we designed comprehensive experiments, of which comparison with baselines indicated the excellent performance of HTINet2 (HR@10 increased by 122.7% and NDCG@10 by 35.7%), ablation experiments illustrated the positive effect of our designed modules of HTINet2, and case study demonstrated the reliability of the predicted targets of Artemisia annua and Coptis chinensis based on the knowledge base, literature, and molecular docking.

In this paper, we present Point Cloud Color Constancy, in short PCCC, an illumination chromaticity estimation algorithm exploiting a point cloud. We leverage the depth information captured by the time-of-flight (ToF) sensor mounted rigidly with the RGB sensor, and form a 6D cloud where each point contains the coordinates and RGB intensities, noted as (x,y,z, r,g, b). PCCC applies the PointNet architecture to the color constancy problem, deriving the illumination vector point-wise and then making a global decision about the global illumination chromaticity. On two popular RGB-D datasets, which we extend with illumination information, as well as on a novel benchmark, PCCC obtains lower error than the state-of-the-art algorithms. Our method is simple andfast, requiring merely 16 x 16-size input and reaching speed over 140 fps (CPU time), including the cost of building the point cloud and net inference.

Illumination estimation refers to estimating the chromaticity vector of illumination, and can be used to recover the surface color under white light. Dual-illuminant is a common scenario in computational illumination estimation tasks. A straightforward way to correct the dual-illuminant image can be estimating a spatially-varying illumination map. However, it is hindered by the lack of large-scale annotated datasets for data-driven methods. In this paper, we propose a novel approach to obtain the dominant dual-illuminant and the pixel-wise illuminant map on real dual-illuminant raw images. Our method consists of 1) dual-illuminant image generator (DIG) to synthesize dual-illuminant images from unique-illuminant datasets assuming the Lambertian model; 2) dual-illuminant estimation network (DE-Net) to estimate illuminant both globally and locally. Quantitative experiments show that with DIG synthesized dual-illuminant images, DE-Net obtains the best accuracy in dual-illumination detection and estimation on the Gehalr-shi dataset and Mutlti-Illuminant Multi-Object dataset.