The field of face sketch-to-photo synthesis involves generating photographic facial images with enhanced details and a heightened sense of style realism. In recent years, the advancement of deep learning techniques has significantly contributed to the development of methods for synthesizing photographic face images from sketches. Nevertheless, challenges remain in synthesizing facial photographs with richer details and more accurate structural representation. This paper introduces a novel architecture for face sketch-to-photo synthesis, using denoising diffusion probabilistic models (DDPM). Our approach simplifies the complex transformation process into sequential forward and backward denoising steps. We incorporate a pretrained coarse generator to effectively encode sketch information, integrating it into each backward step to guide the generative process toward accurate photo space representation. Furthermore, we design a detail diffusion branch to refine the coarse photo face generated from the coarse generator. By deeply fusing multiscale detail features from this branch with a sophisticated conditional noise predictor, our model effectively captures the correlation between detail and stylistic elements both in sketches and in photographic faces. Extensive experimental evaluations on three datasets show the effectiveness of our model, emphasizing its ability to synthesize facial photographs with remarkable realism and rich detail. The synthesized facial images consistently demonstrate superior face recognition accuracy, surpassing that of state-of-the-art methods.

Abstract Small object detection remains a bottleneck because there is little visual information about them, especially in the deep layers. To improve the detection performance of small objects, here, Swin Transformer is introduced as the model backbone network to extract rich features of small objects. Then, a multilevel receptive field expansion network (MRFENet) is proposed based on the characteristics of different stages in the Swin Transformer. Specifically, a receptive field expansion block (RFEB) is designed to acquire contextual cues and extract detailed information. The RFEB is carefully designed to target the required receptive fields of different layers and further refine the features. MRFENet combined with RFEBs implements the retention of small object context cues and the acquisition of receptive fields for the adaptive detection tasks. Finally, a union loss function is designed to enhance the localization ability. Experiments on the MS COCO dataset demonstrate that the proposed MRFENet has a significant improvement against other state‐of‐the‐art methods, which further validates that MRFENet can effectively utilize small object information.

Abstract Digital imaging devices sometimes capture images with abnormal exposure because of the complex lighting conditions and limited dynamic range of luminance. In this work, a new residual dense U‐Net is proposed to predict the information that has been lost in saturated image areas, to enable abnormal exposure restoration from a single image. Full advantage of the multi‐level features is taken from all the convolution layers in the restoration process. Specifically, the densely connected convolutional layers are used in a contracting encoder net to extract abundant local features. The transition layer and local residual learning after each dense block is then applied to adaptively learn more effectively from prior with present local features. Further, an expanding decoder net with dense layers is used and added with skip connections to preserve low‐level information and existing details. Finally, multiple global residual learning is used to adaptively extract hierarchical features and help train the network. It is shown that such a network can be trained end‐to‐end from abnormal exposure images and outperform the prior best method on image enhancement. Experimental results show that the proposed model can greatly enhance the dynamic range of an abnormal exposure image.

In rainy conditions, imaging devices often capture degraded and blurry images. Most existing works on this problem focus on rain streak removal, but these approaches cannot handle the various types of rain found in images. In this paper, we propose a robust rain removal method for use with single images using an attentive composite residual network. We put forth a single-to-dual encoder-decoder structure, which consists of an attentive net that identifies regions containing rain components during encoding, followed by a dual-channel architecture which recovers the background and detail components of the identified regions during decoding. For the detail subnet, we designed a novel building block, namely a composite residual block (CRB), by constructing multiple residual connections among Res2Net modules. Additionally, we designed another attentive-CRB for the attentive net that uses a squeeze-and-excitation (SE)-Res2Net module, to build a channel-wise attention mechanism. We show that such a deep network can be trained end-to-end from rainy images and that it outperforms the previous state-of-the-art methods on datasets containing different types of rainy images. Experimental results also demonstrate the proposed model's superiority over the competitor's on real rain-affected images, recovering visually clean images and retaining good detail.

Multimodal medical image fusion, emerging as a hot topic, aims to fuse images with complementary multi-source information. In this paper, we propose a novel multimodal medical image fusion method based on structural patch decomposition (SPD) and fuzzy logic technology. First, the SPD method is employed to extract two salient features for fusion discrimination. Next, two novel fusion decision maps called an incomplete fusion map and supplemental fusion map are constructed from salient features. In this step, the supplemental map is constructed by our defined two different fuzzy logic systems. The supplemental and incomplete maps are then combined to construct an initial fusion map. The final fusion map is obtained by processing the initial fusion map with a Gaussian filter. Finally, a weighted average approach is adopted to create the final fused image. Additionally, an effective color medical image fusion scheme that can effectively prevent color distortion and obtain superior diagnostic effects is also proposed to enhance fused images. Experimental results clearly demonstrate that the proposed method outperforms state-of-the-art methods in terms of subjective visual and quantitative evaluations.