Despite significant advancements in remote sensing multimodal learning, particularly in image-image feature fusion, the exploration of audio-image feature fusion remains insufficient. Given the complexity and redundancy of ground objects in remote sensing images, accurately aligning audio features with image features during the fusion process is a critical challenge. In this paper, we introduce an object-centric feature fusion method named SlotFusion. By employing a slot attention-based feature decoupling module and a slot-based audiovisual feature fusion module, we transform modality features with complex semantic information into a set of slot features corresponding to object units and use gated activation units to adaptively implement object-centric feature fusion. Experiments on the Audio Visual Aerial Scene Recognition dataset (ADVANCE) demonstrate that the proposed SlotFusion significantly improves remote sensing scene recognition performance, with a 7.04% increase in overall accuracy compared to previous methods, achieving state-of-the-art results.

Creating fine-retouched portrait images is tedious and time-consuming even for professional artists. There exist automatic retouching methods, but they either suffer from over-smoothing artifacts or lack generalization ability. To address such issues, we present StyleRetoucher, a novel automatic portrait image retouching framework, leveraging StyleGAN's generation and generalization ability to improve an input portrait image's skin condition while preserving its facial details. Harnessing the priors of pretrained StyleGAN, our method shows superior robustness: a). performing stably with fewer training samples and b). generalizing well on the out-domain data. Moreover, by blending the spatial features of the input image and intermediate features of the StyleGAN layers, our method preserves the input characteristics to the largest extent. We further propose a novel blemish-aware feature selection mechanism to effectively identify and remove the skin blemishes, improving the image skin condition. Qualitative and quantitative evaluations validate the great generalization capability of our method. Further experiments show StyleRetoucher's superior performance to the alternative solutions in the image retouching task. We also conduct a user perceptive study to confirm the superior retouching performance of our method over the existing state-of-the-art alternatives.

Polarimetric synthetic aperture radar (PolSAR) image classification is a significant research area in the interpretation of PolSAR images. In recent years, PolSAR classification algorithms based on deep learning (DL) have been widely used, and achieved excellent performance in the case of sufficient training samples. However, due to the particularity of radar imaging, the scarcity of annotated data poses a challenge for PolSAR image classification based on DL. Contrastive learning (CL) is an effective approach to solve the problem of insufficient training samples. However, current CL methods have overlooked the utilization of the rich polarimetric information which is unique to PolSAR data. Thus, we propose a novel CL model to address the aforementioned issue. Firstly, inspired by the polarimetric scattering mechanism, the proposed method constructs hard positive samples combining polarimetric information. Then, a mutual learning regularizer (MLR) is used to design a loss function that bridges the semantic gap between different positive instances while introducing the hard positive samples. Finally, a transductive CL framework and gradient stop method are employed for pre-training, preventing model collapse and excessive computing requirements. The proposed method has been tested on two benchmark PolSAR datasets. On the same network backbone, the proposed method achieves up to 4.07% and 5.43% improvement in overall accuracy (OA) and kappa coefficient (KC), and up to 1.08% and 1.43% improvement compared to the the state-of-the-art methods. The experimental results demonstrate impressive performance on few-shot classification task of PolSAR image classification.

In recent years, significant research has been conducted on utilizing simulated data to support Synthetic Aperture Radar Automatic Target Recognition (SAR-ATR) based on deep learning techniques. By distilling the dark knowledge extracted from simulated samples, quality of the learned representations on measured samples can be effectively enhanced. However, our study highlights an important oversight in previous works: unquestioning trust on all simulated samples inevitably introduces the part of dark knowledge that is detrimental to SAR-ATR performance. To address this issue, we introduce evidential learning to estimate the confidence degree of the model after inputting simulated samples, thereby assessing the validity of the dark knowledge to be distilled. Then, the simulated-measured knowledge distillation process will be carried out in a trusted manner. Specifically, we encourage the model to prioritize distilling the dark knowledge with higher validity while avoiding the influence of inferior knowledge through a dynamic confidence weighting method. Additionally, we transform the standard logits-based knowledge distillation loss function into a feature-based one, giving the proposed method the ability to plug-and-play. The above aspects constitute the proposed trusted simulated-measured knowledge distillation method for SAR-ATR. Multiple comparative studies on the Simulated And Measured Paired Labeled Experiment (SAMPLE) dataset demonstrate the effectiveness of our proposed method, which not only achieves superior performance but also maintains the desired computational complexity in the inference phase.

Considering the distinct scattering characteristics observed in diverse frequency bands for terrain surfaces, the utilization of complementary information from multi-band PolSAR data proves advantageous in effectively distinguishing targets that may present challenges when assessed in isolation within a single band. In this study, we employ the polarimetric target decomposition methods to investigate the scattering characteristics of representative targets within the C and L bands. Based on these findings, we construct a random forest fine classification model specifically tailored for different types of urban vegetation. Notably, experimental results demonstrate a noteworthy enhancement in classification accuracy following the integration of multi-band data.

SAR observes only one side of the earth surface and tells different targets by slant range. As a result, in the SAR image some kinds of distortions appear which include compressing in the range direction, shadow, foreshortening and layover. Using the RD locating model, based on the DEM of the corresponding area, a simulated SAR image is produced. Then, get GCPS by registering the simulated SAR image and the original SAR image and transfrom the SAR image with the GCPS. In order to evaluate the effect of orthorectification in the related applications, the building areas are extracted followed by comparing the extraction results of the original image and the rectification image using the detection rate and false alarm rate. Finally it turns out that locating errors decrease and extraction of building areas is more precise.

Exemplar-based portrait stylization is widely attractive and highly desired. Despite recent successes, it remains challenging, especially when considering both texture and geometric styles. In this article, we present the first framework for one-shot 3D portrait style transfer, which can generate 3D face models with both the geometry exaggerated and the texture stylized while preserving the identity from the original content. It requires only one arbitrary style image instead of a large set of training examples for a particular style, provides geometry and texture outputs that are fully parameterized and disentangled, and enables further graphics applications with the 3D representations. The framework consists of two stages. In the first geometric style transfer stage, we use facial landmark translation to capture the coarse geometry style and guide the deformation of the dense 3D face geometry. In the second texture style transfer stage, we focus on performing style transfer on the canonical texture by adopting a differentiable renderer to optimize the texture in a multi-view framework. Experiments show that our method achieves robustly good results on different artistic styles and outperforms existing methods. We also demonstrate the advantages of our method via various 2D and 3D graphics applications.

Abstract Despite recent breakthroughs in deep learning methods for image lighting enhancement, they are inferior when applied to portraits because 3D facial information is ignored in their models. To address this, we present a novel deep learning framework for portrait lighting enhancement based on 3D facial guidance. Our framework consists of two stages. In the first stage, corrected lighting parameters are predicted by a network from the input bad lighting image, with the assistance of a 3D morphable model and a differentiable renderer. Given the predicted lighting parameter, the differentiable renderer renders a face image with corrected shading and texture, which serves as the 3D guidance for learning image lighting enhancement in the second stage. To better exploit the long‐range correlations between the input and the guidance, in the second stage, we design an image‐to‐image translation network with a novel transformer architecture, which automatically produces a lighting‐enhanced result. Experimental results on the FFHQ dataset and in‐the‐wild images show that the proposed method outperforms state‐of‐the‐art methods in terms of both quantitative metrics and visual quality.