Conditional generative adversarial networks (cGANs) have dominated the research of synthetic aperture radar (SAR)-to-optical (S2O) image translation, attributing to the feature generalization ability of residual convolutional blocks. Nevertheless, the fixed geometric structures of convolutional kernels hinder the feature inference from local to global, resulting in unclear contours and missing details in the generated image. To address this challenge, we proposed a contextual inference transformer block in the generator, dubbed CoIT. It enables the network to capture key features in SAR images through context awareness, providing a more comprehensive feature representation from local details to global structure. The proposed CoIT block contextually encodes input keys to establish the relationship between SAR images and optical images, improving the quality of generated images. Experiments on the public datasets WHU-SEN-City and SEN12MS show that the proposed method not only achieves better visual effects but also makes certain progress in evaluation indicators.

Polar sea ice monitoring is essential for climate change analysis and ship navigation route security. The current urgent problem to be solved is how to build a unified sea ice parameter inversion framework to obtain abundant and robust ice charts. However, most machine learning methods are merely suitable for modeling sea ice classification tasks under a single data source. Thus, a multitask sea ice inversion approach is proposed to address this challenge. First, SAR data and microwave scanning radiometer-2 (AMSR-2) data are jointly employed for multitask inversion, e.g., sea ice concentration (SIC), the stages of sea ice development (SOD), and sea ice floe size (FLOE). Then, the proportional-integral-derivative (PID) controllers are introduced into a three branch convolutional neural network to parse detailed, context and boundary information of the sea ice scene, respectively. Finally, three classifiers assign the multilabels to each of tasks for the parameter inversion. Experiments conducted on the AI4Arctic Sea Ice dataset show that our proposal outperforms typical CNN-based methods.

Autoscanning of an unknown environment is the key to many AR/VR and robotic applications. However, autonomous reconstruction with both high efficiency and quality remains a challenging problem. In this work, we propose a reconstruction-oriented autoscanning approach, called ScanBot, which utilizes hierarchical deep reinforcement learning techniques for global region-of-interest (ROI) planning to improve the scanning efficiency and local next-best-view (NBV) planning to enhance the reconstruction quality. Given the partially reconstructed scene, the global policy designates an ROI with insufficient exploration or reconstruction. The local policy is then applied to refine the reconstruction quality of objects in this region by planning and scanning a series of NBVs. A novel mixed 2D-3D representation is designed for these policies, where a 2D quality map with tailored quality channels encoding the scanning progress is consumed by the global policy, and a coarse-to-fine 3D volumetric representation that embodies both local environment and object completeness is fed to the local policy. These two policies iterate until the whole scene has been completely explored and scanned. To speed up the learning of complex environmental dynamics and enhance the agent's memory for spatial-temporal inference, we further introduce two novel auxiliary learning tasks to guide the training of our global policy. Thorough evaluations and comparisons are carried out to show the feasibility of our proposed approach and its advantages over previous methods. Code and data are available at https://github.com/HezhiCao/Scanbot.