Precipitation nowcasting is a long-standing challenge due to the inherent unpredictability, which often lead to significant risks and damage. Traditional approaches that model nonlinear relationships between initial and future precipitation states often fail to accurately capture precipitation dynamics, including distribution and intensity patterns. Current data-driven methods are limited in their ability to represent the chaotic nature of precipitation without guidance from physical theory. To address this, we present Rainfusion, a generative model that integrates Prandtl’s mixing length theory from fluid dynamics with computer vision diffusion models. This integration accounts for nonlinear interactions between large-scale evolution and turbulent fluctuations in precipitation, generating physically plausible predictions. Rainfusion significantly improves forecasting skill on two benchmark dataset over the next 3 hours. Furthermore, we enhance Rainfusion with a control network trained on multi-source data, particularly lightning observations, enabling more accurate and controllable predictions of precipitation’s spatial-temporal patterns. Weather forecasters can utilize Rainfusion to guide predictions toward either growth or decay based on their domain expertise. Our approach advances precipitation nowcasting, offering a robust framework that bridges physical theory with modern deep learning techniques.

In the field of numerical weather prediction, fine-grained precipitation fields play a crucial role in forecasting and analyzing the spatial distribution and intensity of the precipitation. Historically, it is customary to employ the interpolation technique to downscale the low-resolution initial field output by assimilation systems, aligning with the requirements of a high-resolution forecasting model. Currently, data-driven deep learning methods offer novel solutions to address this challenge. In this letter, we propose a spatial downscaling algorithm for precipitation data generated from the North American Land Data Assimilation System (NLDAS), called Gaussian-perceptual-based generative adversarial network (GP-GAN). Specifically, the GP-GAN introduces a Siamese Gaussian-perceptual module (SGPM) which maps the data reconstructed from the generator and ground-truth to Gaussian latent space to learn the distribution of precipitation. Moreover, the adaptive weighted loss function (AWLF) is proposed to strengthen the emphasis and understanding of extreme precipitation events. Experimental results on the RainNet dataset comprising hourly precipitation over the USA demonstrate that GP-GAN provides better performance than other generative adversarial networks (GANs) and diffusion models in improving spatial resolution.

Quantitative precipitation nowcasting (QPN) is a highly challenging task in weather forecasting. The ability to provide precise, immediate, and detailed QPN products is necessary for a variety of situations, including storm warnings, air travel, and large gatherings. To address this challenge, this article proposes a new transformer lightweight physics-informed transformer (LPT)-QPN for QPN tasks, utilizing vertical cumulative liquid water content (VIL) products. This model adopts novel transformer modules to model the long-term evolution of precipitation and incorporates multihead squared attention (MHSA) to model its highly nonlinear relationships while reducing computational complexity. The results of experimental evaluations demonstrate the superiority of LPT-QPN when compared to existing state-of-the-art QPN models. In particular, the LPT-QPN model demonstrates greater accuracy for long lead time and in high-intensity areas, confirmed in both quantitative and qualitative evaluations. In addition, through three customized fine-tuning schemes, we are able to further improve the predictability of the LPT-QPN model for specific precipitation events. By incorporating the physical constraints of the convection-diffusion equation, our approach offers novel perspectives for future explorations that combine physical prior knowledge and deep-learning (DL) techniques.

We propose a SAR and Optical Fusion Network based on the UNet framework (SOF-UNet) for multi-modal land cover classification. The two-stream SOF-UNet consists of three parts: two encoders to extract features, a sharing decoder to upsample the feature maps and specially designed skip connections to fuse multi-modal features. The qualitative and quantitative experimental results show that SOF-UNet has a promising capability to identify different land cover classes and can retain fine details in the prediction maps. Symmetric Cross Entropy (SCE) loss is also verified useful in this framework.

The objective of this research is to realize automatic land cover classification from synthetic aperture radar (SAR) and multispectral remote sensing imagery. We develop a SAR-optical fusion network (SOFNet) with the symmetric cross entropy (SCE) loss to utilize both the SAR and optical information in a novel deep neural network. The proposed framework has been trained on the public SEN12MS dataset and tested on the 2020 IEEE-GRSS Data Fusion Contest (DFC2020) dataset. Experimental results show that our approach takes full advantage of multimodal information and outperforms the state-of-the-art convolutional architectures.

The objective of this research is to detect ocean eddies automatically on Synthetic Aperture Radar (SAR) images. We develop a new approach using Mask Region-based Convolutional Neural Networks (Mask R-CNN) and edge enhancement. First, we use Canny edge detector to extract a wide range of edges in SAR images. Then we put both the edge detection results and the corresponding original images into a Mask R-CNN based model for learning, thereby strengthening edge information. The proposed framework has been trained on a sample dataset of Sentinel-1A SAR-C imagery of the Western Mediterranean Sea. Experimental results revealed that the proposed method improved the performance by 2.3% on the MS COCO metrics compared to the method without edge enhancement.