Compositional Zero-Shot Learning (CZSL) aims to identify unseen state-object compositions by leveraging knowledge learned from seen compositions. Existing approaches often independently predict states and objects, overlooking their relationships. In this paper, we propose a novel framework, learning primitive relations (LPR), designed to probabilistically capture the relationships between states and objects. By employing the cross-attention mechanism, LPR considers the dependencies between states and objects, enabling the model to infer the likelihood of unseen compositions. Experimental results demonstrate that LPR outperforms state-of-the-art methods on all three CZSL benchmark datasets in both closed-world and open-world settings. Through qualitative analysis, we show that LPR leverages state-object relationships for unseen composition prediction.

This study investigates the sensitivity of radiative transfer (RT) simulations to cloud microphysics (MP) assumptions. Experimental RT simulations were used to investigate the impact of various particle size distribution (PSD) and density assumptions for both rainwater and frozen hydrometeors incorporated into the scattering module of Radiative Transfer Models (RTMs). Input atmospheric fields from Weather Research and Forecasting (WRF) model simulations with the Predicted Particle Properties single free-ice category (P31ICE) scheme were held fixed. A plane-parallel, fast RTM with the delta-Eddington approximation was employed to simulate brightness temperatures (TBs) from the Global Precipitation Measuring Mission Microwave Imager (GMI) for five typhoon scenes. Rainwater PSD assumptions had a relatively low impact on radiative signatures due to compensating extinction from small and large particles and the offset between emission and scattering. For frozen hydrometeors, the impacts of PSD assumptions were mitigated by particle-size-dependent scattering compensation and the canceling effect of forward scattering. Density assumptions had the largest impact on radiative signatures. Additionally, sensitivity to variations in atmospheric fields was observed in physically consistent reference RT simulations using WRF outputs with six different MP schemes. Interactions between MP assumptions in the RTM scattering module and those in the WRF model either amplified or suppressed TB responses, underscoring the importance of physical consistency across the RT framework. Comparisons with GMI observations reinforced the need to reconcile both physical realism and consistency throughout RT simulations. These findings provide guidance for improving the application of RTMs in all-sky data assimilation and satellite-based cloud property retrievals.

Accurately representing ice clouds in passive microwave radiative transfer models (RTMs) is considerably challenging as these clouds include numerous ice particles with complex shapes. Current RTMs often oversimplify this complexity by assuming spherical or singular non-spherical habits for these particles. This study improves the representation of ice clouds in RTMs, shifting from the oversimplified “one-shape-fits-all” approach to a more realistic approach describing the inhomogeneous distribution of ice habits. This improved representation is facilitated by the Predicted Particle Properties (P3) microphysics parameterization scheme, which provides natural variability of the ice-phase hydrometeors’ microphysical properties. The impact of this improved representation on microwave scattering is evaluated by comparing the simulated brightness temperatures (TBs) with actual measurements from the Global Precipitation Measurement (GPM) Microwave Imager (GMI) at scattering frequencies between 89 and 183 GHz, mainly focusing on tropical cyclone events in the Northwestern Pacific Ocean. The results show that the improved representation effectively describes the spatiotemporal variability of ice habits, improving the accuracy of TB simulations across the scattering channels. Moreover, investigations are conducted till the frequency of 664 GHz, emphasizing the potential importance of realistic ice habit distribution. Although some limitations exist, primarily relating to the model’s dependency on the P3 scheme and the limited range of the available ice habits, especially for rimed particles, this study takes a significant step toward improving the realism and accuracy of RTMs, providing a deeper understanding of ice clouds and their influence on RTMs.

Text-based motion generation models are drawing a surge of interest for their potential for automating the motion-making process in the game, animation, or robot industries. In this paper, we propose a diffusion-based motion synthesis and editing model named FLAME. Inspired by the recent successes in diffusion models, we integrate diffusion-based generative models into the motion domain. FLAME can generate high-fidelity motions well aligned with the given text. Also, it can edit the parts of the motion, both frame-wise and joint-wise, without any fine-tuning. FLAME involves a new transformer-based architecture we devise to better handle motion data, which is found to be crucial to manage variable-length motions and well attend to free-form text. In experiments, we show that FLAME achieves state-of-the-art generation performances on three text-motion datasets: HumanML3D, BABEL, and KIT. We also demonstrate that FLAME’s editing capability can be extended to other tasks such as motion prediction or motion in-betweening, which have been previously covered by dedicated models.

We propose a text-to-image generation algorithm based on deep neural networks when text captions for images are unavailable during training. In this work, instead of simply generating pseudo-ground-truth sentences of training images using existing image captioning methods, we employ a pretrained CLIP model, which is capable of properly aligning embeddings of images and corresponding texts in a joint space and, consequently, works well on zero-shot recognition tasks. We optimize a text-to-image generation model by maximizing the data log-likelihood conditioned on pairs of image-text CLIP embeddings. To better align data in the two domains, we employ a principled way based on a variational inference, which efficiently estimates an approximate posterior of the hidden text embedding given an image and its CLIP feature. Experimental results validate that the proposed framework outperforms existing approaches by large margins under unsupervised and semi-supervised text-to-image generation settings.

Generalized zero-shot learning (GZSL) is a technique to train a deep learning model to identify unseen classes using the image attribute. In this paper, we put forth a new GZSL technique exploiting Vision Transformer (ViT) to maximize the attribute-related information contained in the image feature. In ViT, the entire image region is processed without the degradation of the image resolution and the local image information is preserved in patch features. To fully enjoy the benefits of ViT, we exploit patch features as well as the CLS feature in the extraction of the attribute-related image feature. In particular, we propose a novel attention-based module, called attribute attention module (AAM), to aggregate the attribute-related information in the patch features. From extensive experiments on benchmark datasets, we demonstrate that the proposed technique outperforms the state-of-the-art GZSL approaches by a large margin.

Face-swapping models have been drawing attention for their compelling generation quality, but their complex architectures and loss functions often require careful tuning for successful training. We propose a new face-swapping model called ‘Smooth-Swap’, which excludes complex handcrafted designs and allows fast and stable training. The main idea of Smooth-Swap is to build smooth identity embedding that can provide stable gradients for identity change. Unlike the one used in previous models trained for a purely discriminative task, the proposed embedding is trained with a supervised contrastive loss promoting a smoother space. With improved smoothness, Smooth-Swap suffices to be composed of a generic U-Net-based generator and three basic loss functions, a far simpler design compared with the previous models. Extensive experiments on face-swapping benchmarks (FFHQ,$Face-Forensics++$) and face images in the wild show that our model is also quantitatively and qualitatively comparable or even superior to the existing methods.

Passive microwave radiative transfer models are strongly influenced by the cloud and precipitation hydrometeor properties. Particularly, they can sensitively interact with frozen hydrometeors through multiple high-frequency channels. However, frozen hydrometeors are one of the most difficult parameters to comprehend due to the lack ofin situdata. Until recently, studies have attempted to describe more reasonable hydrometeor distributions using various microphysics parameterizations coupled with the weather research and forecasting (WRF) models. Herein, we aim to apply the proposed methods to passive microwave radiative transfer simulations. We implemented a passive microwave radiative transfer simulation that considers various microphysical assumptions by creating a new Mie scattering lookup table. Furthermore, we evaluated the bulk microphysics parameterizations [WDM6, Morrison (MORR), Thompson (THOM), and P3 schemes] for the tropical cyclone Krosa (2019) that were observed by the global precipitation measurement microwave imager instrument, specifically concentrating on the rimed and aggregated ice categories (snow, graupel, and P3 ice). Based on the evaluation results, we concluded the following: WDM6 graupel and MORR snow afford excessive scattering signals at 37 GHz. However, at 166 GHz, none of the parameterizations produces sufficient scattering signals for comparison with the observations. The P3 ice affords significantly underestimated scattering signals at 89 GHz and above, despite its sophisticated assumptions. On the contrary, THOM snow affords scattering signals similar to the observations, despite a shape-related error. In summary, this study introduced a method for implementing a microphysical-consistent radiative transfer computation and successfully showed how various microphysical assumptions of clouds can change the passive microwave radiative signatures.

The accuracy of a physically based passive microwave precipitation retrieval algorithm is affected by the quality of the a priori knowledge it employs, which indicates the relationship between the precipitation information obtained from cloud-resolving models (CRMs) and the simulated brightness temperatures (TBs) from radiative transfer models. As various microphysical assumptions reflecting a wide variety of sophisticated microphysical properties are applied to the CRMs, the TBs simulated based on the model-driven 3-D precipitation fields are determined by the selected microphysical assumption. In this article, we developed a prototype precipitation retrieval algorithm that incorporates various cloud microphysics schemes in its a priori knowledge (i.e., databases). In the retrieval process, a specific a priori database is selected for every target precipitation scene by comparing the similarities of the simulated and observed microwave emission and scattering signatures. The prototype algorithm was tested through application to precipitation retrieval for tropical cyclones at various intensity stages, which occurred over the northwestern Pacific region in 2015. The a priori databases constructed using the weather research and forecasting double-moment (WDM6) and Thompson Aerosol Aware schemes are superior when used for weak-to-moderate rainfall systems, whereas the databases constructed with the other schemes are superior within strong rain rate regions. The retrieval results obtained using the best-performing database are generally superior for all rain rate regions. Furthermore, we confirm that the database quality is more important than the number of databases. In comparison with the data from the dual-precipitation radar, the retrieval's correlations, bias, and root mean square are 0.75, 0.14, and 5.62, respectively.

Satellite-based microwave sensors that respond to the vertical distribution of hydrometeors have been continuously employed in the investigation of precipitation systems characteristics. Rain/no-rain classification (RNC) methods often are either applied before retrieving precipitation information from a number of algorithms based on passive microwave measurements or adopted to build the precipitation event-based databases. As a simple rain indicator, the polarized corrected temperature (PCT) at 89-GHz (PCT89) method using the global precipitation measurement (GPM) microwave imager (GMI) has been employed by many researchers, because it can estimate the scattering intensity while minimizing the effects of the surface emissivity at high resolution. This article presents a new consideration using the PCT89-based RNC method through statistical verification. Precipitating clouds were subdivided into 11 types (three stratiform types and four convective types) by the GPM dual frequency precipitation radar (DPR) precipitation classification algorithm. Quantitative comparison of verification results was performed in the tropics from January to December 2015 and major sources of uncertainty were analyzed from the perspective of the precipitation mechanism. Results showed a tendency of false identification for stratiform types except for those located near the convective core, and thus the method was susceptible to failure in the identification of convective types. Consequently, this method leads to an increase of 70% and 54% in the number of two significant stratiform types compared to DPR, while the convective types decreased by up to 53%. This article suggests that the precipitations identified by the PCT89 have features that enhance the bias toward the stratiform type.