Cross-domain recommendation (CDR) mitigates data sparsity and cold-start issues in recommendation systems. While recent CDR approaches using graph neural networks (GNNs) capture complex user-item interactions, they rely on manually designed architectures that are often suboptimal and labor-intensive. Additionally, extracting valuable behavioral information from source domains to improve target domain recommendations remains challenging. To address these challenges, we propose Behavior importance-aware Graph Neural Architecture Search (BiGNAS), a framework that jointly optimizes GNN architecture and data importance for CDR. BiGNAS introduces two key components: a Cross-Domain Customized Supernetwork and a Graph-Based Behavior Importance Perceptron. The supernetwork, as a one-shot, retrain-free module, automatically searches the optimal GNN architecture for each domain without the need for retraining. The perceptron uses auxiliary learning to dynamically assess the importance of source domain behaviors, thereby improving target domain recommendations. Extensive experiments on benchmark CDR datasets and a large-scale industry advertising dataset demonstrate that BiGNAS consistently outperforms state-of-the-art baselines. To the best of our knowledge, this is the first work to jointly optimize GNN architecture and behavior data importance for cross-domain recommendation.

The integration of Large Language Models (LLMs) with retrieval systems has shown promising potential in retrieving documents (docs) or advertisements (ads) for a given query.Existing LLM-based retrieval methods generate numeric or content-based DocIDs to retrieve docs or ads.However, the one-to-few mapping between numeric IDs and docs, along with the time-consuming content extraction, leads to semantic inefficiency and limits the scalability of existing methods on large-scale corpora.In this paper, we propose the Realtime Ad REtrieval (RARE) framework, which leverages LLM-generated text called Commercial Intentions (CIs) as an intermediate semantic representation to directly retrieve ads for queries in real-time.These CIs are generated by a customized LLM injected with commercial knowledge, enhancing its domain relevance.Each CI corresponds to multiple ads, yielding a lightweight and scalable set of CIs.RARE has been implemented in a real-world online system, handling daily search volumes in billions.The online implementation has yielded significant benefits: a 5.04% increase in consumption, a 6.37% increase in gross merchandise volume (GMV), a 1.28% enhancement in click-through rate (CTR) and a 5.29% increase in shallow conversions.Extensive offline experiments show RARE's superiority over ten competitive baselines in four major categories.

Cross-domain recommendation (CDR) has emerged as a promising direction for expanding the applicability of recommendation systems. Recent advances in CDR have demonstrated the effectiveness of the unified distribution paradigm, which leverages shared distributions to transfer knowledge across domains and employs domain-specific adapters for targeted recommendations. While this well-designed paradigm shows promising performance, existing methods require extra supervision signals (e.g. contrastive learning on domain-masked embeddings) to maintain unified distributions across domains, leading to an inherent trade-off between unified objectives and domain-specific preference modeling. To address these limitations, we propose CD-CDR (Conditional Diffusion-CDR), a novel approach that leverages a shared conditional diffusion model to learn unified item distributions and facilitate knowledge transfer across domains. The key insight is to utilize the powerful generative capabilities of diffusion models to learn a shared distribution while naturally incorporating domain-specific characteristics through conditional generation. This design enables CD-CDR to replace traditional adapters with generation conditions as an integral part of the distribution model, thereby eliminating extra supervision signals and fundamentally resolving the trade-off between unified and domain-specific objectives. Extensive experiments on six domain pairs from two real-world datasets demonstrate that CD-CDR significantly outperforms existing methods for both normal and cold-start settings. To the best of our knowledge, this is the first work to explore the unified distribution paradigm in CDR using conditional diffusion models.

With the development of multi-modal modeling techniques, recent sequential recommender systems enhance transferability by incorporating cross-domain universal multi-modal data, e.g., text and image. Existing methods typically adopt pairwise alignment to alleviate the gap between modalities. However, this alignment paradigm has limitations on explainability, consistency, and expansibility, resulting in suboptimal performance. This paper proposes a novel Explainable multi-modality Alignment method for transferable Rec ommender systems, i.e., EARec. Specifically, we design a two-stage framework to achieve explainable modality alignment in the source domain and recommendation based on aligned modality representations in the target domain. In the first stage, we adopt a generative task to align various modalities in parallel to a shared anchor with explainable meaning. All modalities share the same anchor to ensure consistent direction. Additionally, we treat behavior as an independent modality to integrate task-specific information into the alignment framework. In the second stage, we compose multiple item modality representation models trained in the first stage to obtain a unified model capable of understanding various modalities simultaneously, thereby providing high-quality item modality representations for recommendations in the target domain. Benefiting from the approach of parallel modality alignment followed by model composition, the framework shows flexibility in expanding new modalities. Experimental results on multiple public datasets demonstrate the superiority of EARec over baselines, and further analyses indicate the explainability and expansibility of the proposed alignment method.

The quality of features plays an important role in the performance of recommender systems. Recognizing this, feature selection has emerged as a crucial technique in refining recommender systems. Recent advancements leveraging Automated Machine Learning (AutoML) has drawn significant attention, particularly in two main categories: early feature selection and late feature selection, differentiated by whether the selection occurs before or after the embedding layer. The early feature selection selects a fixed subset of features and retrains the model, while the late feature selection, known as adaptive feature selection, dynamically adjusts feature choices for each data instance, recognizing the variability in feature significance. Although adaptive feature selection has shown remarkable improvements in performance, its main drawback lies in its post-embedding layer feature selection. This process often becomes cumbersome and inefficient in large-scale recommender systems with billions of ID-type features, leading to a highly sparse and parameter-heavy embedding layer. To overcome this, we introduce Adaptive Early Feature Selection (AEFS), a very simple method that not only adaptively selects informative features for each instance, but also significantly reduces the activated parameters of the embedding layer. AEFS employs a dual-model architecture, encompassing an auxiliary model dedicated to feature selection and a main model responsible for prediction. To ensure effective alignment between these two models, we incorporate two collaborative training loss constraints. Our extensive experiments on three benchmark datasets validate the efficiency and effectiveness of our approach. Notably, AEFS matches the performance of current state-of-theart Adaptive Late Feature Selection methods while achieving a significant reduction of 37. 5% in the activated parameters of the embedding layer. We believe that this work opens up new possibilities for feature selection.

Large-scale recommender systems usually contain hundreds of multi-valued feature fields, which have different number of values in each field. For easier computation in traditional fixed-shape neural networks, pooling operators are widely used to compress the multi-valued feature into a fixed-dimension vector. Most existing works set a single pooling method for all fields, but this leads to sub-optimal results, because different feature fields have different information distributions and thus require different pooling methods. In this work, we propose an AutoML-based framework, called AutoPooling, which can automatically and efficiently search for the optimal pooling operator for each multivalued feature. Specifically, learnable weights are assigned to all candidate pooling operators in each feature field. Then an AutoML-based algorithm is used to learn both model parameters and the field-aware weights. Finally, the optimal pooling operator can be acquired based on the associated weights. We evaluate the proposed framework on both public and industrial datasets. The results show that AutoPooling significantly outperforms the benchmarks. Further experiment results show that our method is robust in various deep recommend models and different search spaces.

The Earth Polychromatic Camera (EPIC) onboard the Deep Space Climate Observatory (DSCOVR) spacecraft observes the sunlit face of the Earth from a distance of about one-and-a-half million kilometers. Several studies demonstrated that EPIC images often feature sun glint from water surfaces and from horizontally oriented ice crystals occurring inside clouds. This study presents a statistical analysis of a yearlong EPIC dataset to gain insights into sun glints and their impacts on satellite measurements of cloudiness and cloud properties. The first results discussed demonstrate that over land, the observed glints, indeed, come mainly from ice clouds and not from small water bodies. Subsequent results reveal that sun glints affecting EPIC observations (especially at 388 nm) greatly increase the likelihood and sensitivity of cloud detection, particularly of the elusive thin and small ice clouds. Finally, the results indicate that sun glints often cause spurious increases in the cloud optical thickness (COT) and altitude values in the operational EPIC cloud product. Insights into the frequency, magnitude, and causes of glint effects and suggestions for future work are also provided.

Feature selection is crucial in large-scale recommendation system, which can not only reduce the computational cost, but also improve the recommendation efficiency. Most existing works rank the features and then select the top-k ones as the final feature subset. However, they assess feature importance individually and ignore the interrelationship between features. Consequently, multiple features with high relevance may be selected simultaneously, resulting in sub-optimal result. In this work, we solve this problem by proposing an AutoML-based feature selection framework that can automatically search the optimal feature subset. Specifically, we first embed the search space into a weight-sharing Supernet. Then, a two-stage neural architecture search method is employed to evaluate the feature quality. In the first stage, a well-designed sampling method considering feature convergence fairness is applied to train the Supernet. In the second stage, a reinforcement learning method is used to search for the optimal feature subset efficiently. The Experimental results on two real datasets demonstrate the superior performance of new framework over other solutions. Our proposed method obtain significant improvement with a 20% reduction in the amount of features on the Criteo. More validation experiments demonstrate the ability and robustness of the framework.

Overfitting is a common problem in machine learning, which means the model too closely fits the training data while performing poorly in the test data. Among various methods of coping with overfitting, dropout is one of the representative ways. From randomly dropping neurons to dropping neural structures, dropout has achieved great success in improving model performances. Although various dropout methods have been designed and widely applied in past years, their effectiveness, application scenarios, and contributions have not been comprehensively summarized and empirically compared by far. It is the right time to make a comprehensive survey. In this paper, we systematically review previous dropout methods and classify them into three major categories according to the stage where dropout operation is performed. Specifically, more than seventy dropout methods published in top AI conferences or journals (e.g., TKDE, KDD, TheWebConf, SIGIR) are involved. The designed taxonomy is easy to understand and capable of including new dropout methods. Then, we further discuss their application scenarios, connections, and contributions. To verify the effectiveness of distinct dropout methods, extensive experiments are conducted on recommendation scenarios with abundant heterogeneous information. Finally, we propose some open problems and potential research directions about dropout that worth to be further explored.

We present Distributed Equivalent Substitution (DES) training, a novel distributed training framework for large-scale recommender systems with dynamic sparse features. DES introduces fully synchronous training to large-scale recommendation system for the first time by reducing communication, thus making the training of commercial recommender systems converge faster and reach better CTR. DES requires much less communication by substituting the weights-rich operators with the computationally equivalent sub-operators and aggregating partial results instead of transmitting the huge sparse weights directly through the network. Due to the use of synchronous training on large-scale Deep Learning Recommendation Models (DLRMs), DES achieves higher AUC(Area Under ROC). We successfully apply DES training on multiple popular DLRMs of industrial scenarios. Experiments show that our implementation outperforms the state-of-the-art PS-based training framework, achieving up to 68.7% communication savings and higher throughput compared to other PS-based recommender systems.

Although attention-based Neural Machine Translation (NMT) has achieved remarkable progress in recent years, it still suffers from issues of repeating and dropping translations. To alleviate these issues, we propose a novel key-value memory-augmented attention model for NMT, called KVMEMATT. Specifically, we maintain a timely updated keymemory to keep track of attention history and a fixed value-memory to store the representation of source sentence throughout the whole translation process. Via nontrivial transformations and iterative interactions between the two memories, the decoder focuses on more appropriate source word(s) for predicting the next target word at each decoding step, therefore can improve the adequacy of translations. Experimental results on Chinese)English and WMT17 German,English translation tasks demonstrate the superiority of the proposed model.

Cloud detection/screening is a fundamental step in satellite data analysis. For the Ice, Cloud, and land Elevation Satellite (ICESat) and its successor ICESat-2, clouds can significantly affect the accuracy of the surface elevation retrievals. This paper proposes a new method for cloud screening in support of the ICESat-2 mission with focus on the polar ice sheet regions. The method utilizes the apparent surface reflectance (ASR) at the backscattering direction as the cloud screening test. The basis of this method is that clouds produce a strong signal by significantly decreasing the ASR. We show that depending on the height and microphysics of the cloud, the ASR decreases 8%–17% for clouds with an optical depth of 0.1 and 57%–85% for clouds with an optical depth 1.0. Data from ICESat's 1064-nm channel is used to demonstrate the feasibility of the method. It is shown that cloud detectability is a function of surface reflectance variability. Generally, the smaller the surface reflectance variability, the more accurate is cloud detection. Unlike ICESat, which used a 1064-nm laser, ICESat-2 adopts a 532-nm photon counting system for its laser altimeter. With both modeling studies and results from the Moderate Resolution Imaging Spectroradiometer (MODIS), we demonstrate that the ASR variability is much smaller for the 532-nm channel than that for the 1064-nm channel. Hence, the 532-nm channel is better suited for cloud screening than the 1064-nm channel.

This paper establishes a framework that simulates the behavior of a spaceborne 532-nm micropulse photon-counting lidar in cloudy and clear atmospheres in support of the ICESat-2 mission. Adopted by the current mission design, the photon-counting system will be used to obtain surface altimetry for ICESat-2. To investigate how clouds affect surface elevation retrievals, a 3-D Monte Carlo radiative transfer model is used to simulate the photon path distribution and the Poisson distribution is adopted for the number of photon returns. Since the photon-counting system only registers the time of the first arriving photon within the detector “dead time,” the retrieved average surface elevation tends to bias toward higher values. This is known as the first photon bias. With the scenarios simulated here, the first photon bias for clear sky is about 6.5 cm. Clouds affect surface altimetry in two ways: 1) Cloud attenuation lowers the average number of arriving photons and hence reduces the first photon bias, and 2) cloud forward scattering increases the photon path length and makes the surface appear further away from the satellite. Compared with that for clear skies, the average surface elevation detected by the photon-counting system for cloudy skies with optical depth of 1.0 is 4.0-6.0 cm lower for the simulations conducted. The effect of surface roughness on the accuracy of elevation retrievals is also discussed.

In support of the Ice, Cloud, and land Elevation Satellite (ICESat)-II mission, this paper studies the bias in surface-elevation measurements caused by undetected thin clouds. The ICESat-II satellite may only have a 1064-nm single-channel lidar onboard. Less sensitive to clouds than the 532-nm channel, the 1064-nm channel tends to miss thin clouds. Previous studies have demonstrated that scattering by cloud particles increases the photon-path length, thus resulting in biases in ice-sheet-elevation measurements from spaceborne lidars. This effect is referred to as atmospheric path delay. This paper complements previous studies in the following ways: First, atmospheric path delay is estimated over the ice sheets based on cloud statistics from the Geoscience Laser Altimeter System onboard ICESat and the Moderate Resolution Imaging Spectroradiometer (MODIS) onboard Terra and Aqua. Second, the effect of cloud particle size and shape is studied with the state-of-the-art phase functions developed for MODIS cirrus-cloud microphysical model. Third, the contribution of various orders of scattering events to the path delay is studied, and an analytical model of the first-order scattering contribution is developed. This paper focuses on the path delay as a function of telescope field of view (FOV). The results show that reducing telescope FOV can significantly reduce the expected path delay. As an example, the average path delays forFOV= 167 ¿rad (a 100-m-diameter circle on the surface) caused by thin undetected clouds by the 1064-nm channel over Greenland and East Antarctica are illustrated.