The dynamic nature of microservice architectures necessitates robust few-shot anomaly detection (AD) systems to prevent cascading failures. However, current multimodal approaches remain inadequate for cold-start scenarios due to their dependence on extensive training data. The fundamental challenge in few-shot AD lies in two critical aspects: (1) effective feature extraction from intra-modalities, and (2) the alignment of heterogeneous multimodal representations, particularly the distribution discrepancy between discrete token embeddings from logs and continuously evolving representations from traces and metrics. To overcome these challenges, we present FuseGuard, an LLM-enhanced framework that pioneers quality-aware multimodal fusion for few-shot AD. First, our intra-modal feature extraction employs a Dynamic Sparse Correlation Attention mechanism for metrics, Spatio-temporal Encoding for traces, and LLM-enhanced semantic-frequency encoding for logs, enabling adaptive representation of time-evolving patterns. Second, we introduce hierarchical cross-modal alignment via token projection, layer-wise distillation, and consistency learning to preserve modality-specific features while enabling effective knowledge transfer. Finally, a quality-aware fusion mechanism dynamically weights modalities based on uncertainty estimation. Evaluated on three open-source platforms (MSDS, GAIA, TrainTicket) and a production system, FuseGuard's few-shot capability outperforms SOTA by 19.69%-77.07% F1-score across four datasets.

Graph neural networks (GNNs) have proven to be powerful tools for learning from graph-structured data and have achieved great success in many applications. As the sizes of real-world graphs continue to grow, traditional GNN training methods face significant scalability challenges. Recently, disks have gained attention as a cost-effective solution to store large-scale graphs, and several disk-based GNN systems have been proposed to train large-scale graphs on a single machine. However, these systems either overlook the unique data characteristics of GNN workloads when designing cache plans or fail to fully exploit the multilevel hierarchy of storage and computation in system execution, thus resulting in disk I/O bottleneck and resource under-utilization. To address these issues, we present CaliEX, an advanced disk-based GNN system that employs joint optimizations of caching and execution within and across different training stages. CaliEX first designs tailored cache plans and execution policy for both graph topology and features to accelerate neighborhood sampling and feature gathering. Since these two training stages work on different types of data, CaliEX further auto-tunes the cache allocation and pipelines the execution across different stages to improve resource utilization and overall training throughput. Evaluations on multiple GNN models and various large-scale datasets show that CaliEX achieves 3.28 × speedup on average compared to existing disk-based GNN training systems.

Microservices architecture has become increasingly popular in modern software systems, yet its complexity also poses challenges for failure diagnosis. Existing literature has deficiencies in multimodal data mining, mainly in two aspects: first, the mining of multimodal data is insufficient, failing to fully exploit the rich information contained in different types of data; second, the analysis of spatiotemporal features of multimodal data is not thorough, failing to fully explore the potential associations of data in the temporal and spatial dimensions. To address these issues, this paper proposes a new method for anomaly detection in microservices. We transform three types of modal data into the wavelet domain and achieve fusion among modalities based on an attention mechanism, fully utilizing the inherent characteristics of multimodal data in the frequency and temporal domains. This fusion process can extract new information that does not exist in the original modalities, thereby enhancing the model's ability to detect anomalies. In addition, we propose a new diffusion probabilistic model (DDPM) based on spatiotemporal graphs, which combines spatiotemporal learning capabilities with the uncertainty measurement of DDPM to generate future samples in a non-autoregressive manner, achieving multi-horizon prediction. Experimental results show that our method significantly improves anomaly detection performance on three public datasets and a real-world production system, demonstrating its effectiveness in microservices anomaly detection.

With the rise of e-commerce and short videos, online recommender systems that can capture users' interests and update new items in real-time play an increasingly important role. In both online and offline recommendation systems, the cold-start problem caused by interaction sparsity has been impacting the effectiveness of recommendations for cold-start items. Many cold-start scheme based on fine-tuning or knowledge transferring shows excellent performance on offline recommendation. Yet, these schemes are infeasible for online recommendation on streaming data pipelines due to different training method, computational overhead and time constraints. Inspired by the above questions, we propose a model-agnostic recommendation algorithm called Popularity-Aware Meta-learning (PAM), to address the item cold-start problem under streaming data settings. PAM divides the incoming data into different meta-learning tasks by predefined item popularity thresholds. The model can distinguish and reweight behavior-related and content-related features in each task based on their different roles in different popularity levels, thus adapting to recommendations for cold-start samples. These task-fixing design significantly reduces additional computation and storage costs compared to offline methods. Furthermore, PAM also introduced data augmentation and an additional self-supervised loss specifically designed for low-popularity tasks, leveraging insights from high-popularity samples. This approach effectively mitigates the issue of inadequate supervision due to the scarcity of cold-start samples. Experimental results across multiple public datasets demonstrate the superiority of our approach over other baseline methods in addressing cold-start challenges in online streaming data scenarios.

E-commerce recommendation platforms often operate across multiple domains, giving rise to Multi-Domain Recommendation (MDR) methods. These methods leverage auxiliary information from other domains to capture comprehensive user preferences and enhance recommendation accuracy. However, cross-domain knowledge transfer often causes domains to capture similar preferences. Additionally, most MDR models optimize for next-item prediction, which promotes greedy strategies focused on immediate gains within individual domains while neglecting the platform-wide profit. These two factors result in vicious competition, as our empirical analysis reveals: Domains recommend overlapping item lists, competing for the same pool of user attention and platform traffic, reducing the overall orders. To address this, we propose the Direct Preference Optimization from Manual Rules (DPOMR) training paradigm for MDR. Inspired by Reinforcement Learning from Human Feedback (RLHF), DPOMR learns user preferences from pairwise preference data generated through manual rules rooted in business expertise. Furthermore, we design a variant of Direct Preference Optimization (DPO) to bypass expensive RL training. Extensive experiments demonstrate the advantages of DPOMR, including: (1) achieving SOTA performance in boosting overall orders(2) showing compatibility with various backbones, and (3) offering robust extendibility to integrate diverse manual rules tailored to specific business needs.

In modern recommendation systems, leveraging multiple types of user-item interaction behaviors (e.g., click, add-to-cart, and purchase) presents both advantages and challenges. Recent studies organize multi-behavior data into heterogeneous bipartite graphs and used graph neural networks to learn latent representations. However, these methods struggle to model higher-order interactions and capture complex dependencies across various behaviors. In this paper, we propose a novel graph construction method that converts multi-behavior interactions into dual star-expansion hypergraphs by introducing a new type of node called hypernode. Subsequently, we develop StarRec, a hypergraph-based framework for multi-behavior recommendation. StarRec utilizes a spatial-based two-stage intra-behavior message passing strategy and a cross-behavior propagation layer to accurately and efficiently propagate information, modeling both high-order and cross-behavior relationships through the hypergraphs. This approach yields comprehensive representations that enhance recommendation performance. Experimental results on two real-world datasets demonstrate the superiority of StarRec. Our extensive experiments show that StarRec significantly outperforms state-of-the-art methods while maintaining competitive model scalability.

To address the business needs of industrial recommendation systems, an increasing number of Multi-Domain Recommendation (MDR) methods are designed to improve recommendation performance on multiple domains simultaneously. Most MDR methods follow a multi-task learning paradigm, suffering from poor deployability and negative transfer. Due to the great success of large pre-trained models, the pre-train & fine-tune paradigm is attracting increasing attention. The latest methods introduce parameter-efficient fine-tuning techniques like prompt-tuning, showcasing high efficiency and effectiveness. However, these methods neglect the fundamental differences between recommendation and NLP tasks. The inadequate capacity of recommendation models restricts the effectiveness of prompts and adapters. Worse still, traditional natural domain division may group non-identically distributed samples into the same domain, violating the assumption of independent and identically distributed (i.i.d.) data. In this paper, we propose MultiLoRA, a Multi-directional Low Rank Adaptation paradigm for multi-domain recommendation. First we pre-train a universal model using all data samples. Then we conduct multiple domain divisions on the sample space. Under each division, we fine-tune the pre-trained model to obtain a set of domain-specific LoRAs. Finally, we learn a LoRA fusion module to integrate domain-specific preference patterns across multiple divisions. Experimental results on real-world datasets demonstrate notable advantages of MultiLoRA: (1) achieving SOTA performance, (2) showcasing remarkable compatibility, and (3) proving highly efficient, featuring only 2% trainable parameters compared to the backbone.

Cross-Domain Recommendation (CDR) is a promising technique to alleviate data sparsity by transferring knowledge across domains. However, the negative transfer issue in the presence of numerous domains has received limited attention. Most existing methods transfer all information from source domains to the target domain without distinction. This introduces harmful noise and irrelevant features, resulting in suboptimal performance. Although some methods decompose user features into domain-specific and domain-shared components, they fail to consider other causes of negative transfer. Worse still, we argue that simple feature decomposition is insufficient for multi-domain scenarios. To bridge this gap, we propose TrineCDR, the TRIple-level kNowledge transferability Enhanced model for multi-target CDR. Unlike previous methods, TrineCDR captures single domain and targeted cross-domain embeddings to serve multi-domain recommendation. For the latter, we identify three fundamental causes of negative transfer, ranging from micro to macro perspectives, and correspondingly enhance knowledge transferability at three different levels: the feature level, the interaction level, and the domain level. Through these efforts, TrineCDR effectively filters out noise and irrelevant information from source domains, leading to more comprehensive and accurate representations in the target domain. We extensively evaluate the proposed model on real-world datasets, sampled from Amazon and Douban, under both dual-target and multi-target scenarios. The experimental results demonstrate the superiority of TrineCDR over state-of-the-art cross-domain recommendation methods.

The item cold-start problem is crucial for online recommender systems, as the success of the cold-start phase determines whether items can transition into popular ones. Prompt learning, a powerful technique used in natural language processing (NLP) to address zero- or few-shot problems, has been adapted for recommender systems to tackle similar challenges. However, existing methods typically rely on content-based properties or text descriptions for prompting, which we argue may be suboptimal for cold-start recommendations due to 1) semantic gaps with recommender tasks, 2) model bias caused by warm-up items contribute most of the positive feedback to the model, which is the core of the cold-start problem that hinders the recommender quality on cold-start items. We propose to leverage high-value positive feedback, termed pinnacle feedback as prompt information, to simultaneously resolve the above two problems. We experimentally prove that compared to the content description proposed in existing works, the positive feedback is more suitable to serve as prompt information by bridging the semantic gaps. Besides, we propose item-wise personalized prompt networks to encode pinnaclce feedback to relieve the model bias by the positive feedback dominance problem. Extensive experiments on four real-world datasets demonstrate the superiority of our model over state-of-the-art methods. Moreover, PROMO has been successfully deployed on a popular short-video sharing platform, a billion-user scale commercial short-video application, achieving remarkable performance gains across various commercial metrics within cold-start scenarios.

Click-through rate (CTR) prediction, whose purpose is to predict the probability of a user clicking on an item, plays a pivotal role in recommender systems. Capturing users’ accurate preferences from their historical interactions (e.g., clicks) is an essential step for handling this task and has aroused wide concern in both academia and industry. However, most of the previous methods focus on the users with abundant clicks and ill-serve the users who rarely click or purchase items. Though the ratio of these long-tailed users may be small on popular platforms, such as Amazon and Taobao, they are the majority on the newborn e-commerce company like Lazada. To extract the interests of long-tailed users, several works attempt to integrate the side information, such as demographic features. Nevertheless, these features are usually not available and may even lead to privacy concerns. Therefore, how to utilize the noisy and limited clicks becomes the key challenge.In this paper, we propose a novel model called Hierarchical Interest Modeling (HIM). It hierarchically utilizes long-tailed users’ limited behaviors and captures their preferences from both personalized and group-wise perspectives. HIM consists of two main components, including User Behavior Pyramid (UBP) and User Behavior Clustering (UBC). The UBP module utilizes additional negative feedback to reduce the noises in positive feedback, thus obtaining reliable user personalized representations. Then, the UBC module automatically discovers latent user groups with self-supervised reconstruction loss and learns another interest representation for each user in a group-wise aspect. Extensive experiments on both public and industrial datasets verify the superiority of HIM compared with the state-of-the-art baselines. Moreover, HIM has already been deployed on Lazada recommendation scenario and gains 3.38% on CTR prediction on average on the online A/B test. Our codes are available in https://github.com/xiaojin-nj/HIM.

Recommender systems across many Internet services have become a critical part of online businesses, as consumers would refer to them before making decisions. However, the lack of explicit ratings for items on many services makes it challenging to capture user preferences and item characteristics. Both academia and the industry have drawn attention to rating predications as a fundamental problem in recommendation systems. With the emergence of social networks, social recommender systems have been proposed to utilize the relationship between users and items to alleviate the data sparsity problem for rating predictions. However, they either concentrate on the opinion mining for each user and item, or consider the connections between users only. In this paper, we present an effective framework, Triple-hierarchical Attention Graph-based social rating prediction (TAG), to exploit the social relationships between users, the user-item interest relationships, the correlation relationships between items, and reviews for rating predictions. In order to consider opinions from reviews and these complex relationships, we first employ two triple-hierarchical attention to extract user and item features from reviews. We then design an inductive GNN, which generates effective embedding for users and items. Experiments over Yelp show that TAG outperforms state-of-the-art methods across RMSE, MAE, and NDCG metrics.

On-demand food delivery service has widely served people's daily demands worldwide, e.g., customers place over 40 million online orders in Meituan food delivery platform per day in Q3 of 2021. Predicting the food preparation time (FPT) of each order accurately is very significant for the courier and customer experience over the platform. However, there are two challenges, namely incomplete label and huge uncertainty in FPT data, to make the prediction of FPT in practice. In this paper, we apply probabilistic forecasting to FPT for the first time and propose a non-parametric method based on deep learning. Apart from the data with precise label of FPT, we make full use of the lower/upper bound of orders without precise label, during feature extraction and model construction. A number of categories of meaningful features are extracted based on the detailed data analysis to produce sharp probability distribution. For probabilistic forecasting, we propose S-QL and prove its relationship with S-CRPS for interval-censored data for the first time, which serves the quantile discretization of S-CRPS and optimization for the constructed neural network model. Extensive offline experiments over the large-scale real-world dataset, and online A/B test both demonstrate the effectiveness of our proposed method.

Federated learning (FL) is an emerging, privacy-preserving machine learning paradigm, drawing tremendous attention in both academia and industry. A unique characteristic of FL is heterogeneity, which resides in the various hardware specifications and dynamic states across the participating devices. Theoretically, heterogeneity can exert a huge influence on the FL training process, e.g., causing a device unavailable for training or unable to upload its model updates. Unfortunately, these impacts have never been systematically studied and quantified in existing FL literature.

Traffic flow forecasting is of great significance for improving the efficiency of transportation systems and preventing emergencies. Due to the highly non-linearity and intricate evolutionary patterns of short-term and long-term traffic flow, existing methods often fail to take full advantage of spatial-temporal information, especially the various temporal patterns with different period shifting and the characteristics of road segments. Besides, the globality representing the absolute value of traffic status indicators and the locality representing the relative value have not been considered simultaneously. This paper proposes a neural network model that focuses on the globality and locality of traffic networks as well as the temporal patterns of traffic data. The cycle-based dilated deformable convolution block is designed to capture different time-varying trends on each node accurately. Our model can extract both global and local spatial information since we combine two graph convolutional network methods to learn the representations of nodes and edges. Experiments on two real-world datasets show that the model can scrutinize the spatial-temporal correlation of traffic data, and its performance is better than the compared state-of-the-art methods. Further analysis indicates that the locality and globality of the traffic networks are critical to traffic flow prediction and the proposed TSSRGCN model can adapt to the various temporal traffic patterns.

Abstract The advances of mobile equipment and localization techniques put forward the accuracy of the location-based service (LBS) in mobile networks. One core issue for the industry to exploit the economic interest of the LBSs is to make appropriate point-of-interest (POI) recommendation based on users’ interests. Today, the LBS applications expect the recommender systems to recommend the accurate next POI in an anonymous manner, without inquiring users’ attributes or knowing the detailed features of the vast number of POIs. To cope with the challenge, we propose a novel attentive model to recommend appropriate new POIs for users, namely Geographical Attentive Recommendation via Graph (GARG), which takes full advantage of the collaborative, sequential and content-aware information. Unlike previous strategies that equally treat POIs in the sequence or manually define the relationships between POIs, GARG adaptively differentiates the relevance of POIs in the sequence to the prediction, and automatically identifies the POI-wise correlation. Extensive experiments on three real-world datasets demonstrate the effectiveness of GARG and reveal a significant improvement by GARG on the precision, recall and mAP metrics, compared to several state-of-the-art baseline methods.

Benefited from rapid developments of deep learning, 3D shape recognition has become a remarkable subject in computer vision systems.The existing methods of multi-perspective views have shown competitive performance in 3D shape recognition.However, they have not yet fully exploited the information among all views of projection.In this paper, we propose a novel Multi-view Moments Embedding Network(MMEN) for capturing multiple moments information.MMEN obtains the similarity between different views and retains the description of the original view by generating moments matrix for representing the general features of the 3D shape.Additionally, we apply the matrix square-root layer to perform a non-linear scaling to the eigenvalues of the moment embedding matrix.We compare the performance of our proposed network with several state-of-the-art models on the ModelNet datasets, and the results of the average instance/class accuracy demonstrate the promising performance of MMEN on 3D shape recognition.