Monte Carlo Tree Search (MCTS) has been widely used for automated reasoning data exploration, but current supervision extraction methods remain inefficient.Standard approaches retain only the single highest-reward trajectory, discarding the comparative signals present in the many explored paths.Here we introduce Contrastive Reasoning Path Synthesis (CRPS), a framework that transforms supervision extraction from a filtering process into a synthesis procedure.CRPS uses a structured reflective process to analyze the differences between high-and low-quality search trajectories, extracting explicit information about strategic pivots and local failure modes.These insights guide the synthesis of reasoning chains that incorporate success patterns while avoiding identified pitfalls.We show empirically that models fine-tuned on just 60K CRPS-synthesized examples match or exceed the performance of baselines trained on 590K examples derived from standard rejection sampling, a 20× reduction in dataset size.Furthermore, CRPS improves generalization on out-of-domain benchmarks, demonstrating that learning from the contrast between success and failure produces more transferable reasoning capabilities than learning from success alone.

These years have witnessed remarkable progress in modeling user behaviour from personalized online services, especially knowledge graph-based recommendation systems. Meanwhile, more studies are focusing on aspects beyond recommendation performance, since such an observational data-driven paradigm is posing threats to both users and society in terms of trustworthiness. In fact, existing problem-oriented solutions still face significant challenges, as almost all of them suffer from the generality limitations to improve their trustworthiness in a uniform fashion. To address these issues, we propose a plug-and-playDebiasing framework forKnowledgeGraph-basedRecommendationSystems, also known as DiKGRS. Specifically, the Knowledge-augmented Pseudo-Samples Generation (KPSG) method, a novel data augmentation perspective, is proposed to explore more auxiliary information beyond observational user behaviors. Furthermore, the Debiasing Value Networks (DVN), is also developed to evaluate the reliability of generated pseudo-samples by modeling both the item popularity and user demographic bias in the platform. Moreover, an adaptive weighting coordination module is performed to coordinate the proposed DiKGRS framework and its backbones. Experimental results on four real-world datasets from different online service personalization scenarios have illustrated that the proposed framework can significantly improve the trustworthiness of existing knowledge graph-based recommendation systems. The code has been released public available at:https://github.com/alipay/A-Knowledge-augmented-Method-DiKGRS.

Despite the encouraging achievements, the practical application of recommendation systems still faces two key issues. The first is how to better understand the multimodal real-time requests that are the more mainstream request behavior in industrial scenarios; the other is how to effectively capture users' dynamic needs that change with temporal and spatial conditions. The breakthroughs in text understanding and generation capabilities of Large Language Models (LLMs) have demonstrated their tremendous potential in precise recommendation systems, particularly through the enhancement of the understanding of user intent. To address these issues, we propose a novel Spatial-Temporal Multimodal LLM for generative recommendation. Specifically, on the basis of the behavior data constructed from Alipay, spatial-temporal knowledge-guided fine-tuning module is proposed to capture specific needs in user real-time requests. Furthermore, a preference discovery module is developed to learn user preferences in visual queries from multimodal request perspective. Meanwhile, a personalized recommendation module is designed to aggregate spatial-temporal knowledge and user preferences for generative recommendation. Experimental results on a real-world deployed generative recommendation task from the ‘Explore' scenario in Alipay have demonstrated the effectiveness of the proposed framework.

Facing the challenges of sparsity and long tail in thousands of Mini-apps recommendation scenarios deployed on Alipay platform, there is a great need for a simple, effective, and easy-to-deploy industrial solution. To address this issue, we follow the strategy of “divide and conquer” and propose a crowd-based recommendation model by using D eterminantal P oint P rocesse s on C rowd-wise M ixture- o f- E xperts (DPPs-CMoE). Specifically, under the guidance of DPPs-based prototypical tags, the user profiling space is sequentially divided into multiple crowds, with each of them taking on a unique latent specificity; Meanwhile, by treating the modeling of crowd specificity as one of multiple tasks, a crowd-wise architecture is adopted to seamlessly unify the multiple expert networks from the overall user space and the gating network from each of independent crowd spaces. The effectiveness of the proposed method has been illustrated in the experimental results on a mini-apps recommendation scenario deployed in Alipay APPs.

Multimodal electronic health record (EHR) data can offer a holistic assessment of a patient's health status, supporting various predictive healthcare tasks. Recently, several studies have embraced the multitask learning approach in the healthcare domain, exploiting the inherent correlations among clinical tasks to predict multiple outcomes simultaneously. However, existing methods necessitate samples to possess complete labels for all tasks, which places heavy demands on the data and restricts the flexibility of the model. Meanwhile, within a multitask framework with multimodal inputs, how to comprehensively consider the information disparity among modalities and among tasks still remains a challenging problem. To tackle these issues, a unified healthcare prediction model, also named by \textbf{FlexCare}, is proposed to flexibly accommodate incomplete multimodal inputs, promoting the adaption to multiple healthcare tasks. The proposed model breaks the conventional paradigm of parallel multitask prediction by decomposing it into a series of asynchronous single-task prediction. Specifically, a task-agnostic multimodal information extraction module is presented to capture decorrelated representations of diverse intra- and inter-modality patterns. Taking full account of the information disparities between different modalities and different tasks, we present a task-guided hierarchical multimodal fusion module that integrates the refined modality-level representations into an individual patient-level representation. Experimental results on multiple tasks from MIMIC-IV/MIMIC-CXR/MIMIC-NOTE datasets demonstrate the effectiveness of the proposed method. Additionally, further analysis underscores the feasibility and potential of employing such a multitask strategy in the healthcare domain. The source code is available at https://github.com/mhxu1998/FlexCare.

Graph neural networks (GNNs) have shown remarkable performance on homophilic graph data while being far less impressive when handling non-homophilic graph data due to the inherent low-pass filtering property of GNNs. In general, since real-world graphs are often complex mixtures of diverse subgraph patterns, learning a universal spectral filter on the graph from the global perspective as in most current works may still suffer from great difficulty in adapting to the variation of local patterns. On the basis of the theoretical analysis of local patterns, we rethink the existing spectral filtering methods and propose theNode-oriented spectralFiltering forGraphNeuralNetwork (namely NFGNN). By estimating the node-oriented spectral filter for each node, NFGNN is provided with the capability of precise local node positioning via the generalized translated operator, thus discriminating the variations of local homophily patterns adaptively. Meanwhile, the utilization of re-parameterization brings a good trade-off between global consistency and local sensibility for learning the node-oriented spectral filters. Furthermore, we theoretically analyze the localization property of NFGNN, demonstrating that the signal after adaptive filtering is still positioned around the corresponding node. Extensive experimental results demonstrate that the proposed NFGNN achieves more favorable performance.

Multi-task learning (MTL) is widely used in the online recommendation and financial services for multi-step conversion estimation, but current works often overlook the sequential dependence among tasks. In particular, sequential dependence multi-task learning (SDMTL) faces challenges in dealing with complex task correlations and extracting valuable information in real-world scenarios, leading to negative transfer and a deterioration in the performance. Herein, a systematic learning paradigm of the SDMTL problem is established for the first time, which applies to more general multi-step conversion scenarios with longer conversion paths or various task dependence relationships. Meanwhile, an SDMTL architecture, named Task-Aware Feature Extraction (TAFE), is designed to enable the dynamic task representation learning from a sample-wise view. TAFE selectively reconstructs the implicit shared information corresponding to each sample case and performs the explicit task-specific extraction under dependence constraints, which can avoid the negative transfer, resulting in more effective information sharing and joint representation learning. Extensive experiment results demonstrate the effectiveness and applicability of the proposed theoretical and implementation frameworks. Furthermore, the online evaluations at MYbank showed that TAFE had an average increase of 9.22% and 3.76% in various scenarios on the post-view click-through & conversion rate (CTCVR) estimation task. Currently, TAFE is deployed in an online platform to provide various traffic services.

Risk prediction, usually achieved by learning representations from patient’s physiological sequence or user’s behavioral sequence data, and has been widely applied in healthcare and finance. Despite that, some recent time-aware deep learning methods have led to superior performances in such sequence representation learning tasks, such improvement is limited due to a lack of guidance from hierarchical global view. To address this issue, we propose a novel end-to-end H ierarchical G lobal V iew-guided (HGV) sequence representation learning framework. Specifically, the Global Graph Embedding (GGE) module is proposed to learn sequential clip-aware representations from temporal correlation graph (TCG) at instance level. Furthermore, following the way of key-query attention, the harmonic β-attention (β-Attn) is also developed for making a global tradeoff between time-aware decay and observation significance at channel level adaptively. Moreover, the hierarchical representations at both instance level and channel level can be coordinated by the heterogeneous information aggregation under the guidance of global view. Experimental results on both healthcare risk prediction benchmark and SMEs credit overdue risk prediction task from the real-world industrial scenario in MYBank, Ant Group, have illustrated that the proposed model can achieve competitive prediction performance compared with other known baselines. The code has been released public available at: https://github.com/LiYouru0228/HGV.

While large enterprises are benefiting from their global supply chains in these years, it is not easy for Small and Medium-sized Enterprises (SMEs) to find supply chain partners. Treating it as a supply chain mining problem, some deep learning methods, especially knowledge graph (KG) enhanced ones, can achieve workable performance by utilizing explicit structure information from KG while considering effectiveness. However, such improvement is limited when facing the challenges of scalability, complexity, and noisiness in large-scale KGs. To address these issues, we propose a novelMeta-tagSupportedConnectivity representationLearning framework, also known as MSCL. Specifically, a Meta-tag Collaborative Filtering (MCF) method is proposed to highlight the representative schema from huge number of paths connecting two enterprises in large-scale KG. Furthermore, the DPPs-induced Hierarchical Path Sampling (DHPS), a novel sampling framework, is also developed to capture the latent connectivity pattern in KG more effectively. Moreover, the path-wise knowledge representations and the underlying information inherent in pairwise enterprises are aggregated by a connectivity representation learning (CRL) approach for SMEs supply chain mining. Experimental results from two real-world industries have illustrated that the proposed model can achieve competitive performance compared with other existing baselines.

Recent years have witnessed the successful application of knowledge graph techniques in structured data processing, while how to incorporate knowledge from visual and textual modalities into knowledge graphs has been given less attention. To better organize them, Multimodal Knowledge Graphs (MKGs), comprising the structural triplets of traditional Knowledge Graphs (KGs) together with entity-related multimodal data (e.g., images and texts), have been introduced consecutively. However, it is still a great challenge to explore MKGs due to their inherent incompleteness. Although most existing Multimodal Knowledge Graph Completion (MKGC) approaches can infer missing triplets based on available factual triplets and multimodal information, they almost ignore the modal conflicts and supervisory effect, failing to achieve a more comprehensive understanding of entities. To address these issues, we propose a novel H ierarchical K nowledge A lignment ( HKA ) framework for MKGC. Specifically, a macro-knowledge alignment module is proposed to capture global semantic relevance between modalities for dealing with modal conflicts in MKG. Furthermore, a micro-knowledge alignment module is also developed to reveal the local consistency information through inter- and intra-modality supervisory effects more effectively. By integrating different modal predictions, a final decision can be made. Experimental results on three benchmark MKGC tasks have demonstrated the effectiveness of the proposed HKA framework.

To effectively explore the supply chain relationships among Small and Medium-sized Enterprises (SMEs), some remarkable progress in such a relation modeling problem, especially knowledge graph-based methods have been witnessed during these years. As a typical link prediction task, supply chain prediction can usually predict the unknown future relationship facts between SMEs by utilizing the historical semantic connections between entities in knowledge graphs (KGs). However, it is still a great challenge for existing models as seldom of them can consider both temporal dependency and cooperative correlation of the connectivity pattern along the timeline synergistically. Accordingly, we propose a novel framework to learn joint relational co-evolution in Spatial-Temporal Knowledge Graphs (STKG). Specifically, on the base of the constructed large-scale financial STKG, a multi-view relational sequences mining method is proposed to reveal the semantic information from ontological concepts. Furthermore, a relational co-evolution learning module is also developed to capture the regularity of evolving connectivity patterns from the spatial-temporal view. Meanwhile, a multiple random subspace representation learning layer is also designed to improve both compatibility and complementarity during knowledge aggregation. Experimental results on large-scale SMEs supply chain prediction tasks from four real-world industries in China have illustrated the effectiveness of the proposed model.

Accurate forecasting of time series mitigates the uncertainty of future outlooks and is a great help in reducing errors in decisions. Despite years of researches, there are still some challenges to accurate forecasting of time series, including the difficulty of dynamic modeling, the problem of capturing short-term correlations, and the conundrum of long-term forecasting. This paper offers an A dversarial Truncated Cauchy S elf- A ttentive T ime- V ariant Neural N etwork (ASATVN) for multi-step ahead time series forecasting. Specifically, the proposed model builds on Generative Adversarial Networks, in which the generator is composed of a novel time-variant model. The time-variant model contributes to learning dynamic time-series changes with its time-variant architecture and employs a newly proposed Truncated Cauchy Self-Attention block to capture the local sequential dependencies better. For the discriminator , two self-attentive discriminators are presented to regularize predictions with fidelity and continuity, which is beneficial to predicting sequence over longer time horizons. Our proposed ASATVN model outperforms the state-of-the-art predictive models on eleven real-world benchmark datasets, demonstrating its effectiveness.

Despite that path-based and embedding-based models with knowledge graphs (KGs) achieve better recommendation performance compared with other deep learning based methods, such improvement is limited due to a lack of modeling user's dynamic interest. To address this issue, we explore a principled model to provide semantic understanding of each item in user's historical interest sequence in KGs. Specifically, we propose a multi-granularity dynamic interest sequence learning method, which is based on knowledge-enhanced path mining and interest fluctuation signal discovery, to obtain semantic-enhanced paths. Furthermore, the paths are embedded by the SEP2Vec, and merged through the proposed entropy-aware pooling layer to obtain the user preference representation, which is then used to learn dynamic user interest sequence. Experimental results on two public datasets of movie and music recommendation, and two industrial datasets of personalized local service recommendation in Alipay App have illustrated that the proposed model can achieve significantly better prediction performance compared with other known baselines.

For an actual recommendation system, it generally involves a variety of heterogeneous interactive relationships, such as the typical user-user (U2U), item-item (I2I), and user-item (U2I) interaction relationships. With the application of graph neural networks (GNNs) in embedding various interactive relations, recommendation technology has made gratifying progress in recent years, which benefits lot from its powerful ability in relation modeling. However, most of the existing GNN-based methods fail to collaboratively explore the above heterogeneous multiple interactive relationships, including the internal correlations among multiple relationships and the intrinsic association behind different relationships. As a consequence, the user's personalized preference for the items to be recommended will not be well captured. In this paper, we propose aSylvester equation inducedCollaborativeRepresentationLearning framework (S-CRL) for recommendation system by utilizing the heterogeneous multiple interactive relationships. In particular, we ingeniously define a novel Sylvester equation to associate tactfully the multiple heterogeneous relations together. From the perspective of rating propagation, such Sylvester equation is shown theoretically to be the optimal solution of a local structure sensitive rating propagation function. Additionally, to seek more expressive embeddings about user and item, a layer-wise attention is introduced to aggregate the multi-hop information from U2U and I2I graphs, respectively, so as to promote the aggregation with the corresponding embeddings from the U2I interaction graph. Extensive experiments on three real-world datasets verify that our model achieves more favorable performance over currently representative methods.

Bike-sharing systems, aiming at meeting the public’s need for ”last mile” transportation, are becoming popular in recent years. With an accurate demand prediction model, shared bikes, though with a limited amount, can be effectively utilized whenever and wherever there are travel demands. Despite that some deep learning methods, especially long shortterm memory neural networks (LSTMs), can improve the performance of traditional demand prediction methods only based on temporal representation, such improvement is limited due to a lack of mining complex spatial-temporal relations. To address this issue, we proposed a novel model named STG2Vec to learn the representation from heterogeneous spatial-temporal graph. Specifically, we developed an event-flow serializing method to encode the evolution of dynamic heterogeneous graph into a special language pattern such as word sequence in a corpus. Furthermore, a dynamic attention-based graph embedding model is introduced to obtain an importance-awareness vectorized representation of the event flow. Additionally, together with other multi-source information such as geographical position, historical transition patterns and weather, e.g., the representation learned by STG2Vec can be fed into the LSTMs for temporal modeling. Experimental results from Citi-Bike electronic usage records dataset in New York City have illustrated that the proposed model can achieve competitive prediction performance compared with its variants and other baseline models.