Multi-hop question answering (QA) typically involves retrieving multiple relevant passages as evidence, which a reader then uses to derive the final answer. Existing methods often decompose complex questions into subquestions, leveraging large language models for step-by-step inference. However, these approaches fail to address the interdependence between retrieval and decomposition, resulting in suboptimal performance: Incomplete or inaccurate evidence can lead to poorly crafted subquestions, which, in turn, amplify the retrieval of irrelevant information and impede the reasoning process. To tackle this, we introduce BidLoop, a multi-step reasoning framework that explicitly models Bid irectional Loop between decomposition and retrieval. BidLoop employs four specialized modules: the Planner, Evaluator, Retriever, and Reader. In each reasoning round, the Planner generates a new subquestion based on prior evidence and subquestion-answer pairs. The Evaluator assesses whether the gathered evidence is sufficient to produce the final answer or if further reasoning is needed. Guided by the Planner's subquestion, the Retriever fetches relevant evidence, while the Reader answers the subquestion, adding new evidence for the next round. Our approach excels in generalization, performing strongly on unseen datasets without training on them. Extensive experiments across diverse multi-hop QA datasets demonstrate that BidLoop significantly surpasses existing state-of-the-art models.

Collaborative Filtering (CF) methods commonly use negative sampling to improve preference learning by contrasting observed interactions with unobserved items. While effective, conventional practice implicitly treats all unclicked items as equally informative, regardless of their semantic group (e.g., genre or category). This overlooks a critical limitation: models may exploit coarse group distinctions rather than genuine fine-grained preferences, thereby inducing exposure imbalances across item groups. Such imbalances constitute a violation of item-side fairness, which seeks equitable exposure and evaluation for items from different semantic groups. When negative samples are drawn predominantly from groups semantically distant from a user’s positives, the learning signal becomes biased and comparisons unfair. We therefore revisit negative sampling through the lens of item-side fairness and argue that genuine fairness requires context-aware sampling that ensures like-for-like comparisons within each semantic group. To this end, we introduce FairNS, a diffusion-based sampling framework that generates negative samples within the same semantic group as the user’s positives, encouraging fair intra-group contrasts that respect group integrity. By centering training on these intra-group comparisons, FairNS mitigates cross-group bias and enables the recommender to learn more precise user preferences. FairNS is optimized via a bi-level objective that jointly refines the sampling mechanism and the recommendation model. Experiments on three benchmark datasets show that FairNS achieves a favorable fairness–accuracy tradeoff.

Spatio-temporal tasks often encounter incomplete data arising from missing or inaccessible sensors, making spatio-temporal kriging crucial for inferring the completely missing temporal information. However, current models struggle with ensuring the validity and generalizability of inferred spatio-temporal patterns, especially in capturing dynamic spatial dependencies and temporal shifts, and optimizing the generalizability of unknown sensors. To overcome these limitations, we propose Spatio-Temporal Aware Graph Adversarial Neural Network (STA-GANN), a novel GNN-based kriging framework that improves spatio-temporal pattern validity and generalization. STA-GANN integrates (i) Decoupled Phase Module that senses and adjusts for timestamp shifts. (ii) Dynamic Data-Driven Metadata Graph Modeling to update spatial relationships using temporal data and metadata; (iii) An adversarial transfer learning strategy to ensure generalizability. Extensive validation across nine datasets from four fields and theoretical evidence both demonstrate the superior performance of STA-GANN.

As trustworthy AI continues to advance, the fairness issue in recommendations has received increasing attention. A recommender system is considered unfair when it produces unequal outcomes for different user groups based on user-sensitive attributes (e.g., age, gender). Some researchers have proposed data augmentation-based methods aiming at alleviating user-level unfairness by altering the skewed distribution of training data among various user groups. Despite yielding promising results, they often rely on fairness-related assumptions that may not align with reality, potentially reducing the data quality and negatively affecting model effectiveness. To tackle this issue, in this paper, we study how to implement high-quality data augmentation to improve recommendation fairness. Specifically, we proposeFairDgcl, a dynamic graph adversarial contrastive learning framework aiming at improving fairness in recommender system. First, FairDgcl develops an adversarial contrastive network with a view generator and a view discriminator to learn generating fair augmentation strategies in an adversarial style. Then, we propose two dynamic, learnable models to generate contrastive views within contrastive learning framework, which automatically fine-tune the augmentation strategies. Meanwhile, we theoretically show that FairDgcl can simultaneously generate enhanced representations that possess both fairness and accuracy. Lastly, comprehensive experiments conducted on four real-world datasets demonstrate the effectiveness of the proposed FairDgcl. The code can be found athttps://github.com/cwei01/FairDgcl.

Knowledge Graph Embedding (KGE) aims to learn dense embeddings as the representations for entities and relations in KGs. Indeed, the entities in existing KGs suffer from the data imbalance issue, i.e., there exists a substantial disparity in the occurrence frequencies among various entities. Existing KGE models pre-define a unified and fixed dimension size for all entity embeddings. However, embedding sizes of entities are highly desired for their frequencies, while a uniform embedding size may result in inadequate expression of entities, i.e., leading to overfitting for low-frequency entities and underfitting for high-frequency ones. A straight-forward idea is to set the embedding sizes for each entity before KGE training. However, manually selecting different embedding sizes is labor-intensive and time-consuming, which is difficult to achieve in real-world scenarios. To tackle this problem, we propose AdaE, which adaptively learns KG embeddings with different embedding sizes during training. In particular, AdaE is capable of selecting appropriate dimension sizes for each entity from a continuous integer space. To this end, we specially tailor bilevel optimization for the KGE task, which alternately learns representations and embedding sizes of entities. Moreover, it is worth noting that our framework is general and flexible, which is suitable for various existing KGE models. Extensive experiments demonstrate the effectiveness and compatibility of AdaE.

Multivariate Time Series (MTS) analysis is crucial to understanding and managing complex systems, such as traffic and energy systems, and a variety of approaches to MTS forecasting have been proposed recently. However, we often observe inconsistent or seemingly contradictory performance findings across different studies. This hinders our understanding of the merits of different approaches and slows down progress. We address the need for means of assessing MTS forecasting proposals reliably and fairly, in turn enabling better exploitation of MTS as seen in different applications. Specifically, we first propose BasicTS+, a benchmark designed to enable fair, comprehensive, and reproducible comparison of MTS forecasting solutions. BasicTS+ establishes a unified training pipeline and reasonable settings, enabling an unbiased evaluation. Second, we identify the heterogeneity across different MTS as an important consideration and enable classification of MTS based on their temporal and spatial characteristics. Disregarding this heterogeneity is a prime reason for difficulties in selecting the most promising technical directions. Third, we apply BasicTS+ along with rich datasets to assess the capabilities of more than 30 MTS forecasting solutions. This provides readers with an overall picture of the cutting-edge research on MTS forecasting.

Spatial-Temporal Graph Neural Networks (STGNNs) have been widely utilized in multivariate time series forecasting (MTSF), but they rely on the assumption of data completeness. In practice, due to factors such as natural disaster, STGNNs frequently encounter the challenge of missing data resulting from numerous malfunctioning data collectors. In this case, on the one hand, due to the presence of missing values, STGNNs easily generate incorrect spatial correlations, leading to the performance degradation. On the other hand, STGNNs require separate training of models for different missing rates, limiting their robustness. To address these challenges, we first propose two important components (interpolation attention and adaptive graph convolution), which utilize normal values to recover missing values into reliable representations and reconstruct spatial correlations. Then, we replace the fully connected layers in simple recursive units with these two components and propose Graph Interpolation Attention Recursive Network (GinAR), aiming to recursively correct spatial correlations and achieve end-to-end MTSF with missing values. Finally, we use data with different missing rates as positive and negative data pairs. By employing contrastive learning to train GinAR, we propose GinAR+ and enhance its robustness to data with different missing rates. Experiments validate the superiority of GinAR+ and our motivation.

Classical sequential recommendation models generally adopt ID embeddings to store knowledge learned from user historical behaviors and represent items. However, these unique IDs are challenging to be transferred to new domains. With the thriving of pre-trained language model (PLM), some pioneer works adopt PLM for pre-trained recommendation, where modality information is considered universal across domains via PLM. Unfortunately, the behavioral information in ID embeddings is verified to currently dominate in recommendation compared to modality information and thus limits these models’ performance. In this work, we propose a novel ID-centric recommendation pre-training paradigm (IDP), which directly transfers informative ID embeddings learned in pre-training domains to item representations in new domains. Specifically, in pre-training stage, besides the ID-based sequential recommendation model, we also build a Cross-domain ID-matcher (CDIM) learned by both behavioral and modality information. In the tuning stage, modality information of new domain items is regarded as a cross-domain bridge built by CDIM. They first adopted to retrieve behaviorally and semantically similar items from pre-training domains using CDIM. Next, these retrieved items’ pre-trained ID embeddings are directly adopted to generate downstream new items’ embeddings. Through extensive experiments on real-world datasets, we demonstrate that our proposed model significantly outperforms all baselines.

Recently, there has been an emergence of self-supervised recommendation methods that integrate knowledge graphs. Upon conducting a comprehensive review of contrastive learning (CL) in recommender systems, we conclude that existing methods solely focus on data view generation (the first phase) while neglecting the equally pivotal data view alignment (the second phase). However, due to the complexity and variability of real-world graph data, regardless of the graph augmentation strategy employed, it may be unrealistic to expect all entities to benefit from CL. In this article, we propose a H ierarchical E nhanced K nowledge-Aware C ontrastive L earning (HEK-CL) method for recommendation. Overall, we aim to hierarchically carry out enhancement strategies in both the first and second phases of knowledge-aware CL: (1) From the perspective of enhancing data view generation, we focus on combining non-Euclidean representation learning with graph denoising modules. Owing to the unified space’s ability to learn the ideal curvature from data distributions, the quality of embeddings for graph data has seen enhancements; (2) From the perspective of enhancing data view alignment, we propose a hyperbolic robust contrastive loss, named HRCL. Through rigorous theoretical analysis and experiments, we demonstrate that HRCL provides a more balanced and equitable training process for all entities than InfoNCE. Numerous experiments on the three real-world datasets show that our HEK-CL outperforms state-of-the-art baselines.

Knowledge Graph (KG) has proven its effectiveness in recommendation systems. Recent knowledge-aware recommendation methods, which utilize graph neural networks and contrastive learning, underestimate two issues: 1) The neglect of modeling the latent relationships between users and entities; 2) The insufficiency of traditional cross-view contrastive learning whose domain is incapable of covering all nodes in a graph. To address these issues, we propose a novel model named Knowledge-aware User Preference Network (KUPN). Specifically, KUPN first constructs the relational preference view containing a new graph named User Preference Graph (UPG) to model the potential relationships between users and entities. Then, we adopt a novel attentive information aggregation to learn the UPG. In addition, we obtain semantic information of users and entities from collaborative knowledge view which consists of KG and Interaction Graph (IG) as supplementary. Finally, we apply a cross-view contrastive learning for complete domains between dynamic relational preference view and collaborative knowledge view. Extensive experiments on three real-world datasets demonstrate the superiority of KUPN against the state-of-the-art methods.

Temporal Knowledge Graph (TKG) reasoning is a crucial task that aims to predict future facts based on historical information. In the process of reasoning over TKGs, we identify two types of facts that need to be predicted: 1) recurring facts and 2) unknown facts. While existing models emphasize reasoning about recurring facts, they inadvertently overlook the importance of unknown facts. To make better predictions on both facts, we introduce a novel TKG reasoning model, named Multi-view Recurrent Network (MV-NET), which generates different views to capture reasoning patterns for both recurring and unknown facts. Specifically, MV-NET comprises three views: a recurring history view that captures repetitive features, an exploring history view that focuses on exploring new information for unknown facts, and a full history view that assimilates historical information comprehensively. Then, the historical information of each view is encoded by a multi-view recurrent network. To better integrate the embeddings of three views, we employ an adaptive scoring module, which consists of a query-aware attentive fusion mechanism to incorporate the predicted scores from three views, thus obtaining fused scores for prediction. Extensive experiments on three commonly used datasets demonstrate the superiority of MV-NET compared to many state-of-the-art baselines.

Online games often house virtual shops where players can acquire character skins. Our task is centered on tailoring skin recommendations across diverse scenarios by analyzing historical interactions such as clicks, usage, and purchases. Traditional multi-behavior recommendation models employed for this task are limited. They either only predict skins based on a single type of behavior or merely recommend skins for target behavior type/task. These models lack the ability to control predictions of skins that are associated with different scenarios and behaviors. To overcome these limitations, we utilize the pretraining capabilities of Large Sequential Models (LSMs) coupled with a novel stimulus prompt mechanism and build a controllable multi-behavior recommendation (CMBR) model. In our approach, the pretraining ability is used to encapsulate users' multi-behavioral sequences into the representation of users' general interests. Subsequently, our designed stimulus prompt mechanism stimulates the model to extract scenario-related interests, thus generating potential skin purchases (or clicks and other interactions) for users. To the best of our knowledge, this is the first work to provide controlled multi-behavior recommendations, and also the first to apply the pretraining capabilities of LSMs in game domain. Through offline experiments and online A/B tests, we validate our method significantly outperforms baseline models, exhibiting about a tenfold improvement on various metrics during the offline test.

The graph-based recommendation has achieved great success in recent years. However, most existing graph-based recommendations focus on capturing user preference based on positive edges/feedback, while ignoring negative edges/feedback (e.g., dislike, low rating) that widely exist in real-world recommender systems. How to utilize negative feedback in graph-based recommendations still remains underexplored. In this study, we first conducted a comprehensive experimental analysis and found that (1) existing graph neural networks are not well-suited for modeling negative feedback, which acts as a high-frequency signal in a user-item graph. (2) The graph-based recommendation suffers from the representation degeneration problem. Based on the two observations, we propose a novel model that models positive and negative feedback from a frequency filter perspective called Dual-frequency Graph Neural Network for Sign-aware Recommendation (DFGNN). Specifically, in DFGNN, the designed dual-frequency graph filter (DGF) captures both low-frequency and high-frequency signals that contain positive and negative feedback. Furthermore, the proposed signed graph regularization is applied to maintain the user/item embedding uniform in the embedding space to alleviate the representation degeneration problem. Additionally, we conduct extensive experiments on real-world datasets and demonstrate the effectiveness of the proposed model. Codes of our model will be released upon acceptance.

Multivariate time series forecasting (MTSF) is crucial for decision-making to precisely forecast the future values/trends, based on the complex relationships identified from historical observations of multiple sequences. Recently, Spatial-Temporal Graph Neural Networks (STGNNs) have gradually become the theme of MTSF model as their powerful capability in mining spatial-temporal dependencies, but almost of them heavily rely on the assumption of historical data integrity. In reality, due to factors such as data collector failures and time-consuming repairment, it is extremely challenging to collect the whole historical observations without missing any variable. In this case, STGNNs can only utilize a subset of normal variables and easily suffer from the incorrect spatial-temporal dependency modeling issue, resulting in the degradation of their forecasting performance. To address the problem, in this paper, we propose a novel Graph Interpolation Attention Recursive Network (named GinAR) to precisely model the spatial-temporal dependencies over the limited collected data for forecasting. In GinAR, it consists of two key components, that is, interpolation attention and adaptive graph convolution to take place of the fully connected layer of simple recursive units, and thus are capable of recovering all missing variables and reconstructing the correct spatial-temporal dependencies for recursively modeling of multivariate time series data, respectively. Extensive experiments conducted on five real-world datasets demonstrate that GinAR outperforms 11 SOTA baselines, and even when 90% of variables are missing, it can still accurately predict the future values of all variables.

Temporal Knowledge Graph (TKG) reasoning involves predicting future facts based on historical information by learning correlations between entities and relations. Recently, many models have been proposed for the TKG reasoning task. However, most existing models cannot efficiently utilize historical information, which can be summarized in two aspects: 1) Many models only consider the historical information in a fixed time range, resulting in a lack of useful information; 2) some models use all the historical facts, thus some noise or invalid facts are introduced during reasoning. In this regard, we propose a novel TKG reasoning model with dynamic memory enhancement (DyMemR). Inspired by human memory, we introduce memory capacity, memory loss, and repetition stimulation to design a human-like memory pool that could remember potentially useful historical facts. To fully leverage the memory pool, we utilize a two-stage training strategy.The first stage is guided by the memory-based encoding module which learns embeddings from memory-based subgraphs generated through the memory pool. The second stage is the memory-based scoring module that emphasizes the historical facts in the memory pool. Finally, we extensively validate the superiority of DyMemR against various state-of-the-art baselines.

The emergence of Graph Neural Networks (GNNs) has greatly advanced the development of recommendation systems. Recently, many researchers have leveraged GNN-based models to learn fair representations for users and items. However, current GNN-based models suffer from biased user–item interaction data, which negatively impacts recommendation fairness. Although there have been several studies employing adversarial learning to mitigate this issue in recommendation systems, they mostly focus on modifying the model training approach with fairness regularization and neglect direct intervention of biased interaction. In contrast to these models, this article introduces a novel perspective by directly intervening in observed interactions to generate a counterfactual graph (called FairGap) that is not influenced by sensitive node attributes, enabling us to learn fair representations for users and items easily. We design FairGap to answer the key counterfactual question: “Would interactions with an item remain unchanged if a user’s sensitive attributes were concealed?”. We also provide theoretical proofs to show that our learning strategy via the counterfactual graph is unbiased in expectation. Moreover, we propose a fairness-enhancing mechanism to continuously improve user fairness in the graph-based recommendation. Extensive experimental results against state-of-the-art competitors and base models on three real-world datasets validate the effectiveness of our proposed model.

Knowledge graphs (KGs) play a pivotal role in AI-related applications.In order to construct or continuously enrich KGs, automatic knowledge construction and update mechanisms are usually utilized, which inevitably bring in plenty of noise, and noise would degrade the performance of downstream applications.Existing KG error detection methods utilize the embeddings of entities and relations, or directly leverage the paths between entities to test the plausibility of triples, while ignore the valuable hierarchical information contained in the paths between entities.Indeed, the paths between a pair of entities conform to a hierarchical structure.Specifically, there may be a number of paths between two entities, and each path is comprised of several relations.The hierarchical structure is able to provide precious information, and is beneficial to leverage the path information in a fine-grained manner.To this end, in this paper, we propose a novel model named KG error detection with HiErarchical pAth stRucture (HEAR for short).Particularly, for a given triple, HEAR first learns path representations with the relations contained in the path, then integrates all path representations, and at last predicts the plausibility of the triple.Finally, we extensively validate the superiority of HEAR against various state-of-the-art baselines.

Since Knowledge Graphs (KGs) contain rich semantic information, recently there has been an influx of KG-enhanced recommendation methods. Most of existing methods are entirely designed based on euclidean space without considering curvature. However, recent studies have revealed that a tremendous graph-structured data exhibits highly non-euclidean properties. Motivated by these observations, in this work, we propose a knowledge-based multiple adaptive spaces fusion method for recommendation, namely MCKG. Unlike existing methods that solely adopt a specific manifold, we introduce the unified space that is compatible with hyperbolic, euclidean and spherical spaces. Furthermore, we fuse the multiple unified spaces in an attention manner to obtain the high-quality embeddings for better knowledge propagation. In addition, we propose a geometry-aware optimization strategy which enables the pull and push processes benefited from both hyperbolic and spherical spaces. Specifically, in hyperbolic space, we set smaller margins in the area near to the origin, which is conducive to distinguishing between highly similar positive items and negative ones. At the same time, we set larger margins in the area far from the origin to ensure the model has sufficient error tolerance. The similar manner also applies to spherical spaces. Extensive experiments on three real-world datasets demonstrate that the MCKG has a significant improvement over state-of-the-art recommendation methods. Further ablation experiments verify the importance of multi-space fusion and geometry-aware optimization strategy, justifying the rationality and effectiveness of MCKG.

Recently, a series of pioneer studies have shown the potency of pre-trained models in sequential recommendation, illuminating the path of building an omniscient unified pre-trained recommendation model for different downstream recommendation tasks. Despite these advancements, the vulnerabilities of classical recommender systems also exist in pre-trained recommendation in a new form, while the security of pre-trained recommendation model is still unexplored, which may threaten its widely practical applications. In this study, we propose a novel framework for backdoor attacking in pre-trained recommendation. We demonstrate the provider of the pre-trained model can easily insert a backdoor in pre-training, thereby increasing the exposure rates of target items to target user groups. Specifically, we design two novel and effective backdoor attacks: basic replacement and prompt-enhanced, under various recommendation pre-training usage scenarios. Experimental results on real-world datasets show that our proposed attack strategies significantly improve the exposure rates of target items to target users by hundreds of times in comparison to the clean model. The source codes are released in https://github.com/wyqing20/APRec.

Knowledge graph embedding (KGE) aims to project both entities and relations in a knowledge graph (KG) into low-dimensional vectors. Indeed, existing KGs suffer from the data imbalance issue, i.e., entities and relations conform to a long-tail distribution, only a small portion of entities and relations occur frequently, while the vast majority of entities and relations only have a few training samples. Existing KGE methods assign equal weights to each entity and relation during the training process. Under this setting, long-tail entities and relations are not fully trained during training, leading to unreliable representations. In this paper, we propose WeightE, which attends differentially to different entities and relations. Specifically, WeightE is able to endow lower weights to frequent entities and relations, and higher weights to infrequent ones. In such manner, WeightE is capable of increasing the weights of long-tail entities and relations, and learning better representations for them. In particular, WeightE tailors bilevel optimization for the KGE task, where the inner level aims to learn reliable entity and relation embeddings, and the outer level attempts to assign appropriate weights for each entity and relation. Moreover, it is worth noting that our technique of applying weights to different entities and relations is general and flexible, which can be applied to a number of existing KGE models. Finally, we extensively validate the superiority of WeightE against various state-of-the-art baselines.

In recent years, graph neural networks (GNNs)-based methods have been widely adopted for heterogeneous graph (HG) embedding, due to their power in effectively encoding rich information from a HG into the low-dimensional node embeddings. However, previous works usually easily fail to fully leverage the inherent heterogeneity and rich semantics contained in the complex local structures of HGs. On the one hand, most of the existing methods either inadequately model the local structure under specific semantics, or neglect the heterogeneity when aggregating information from the local structure. On the other hand, representations from multiple semantics are not comprehensively integrated to obtain node embeddings with versatility. To address the problem, we propose aHeterogeneous Graph Neural Networkfor HG embeddingwithin a Multi-View representation learning framework(named MV-HetGNN), which consists of a view-specific ego graph encoder and auto multi-view fusion layer. MV-HetGNN thoroughly learns complex heterogeneity and semantics in the local structure to generate comprehensive and versatile node representations for HGs. Extensive experiments on three real-world HG datasets demonstrate the significant superiority of our proposed MV-HetGNN compared to the state-of-the-art baselines in various downstream tasks, e.g., node classification, node clustering, and link prediction.

Nowadays, Knowledge graphs (KGs) have been playing a pivotal role in AI-related applications. Despite the large sizes, existing KGs are far from complete and comprehensive. In order to continuously enrich KGs, automatic knowledge construction and update mechanisms are usually utilized, which inevitably bring in plenty of noise. However, most existing knowledge graph embedding (KGE) methods assume that all the triple facts in KGs are correct, and project both entities and relations into a low-dimensional space without considering noise and knowledge conflicts. This will lead to low-quality and unreliable representations of KGs. To this end, in this paper, we propose a general multi-task reinforcement learning framework, which can greatly alleviate the noisy data problem. In our framework, we exploit reinforcement learning for choosing high-quality knowledge triples while filtering out the noisy ones. Also, in order to take full advantage of the correlations among semantically similar relations, the triple selection processes of similar relations are trained in a collective way with multi-task learning. Moreover, we extend popular KGE models TransE, DistMult, ConvE and RotatE with the proposed framework. Finally, the experimental validation shows that our approach is able to enhance existing KGE models and can provide more robust representations of KGs in noisy scenarios.

Recently, recommender systems based on knowledge graphs (KGs) have become a popular research direction. Graph neural network (GNN) is the key technology of KG-based recommendation systems. However, existing GNNs have a significant flaw: They cannot explicitly model users’ intent in recommendations. Intent plays an essential role in users’ behaviors. For example, users may first generate an intent to purchase a certain group of items and then select a specific item from the group based on their preferences. Therefore, explicitly modeling intent has a positive significance for improving recommendation performance and providing explanations for recommendations. In this article, we propose a new model called Topic-aware Intention Network (TIN) for explainable recommendations with KGs. TIN models user representations from both preference and intent views. Specifically, we design a relational attention graph neural network to selectively aggregate information in KG to learn user preferences, and we propose a knowledge-enhanced topic model to learn user intent, which is viewed as topics hidden in user behavior sequences. Finally, we obtain the user representation by fusing user preference and intent through an attention network. The experimental results show that our proposed model outperforms the state-of-the-art methods and can generate reasonable explanations for the recommendation results.

Multivariate Time Series (MTS) forecasting plays a vital role in a wide range of applications. Recently, Spatial-Temporal Graph Neural Networks (STGNNs) have become increasingly popular MTS forecasting methods due to their state-of-the-art performance. However, recent works are becoming more sophisticated with limited performance improvements. This phenomenon motivates us to explore the critical factors of MTS forecasting and design a model that is as powerful as STGNNs, but more concise and efficient. In this paper, we identify the indistinguishability of samples in both spatial and temporal dimensions as a key bottleneck, and propose a simple yet effective baseline for MTS forecasting by attaching Spatial and Temporal IDentity information (STID), which achieves the best performance and efficiency simultaneously based on simple Multi-Layer Perceptrons (MLPs). These results suggest that we can design efficient and effective models as long as they solve the indistinguishability of samples, without being limited to STGNNs.

Recent years have witnessed remarkable progress on knowledge graph embedding (KGE) methods to learn the representations of entities and relations in static knowledge graphs (SKGs). However, knowledge changes over time. In order to represent the facts happening in a specific time, temporal knowledge graph (TKG) embedding approaches are put forward. While most existing models ignore the independence of semantic and temporal information. We empirically find that current models have difficulty distinguishing representations of the same entity or relation at different timestamps. In this regard, we propose a TimeLine-Traced Knowledge Graph Embedding method (TLT-KGE) for temporal knowledge graph completion. TLT-KGE aims to embed the entities and relations with timestamps as a complex vector or a quaternion vector. Specifically, TLT-KGE models semantic information and temporal information as different axes of complex number space or quaternion space. Meanwhile, two specific components carving the relationship between semantic and temporal information are devised to buoy the modeling. In this way, the proposed method can not only distinguish the independence of the semantic and temporal information, but also establish a connection between them. Experimental results on the link prediction task demonstrate that TLT-KGE achieves substantial improvements over state-of-the-art competitors. The source code will be available on https://github.com/zhangfw123/TLT-KGE.

Multivariate Time Series (MTS) forecasting plays a vital role in a wide range of applications. Recently, Spatial-Temporal Graph Neural Networks (STGNNs) have become increasingly popular MTS forecasting methods. STGNNs jointly model the spatial and temporal patterns of MTS through graph neural networks and sequential models, significantly improving the prediction accuracy. But limited by model complexity, most STGNNs only consider short-term historical MTS data, such as data over the past one hour. However, the patterns of time series and the dependencies between them (i.e., the temporal and spatial patterns) need to be analyzed based on long-term historical MTS data. To address this issue, we propose a novel framework, in which STGNN is Enhanced by a scalable time series Pre-training model (STEP). Specifically, we design a pre-training model to efficiently learn temporal patterns from very long-term history time series (e.g., the past two weeks) and generate segment-level representations. These representations provide contextual information for short-term time series input to STGNNs and facilitate modeling dependencies between time series. Experiments on three public real-world datasets demonstrate that our framework is capable of significantly enhancing downstream STGNNs, and our pre-training model aptly captures temporal patterns.

We all depend on mobility, and vehicular transportation affects the daily lives of most of us. Thus, the ability to forecast the state of traffic in a road network is an important functionality and a challenging task. Traffic data is often obtained from sensors deployed in a road network. Recent proposals on spatial-temporal graph neural networks have achieved great progress at modeling complex spatial-temporal correlations in traffic data, by modeling traffic data as a diffusion process. However, intuitively, traffic data encompasses two different kinds of hidden time series signals, namely the diffusion signals and inherent signals. Unfortunately, nearly all previous works coarsely consider traffic signals entirely as the outcome of the diffusion, while neglecting the inherent signals, which impacts model performance negatively. To improve modeling performance, we propose a novel Decoupled Spatial-Temporal Framework (DSTF) that separates the diffusion and inherent traffic information in a data-driven manner, which encompasses a unique estimation gate and a residual decomposition mechanism. The separated signals can be handled subsequently by the diffusion and inherent modules separately. Further, we propose an instantiation of DSTF, Decoupled Dynamic Spatial-Temporal Graph Neural Network (D 2 STGNN), that captures spatial-temporal correlations and also features a dynamic graph learning module that targets the learning of the dynamic characteristics of traffic networks. Extensive experiments with four real-world traffic datasets demonstrate that the framework is capable of advancing the state-of-the-art.

Transformer-based language models (e.g. BERT, RoBERT, GPT, etc) have shown remarkable performance in many natural language processing tasks and their multilingual variants make it easier to handle cross-lingual tasks without using machine translation system. In this paper, we apply multilingual BERT in cross-lingual information retrieval (CLIR) task with triplet loss to learn the relevance between queries and documents written in different languages. Moreover, we align the token embeddings from different languages via adversarial networks to help the language model to learn cross-lingual sentence representation. We achieve the state-of-the-art result on the newly published CLIR dataset: CLIRMatrix. Furthermore, we show that the adversarial multilingual BERT can also get the competitive result in the zero-shot setting in some specific languages when we are lack of CLIR training data in a specific language.

Completing knowledge bases (KBs) with missing facts is of great importance, since most existing KBs are far from complete. To this end, many knowledge base completion (KBC) methods have been proposed. However, most existing methods embed each relation into a vector separately, while ignoring the correlations among different relations. Actually, in large-scale KBs, there always exist some relations that are semantically related, and we believe this can help to facilitate the knowledge sharing when learning the embedding of related relations simultaneously. Along this line, we propose a novel KBC model by Multi -Task E mbedding, named MultiE. In this model, semantically related relations are first clustered into the same group, and then learning the embedding of each relation can leverage the knowledge among different relations. Moreover, we propose a three-layer network to predict the missing values of incomplete knowledge triples. Finally, experiments on three popular benchmarks FB15k, FB15k-237 and WN18 are conducted to demonstrate the effectiveness of MultiE against some state-of-the-art baseline competitors.