Patents are crucial for protecting technological innovations and fostering competitive advancements in industry. Patent prediction, a novel task in the field of patent mining, aims to forecast future technological trends, providing valuable insights for strategic planning and innovation in the industry. However, the complexity of patent data and the diversity of technological fields make effective patent prediction a significant challenge. Existing methods for predicting scientific research trends struggle to effectively model patent structures and capture dependencies between patents, resulting in suboptimal patent trend predictions. In this article, we propose a novel method, patent prediction with prompt learning (P3L), to achieve effective and accurate prediction of future patent developments based on a pretrained language model (PLM). P3L includes a patent similarity path extraction module to extract multiple patent development paths from extensive datasets. Following this, we design a patent prompt learning approach that integrates patent development paths, keywords, and patent similarities into the prompts. To mitigate potential noise introduced by this integration, we introduce an attention mask matrix for prompt denoising. Finally, we introduce three patent datasets with rich structures, and conduct extensive experiments on these datasets as well as a public dataset, demonstrating the superiority of the proposed method. The dataset and code have been made publicly available athttps://github.com/AllminerLab/P3L

In recent years, Large Language Models (LLMs) have gained increasing attention in personalized recommendation systems due to their strong capabilities to understand contextual and semantic information. However, most existing LLM-based recommendation methods primarily rely on textual features, often neglecting the unique structural information embedded in user and item IDs, which are critical for capturing personalized preferences. To bridge this gap, we propose a novel method called CL 3M - Rec, which enhances the semantic representation of user and item IDs through Contrastive Learning (CL) and optimizes LLM recommendation performance via prompt construction. By effectively integrating structured ID information with textual semantics, our approach enables the LLMs to better model individual user interests and improve personalized prediction. Experiments on four public datasets demonstrate that CL3M-Rec consistently outperforms existing LLM-based and CL-based recommendation approaches, achieving superior predictive accuracy and robust generalization across diverse recommendation scenarios. The code of our method is available at https://github.com/Lewis44zhou/CL-3M-Rec.

Abstract Recommendation system technologies predominantly focus on user-item interaction data, which are mapped into shared vector spaces for digital representation. These representations are then analyzed to uncover the relationships between users and items. As recommendation technologies have seen widespread adoption, a novel challenge has emerged in supply-demand matching contexts: the dual-triangular recommendation problem, involving four key entities, i.e., users with their demands, and suppliers with their offered items, forming a heterogeneous information network. In this work, we introduce the concept of dual-triangular recommendation and formally define this scientific problem. We propose a dual-triangular recommendation algorithm, enhanced by large language models, which utilizes knowledge graph encoder and LLM-augmented encoder to generate embedding representations for the four entities. A multi-task framework is employed to enable the sharing of underlying parameters across multiple recommendation tasks within the dual-triangular context. Through extensive experiments conducted on a real-world technology commercialization platform dataset, patent transfer dataset, and talent recruitment dataset, we demonstrate the effectiveness of our approach, offering a feasible and scalable solution to the dual-triangular recommendation problem.

Due to the wide existence of unlabeled graph-structured data (e.g., molecular structures), the graph-level clustering has recently attracted increasing attention, whose goal is to divide the input graphs into several disjoint groups. However, the existing methods habitually focus on learning the graphs embeddings with different graph reguralizations, and seldom refer to the obvious differences in data distributions of distinct graph-level datasets. How to characteristically consider multiple graph-level datasets in a general well-designed model without prior knowledge is still challenging. In view of this, we propose a novel Graph Prompt Clustering (GPC) method. Within this model, there are two main modules, i.e., graph model pretraining as well as prompt and finetuning. In the graph model pretraining module, the graph model is pretrained by a selected source graph-level dataset with mutual information maximization and self-supervised clustering regularization. In the prompt and finetuning module, the network parameters of the pretrained graph model are frozen, and a groups of learnable prompt vectors assigned to each graph-level representation are trained for adapting different target graph-level datasets with various data distributions. Experimental results across six benchmark datasets demonstrate the impressive generalization capability and effectiveness of GPC compared with the state-of-the-art methods.

Drug combination therapy with significant advantages is a well-established concept in cancer treatment. Some related efforts have been made with multiple artful deep learning techniques. However, they are usually based on data for drug synergy prediction, ignoring the professional characteristics of data and the systematic knowledge accumulation. Meanwhile, integrating the dispersed professional knowledge and effectively utilizing it in data remains a crucial technical challenge. In this study, we propose KSDDC, a novel model for knowledge-aware synergistic discovery of drug combinations from a large language model (LLM) perspective (i.e., from the continuously learnable and refined large database). Within this framework, three main modules are well-designed, i.e., knowledge-aware drug feature auto-encoding, knowledge-aware cell line feature encoding and drug-drug synergy prediction. Informative embeddings of samples are discovered and combined to make accurate drug synergy prediction. Overall, KSDDC is superior compared with the other shallow machine learning based methods and deep learning based methods on several synergistic prediction benchmarks, where about 19% F1-score improvements over the second best method on DrugComb_1 can be observed. Starting with drug synergy prediction, our studies with knowledge-enabled data mining offer valuable insights and serve as a reference method for future research in this field.

Cross-domain recommendation (CDR) has received attention to solve the cold-start and data sparsity problems. Existing methods mainly focus on the information about overlapping users or items, neglecting to effectively and efficiently utilize the information about nonoverlapping users or items in the source domain. Currently, graph prompt learning is proposed to bridge the gap between the pretrained tasks and downstream tasks, which can fully use the information of the source domain. However, existing graph prompt based CDR methods are rare and solely focus on the user-item interaction graphs without considering extra auxiliary information. Therefore, in this article, we construct knowledge graphs (KGs) as auxiliary information and propose a novel knowledge-aware graph prompt tuning for CDR (KGP-CDR) model. First, the KGs of the source and target domains are constructed, respectively, and a graph encoder is pretrained on the KG of the source domain. In addition, two types of graph prompts are designed: soft graph prompts and personalized graph prompts. These prompts are finetuned in the target domain along with the pretrained graph encoder. Ultimately, the predicted rating can be acquired through the finetuned graph prompts. Experiments on five real-world datasets show that the proposed method performs better than the state-of-the-art methods.

Attributed graph clustering, aiming to discover the underlying graph structure and partition the graph nodes into several disjoint categories, is a basic task in graph data analysis. Although recent efforts over graph contrastive clustering have achieved decent performances, most of them get accustomed to construct the positive neighbor set by the generated pseudo clustering information, directly ignoring the ready-made neighbor nodes and the underlying semantics of edges in a graph. How to well deal with the graph neighbor-specific information to facilitate the performance of graph contrastive clustering is still a challenging problem. Therefore, in this work, we propose a novel Homophily Induced Contrastive Attributed Graph Clustering (HomoCAGC) method, where the power of homophily is exploited in facilitating the performance of contrastive attributed graph clustering, while the pseudo homophily in a graph is also explored and distinguished. Especially, the node feature as reliable information guidance is used to compute the underlying feature-oriented pair-wise node similarity, based on which the positive node pairs in contrastive regularizer are adjusted for better node representation characterization. According to the refined node representations, a triplet self-supervised clustering objective is well-designed to ensure the output embedding is cluster-oriented, and suitable for the downstream clustering task. Extensive experiments on seven benchmarks are conducted to demonstrate the effectiveness of HomoCAGC.

Clinical staging is crucial for treatment strategies and improving 5-year survival rates in hepatocellular carcinoma (HCC) patients. However, existing methods struggle to distinguish stages with highly similar textual features. Additionally, their lack of interpretability hampers their practical application in medical scenarios. Here, we introduce KnowST, a joint-knowledge network designed to leverage task relevance to explore implicit knowledge for interpretable staging prediction of liver cancer. First, the relevance of auxiliary tasks and the main task is established from two perspectives to guide the model's focus on staging-related implicit knowledge in radiology reports. Stages-to-stages: KnowST learns the inter-stage distinctions between different stages and the similarities within the same stages, using these as important references for staging differentiation. Factors-to-stages: Clinically, staging is determined by multiple tumor factors. These factors can serve as effective clues to assist KnowST in predicting the correct stage, especially in the case of confusing stages. Second, domain-specific word embeddings are introduced to bridge the gap between pre-trained language models and Chinese radiology reports. Lastly, tumor factor prediction enhances the credibility of the deep model in staging prediction, and its visualized results effectively demonstrate the model's interpretability. Overall, KnowST leverages the joint-knowledge from these two perspectives, effectively utilizing implicit information in radiology reports to achieve interpretable clinical staging. Compared to the optimal baselines, KnowST improves AUC by 7.69% and achieves 90.52% accuracy on 573 real-world radiology reports, while also demonstrating superior stage identification and stable performance across various metrics.

Patent data generally includes information from different perspectives or different types, and its heterogeneous attributes can be greatly beneficial to data clustering analysis. However, the existing patent analysis method always focus on the patent text cues, and such a strategy merely depends on the feature information to capture the data characteristics, failing to multi-type informative patent representation. Therefore, in this paper, to model the underlying structure/relationships of patent data, we employ the knowledge graph to depict the heterogeneous attributes of patent, and propose a novel Knowledge Graph-based Patent Clustering (KGPC) method, where the relationship reconstruction in knowledge graph as well as clustering-oriented representation refinement for patent clustering are jointly considered. With this model, there are three components, i.e., entity representation refinement, relationship reconstruction and self-supervised entity clustering. Given a patent knowledge graph as input, the entity representation refinement can be mutually boosted by the relationship reconstruction and self-supervised clustering objective, thereby leading to a balanced clustering-oriented output. Extensive experiments on several real-world patent knowledge graph datasets validate the effectiveness of KGPC while compared with the state-of-the-art.

Reciprocal recommender systems (RRS) have been widely used in online platforms such as online dating and recruitment. They can simultaneously fulfill the needs of both parties involved in the recommendation process. Due to the inherent nature of the task, interaction data is relatively sparse compared to other recommendation tasks. Existing works mainly address this issue through content-based recommendation methods. However, these methods often implicitly model textual information from a unified perspective, making it challenging to capture the distinct intentions held by each party, which further leads to limited performance and the lack of interpretability. In this paper, we propose a Knowledge-Aware Explainable Reciprocal Recommender System (KAERR), which models metapaths between two parties independently, considering their respective perspectives and requirements. Various metapaths are fused using an attention-based mechanism, where the attention weights unveil dual-perspective preferences and provide recommendation explanations for both parties. Extensive experiments on two real-world datasets from diverse scenarios demonstrate that the proposed model outperforms state-of-the-art baselines, while also delivering compelling reasons for recommendations to both parties.

Traffic prediction is a complex task. In the traditional forecasting methodologies, there are limitations regarding generalization, as specifically observed in the pretraining approaches. This is particularly evident in models involving temporal and spatial attributes, where finetuning often encounters numerous challenges. With the emergence of graph prompt learning, a new finetuning paradigm has gained increasing application. In the context of the traffic prediction methods involving spatio-temporal attributes, we endeavor to enhance its performance in the downstream tasks through graph pretraining and prompt-based finetuning. In particular, we propose a traffic prediction model based on graph prompt learning. This model deeply mines the hierarchical information of traffic data and its temporal attributes, by employing the method of graph pretraining and prompt learning to enhance the model’s generalizability and accuracy. Extensive experiments conducted on two real-world traffic prediction datasets confirm the effectiveness of the proposed method.

Due to the collective decision-making nature of enterprises, the process of accepting recommendations is predominantly characterized by an analytical synthesis of objective requirements and cost-effectiveness, rather than being rooted in individual interests. This distinguishes enterprise recommendation scenarios from those tailored for individuals or groups formed by similar individuals, rendering traditional recommendation algorithms less applicable in the corporate context. To overcome the challenges, by taking the corporate volunteer as an example, which aims to recommend volunteer activities to enterprises, we propose a novel recommendation model calledAttributeKnowledgeGraphNeuralNetworks (AKGNN). Specifically, a novel comprehensive attribute knowledge graph is constructed for enterprises and volunteer activities, based on which we obtain the feature representation. Then we utilize anextendedVariationalAuto-Encoder (eVAE) model to learn the preferences representation and then we utilize a GNN model to learn the comprehensive representation with representation of the similar nodes. Finally, all the comprehensive representations are input to the prediction layer. Extensive experiments have been conducted on real datasets, confirming the advantages of the AKGNN model. We delineate the challenges faced by recommendation algorithms in Business-to-Business (B2B) platforms and introduces a novel research approach utilizing attribute knowledge graphs.

Word polysemy poses a formidable challenge in the semantic similarity task, especially for complex Chinese semantic information. However, most existing methods tend to emphasize information expansion, often overlooking the fact that the added information may be either irrelevant or only weakly correlated. In view of this, we propose a novel approach that fuses knowledge enhancement and context filtering to achieve self-selective semantic expansion. This approach, termed mutual-scenario-aware multimodal-enhanced representation learning (MuSAM), integrates information across multiple modalities. Specifically, we extend individual words within three modalities, and the extended information with weak correlation is filtered and denoised, to get the mutual-scenario extended information. The extended information with strong correlation in each modality will be fused to obtain the multimodal fusion representation vector of the word pair. Experimental evaluations conducted on five datasets underscore the superiority of the MuSAM model over state-of-the-art methods, showcasing a performance improvement ranging from a minimum of 4% to a maximum of 21%. Remarkably, our model is designed in a completely engineering manner, which can be applied to real scenarios directly without manual intervention.

With the rapid growth of online platforms and the abundance of available information, personalized recommender systems have become essential for assisting users in discovering relevant and interesting content. Among the various methods, knowledge-aware recommendation model has achieved notable success by leveraging the rich semantic information encoded in knowledge graphs. However, it overlooks the fact that users’ historical click sequences can better reflect their preferences within a period of time, thus imposing certain limitations on the recommendation performance. On the other hand, the application of pre-trained language models in recommender systems has demonstrated increasingly significant potential, as they can capture sequential patterns and dependencies within users’ historical click sequences and effectively capture contextual information in user-item interactions. To this end, we propose a hybrid recommendation model that leverages Pre-training in the collaborative Knowledge graph Attention neTwork (PKAT), to extract both the high-order connectivity information in collaborative knowledge graphs and the contextual information in users’ historical click sequences captured by Bidirectional Encoder Representations from Transformers (BERT). The collaborative knowledge graph attention network enables the model to effectively capture the intricate relationships between users, items, and knowledge entities, thus enhancing the representation learning process. Furthermore, what sets PKAT apart from other state-of-the-art knowledge-aware recommendation methods is the incorporation of the BERT language model. This integration allows PKAT to capture the contextual sequence information of user behavior, enabling it to generate more accurate and personalized recommendations. Extensive experiments are conducted on multiple benchmark datasets. And the results demonstrate that our PKAT model outperforms several state-of-the-art baselines.

Most existing recommendation research has been concentrated on unidirectional recommendation, i.e. only recommending items to users. However, in many real-world scenarios, the platform needs to achieve bi-directional recommendation. For example, in an online recruitment scenario, the recommender system not only needs to recommend positions to candidates, but also recommend candidates to enterprises. In this paper, we first formalize a new recommendation problem called bi-directional recommendation and contribute a new bidirectional recommendation model named BiROR (Bi-directional Recommendation for Online Recruitment). In BiROR, an encoder component is utilized to learn the text embeddings, and a graph learning component is designed to learn the graph embeddings. In addition, a multi-task learning framework is designed to achieve bi-directional recommendation. In the multi-task learning framework, we share the text embeddings and graph embeddings to alleviate the problems of data sparsity and data asymmetry in online recruitment. Extensive experiments in a real-world task show that BiROR outperforms the state-of-the-art methods, verifying the effectiveness of the designs of our model.