Graph neural networks (GNNs) can effectively model structural information of graphs, making them widely used in knowledge graph (KG) reasoning. However, existing studies on the expressive power of GNNs mainly focuses on simple single-relation graphs, and there is still insufficient discussion on the power of GNN to express logical rules in KGs. How to enhance the logical expressive power of GNNs is still a key issue. Motivated by this, we propose Path-Neighbor enhanced GNN (PN-GNN), a method to enhance the logical expressive power of GNN by aggregating node-neighbor embeddings on the reasoning path. First, we analyze the logical expressive power of existing GNN-based methods and point out the shortcomings of the expressive power of these methods. Then, we theoretically investigate the logical expressive power of PN-GNN, showing that it not only has strictly stronger expressive power than C-GNN but also that its (k+1)-hop logical expressiveness is strictly superior to that of k-hop. Finally, we evaluate the logical expressive power of PN-GNN on six synthetic datasets and two real-world datasets. Both theoretical analysis and extensive experiments confirm that PN-GNN enhances the expressive power of logical rules without compromising generalization, as evidenced by its competitive performance in KG reasoning tasks.

Among various temporal knowledge graph (TKG) extrapolation methods, rule-based approaches stand out for their explicit rules and transparent reasoning paths. However, the vast search space for rule extraction poses a challenge in identifying high-quality logic rules. To navigate this challenge, we explore the use of generation models to generate new rules, thereby enriching our rule base and enhancing our reasoning capabilities. In this paper, we introduce LLM-DR, an innovative rule-based method for TKG extrapolation, which harnesses diffusion models to generate rules that are consistent with the distribution of the source data, while also amalgamating the rich semantic insights of Large Language Models (LLMs). Specifically, our LLM-DR generates semantically relevant and high-quality rules, employing conditional diffusion models in a classifier-free guidance fashion and refining them with LLM-based constraints. To assess rule efficacy, we meticulously design a coarse-to-fine evaluation strategy that initiates with coarse-grained filtering to eliminate less plausible rules and proceeds with fine-grained scoring to quantify the reliability of the retained. Extensive experiments demonstrate the promising capacity of our LLM-DR.

Traditional recommendation system focus more on the correlations between users and items (user-item relationships), while research on user-user relationships has received significant attention these years, which is also known as social recommendation. Graph-based models have achieved a great success in this task by utilizing the complex topological information of the social networks. However, these models still face the insufficient expressive and overfitting problems. Counterfactual approaches are proven effective as information augmentation strategies towards above issues in various scenarios, but not fully utilized in social recommendations. To this end, we propose a novel social recommendation method, termed SR-GCA, via a plug-and-play Graph-Level Counterfactual Augmentation mechanism. Specifically, we first generate counterfactual social and item links by constructing a counterfactual matrix for data aug- mentation. Then, we employ a supervised learning strategy to refine data both factual and counterfactual links. Thirdly, we enhance representations learning between users via an alignment and self-supervised optimization techniques. Extensive experiments demonstrate the promising capacity of our model from five aspects, including superiority, effectively, transfer- ability, complexity, sensitively. In particular, the transferability is well-proven by extending our GCA module to three typical social recommendation models.

Accurate prediction of opinion evolution is crucial for understanding public opinion dynamics. The neural message passing mechanisms, which exhibit conceptual similarities with opinion dynamics, are capable of autonomously updating parameters through gradient-based optimization techniques, thereby establishing themselves as a surrogate model for investigating the evolution of opinions. However, existing methods face three key limitations: (1) Social weights between individuals rely solely on node attribute similarity, ignoring temporal dynamics; (2) Temporal features are modeled only through time derivatives, and lacking robustness to external noise; (3) There is a lack of research on real-world opinion data with network structures, especially open-source datasets. To tackle these challenges, we propose Temporal Adaptive Neural Message Passing, a deep learning framework that addresses opinion dynamics through adaptive message passing, noise-aware data fusion to enhance long-term forecasting. Furthermore, we release three large-scale real-world datasets to serve as benchmarks for future research. For evaluation, we conducted experiments on three real-world and three synthetic datasets, comparing our method with existing mechanistic models (including hybrid models) and non-mechanistic models. The results demonstrate that our method achieves the best performance across all datasets.

Open-world knowledge graph completion (KGC) aims to infer novel facts by enriching existing graphs with external knowledge sources while maintaining semantic consistency under the open-world assumption (OWA).Generation-based KGC methods leverage the inherent strengths of large language models (LLMs) in language understanding and creative problem-solving, making them promising approaches.However, they face limitations: (1) The unreliable external knowledge from LLMs can lead to hallucinations and undermine KGC reliability.(2) The lack of an automated and rational evaluation strategy for new facts under OWA results in the exclusion of some new but correct entities.In the paper, we propose MusKGC, a novel multi-source knowledge enhancement framework based on an LLM for KGC under OWA.We induce relation templates with entity type constraints to link structured knowledge with natural language, improving the comprehension of the LLM.Next, we combine intrinsic KG facts with reliable external knowledge to guide the LLM in accurately generating missing entities with supporting evidence.Lastly, we introduce a new evaluation strategy for factuality and consistency to validate accurate inferences of new facts, including unknown entities.Extensive experiments show that our proposed framework achieves SOTA performance across benchmarks, and our evaluation strategy effectively assesses new facts under OWA.

Large models require larger datasets. While people gain from using massive amounts of data to train large models, they must be concerned about privacy issues. To address this issue, we propose a novel approach for private generative modeling using the Sliced Wasserstein Distance (SWD) metric in a Differential Private (DP) manner. We propose Normalized Clipping, a parameter-free clipping technique that generates higher-quality images. We demonstrate the advantages of Normalized Clipping over the traditional clipping method in parameter tuning and model performance through experiments. Moreover, experimental results indicate that our model outperforms previous methods in differentially private image generation tasks.

Temporal knowledge graph (TKG) reasoning has two settings: interpolation reasoning and extrapolation reasoning.Both of them draw plenty of research interest and have great significance.Methods of the former deemphasize the temporal correlations among facts sequences, while methods of the latter require strict chronological order of knowledge and ignore inferring clues provided by missing facts of the past.These limit the practicability of TKG applications as almost all of the existing TKG reasoning methods are designed specifically to address either one setting.To this end, this paper proposes an original Temporal PAth-based Reasoning (TPAR) model for both the interpolation and extrapolation reasoning.TPAR performs a neural-driven symbolic reasoning fashion that is robust to ambiguous and noisy temporal data and with fine interpretability as well.Comprehensive experiments show that TPAR outperforms SOTA methods on the link prediction task for both the interpolation and the extrapolation settings.A novel pipeline experimental setting is designed to evaluate the performances of SOTA combinations and the proposed TPAR towards interpolation and extrapolation reasoning.More diverse experiments are conducted to show the robustness and interpretability of TPAR.

Extrapolation reasoning on Temporal Knowledge Graphs (TKGs) aims to predict future events from a set of historical Knowledge Graphs (KGs) in a chronological order. The temporally adjacent facts in TKGs naturally form event sequences, implying informative temporal event dependencies. Recently, many extrapolation works have been devoted to modelling these dependencies, but the task is still far from resolved because existing works primarily rely on encoding event information into entity representations to achieve this purpose, while overlooking the significant temporal event dependencies implied by relations. In this work, we aim to learn relational temporal context to explore the temporal event dependencies implicit in relations and propose a Temporal relational context-based temporal dependencies learning Network (Trend) to capture the temporal dependencies both semantically and structurally. Experimental results on benchmark datasets demonstrate the superiority of Trend.

The large-scale Multi-Agent Path Finding (MAPF) problem presents a significant challenge in combinatorial optimization. Currently, one of the advanced, near-optimal algorithms is Large Neighborhood Search (LNS), which can handle instances with thousands of agents. Although a basic portfolio parallel search based on multiple independent LNS solvers enhances speed and robustness, it encounters scalability issues with increasing CPU cores. To address this limitation, we propose the Cooperative Parallel LNS (CPLNS) algorithm, aimed at boosting parallel efficiency. The main challenge in cooperative parallel search lies in designing suitable portfolio and cooperative strategies that balance search diversification and intensification. To address this, we first analyze the characteristics of LNS. We then introduce a flexible group-based cooperative parallel strategy, where the current best solution is shared within each group to aid intensification, while maintaining diversification through independent group computations. Furthermore, we augment search diversification by integrating a simulated annealing-based LNS and bounded suboptimal single-agent pathfinding. We also introduce a rule-based methodology for portfolio construction to simplify parameter settings and improve search efficiency. Finally, we enhance communication and memory efficiency through a shared data filtering technique and optimized data structures. In benchmarks on 33 maps with 825 instances, CPLNS achieved a median speedup of 21.95 on a 32-core machine, solving 96.97% of cases within five minutes and reducing the average suboptimality score from 1.728 to 1.456. Additionally, tests with up to 10,000 agents verify CPLNS's scalability for large-scale MAPF problems.

Acronym disambiguation is the process of determining the correct expansion of an acronym in given context, which can assist many downstream natural language processing tasks. Typically, existing methods on this task will directly perform semantic comparisons between the candidate expansions and the original sentence, ignoring the relevance of contextual information to expansions. To solve this issue, this paper proposes a context‐aware acronym disambiguation method with Siamese BERT network (ContextAD). First, we combine each candidate expansion with corresponding acronym’s context to form a new sentence set. Then, the new and original sentences are input into a Siamese BERT network that can obtain the semantic similarity. The new sentences and the separate candidate expansions are input into the Siamese BERT network, respectively, along with the original sentences, which can obtain another semantic similarity. Finally, the two different semantic similarities are combined to determine the most suitable expansion. We quantify the improvement of our proposed ContextAD model against a state‐of‐the‐art baseline using the public dataset of the shared tasks of acronym disambiguation (AD) held under AAAI‐2021 workshop on SDU and show that it achieves a better performance based on the same BERT model.

Temporal factors are tied to the growth of facts in realistic applications, such as the progress of diseases and the development of political situation, therefore, research on Temporal Knowledge Graph (TKG) attracks much attention.In TKG, relation patterns inherent with temporality are required to be studied for representation learning and reasoning across temporal facts.However, existing methods can hardly model temporal relation patterns, nor can capture the intrinsic connections between relations when evolving over time, lacking of interpretability.In this paper, we propose a novel temporal modeling method which represents temporal entities as Rotations in Quaternion Vector Space (RotateQVS) and relations as complex vectors in Hamilton's quaternion space.We demonstrate our method can model key patterns of relations in TKG, such as symmetry, asymmetry, inverse, and can further capture time-evolved relations by theory.Empirically, we show that our method can boost the performance of link prediction tasks over four temporal knowledge graph benchmarks.

This paper presents an approach to Alzheimer’s disease (AD) diagnosis from spontaneous speech using an end-to-end ASR-enhanced neural network. Under the condition that only audio data are provided and accurate transcripts are unavailable, this paper proposes a system that can analyze utterances to differentiate between AD patients, healthy controls, and individuals with mild cognitive impairment. The ASR-enhanced model comprises automatic speech recognition (ASR) with an encoder-decoder structure and the encoder followed by an AD classification network. The encoder takes a Mel spectrogram as input and transforms it into high-level acoustic features that correlate with AD. The classification network then maps intermediate acoustic features to three categories. In the training phase, the AD classification and speech recognition tasks are optimized simultaneously. Experimental results obtained from an AD recognition dataset of Chinese spontaneous speech1illustrate the effectiveness of integrating ASR into AD diagnosis in an end-to-end manner. Further, our model has low dependency on accurate ASR transcripts. This work achieved accuracy scores of 89.1% and 82.6% for long and short utterance tracks, respectively.

Recently, large-scale datasets have vastly facilitated the development in nearly domains of Natural Language Processing. However, lacking large scale Chinese corpus is still a critical bottleneck for further research on deep text summarization methods. In this paper, we publish a large-scale Chinese Long-text Extractive Summarization corpus named CLES. The CLES contains about 104Kpairs, which is originally collected from Sina Weibo1. To verify the quality of the corpus, we also manually tagged the relevance score of 5,000pairs. Our benchmark models on the proposed corpus include conventional deep learning based extractive models and several pre-trained Bert-based algorithms. Their performances are reported and briefly analyzed to facilitate further research on the corpus. We will release this corpus for further research2.

Depression is one of the most common mental health disorders, it’s crucial to design an effective and robust model for automatic depression detection (ADD). Although current approaches rely on extra topic models or manually topic-selection procedures which is time-consuming, they still haven’t thoroughly explored the sufficient context information among clinical interviews. In this paper, we propose HCAG, a novel Hierarchical Context-Aware Graph attention model for ADD. Our model mirrors the hierarchical structure of depression assessment and leverages the Graph Attention Network (GAT) to grasp relational contextual information of text/audio modality. Experiments on the DAIC-WOZ dataset show a great performance improvement, with the Fl-score of 0.92, a Mean Absolute Error (MAE) of 2.94, and a Root Mean Square Error (RMSE) of 3.80. To the best of our knowledge, our model outperforms the existing state-of-the-art methods.

Universal constructions are attractive as they can turn a sequential implementation of any data structure into a concurrent implementation. However, existing universal constructions have limitations, such as imposing high copying overhead, or poor scalability on NUMA systems mainly due to their lack of NUMA-aware design principles. To overcome these limitations, this paper introduces CR, a universal construction that provides highly scalable updates on NUMA systems while offering fast read-side performance. CR achieves NUMA-awareness by utilizing delegation within a NUMA node and a global shared log to maintain the consistency of replicas of data structures across nodes. Using CR does not require expertise in concurrent data structure design. Our evaluation shows that CR has up to 11.2 times better performance compared to a state-of-the-art universal construction CX on our tested sequential data structures. To demonstrate the effectiveness and applicability of CR, we have applied CR to an in-memory database system. The database shows up to 18.1 times better performance compared to the original version.

In the last few years, the availability of temporal knowledge graphs has stimulated extensive research in temporal knowledge graph completion (TKGC) and temporal knowledge graph embedding (TKGE), where temporal information is added to static knowledge graphs that have been widely applied previously. However, most existing methods, such as current state-of-the-art DE-SimplE and TeRo, learn embeddings of temporal-evolving attributes, overlooking the inherent attributes inside entities, where some essential and inherent features are included. In this paper, we introduce a novel method utilizing Inherent Attributes with a Graph Attention network (IAGAT) for TKGC. Our IAGAT extracts inherent attributes from sufficient features corresponding to various facts at different time stamps, to obtain the inherent embeddings. And we take advantages of previous rotation based methods to obtain the temporal-evolving embed-dings. Through extensive experiments and sufficient comparisons, we demonstrate our model outperforms the current state-of-the-art models on link prediction task. Furthermore, we evaluate and prove the necessity of the inherent attributes in performance improvement, and study how our model functions in extracting inherent features.

Knowledge Graphs represent facts with triples containing head entity$h$, tail entity$t$and relation$r$which facilitate applications of many scenarios, for example intelligent web search, community detection and question answering. Knowledge Graph Embedding (KGE) represents elements of triples in a low-dimensional continuous vector space. Though it has been widely studied by both academic and industry communities, most researches focus on learning embeddings of entities and relations separately rather than considering the interactive semantic information between them. However, neither relations nor entities exist lonely out of context. In this paper, we define an interactive semantic space to model the context of triples and propose a novel Bi-linear Semantic Matching Method using Convolutional networks (BiSC). Specifically, we use 1D convolutional neural networks to extract features of the interactive semantics and then compute the similarity scores in the bi-linear space. Compared to existing complex graph network methods, BiSC needs lower computational cost to reach competitive results on link prediction task. The consistent state-of-the-art performance through extensive experiments over two benchmarks demonstrates the advantages of the proposed BiSC model. Further analysis on convergence study and case study of interactive semantic space show the efficiency of our model.

End-to-end face quality assessment based on deep learning can directly predict the overall quantitative score of face quality, thus helping to control the risk of face recognition system. Thanks to the development of automatic quality pseudo-label generation, most recent methods can use large-scale face datasets to learn the quality model. However, existing methods use regression models to fit the pseudo-labels, which lack attention to samples that are easy to be misidentified, and require large models for training. The paper treats the quality assessment as a classification problem, focusing on difficult samples near the classification boundary. Specifically, pairwise binary quality pseudo-label is generated based on the face similarity score without additional manual annotation. An identification quality loss is used to decouple the pairwise network training. In addition, a lightweight quality network is trained by performing knowledge distillation on the quality prediction branch of the face recognition network. Experiments show that the proposed quality network achieves state-of-the-art results with only 0.45M parameters and 77M FLOPs.

A mispronunciation detection and diagnosis (MD&D) system typically consists of multiple stages, such as an acoustic model, a language model and a Viterbi decoder. In order to integrate these stages, we propose SED-MDD, an end-to-end model for sentence dependent mispronunciation detection and diagnosis (MD&D) . Our proposed model takes mel-spectrogram and characters as inputs and outputs the corresponding phone sequence. Our experiments prove that SED-MDD can implicitly learn the phonological rules in both acoustic and linguistic features directly from the phonological annotation and transcription in the training data. To the best of our knowledge, SED-MDD is the first model of its kind and it achieves an accuracy of 86.35% and a correctness of 88.61% on L2-ARCTIC which significantly outperforms the existing end-to-end mispronunciation detection and diagnosis (MD&D) model CNN-RNN-CTC.

Existing ranking ELM algorithms bias to imbalanced queries since they equally treat each pairwise error. In this study we propose a flexible ranking ELM method based on matrix-centering transformation to replace the traditional graph Laplacian matrix based methods. Specifically, we introduce a useful query-level normalized loss function and enforce the matrix-centering transformation to it to avoid training a bias model. Fortunately, by this setting, we can also greatly simplify the learning process of ELM because of the symmetry and idempotence of the centering matrix. Based on the proposed framework, three different ranking ELM variants are implemented: (a) a regularized ranking ELM model; (b) an enhanced incremental ranking ELM model; and (c) an online sequential ranking ELM model. Experimental results demonstrate that our proposed ranking ELM algorithms can obtain comparable or better performances than the state-of-the-art ranking algorithms.

Drug-drug interactions (DDIs) are crucial for healthcare. Besides DDIs reported in medical knowledge bases such as DrugBank, a large number of latest DDI findings are also reported in unstructured biomedical literature. Extracting DDIs from unstructured biomedical literature is a worthy addition to the existing knowledge bases. Currently, convolutional neural network (CNN) is a state-of-the-art method for DDI extraction. One limitation of CNN is that it neglects long distance dependencies between words in candidate DDI instances, which may be helpful for DDI extraction. In order to incorporate the long distance dependencies between words in candidate DDI instances, in this work, we propose a dependency-based convolutional neural network (DCNN) for DDI extraction. Experiments conducted on the DDIExtraction 2013 corpus show that DCNN using a public state-of-the-art dependency parser achieves an F-score of 70.19%, outperforming CNN by 0.44%. By analyzing errors of DCNN, we find that errors from dependency parsers are propagated into DCNN and affect the performance of DCNN. To reduce error propagation, we design a simple rule to combine CNN with DCNN, that is, using DCNN to extract DDIs in short sentences and CNN to extract DDIs in long distances as most dependency parsers work well for short sentences but bad for long sentences. Finally, our system that combines CNN and DCNN achieves an F-score of 70.81%, outperforming CNN by 1.06% and DNN by 0.62% on the DDIExtraction 2013 corpus.