Gang Hu, Yating Chen, Haiyan Ding, Wang Gao, Huang Jiajia, Min Peng, Qianqian Xie, Kun Yue. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Knowledge Graph Question Answering (KGQA) aims to answer natural language questions by reasoning over structured knowledge graphs (KGs). While large language models (LLMs) have advanced KGQA through their strong reasoning capabilities, existing methods continue to struggle to fully exploit both the rich knowledge encoded in KGs and the reasoning capabilities of LLMs, particularly in complex scenarios. They often assume complete KG coverage and lack mechanisms to judge when external information is needed, and their reasoning remains locally myopic, failing to maintain coherent multi-step planning, leading to reasoning failures even when relevant knowledge exists. We propose Graph-RFT, a novel two-stage reinforcement fine-tuning KGQA framework with a ''plan–KGsearch–and–Websearch–during–think'' paradigm, that enables LLMs to perform autonomous planning and adaptive retrieval scheduling across KG and web sources under incomplete knowledge conditions. Graph-RFT introduces a chain-of-thought (CoT) fine-tuning method with a customized plan–retrieval dataset activates structured reasoning and resolves the GRPO cold-start problem. It then introduces a novel plan–retrieval guided reinforcement learning process integrates explicit planning and retrieval actions with a multi-reward design, enabling coverage-aware retrieval scheduling. It employs a Cartesian-inspired planning module to decompose complex questions into ordered sub-questions, and logical expression to guide tool invocation for globally consistent multi-step reasoning. This reasoning–retrieval process is optimized with a multi-reward combining outcome and retrieval-specific signals, enabling the model to learn when and how to combine KG and web retrieval effectively. Experiments on multiple KGQA benchmarks demonstrate that Graph-RFT achieves superior performance over strong baselines, even with smaller LLM backbones, and substantially improves complex question decomposition, factual coverage, and tool coordination.

AI has found extensive application in edge analytics, with Deep Neural Networks effectively processing and analyzing data. Some applications need to deal with unstructured data like graphs. This complexity and diversity of unstructured features pose challenges for traditional DNN models. Therefore, Graph Neural Networks have emerged as a significant solution. However, GNNs encounter two primary challenges in edge analytics scenarios: scalability and efficiency. These challenges arise primarily from significant structural differences among various graphs and models. To address these issues, we propose AGF, an Adaptive GNN Framework. AGF consists of three components: subgraph partitioning and model partitioning, scalable hardware architecture, and design space exploration. Subgraph partitioning and model partitioning transform complex and diverse GNNs into a uniform computing flow and subgraphs. These can be directly mapped onto the hardware kernel, leveraging GNN characteristics to achieve independent parallelism. The hardware architecture is based on a multi-level processing element structure, which can efficiently parallelize subgraphs and vertices based on the partitioning. The design space exploration selects kernel allocation strategies and setups based on the hardware, dataset, and model. We validated AGF using six datasets and three GNN models. The results demonstrate that our framework achieves an acceleration ranging from 4.54 to 53.17× compared to the GPU-based GNN framework. When compared to other FPGA-based GNN accelerators, AGF achieves a latency reduction ranging from 1.16 to 33.6× in model execution and from 1.09 to 2.37× in end-to-end scenarios.

Large Language Models (LLMs) present massive inherent knowledge and superior semantic comprehension capability, which have revolutionized various tasks in natural language processing. Despite their success, a critical gap remains in enabling LLMs to perform knowledge graph completion (KGC). Empirical evidence suggests that LLMs consistently perform worse than conventional KGC approaches, even through sophisticated prompt design or tailored instruction-tuning. Fundamentally, applying LLMs on KGC introduces several critical challenges, including a vast set of entity candidates, hallucination issue of LLMs, and under-exploitation of the graph structure. To address these challenges, we propose a novel instruction-tuning-based method, namely FtG. Specifically, we present a filter-then-generate paradigm and formulate the KGC task into a multiple-choice question format. In this way, we can harness the capability of LLMs while mitigating the issue casused by hallucinations. Moreover, we devise a flexible ego-graph serialization prompt and employ a structure-text adapter to couple structure and text information in a contextualized manner. Experimental results demonstrate that FtG achieves substantial performance gain compared to existing state-of-the-art methods. The instruction dataset and code are available at https://github.com/LB0828/FtG.

In recent years, Temporal Knowledge Graphs (TKGs) have developed rapidly and Temporal Knowledge Graph Completion (TKGC) aims at completing the missing parts of incomplete TKGs. Many methods have been proposed for TKGC, but there are still unresolved issues. Current methods have not fully explored the temporal distribution features of different entities, and the modeling approaches for historical graphs overlook internal temporal correlations. To solve these issues, we propose LeREC, Learning from REConstruction: A Two-Stage Global-to-Local Framework for TKGC. In the first stage, we use the temporal distribution features of entities as input and employ VGAE to reconstruct the simplified global historical graph. In the second stage, we introduce time-aware relational attributes to integrate multiple local historical graphs and predict missing entities in the query. Experiments conducted on benchmark datasets demonstrate the superiority of our model.

Graph foundation models (GFMs) have emerged as a promising paradigm for learning transferable knowledge across diverse graph-structured data. The inherent heterogeneity in features and graph structures poses significant challenges for building scalable and generalizable GFMs. Existing research has employed mixture-of-experts (MoE) models to handle the challenges, assigning the most suitable expert to each graph. Despite this, the underlying mechanisms of MoE within the context of GFMs remain insufficiently explored. In this work, we conduct an in-depth experimental study on an MoE-based GFM and uncover an intriguing finding: the experts ranked second and third assigned by the router perform better than the top-ranked expert. This insight motivates us to investigate the potential of leveraging knowledge embedded across multiple experts. However, directly ensembling the outputs of multiple experts would incur substantial computational overhead, while applying a standard expert merging strategy risks suboptimal performance. To address these challenges, we introduce two enhanced expert merging strategies that retain the computational efficiency of expert merging, while improving performance to approach the effectiveness of expert ensembling. Specifically, we propose (i) a knowledge distillation-inspired expert merging method that aligns the behavior of parameter-fused experts with expert ensembles, and (ii) a theoretical parameter proximity approach that leverages the similarity of expert parameters to approximate ensemble outputs while preserving diversity. Extensive experiments demonstrate that our methods effectively enhance model performance.

The application of AI in psychiatric diagnosis faces significant challenges, including the subjective nature of mental health assessments, symptom overlap across disorders, and privacy constraints limiting data availability. To address these issues, we present MoodAngels, the first specialized multi-agent framework for mood disorder diagnosis. Our approach combines granular-scale analysis of clinical assessments with a structured verification process, enabling more accurate interpretation of complex psychiatric data. Complementing this framework, we introduce MoodSyn, an open-source dataset of 1,173 synthetic psychiatric cases that preserves clinical validity while ensuring patient privacy. Experimental results demonstrate that MoodAngels outperforms conventional methods, with our baseline agent achieving 12.3\% higher accuracy than GPT-4o on real-world cases, and our full multi-agent system delivering further improvements. Together, these contributions provide both an advanced diagnostic tool and a critical research resource for computational psychiatry, bridging important gaps in AI-assisted mental health assessment.

Recent advancements have highlighted that Large Language Models (LLMs) are prone to hallucinations when solving complex reasoning problems, leading to erroneous results. To tackle this issue, researchers incorporate Knowledge Graphs (KGs) to improve the reasoning ability of LLMs. However, existing methods face two limitations: 1) they typically assume that all answers to the questions are contained in KGs, neglecting the incompleteness issue of KGs, and 2) they treat the KG as a static repository and overlook the implicit logical reasoning structures inherent in KGs. In this paper, we introduce SymAgent, an innovative neural-symbolic agent framework that achieves collaborative augmentation between KGs and LLMs. We conceptualize KGs as dynamic environments and transform complex reasoning tasks into a multi-step interactive process, enabling KGs to participate deeply in the reasoning process. SymAgent consists of two modules: Agent-Planner and Agent-Executor. The Agent-Planner leverages LLM's inductive reasoning capability to extract symbolic rules from KGs, guiding efficient question decomposition. The Agent-Executor autonomously invokes predefined action tools to integrate information from KGs and external documents, addressing the issues of KG incompleteness. Furthermore, we design a self-learning framework comprising online exploration and offline iterative policy updating phases, enabling the agent to automatically synthesize reasoning trajectories and improve performance. Experimental results demonstrate that SymAgent with weak LLM backbones (i.e., 7B series) yields better or comparable performance compared to various strong baselines. Further analysis reveals that our agent can identify missing triples, facilitating automatic KG updates.

Graph Neural Networks (GNNs) have become pivotal for analyzing relational data in embedded intelligent systems such as IOT devices. However, their deployment on resource-constrained devices faces critical barriers: traditional graph partitioning methods induce unbalanced computational loads due to rigid granularity, while hardware mapping strategies cause inefficient resource utilization under dynamic graph structures. These limitations conflict with the requirements of embedded systems for resource efficiency and scalability. To address this, we present GNNmap, a hardware-software co-design framework that synergizes multi-granular graph partitioning with topology-aware GNN mapping. The framework first reconstructs input graphs into balanced kernel groups comprising cohesive supernodes (corresponding to parallelizable subgraphs). By combining coarse-grained partitioning with fine-grained optimization, GNNmap ensures load balance while dramatically reducing cross-subgraph communication. Concurrently, a subgraph-PE mapping based on coarse-grained reconfigurable architectures (CGRAs) enables efficient graph-to-hardware matching through the joint modeling of graph topological features and hardware resource constraints. By dynamically coordinating graph reorganization and hardware resource allocation, GNNmap resolves the intrinsic mismatch between irregular graph computations and static hardware configurations. Experimental results demonstrate that GNNmap achieves improvements over existing works, improving inference performance by 1.47× to 62.8×, resource efficiency by 1.15× to 3.06×, and energy efficiency by 1.34× to 3.50×.

Graph Neural Networks (GNNs) have seen considerable advancements across various applications. However, the computationally and storage-intensive nature of GNNs presents unique challenges for hardware design. Existing GNN accelerators frequently grapple with low execution efficiency and workload imbalance. Therefore, this article introduces DGMF, a unified dynamic mapping approach that optimizes dataflow and computation mode based on GNN models and datasets. To support this dynamic mapping approach, we also present a dedicated hardware architecture. This architecture dynamically adjusts resource allocation and parallelism based on models and datasets, thereby ensuring optimal utilization. Additionally, we propose a Design Space Exploration (DSE) algorithm that traverses the defined design space to identify the optimal design solution, effectively managing diverse design constraints and objectives. To validate our approach, we developed an accelerator based on three datasets and five types of GNN models. Experimental results demonstrate that DGMF achieves performance by 10.5×–2,548×, and 1.01×–51.7× compared to GPUs and existing accelerators. It also improves resource efficiency, energy efficiency, and DRAM access by 0.87×–1.47×, 0.92×–2.18×, and 1.01×–47.25×, respectively, compared to other works.

Mengxi Xiao, Qianqian Xie, Ziyan Kuang, Zhicheng Liu, Kailai Yang, Min Peng, Weiguang Han, Jimin Huang. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Background: Code pre-training and large language models are heavily dependent on data quality. These models require a vast, high-quality corpus matching text descriptions with codes to establish semantic correlations between natural and programming languages. Unlike NLP tasks, code comment heavily relies on specialized programming knowledge and is often limited in quantity and variety. Thus, most widely available open-source datasets are established with compromise and noise from platforms, such as StackOverflow, where code snippets are often incomplete. This may lead to significant errors when deploying the trained models in real-world applications. Aims: Comments as a substitute for queries are used to build code search datasets from GitHub. While comments describe code functionality and details, they often contain noise and differ from queries. Thus, our research focuses on improving the syntactic and semantic quality of code comments. Method: We propose a comment-based data refinement framework CoCoRF 1 via an unsupervised and supervised co-learning technique. It applies manually defined rules for syntax filtering and constructs a bootstrap query corpus via the WTFF algorithm for training the TVAE model for further semantic filtering. Results: Our study shows that CoCoRF achieves high efficiency with less computational resource, and outperforms comparison models in DeepCS code search task. Conclusions: Our findings indicate that the CoCoRF framework significantly improves the performance of code search tasks by enhancing the quality of code datasets.

The assessment of image aesthetics involves both objective and subjective elements. While recent studies have utilized textual cues from image comments for assessment, they encounter challenges in establishing an intimate connection between subjective aesthetic aspects in text and the corresponding visual elements in images. Furthermore, the absence of annotated comments introduces abundant noises from unrelated aesthetic comments, adversely affecting model performance. To overcome these challenges, this paper innovates an automated annotation pipeline (TASP) that leverages a large-scale language model to extract Textual Subjective Aesthetic Attributes. In contrast to manual methods, our pipeline achieves high accuracy while being significantly more cost-efficient and transparent for reproducibility. Additionally, we propose a Multi-Channel Fusion Network (MACFAN) to effectively capture Multi-modal Subjective Aesthetic Attributes, enhancing the model’s understanding of universal aesthetics. Experimental results demonstrate the superior performance of our model over existing state-of-the-art models.

In inductive link prediction for evolving knowledge graphs (KGs), interpretability is crucial yet often overlooked in relational message aggregation methods. Previous approaches typically neglect deeper relational insights, limiting their explanatory power. In this paper, we propose CAPM (Constrained Aggregation and Predictive Modeling) to address this critical gap by uniquely incorporating semantic descriptions of entities. This integration allows for a clearer understanding of how and why certain links are predicted and strengthens its ability to contextualize and clarify the relationships within the KG. In particular, CAPM combines entity type and an attention mechanism during aggregation, ensuring a sophisticated blend of structured and semantic information. This method also improves relevance assessment in relational paths, leveraging prior knowledge in adjacent relations. Tested on sparse KGs, CAPM demonstrates exceptional performance in inductive link prediction scenarios. Ablation studies confirm its superiority, particularly when combined with embedding-based methods for entity-type representation, highlighting its effectiveness in evolving KGs. Through this innovative approach, CAPM offers a comprehensive solution that balances the dynamic nature of KGs with the essential need for interpretability in link prediction.

Despite Spanish's pivotal role in the global finance industry, a pronounced gap exists in Spanish financial natural language processing (NLP) and application studies compared to English, especially in the era of large language models (LLMs).To bridge this gap, we unveil Toisón de Oro, the first bilingual framework that establishes instruction datasets, finetuned LLMs, and evaluation benchmark for financial LLMs in Spanish joint with English.We construct a rigorously curated bilingual instruction dataset including over 144K Spanish and English samples from 15 datasets covering 7 tasks.Harnessing this, we introduce FinMA-ES, an LLM designed for bilingual financial applications.We evaluate our model and existing LLMs using FLARE-ES, the first comprehensive bilingual evaluation benchmark with 21 datasets covering 9 tasks.The FLARE-ES benchmark results

LLMs have transformed NLP and shown promise in various fields, yet their potential in finance is underexplored due to a lack of comprehensive benchmarks, the rapid development of LLMs, and the complexity of financial tasks. In this paper, we introduce FinBen, the first extensive open-source evaluation benchmark, including 42 datasets spanning 24 financial tasks, covering eight critical aspects: information extraction (IE), textual analysis, question answering (QA), text generation, risk management, forecasting, decision-making, and bilingual (English and Spanish). FinBen offers several key innovations: a broader range of tasks and datasets, the first evaluation of stock trading, novel agent and Retrieval-Augmented Generation (RAG) evaluation, and two novel datasets for regulations and stock trading. Our evaluation of 21 representative LLMs, including GPT-4, ChatGPT, and the latest Gemini, reveals several key findings: While LLMs excel in IE and textual analysis, they struggle with advanced reasoning and complex tasks like text generation and forecasting. GPT-4 excels in IE and stock trading, while Gemini is better at text generation and forecasting. Instruction-tuned LLMs improve textual analysis but offer limited benefits for complex tasks such as QA. FinBen has been used to host the first financial LLMs shared task at the FinNLP-AgentScen workshop during IJCAI-2024, attracting 12 teams. Their novel solutions outperformed GPT-4, showcasing FinBen's potential to drive innovations in financial LLMs. All datasets and code are publicly available for the research community, with results shared and updated regularly on the Open Financial LLM Leaderboard.

Link prediction (LP) in knowledge graph (KG) is a crucial task that has received increasing attention recently. Due to the heterogeneous structures of KGs, various application scenarios, and demand-specific downstream objectives, there exist multiple subtasks in LP. Most studies only focus on designing a dedicated architecture for a specific subtask, which results in various complicated LP models. The isolated architectures and chaotic situations make it significant to construct a unified model that can handle multiple LP subtasks simultaneously. However, unifying all subtasks in LP presents numerous challenges, including unified input forms, task-specific context modeling, and topological information encoding. To address these challenges, we propose a topology-aware generative framework, namely UniLP, which utilizes a generative pre-trained language model to accomplish different LP subtasks universally. Specifically, we introduce a context demonstration template to convert task-specific context into a unified generative formulation. Based on the unified formulation, to address the limitation of transformer architecture that may overlook important structural signals in KGs, we design novel topology-aware soft prompts to deeply couple topology and text information in a contextualized manner. Extensive experiment results demonstrate that our framework achieves substantial performance gain and provides a real unified end-to-end solution for the whole LP subtasks. We also perform comprehensive ablation studies to support in-depth analysis of each component in UniLP.

In computer vision, the joint development of the algorithm and computing dimensions cannot be separated. Models and algorithms are constantly evolving, while hardware designs must adapt to new or updated algorithms. Reconfigurable devices are recognized as important platforms for computer vision applications because of their reconfigurability. There are two typical design approaches: customized and overlay design. However, existing work is unable to achieve both efficient performance and scalability to adapt to a wide range of models. To address both considerations, we propose a design framework based on reconfigurable devices to provide unified support for computer vision models. It provides software-programmable modules while leaving unit design space for problem-specific algorithms. Based on the proposed framework, we design a model mapping method and a hardware architecture with two processor arrays to enable dynamic and static reconfiguration, thereby relieving redesign pressure. In addition, resource consumption and efficiency can be balanced by adjusting the hyperparameter. In experiments on CNN, vision Transformer, and vision MLP models, our work’s throughput is improved by 18.8x–33.6x and 1.4x–2.0x compared to CPU and GPU. Compared to others on the same platform, accelerators based on our framework can better balance resource consumption and efficiency.

Slot filling is a major problem in spoken language understanding (SLU) task. However, the current SLU models may experience performance degradation when encountering unfamiliar data in different datasets. Meanwhile, as recent models are becoming more complex, retraining the model in a new application scenario would be too expensive. So we think it is important to study the robustness and generalization capability of SLU models. Then we propose the Natural Adversarial Slot Evaluator (NASE), a benchmark with adversarial SLU data to evaluate the robustness and generalization capability of SLU models on the task of slot filling. Our experiments and analysis reveal that all of the six SLU models have a significant performance degradation on NASE. The further analysis points out that the models rely more on the context of the slots than the slot values themselves to make predictions. In addition, the widespread use of joint learning strategy makes unfamiliar intents also affect the slot filling. Based on our findings, we also propose a simple data augmentation method to improve the robustness of SLU models in slot filling. The F1 Scores improve up to about 30% compared to the original model.

Although large language models (LLMs) have shown great performance in natural language processing (NLP) in the financial domain, there are no publicly available financially tailored LLMs, instruction tuning datasets, and evaluation benchmarks, which is critical for continually pushing forward the open-source development of financial artificial intelligence (AI). This paper introduces PIXIU, a comprehensive framework including the first financial LLM based on fine-tuning LLaMA with instruction data, the first instruction data with 128K data samples to support the fine-tuning, and an evaluation benchmark with 8 tasks and 15 datasets. We first construct the large-scale multi-task instruction data considering a variety of financial tasks, financial document types, and financial data modalities. We then propose a financial LLM called FinMA by fine-tuning LLaMA with the constructed dataset to be able to follow instructions for various financial tasks. To support the evaluation of financial LLMs, we propose a standardized benchmark that covers a set of critical financial tasks, including six financial NLP tasks and two financial prediction tasks. With this benchmark, we conduct a detailed analysis of FinMA and several existing LLMs, uncovering their strengths and weaknesses in handling critical financial tasks. The model, datasets, benchmark, and experimental results are open-sourced to facilitate future research in financial AI.

Abstract Lung squamous cell carcinoma (LUSC) is associated with a worse prognosis than other histological subtypes of non-small cell lung cancer. Due to the vital role of CD8+ T cells in anti-tumor immunity, the characterization of CD8+ T cell infiltration-related (CTLIR) gene signature in LUSC is worthy of in-depth exploration. In our study, tumor tissues of LUSC patients from Renmin Hospital of Wuhan University were stained by multiplex immunohistochemistry to evaluate the density of infiltrated CD8+ T cells and explore the correlation with immunotherapy response. We found that the proportion of LUSC patients who responded to immunotherapy was higher in the high density of CD8+ T cell infiltration group than in the low density of CD8+ T cell infiltration group. Subsequently, we collected bulk RNA-sequencing data from The Cancer Genome Atlas (TCGA) database. The abundance of infiltrating immune cells in LUSC patients was analyzed by using CIBERSORT algorithm, and weighted correlation network analysis was performed to identify the co-expressed gene modules related to CD8+ T cells. We then developed a prognostic gene signature based on CD8+ T cell co-expressed genes and calculated the CTLIR risk score, which stratified LUSC patients into high-risk and low-risk groups. With univariate and multivariate analyses, the gene signature was identified as an independent prognostic factor in LUSC patients. The overall survival of LUSC patients in the high-risk group was significantly shorter than that of the low-risk group in the TCGA cohort, which was validated in Gene Expression Omnibus datasets. We analyzed immune cell infiltration in the tumor microenviroment and found fewer CD8+ T cells and more regulatory T cell infiltration in the high-risk group, which is characterized as an immunosuppressive phenotype. Furthermore, the LUSC patients in the high-risk group were predicted to have a better response to immunotherapy than those in the low-risk group when treated with PD-1 and CTLA4 inhibitors. In conclusion, we performed a comprehensive molecular analysis of the CTLIR gene signature in LUSC and constructed a risk model for LUSC patients to predict prognosis and immunotherapy response.

Emerging non-volatile memory (NVM) has been considered as a promising candidate for the next generation memory architecture because of its excellent characteristics. However, the endurance of NVM is much lower than DRAM. Without additional wear management technology, its lifetime can be very short, which extremely limits the use of NVM. This paper observes that the tail wear with a very small percentage of extreme deviation significantly hurts the lifetime of NVM, which the existing methods do not effectively solve. We present Lamina to address the tail wear issue, in order to improve the lifetime of NVM. Lamina consists of two parts: bounded tail wear leveling (BTWL) and lightweight wear enhancement (LWE). BTWL is used to make the wear degree of all pages close to the average value and control the upper limit of tail wear. LWE improves the accuracy of BTWL by exploiting the locality to interpolate low-frequency sampling schemes in virtual memory space. Our experiments show that compared with the state-of-the-art methods, Lamina can significantly improve the lifetime of NVM with low overhead.

We propose a scalable graph-level optimizer named GLite to speed up search-based optimizations on large neural networks. GLite leverages a potential-based partitioning strategy to partition large computation graphs into small subgraphs without losing profitable substitution patterns. To avoid redundant subgraph matching, we propose a dynamic programming algorithm to reuse explored matching patterns. The experimental results show that GLite reduces the running time of search-based optimizations from hours to milliseconds, without compromising in inference performance.

To tackle the understanding of complex questions, Question Decomposition Meaning Representation (QDMR) decomposes a complex question into a sequence of atomic simple questions. However, state-of-the-art QDMR parsers neglect the type information of simple questions and the dependency information between simple questions, leading to limited overall performance. In this paper, we propose a Decomposition Graph Reconstruction (DGR) model to induce the information by introducing two additional tasks: simple question type classification and dependency relationship detection. DGR can generate sub-questions with the expected type and organize sub-questions with correct logical connections. Experimental results on the BREAK QDMR dataset demonstrate the effectiveness of our method compared with previous methods.

Few-shot slot tagging is an important task in developing dialogue system. Most previous few-shot slot tagging models classify an item according to its similarity to the representation of each class. These models leverage context information implicitly through each words' contextual embedding. However, the entangled language features of words may interfere with context information, misleading the utilization of crucial slot features in few-shot scenario. To tackle these problems, we propose the Decoupled Context Enhanced Network (DCEN) for few-shot slot tagging. Different from previous models, we extract decoupled context explicitly to make full use of slot features contained in the context. Decoupled context includes two parts, local and global decoupled context information. We introduce a local extractor to extract local decoupled context by integrating information from adjacent words, and a global extractor based on transformer to extract global decoupled information by orthogonalization. Experimental results on SNIPS show that our model achieves the state-of-the-art performance with considerable improvements.

Qianqian Xie, Jimin Huang, Pan Du, Min Peng, Jian-Yun Nie. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Graph Neural Networks (GNNs) such as GCN can effectively learn document representations via the semantic relation graph among documents and words. However, despite a few exceptions, most of the previous work in this line of research does not consider the underlying topical semantics inherited in document contents and the relation graph, making the representations less effective and hard to interpret. In a few recent studies trying to incorporate latent topics into GNNs, the topics have been learned independently from the relation graph modeling. Intuitively, topic extraction can benefit much from the information propagation of the relation graph structure - directly and indirectly connected documents and words have similar topics. In this paper, we propose a novel Graph Topic Neural Network (GTNN) model to mine latent topic semantics for interpretable document representation learning, taking into account the document-document, document-word, and word-word relationships in the graph. We also show that our model can be viewed as semi-amortized inference for relational topic model based on Poisson distribution, with high order correlations. We test our model in several settings: unsupervised, semi-supervised, and supervised representation learning, for both connected and unconnected documents. In all the cases, our model outperforms the state-of-the-art models for these tasks.

In recent years, code search techniques on software Q&A sites have become increasingly attractive due to the need for software development. Most of the existing work treats code snippets as text fragments, ignoring the effect of the structured information (i.e. sequential information) of the code. Meanwhile, much of the existing work does not take into account the interactive between code snippets and queries. In this paper, we propose a novel deep neural network named HECS1(Hierarchical embedding for code search) to solve the problems mentioned above. Our method divides the embedding process of code and query into two hierarchies, that is, the potential information is captured by two modules (the Intra-language encoding module and the Cross-language encoding module). In particular, our approach uses special LSTM (Long Short-Term Memory) variants, which is ON-LSTM (ordered neurons LSTM) to capture the keyword order structure of the code. The Intra-language encoding module is implemented by the LSTM variant and the Cross-language encoding module is an interactive information calculation module implemented by the attention mechanism. In this way, the similarity between the query and the corresponding code snippets in the vector space could be better captured. HECS can understand the difference between positive and negative samples more accurately. We empirically evaluate HECS, using a large scale codebase collected from StackOverflow. The experimental results show that our approach achieves state-of-the-art performance.

Summary Software repositories are crucial resources for many software tasks, including code retrieval and annotation. Programming forums provide questions and answers (Q&A) from software developers, containing abundant code‐description posts for exchanging knowledge about programming issues. However, most posts provide personal opinions of users that are often not adequately confirmed or outdated. Mining software repositories in such open and unrestricted forums is challenging. Since the posts can be arbitrary and noisy, it is difficult to get unified labels for supervised noise elimination. Different from existing mining approaches, this paper proposes Code‐Description Mining Framework (CodeMF), an unsupervised framework to eliminate noisy posts and extract high quality software repositories from programming forums. CodeMF treats all social features of the posts as discrete‐time signals for kernel principal component analysis and further performs wavelet transform feature fusion to find the delicate changes (noises in temporal signals). We conduct comprehensive experiments on StackOverflow. Experimental results demonstrate that CodeMF can effectively reduce running time and improve precision via mining high‐quality software repositories for various programming languages, especially for the large‐scale codebases. To further illustrate the effect of CodeMF applied in software tasks, we introduce it to improve the performance of query‐expansion code search. Meanwhile, for SQL and C# programs, compared to the state‐of‐the‐art query‐expansion method QECK, the improvement of QECK CodeMF is 2% and 6% on Recall@10, and 4% and 14% on mean reciprocal rank, respectively.

Deep generative models (DGMs) have shown strong performance in semi-supervised learning (SSL), which incorporate discrete class information into the learning process. Yet existing methods generally overfit to the given labeled data, for only considering the conditional probability of labels. In this paper, we propose a novel discriminative regularized deep generative method for SSL, which fully exploits the discriminative and geometric information of data to address the aforementioned issue. Our method introduces the cluster and manifold assumption that maximizes the classification margin between clusters and simultaneously smooths the predictions of the data which is close in the sub-manifold of each cluster, to regularize the learning of the classifier in DGMs. To derive the regularization based on introduced assumptions, we adopt the generated data of DGMs along with labelled and unlabelled data, to model the data manifold and yield clusters based on the Gumbel-softmax distribution. Experimental results on both text and image datasets demonstrate the effectiveness and flexibility of our method, and prove that two introduced assumptions are complementary in guiding the classification boundary, thus improving the discriminative ability of the classifier.

Although distant supervised relation extraction has been widely used, its performance is weakened by the wrong labeling problem. During eliminating noise, many previous work failed to grasp the trade-off between long dependencies and computational complexity while using neural networks, and ignored the potential noise in external knowledge. In this paper, we propose a neural model incorporating dilated convolutional neural networks with soft entity type constraints to jointly tackle the above two issues. Rather than using traditional convolutional neural networks or recurrent neural networks, we propose dilated convolutional networks as the sentence encoder so as to capture long dependencies in large scale context and improve the robustness against local noise while keeping the efficiency of computation. Furthermore, we utilize entity types, which is considerd as external knowledge with noise, to denoise in relation classification by taking the constraints between entity types and relations into account and learning more precise entity types and attention weights simultaneously. Experimental results on the New York Times dataset show that, our model incorporating dilated convolutional networks with soft entity type constraints promotes distant supervised relation extraction and achieves state-of-art results compared with baselines.

Existing generative adversarial networks (GANs) with manifold regularization for semi-supervised learning (SSL) have shown promising performance in image generation and semi-supervised learning (SSL), which penalize the smoothness of classifier over data manifold based on the smoothness assumption. However, the smoothness assumption is valid for data points in high density region while not hold for data points in low density region, thus they tend to misclassify boundary instances in low density region. In this paper, we propose a novel discriminative regularization method for semi-supervised learning with conditional generative adversarial nets (CGANs). In our method, the discriminative information from class conditional data distribution captured by CGANs is utilized to improve the discrimination of classifier. Different from regular manifold regularization, the discriminative regularization encourages the classifier invariance to local perturbations on the sub-manifold of each cluster, and distinct classification outputs for data points in different clusters. Moreover, our method can be easily implemented via the stochastic approximation without constructing the Laplacian graph or computing the Jacobian of classifier explicitly. Experimental results on benchmark datasets show that our method can achieve competitive performance against previous advanced methods.

Sparse topic models (STMs) are widely used for learning a semantically rich latent sparse representation of short texts in large scale, mainly by imposing sparse priors or appropriate regularizers on topic models. However, it is difficult for these STMs to model the sparse structure and pattern of the corpora accurately, since their sparse priors always fail to achieve real sparseness, and their regularizers bypass the prior information of the relevance between sparse coefficients. In this paper, we propose a novel Bayesian hierarchical topic models called Bayesian Sparse Topical Coding with Poisson Distribution (BSTC-P) on the basis of Sparse Topical Coding with Sparse Groups (STCSG). Different from traditional STMs, it focuses on imposing hierarchical sparse prior to leverage the prior information of relevance between sparse coefficients. Furthermore, we propose a sparsity-enhanced BSTC, Bayesian Sparse Topical Coding with Normal Distribution (BSTC-N), via mathematic approximation. We adopt superior hierarchical sparse inducing prior, with the purpose of achieving the sparsest optimal solution. Experimental results on datasets of Newsgroups and Twitter show that both BSTC-P and BSTC-N have better performance on finding clear latent semantic representations. Therefore, they yield better performance than existing works on document classification tasks.

Min Peng, Qianqian Xie, Yanchun Zhang, Hua Wang, Xiuzhen Zhang, Jimin Huang, Gang Tian. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.

Emerging topic detection from microblogs has developed into an attractive task because events usually break on social channels. However, due to the features of high noise, short length, fast arriving rate and irregular writing style of microblogs, it has been proven to be a challenge to detect emerging topics from microblog streams early and accurately in a scalable way. Several approaches have been proposed to tackle this problem and have achieved sound performance in some aspects. However, from the point of novelty and scalability, there is still considerable space for improvement. Inspired by the consideration, we propose an emerging topic detection framework based on emerging pattern mining. Via encoding the term novelty into an efficient high utility itemset mining (HUIM) algorithm, a group of emerging patterns which are concise and interpretive representations of topics can be first detected, decreasing the computational cost of the clustering part.

Learning low dimensional representations from a large number of short corpora has a profound practical significance but with vital challenge in content analysis and data mining applications. In this paper, we propose a novel topic model called Block Bayesian Sparse Topic Coding (Block-BSTC), which is capable of discovering the latent semantic representation of short texts. The Block-BSTC relaxes the normalization constraint of the inferred representations with word embeddings and block sparse Bayesian learning, which is convenient to directly control the sparsity of word codes with exploiting the intra-block correlations. Furthermore, the experimental results show that Block-BSTC achieves great performance on the sparsity ratio of word codes. Meanwhile, it can improve the accuracy of document classification.

This article presents an unsupervised multi-view hierarchical embedding (UMHE) framework to sufficiently reveal the intrinsic topical knowledge in social events. Event-oriented topics are highly related to such events as it can provide explicit descriptions of what have happened in social community. In many real-world cases, however, it is difficult to include all attributes of microblogs, more often, textual aspects only are available. Traditional topic modelling methods have failed to generate event-oriented topics with the textual aspects, since the inherent relations between topics are often overlooked in these methods. Meanwhile, the metrics in original word vocabulary space might not effectively capture semantic distances. Our UMHE framework overcomes the severe information deficiency and poor feature representation. The UMHE first develops a multi-view Bayesian rose tree to preliminarily generate prior knowledge for latent topics and their relations. With such prior knowledge, we design an unsupervised translation-based hierarchical embedding method to make a better representation of these latent topics. By applying self-adaptive spectral clustering on the embedding space and the original space concomitantly, we eventually extract event-oriented topics in word distributions to express social events. Our framework is purely data-driven and unsupervised, without any external knowledge. Experimental results on TREC Tweets2011 dataset and Sina Weibo dataset demonstrate that the UMHE framework can construct hierarchical structure with high fitness, but also yield topic embeddings with salient semantics; therefore, it can derive event-oriented topics with meaningful descriptions.

Compressing textstreams generated by social networks can both reduce storage consumption and improve efficiency such as fast searching. However, the compression process is a challenge due to the large scale of textstreams. In this article, we propose a textstream compression framework based on compressed sensing theory and design a series of matching parallel procedures. The new approach uses a linear projection technique in the textstream compression process, achieving fast compression speed and low compression ratio. Two processes are executed by designing elaborated parallel procedures for efficient compressing and decompressing of large-scale textstreams. The decompression process is implemented for approximate solutions of underdetermined linear systems. Experimental results show that the new method can efficiently achieve the compression and decompression tasks on a large amount of text generated by social networks.

To date, data generates and arrives in the form of stream to propagate discussions of public events in microblog services. Discovering event-oriented topics from the stream will lead to a better understanding of the change of public concern. However, as the massive scale of the data stream, traditional static topic models, such as LDA, are no longer fit for topic detection and tracking tasks. In this paper, we propose a central topic model (CenTM), where a Multi-view Clustering algorithm with Two-phase Random Walk (MC-TRW) is devised to aggregate the LDA's latent topics into central topics. Furthermore, we leverage the aggregation of central topics alternately with MC-TRW and sequential topic inference to improve the scalability in the stream fashion, so as to derive the dynamic central topic model (DCenTM). Specifically, our model is able to uncover the intrinsic characteristics of the central topics and predict the trend of their intensity along a life cycle. Experimental results demonstrate that the proposed central topic model is event-oriented and of high generalization, it therefore can dispose the topic trend prediction effectively and precisely in massive data stream.

Abstract In this paper we focus on the mobile GIS management system based on multi‐agents, and wider range of client devices with J2ME‐MIDP kernel are supported in it. A kit (Map Database Manager) is developed and described in detail. The kit can convert GIS data of commercial formats such as ArcGIS and Mapinfo into Geospatial data in GML, and provide Geospatial data for handheld devices freely without commercial plug‐in software. A peer to peer multi‐agents mechanism on mobile GIS services is the main subject of this paper. According to user's requirement and known buffer area radius on map, agent can define the factors to calculate the optimal path from one node to target node in a map, which considers not only the length of the road but also traffic flow, road cost, and other factors. We use the integrated cost to re‐define the road path weights, and select the optimal path among the road based on the minimum weight. The actual experimental results shown that in the quantified traffic flow and road condition, from node to other target nodes, we can get an optimal path on map, which satisfies the condition mostly. The results can greatly assist the searching of the optimal path in GIS services. Copyright © 2010 John Wiley & Sons, Ltd.

Multi-domain traffic engineering is a major focus area for carriers and crankback signaling offers a very promising alternative. However, even though various crankback studies have been done, there remains significant latitude for improved multi-domain designs. To address these challenges, this work develops a novel solution for intra/inter-domain signaling crankback in IP/MPLS networks. Namely, dynamic intra-domain link-state routing information is coupled with inter-domain path/distance-vector routing state to improve the search process. Mechanisms are also added to limit setup signaling overheads and track crankback history from congested links. The performance of the proposed solution is analyzed using simulation and compared against other techniques including hierarchical inter-domain routing.

Multi-domain traffic engineering is a major focus area for carriers and crankback signaling offers a very promising alternative. However, even though various crankback studies have been done, there remains significant latitude for improved multi-domain designs. To address these challenges, this work develops a novel solution for intra/inter-domain signaling crankback in IP/MPLS networks. Namely, dynamic intra-domain link-state routing information is coupled with inter-domain path/distance-vector routing state to improve the search process. Mechanisms are also added to limit setup signaling overheads and track crankback history from congested links. The performance of the proposed solution is analyzed using simulation and compared against other techniques including hierarchical inter-domain routing.

Role-based access control (RBAC) has been proven to be a flexible and useful access control model for information sharing in distributed collaborative environments. Permission-role assignments (PRA) is one important process in the access model. However, problems may arise during the procedures of PRA Conflicting permissions may assign to one role, and as a result, the role with the permissions can derive unexpected access capabilities. This paper aims to analyze the problems during the procedures of permission-role assignments in distributed collaborative environments and to develop authorization allocation algorithms to address the problems within permission-role assignments. The algorithms are extended to the case of PRA with the mobility of permission-role relationship. Finally, comparisons with other related work are discussed to demonstrate the effective work of the paper.