Zheyuan Zhang, Kaiwen Shi, Zhengqing Yuan, Zehong Wang, Tianyi Ma, Keerthiram Murugesan, Vincent Galassi, Chuxu Zhang, Yanfang Ye. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Zheyuan Zhang, Yiyang Li, Nhi Ha Lan Le, Zehong Wang, Tianyi Ma, Vincent Galassi, Keerthiram Murugesan, Nuno Moniz, Werner Geyer, Nitesh V Chawla, Chuxu Zhang, Yanfang Ye. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Hypergraphs, in contrast to general graphs, utilize hyperedges to connect multiple nodes, thereby inherently facilitating the representation of higher-order relational structures. To leverage the benefits of hypergraphs, several Hypergraph Neural Networks (HyGNNs) have been proposed to model hypergraph structures. Although existing HyGNNs excel at capturing complex relationships in homophilic hypergraphs, they still face challenges in modeling heterophilic hypergraphs, as most existing HyGNNs are designed based on the homophily principle. Recent studies have attempted to leverage attention mechanisms that are less reliant on the homophily principle. However, these attention mechanisms remain ineffective for nodes in heterophilic hypergraphs. To tackle the aforementioned challenges, we propose a novel Broadcast HyperGraph Neural Network (BHyGNN) to adaptively broadcast node information to learn more effective node representations in heterophilic hypergraphs. Specifically, we devise a novel Variational Broadcast Autoencoder Network to sample the broadcast and receive actions to propagate information between nodes and hyperedges. Moreover, we design an incorporation transformer mechanism to perform the estimated broadcast or receive actions to learn the hyperedge or node representations, incorporating the information from both sides. Extensive experiments over five benchmark heterophilic hypergraph datasets and six homophilic hypergraph datasets demonstrate the effectiveness of BHyGNN over all baseline methods. Our source code and datasets are available at https://github.com/Tianyi-Billy-Ma/BHyGNN.

Foundation models are pretrained on large-scale corpora to learn generalizable patterns across domains and tasks---such as contours, textures, and edges in images, or tokens and sentences in text. In contrast, discovering such generalities in graph-structured data, especially across heterogeneous graph tasks, remains an open challenge. To address this, we propose a novel approach to cross-task generalization in graphs via task-trees, which serve as unified learning instances aligning node-, edge-, and graph-level tasks. We theoretically analyze the stability, transferability, and generalization properties of task-trees, showing that pretraining a graph neural network (GNN) on diverse task-trees with a reconstruction objective induces transferable knowledge. This enables efficient adaptation to downstream tasks with minimal fine-tuning. To validate our framework, we introduce Graph Generality Identifier on Task-Trees (GIT), a graph foundation model that demonstrates strong performance on over 30 graphs across five domains via fine-tuning, in-context learning, and zero-shot generalization. Code and data are available at https://github.com/Zehong-Wang/GIT.

Graph learning tasks often hinge on identifying key substructure patterns—such as triadic closures in social networks or benzene rings in molecular graphs—that underpin downstream performance. However, most existing graph neural networks (GNNs) rely on message passing, which aggregates local neighborhood information iteratively and struggles to explicitly capture such fundamental motifs, like triangles, $k$-cliques, and rings. This limitation hinders both expressiveness and long-range dependency modeling. In this paper, we introduce the Neural Graph Pattern Machine (GPM), a novel framework that bypasses message passing by learning directly from graph substructures. GPM efficiently extracts, encodes, and prioritizes task-relevant graph patterns, offering greater expressivity and improved ability to capture long-range dependencies. Empirical evaluations across four standard tasks—node classification, link prediction, graph classification, and graph regression—demonstrate that GPM outperforms state-of-the-art baselines. Further analysis reveals that GPM exhibits strong out-of-distribution generalization, desirable scalability, and enhanced interpretability. Code and datasets are available at: https://github.com/Zehong-Wang/GPM.

Graph learning models have demonstrated great prowess in learning expressive representations from large-scale graph data in a wide variety of real-world scenarios. As a prevalent strategy for training powerful graph learning models, the ''pre-training, adaptation'' scheme first pre-trains graph learning models on unlabeled graph data in a self-supervised manner and then adapts them to specific downstream tasks. During the adaptation phase, graph prompting emerges as a promising approach that learns trainable prompts while keeping the pre-trained graph learning models unchanged. In this paper, we present a systematic review of recent advancements in graph prompting. First, we introduce representative graph pre-training methods that serve as the foundation step of graph prompting. Next, we review mainstream techniques in graph prompting and elaborate on how they design learnable prompts for graph prompting. Furthermore, we summarize the real-world applications of graph prompting from different domains. Finally, we discuss several open challenges in existing studies with promising future directions in this field.

Foundation models have revolutionized machine learning by enabling general-purpose reasoning across diverse tasks and domains. These models, pretrained on large-scale data, demonstrate strong adaptability with minimal task-specific supervision, leading to breakthroughs in natural language processing and computer vision. Inspired by this paradigm, Graph Foundation Models (GFMs) have emerged to extend the benefits of foundation models to graph-structured data, which is pretrained on massive graphs and can be fast adapted to different downstream tasks. In this paper, we provide a comprehensive survey of the state-of-the-art techniques of graph foundation models. In particular, we (1) formally categorize the challenges in designing graph foundation models; (2) comprehensively review the existing and recent advances of graph foundation models; (3) extend the graph foundation models in real-world problems; and (4) elucidate open questions and future research directions. Our systematic review summarizes representative models, highlights key design principles, and provides comparative analyses. This survey introduces major topics within foundation models and offers a guide to a new frontier of graph learning. Our extended survey is available at https://arxiv.org/abs/2505.15116.

The prevalence of unhealthy eating habits has become a growing concern in the United States. However, popular food recommendation platforms, such as Yelp, tend to prioritize users' dietary preferences over the healthiness of their choices. While some efforts have focused on developing health-aware food recommendation systems, personalization based on specific health conditions remains underexplored. Additionally, the lack of interpretability in these systems prevents users from evaluating the reliability of recommendations, limiting their practical adoption. To address these issues, we introduce two large-scale personalized health-aware food recommendation benchmarks at the first attempt. Building on this, we propose a novel framework called the Multi-Objective Personalized Interpretable Health-aware Food Recommendation System (MOPI-HFRS). This system generates food recommendations by jointly optimizing three objectives: user preference, personalized healthiness, and nutritional diversity. It also incorporates a reasoning module enhanced by large language models (LLMs) to provide interpretable recommendations that promote healthy dietary knowledge. The framework integrates descriptive features and health data using two structure learning and pooling modules within a graph learning framework. Pareto optimization is applied to balance the multi-faceted objectives. To further enhance healthy dietary knowledge, the system leverages LLMs by infusing knowledge from the recommendation model, generating meaningful interpretations for the recommendations. Extensive experiments on the proposed benchmarks demonstrate that MOPI-HFRS outperforms state-of-the-art methods by delivering diverse, healthy food recommendations alongside reliable explanations.

Zehong Wang, Sidney Liu, Zheyuan Zhang, Tianyi Ma, Chuxu Zhang, Yanfang Ye. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

The exponential growth of data-driven systems and AI technologies has intensified the demand for high-quality web-sourced datasets. While existing datasets have proven valuable, conventional web data collection approaches face significant limitations in terms of human effort and scalability. Current data collecting solutions fall into two categories: wrapper-based methods that struggle with adaptability and reproducibility, and large language model (LLM)-based approaches that incur substantial computational and financial costs. To address these challenges, we propose AutoData, a novel multi-agent system for Automated web Data collection, that requires minimal human intervention, i.e., only necessitating a natural language instruction specifying the desired dataset. In addition, AutoData is designed for a robust multi-agent architecture, featuring a novel oriented message hypergraph coordinated by a central task manager, to efficiently organize agents across research and development squads. Besides, we introduce a novel hypergraph cache system to advance the multi-agent collaboration process that enables efficient automated data collection and mitigates the token cost issues prevalent in existing LLM-based systems. Moreover, we introduce Instruct2DS, a new benchmark dataset supporting live data collection from web sources across three domains: academic, finance, and sports. Comprehensive evaluations over Instruct2DS and three existing benchmark datasets demonstrate AutoData's superior performance compared to baseline methods. Case studies on challenging tasks such as picture book collection and paper extraction from surveys further validate its applicability.

Graph neural networks (GNNs) have been predominantly driven by message-passing, where node representations are iteratively updated via local neighborhood aggregation. Despite their success, message-passing suffers from fundamental limitations---including constrained expressiveness, over-smoothing, over-squashing, and limited capacity to model long-range dependencies. These issues hinder scalability: increasing data size or model size often fails to yield improved performance. To this end, we explore pathways beyond message-passing and introduce Generative Graph Pattern Machine (G$^2$PM), a generative Transformer pre-training framework for graphs. G$^2$PM represents graph instances (nodes, edges, or entire graphs) as sequences of substructures, and employs generative pre-training over the sequences to learn generalizable and transferable representations. Empirically, G$^2$PM demonstrates strong scalability: on the ogbn-arxiv benchmark, it continues to improve with model sizes up to 60M parameters, outperforming prior generative approaches that plateau at significantly smaller scales (e.g., 3M). In addition, we systematically analyze the model design space, highlighting key architectural choices that contribute to its scalability and generalization. Across diverse tasks---including node/link/graph classification, transfer learning, and cross-graph pretraining---G$^2$PM consistently outperforms strong baselines, establishing a compelling foundation for scalable graph learning. The code and dataset are available at https://github.com/Zehong-Wang/G2PM.

Graph Neural Networks (GNNs) have demonstrated their effectiveness in various graph learning tasks, yet their reliance on neighborhood aggregation during inference poses challenges for deployment in latency-sensitive applications, such as real-time financial fraud detection. To address this limitation, recent studies have proposed distilling knowledge from teacher GNNs into student Multi-Layer Perceptrons (MLPs) trained on node content, aiming to accelerate inference. However, these approaches often inadequately explore structural information when inferring unseen nodes. To this end, we introduce SimMLP, a Self-supervised framework for learning MLPs on graphs, designed to fully integrate rich structural information into MLPs. Notably, SimMLP is the first MLP-learning method that can achieve equivalence to GNNs in the optimal case. The key idea is to employ self-supervised learning to align the representations encoded by graph context-aware GNNs and neighborhood dependency-free MLPs, thereby fully integrating the structural information into MLPs. We provide a comprehensive theoretical analysis, demonstrating the equivalence between SimMLP and GNNs based on mutual information and inductive bias, highlighting SimMLP's advanced structural learning capabilities. Additionally, we conduct extensive experiments on 20 benchmark datasets, covering node classification, link prediction, and graph classification, to showcase SimMLP's superiority over state-of-the-art baselines, particularly in scenarios involving unseen nodes (e.g., inductive and cold-start node classification) where structural insights are crucial. Our codes are available at: https://github.com/Zehong-Wang/SimMLP.

Contrastive learning (CL) has emerged as a powerful approach for self-supervised learning. However, it suffers from sampling bias, which hinders its performance. While the mainstream solutions, hard negative mining (HNM) and supervised CL (SCL), have been proposed to mitigate this critical issue, they do not effectively address graph CL (GCL). To address it, we propose graph positive sampling (GPS) and three contrastive objectives. The former is a novel learning paradigm designed to leverage the inherent properties of graphs for improved GCL models, which utilizes four complementary similarity measurements, including node centrality, topological distance, neighborhood overlapping, and semantic distance, to select positive counterparts for each node. Notably, GPS operates without relying on true labels and enables preprocessing applications. The latter aims to fuse positive samples and enhance representative selection in the semantic space. We release three node-level models with GPS and conduct extensive experiments on public datasets. The results demonstrate the superiority of GPS over state-of-the-art (SOTA) baselines and debiasing methods. In addition, the GPS has also been proven to be versatile, adaptive, and flexible.

Large Language Models (LLMs) have demonstrated zero-shot generalization capabilities in analyzing and predicting protein properties through natural language interactions. However, existing protein-focused datasets for LLM fine-tuning, such as ProteinLMDataset with its 17.46 billion tokens for pre-training and 893,000 instructions for fine-tuning, face limitations. Specifically, they include insufficient coverage of protein functional properties, inadequate protein-protein interaction data, and limited integration of contextual information from biomedical literature. To overcome these challenges, we present ProteinPFAIDataset, which integrates data from UniProt and PubMed, comprising 72.8 million tokens for Supervised Fine-Tuning (SFT). ProteinPFAIDataset encompasses important protein characteristics including enzyme activities, molecular functions, pH dependence, tissue specificity, temperature sensitivity, subunit structure, and disease associations. Additionally, we propose a novel knowledge graph-based approach that incorporates over 300,000 biomedical literature entries, providing rich contextual information about protein functions and interactions. To validate the effectiveness of our dataset, we fine-tuned Falcon2-11B LLM, resulting in a model we call Falcon2-11B-PFAI. The fine-tuned model achieved state-of-the-art performance on ProteinLMBench, improving accuracy from 47.10% to 58.37%. The dataset is available at https://huggingface.co/datasets/xiaorui1/PFAI. The fine-tuned model is available at https://huggingface.co/xiaorui1/PFAI/tree/main.

Large-scale datasets have fueled recent advancements in AI-based autonomous vehicle research. However, these datasets are usually collected from a single vehicle's one-time pass of a certain location, lacking multiagent inter-actions or repeated traversals of the same place. Such information could lead to transformative enhancements in autonomous vehicles' perception, prediction, and planning capabilities. To bridge this gap, in collaboration with the self-driving company May Mobility, we present MARS dataset which unifies scenarios that enable MultiAgent, multitraveRSal, and multimodal autonomous vehicle re-search. More specifically, MARS is collected with a fleet of autonomous vehicles driving within a certain geographical area. Each vehicle has its own route and different vehicles may appear at nearby locations. Each vehicle is equipped with a LiDAR and surround-view RGB cameras. We curate two subsets in MARS: one facilitates collaborative driving with multiple vehicles simultaneously present at the same location, and the other enables memory retrospection through asynchronous traversals of the same location by multiple vehicles. We conduct experiments in place recog-nition and neural reconstruction. More importantly, MARS introduces new research opportunities and challenges such as multitraversal 3D reconstruction, multiagent perception, and unsupervised object discovery. Our data and codes can be found at https://ai4ce.github.io/MARS/.

The opioid crisis has been one of the most critical society concerns in the United States. Although the medication assisted treatment (MAT) is recognized as the most effective treatment for opioid misuse and addiction, the various side effects can trigger opioid relapse. In addition to MAT, the dietary nutrition intervention has been demonstrated its importance in opioid misuse prevention and recovery. However, research on the alarming connections between dietary patterns and opioid misuse remain under-explored. In response to this gap, in this paper, we first establish a large-scale multifaceted dietary benchmark dataset related to opioid users at the first attempt and then develop a novel framework - i.e., namely Opioid Misuse Detection with INterpretable Dietary Patterns (Diet-ODIN) - to bridge heterogeneous graph (HG) and large language model (LLM) for the identification of users with opioid misuse and the interpretation of their associated dietary patterns. Specifically, in Diet-ODIN, we first construct an HG to comprehensively incorporate both dietary and health-related information, and then we devise a holistic graph learning framework with noise reduction to fully capitalize both users' individual dietary habits and shared dietary patterns for the detection of users with opioid misuse. To further delve into the intricate correlations between dietary patterns and opioid misuse, we exploit an LLM by utilizing the knowledge obtained from the graph learning model for interpretation. The extensive experimental results based on our established benchmark with quantitative and qualitative measures demonstrate the outstanding performance of Diet-ODIN on exploring the complex interplay between opioid misuse and dietary patterns, by comparison with state-of-the-art baseline methods. Our code, built benchmark and system demo are available at https://github.com/JasonZhangzy1757/Diet-ODIN.

Humans naturally retain memories of permanent elements, while ephemeral moments often slip through the cracks of memory. This selective retention is crucial for robotic perception, localization, and mapping. To endow robots with this capability, we introduce 3D Gaussian Mapping (3DGM), a self-supervised, camera-only offline mapping framework grounded in 3D Gaussian Splatting. 3DGM converts multitraverse RGB videos from the same region into a Gaussian-based environmental map while concurrently performing 2D ephemeral object segmentation. Our key observation is that the environment remains consistent across traversals, while objects frequently change. This allows us to exploit self-supervision from repeated traversals to achieve environment-object decomposition. More specifically, 3DGM formulates multitraverse environmental mapping as a robust 3D representation learning problem, treating pixels of the environment and objects as inliers and outliers, respectively. Using robust feature distillation, feature residual mining, and robust optimization, 3DGM simultaneously performs 2D segmentation and 3D mapping without human intervention. We build the Mapverse benchmark, sourced from the Ithaca365 and nuPlan datasets, to evaluate our method in unsupervised 2D segmentation, 3D reconstruction, and neural rendering. Extensive results verify the effectiveness and potential of our method for self-driving and robotics.

Inspired by the success of foundation models in applications such as ChatGPT, as graph data has been ubiquitous, one can envision the far-reaching impacts that can be brought by Graph Foundation Models (GFMs) with broader applications in the areas such as scientific research, social network analysis, drug discovery, and e-commerce. Despite the significant progress of pre-trained graph neural networks, there haven’t been GFMs that can achieve desired performance on various graph-learning-related tasks. Building GFMs may rely on a vocabulary that encodes transferable patterns shared among different tasks and domains. Unlike image and text, defining such transferable patterns for graphs remains an open question. In this paper, we aim to bridge this gap by rethinking the transferable patterns on graphs as computation trees -- i.e., tree structures derived from the message-passing process. Based on this insight, we propose a cross-task, cross-domain graph foundation model named GFT, short for Graph Foundation model with transferable Tree vocabulary. By treating computation trees as tokens within the transferable vocabulary, GFT improves model generalization and reduces the risk of negative transfer. The theoretical analyses and extensive experimental studies have demonstrated the transferability of computation trees and shown the effectiveness of GFT across diverse tasks and domains in graph learning. The open source code and data are available at https://github.com/Zehong-Wang/GFT.

Differentiable architecture search is a popular gradient-based method for neural architecture search, and has achieved great success in automating design of neural network architectures. However, it still has some limitations such as performance collapse, which seriously affects network architecture performance. To solve this issue, we propose an algorithm called self-adaptive weight based on dual-attention for differentiable neural architecture search (SWD-NAS) in this article. SWD-NAS utilizes a dual-attention mechanism to measure architectural weights. Specifically, an upper-attention module is used to adaptively select channels based on their weights before inputting into the search space. A lower-attention (LA) module is utilized to calculate architectural weights. In addition, we propose an architectural weight normalization to alleviate the unfair competition among connection edges. Finally, we evaluate the architectures searched on CIFAR-10 and CIFAR-100, achieving test errors of 2.51% and 16.13%, respectively. Furthermore, we transfer the architecture searched on CIFAR-10 to ImageNet, achieving top-one and top-five errors of 24.5% and 7.6%, respectively. This demonstrates the superior performance of the proposed algorithm compared to many gradient-based algorithms.

Inspired by the success of Contrastive Learning (CL) in computer vision and natural language processing, Graph Contrastive Learning (GCL) has been developed to learn discriminative node representations on graph datasets. However, the development of GCL on Heterogeneous Information Networks (HINs) is still in the infant stage. For example, it is unclear how to augment the HINs without substantially altering the underlying semantics, and how to design the contrastive objective to fully capture the rich semantics. Moreover, early investigations demonstrate that CL suffers from sampling bias, whereas conventional debias- ing techniques are empirically shown to be inadequate for GCL. How to mitigate the sampling bias for heterogeneous GCL is another important problem. To address the aforementioned challenges, we propose a novel Heterogeneous Graph Contrastive Multi-view Learning (HGCML) model. In particular, we use metapaths as the augmentation to generate multiple subgraphs as multi-views, and propose a contrastive objective to maximize the mutual information between any pairs of metapath-induced views. To alleviate the sampling bias, we further propose a positive sampling strategy to explicitly select positives for each node via jointly considering semantic and structural information preserved on each metapath view. Extensive experiments demonstrate HGCML consistently outperforms state-of-the-art baselines on five real-world benchmark datasets. To enhance the repro- ducibility of our work, we make all the code publicly available at https://github.com/Zehong-Wang/HGCML.

Restaurant recommendation is one of the most recommendation problems because the result of recommendation varies in different environments. Many methods have been proposed to recommend restaurants in a mobile environment by considering user preference, restaurant attributes, and location. However, there are few restaurant recommender systems according to the internet of vehicles environment. This paper presents a recommender system based on the prediction of traffic conditions in the internet of vehicles environment. This recommender system uses a phased selection method to recommend restaurants. The first stage is to screen restaurants that are on the user’s driving route; the second stage is to recommend restaurants from the user attributes, restaurant attributes (with traffic conditions), and vehicle context, using a deep learning model. The experimental evaluation shows that the proposed recommender system is both efficient and effective.