Spatial-temporal graph modeling is challenging due to the diverse node interactions across spatial and temporal dimensions. Recent studies typically adopt Graph Neural Networks (GNNs) to perform node-level aggregation at different time steps, acting as a series of low-pass graph spectral filters, for node interaction modeling. However, these filters, confined to the spatial dimension, are ill-suited for processing signals of nodes with inherent spatial-temporal interdependencies. Moreover, oversimplified low-pass filtering fails to fully exploit information from diverse node interactions. To address these issues, we propose a Spatial-Temporal Spectral Graph Neural Network (STSGNN), which designs specialized two-dimensional (2-D) graph spectral filters for comprehensive spatial-temporal graph modeling. First, based on the normalized Laplacian spectrum of spatial and temporal graphs, we extend the existing graph spectral theory from a univariate spatial dimension to a bivariate spatial-temporal dimension through a 2-D Discrete Graph Fourier Transform (2-D DGFT). Then, we leverage the bivariate Bernstein polynomial approximation, with learned basis coefficients, to design 2-D filters with specialized spectral properties for unified spatial-temporal signal filtering. Finally, the filtered signals, with refined spatial-temporal representations, are fed into well-designed pyramidal gated convolution modules to acquire multiple ranges of spatial-temporal dependencies. Experiments on traffic and meteorological prediction tasks demonstrate that STSGNN achieves state-of-the-art performance. Additionally, we visualize the 2-D filters learned from inputs with distinct spatial-temporal characteristics to enhance the model's interpretability.

Fangru Lin, Shaoguang Mao, Emanuele La Malfa, Valentin Hofmann, Adrian de Wynter, Xun Wang, Si-Qing Chen, Michael J. Wooldridge, Janet B. Pierrehumbert, Furu Wei. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Temporal reasoning and planning are essential capabilities for large language models (LLMs), yet most existing benchmarks evaluate them in isolation and under limited forms of complexity.To address this gap, we introduce the Temporal Constraint-based Planning (TCP) benchmark, that jointly assesses both capabilities.Each instance in TCP features a naturalistic dialogue around a collaborative project, where diverse and interdependent temporal constraints are explicitly or implicitly expressed, and models must infer an optimal schedule that satisfies all constraints.To construct TCP, we generate abstract problem prototypes that are then paired with realistic scenarios from various domains and enriched into dialogues using an LLM.A human quality check is performed on a sampled subset to confirm the reliability of our benchmark.We evaluate state-of-the-art LLMs and find that even the strongest models may struggle with TCP, highlighting its difficulty and revealing limitations in LLMs' temporal constraint-based planning abilities.We analyze underlying failure cases, open source our benchmark 1 , and hope our findings can inspire future research.

Multivariate Time-Series (MTS) forecasting is challenging due to the complex spatial-temporal dependencies inherent in MTS data. Recent studies typically adopt spatial-temporal graph models to leverage this information. However, most of these approaches assume homophily in graphs and perform only immediate message passing, which limits their ability to fully capture spatial-temporal dependencies. In this paper, we propose a novel Dual-Path Spatial-Temporal Graph Neural Network (DPSTGNN). We first extract and cluster representative short-term series patterns from the MTS data. Then, by comparing historical series segments with the extracted representative patterns, we generate pattern attribution embeddings for nodes, enabling the construction of Homophily and Heterophily Spatial-Temporal Crossed Graphs (STCGs), which depict diverse node-wise relationships across different time segments. Finally, we introduce dilated spatial-temporal crossed graph convolutions on both types of STCGs to capture a wide range of spatial-temporal dependencies for enhanced MTS forecasting. Experiments on six real-world datasets demonstrate that our method outperforms state-of-the-art models.

Graph Neural Networks (GNNs) and transformers are prevalent in multivariate time series (MTS) forecasting. They typically treat variable values at different timestamps as nodes and employ separate modules along the spatial and temporal dimensions to capture spatial-temporal dependencies. However, this separate modeling approach contradicts the fundamental nature of MTS data generation and fails to fully exploit spatial-temporal joint information within MTS, as it overlooks interactions between different variables across different timestamps. An intuitive solution to these issues is to apply an attention mechanism directly to all nodes, regardless of their temporal or spatial alignment, thus enabling unified and comprehensive spatial-temporal learning. Given the high computational complexity involved in exploiting spatial-temporal information this way, we propose Fourier-based Adaptive Global Fusion Network (FAGFNet) for efficient and comprehensive MTS forecasting. Specifically, we first transform the input into the frequency domain via a Fourier transform. Then we feed the obtained frequency representations into the well-designed FAGF module, which extracts key frequency representations with an adaptive global-scope mask, for refined spatial-temporal learning. By leveraging the convolution theorem in time-frequency analysis, FAGFNet fuses information in the frequency domain, reducing complexity compared to time-domain attention mechanisms. Extensive experiments on various datasets validate the superior performance of FAGFNet.

Multivariate Time Series (MTS) classification is a critical yet challenging task in the field of data mining. Recently, Graph Neural Networks (GNNs) have emerged as powerful tools for this task due to their exceptional capability to capture dependencies among variables (nodes). However, the nodes in the graph exhibit heterogeneity where each node plays a distinct role within the network. Existing GNN-based methods predominantly employ fixed-size receptive fields to capture spatial information, which limits the models’ ability to flexibly adjust the scope of information aggregation based on individual node characteristics and importance. To address these issues, we propose Receptive Field-Aware Contrastive Learning (RFACL) for MTS classification. Specifically, we design the Dynamic Selection Hop (DSH) block and integrate it into the contrastive learning framework. The DSH block assigns different weights to various-hop neighborhoods, allowing each node to adaptively adjust the size of its receptive field according to the characteristics of the input data. To further capture the spatial relationships corresponding to each time segment, RFACL stacks multiple DSH blocks together, enabling the dynamic selection and integration of multi-scale information. Extensive experiments demonstrate that our proposed method achieves state-of-the-art performance on various MTS classification tasks. The code is available at https://github.com/LinHuai01/RFACL.

Evaluating large language models (LLMs) is crucial for both assessing their capabilities and identifying safety or robustness issues prior to deployment. Reliably measuring abstract and complex phenomena such as safety' androbustness' requires strong construct validity, that is, having measures that represent what matters to the phenomenon. With a team of 29 expert reviewers, we conduct a systematic review of 445 LLM benchmarks from leading conferences in natural language processing and machine learning. Across the reviewed articles, we find patterns related to the measured phenomena, tasks, and scoring metrics which undermine the validity of the resulting claims. To address these shortcomings, we provide eight key recommendations and detailed actionable guidance to researchers and practitioners in developing LLM benchmarks.

Scalar adjectives pertain to various domain scales and vary in intensity within each scale (e.g. certain is more intense than likely on the likelihood scale). Scalar implicatures arise from the consideration of alternative statements which could have been made. They can be triggered by scalar adjectives and require listeners to reason pragmatically about them. Some scalar adjectives are more likely to trigger scalar implicatures than others. This phenomenon is referred to as scalar diversity. In this study, we probe different families of Large Language Models such as GPT-4 for their knowledge of the lexical semantics of scalar adjectives and one specific aspect of their pragmatics, namely scalar diversity. We find that they encode rich lexical-semantic information about scalar adjectives. However, the rich lexical-semantic knowledge does not entail a good understanding of scalar diversity. We also compare current models of different sizes and complexities and find that larger models are not always better. Finally, we explain our probing results by leveraging linguistic intuitions and model training objectives.

Planning is a fundamental property of human intelligence. Reasoning about asynchronous plans is challenging since it requires sequential and parallel planning to optimize time costs. Can large language models (LLMs) succeed at this task? Here, we present the first large-scale study investigating this question. We find that a representative set of closed and open-source LLMs, including GPT-4 and LLaMA-2, behave poorly when not supplied with illustrations about the task-solving process in our benchmark AsyncHow. We propose a novel technique called *Plan Like a Graph* (PLaG) that combines graphs with natural language prompts and achieves state-of-the-art results. We show that although PLaG can boost model performance, LLMs still suffer from drastic degradation when task complexity increases, highlighting the limits of utilizing LLMs for simulating digital devices. We see our study as an exciting step towards using LLMs as efficient autonomous agents. Our code and data are available at https://github.com/fangru-lin/graph-llm-asynchow-plan.