Xingrun Xing, Zheng Liu, Shitao Xiao, Boyan Gao, Yiming Liang, Haokun Lin, Xianlin Zeng, Guoqi Li, Jiajun Zhang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Graph Neural Networks (GNNs) have recently gained widespread attention as a successful tool for analyzing graph-structured data. However, imperfect graph structure with noisy links lacks enough robustness and may damage graph representations, therefore limiting the GNNs' performance in practical tasks. Moreover, existing generative architectures fail to fit discriminative graph-related tasks. To tackle these issues, we introduce an unsupervised method based on a joint of generative training and discriminative training to learn graph structure and representation, aiming to improve the discriminative performance of generative models. We propose an Energy-based Contrastive Learning (ECL) guided Graph Structure Refinement (GSR) framework, denoted as ECL-GSR. To our knowledge, this is the first work to combine energy-based models with contrastive learning for GSR. Specifically, we leverage ECL to approximate the joint distribution of sample pairs, which increases the similarity between representations of positive pairs while reducing the similarity between negative ones. Refined structure is produced by augmenting and removing edges according to the similarity metrics among node representations. Extensive experiments demonstrate that ECL-GSR outperforms the state-of-the-art on eight benchmark datasets in node classification. ECL-GSR achieves faster training with fewer samples and memories against the leading baseline, highlighting its simplicity and efficiency in downstream tasks.

Super-resolution (SR) of remote sensing images (RSIs) has been improved significantly with the development of deep learning. However, better performances usually come from complex network architectures and require a substantial number of parameters. Moreover, many methods can only deal with SR of single and fixed-scale factors. As such, we propose a binary lightweight SR (BLiSR) method to decrease the computation and storage burden and increase the practicality of SR, where we employ a binary neural network (BNN) as the backbone and leverage a binary continuous up-sampling module (BCUM) to achieve arbitrary scale RSI SR. Specifically, we introduce an adaptive binary convolution (ABConv) as the basic unit of BLiSR, which can adaptively adjust the learnable parameters to fit the distribution of full-precision weights and activations. Then, a scalable hyperbolic tangent function is presented to approximate the Sign function in backpropagation and increase the learning capability of BNN. Furthermore, we design a lightweight SR network that considers the full-precision information flow of BNN. The network comprises several basic binary units and a multilayer group fusion block (MGFB), which can extract and fuse the multilevel information from LR images, respectively. Finally, BCUM can predict the pixel values of HR images based on the frequency implicit representation network (IRN) and reconstruct the LR images at arbitrary scales. Extensive experiments on four RSI datasets demonstrate that the proposed BLiSR is superior to several lightweight state-of-the-art (SOTA) methods on both fixed and arbitrary scale SR settings, with a better balance of complexity and performance.

Distributed stochastic optimization (DSO) with local set constraints and coupled inequality constraints over a multiagent network is considered in this article. Usually, such problems are tackled by projected primal-dual methods, which require expensive projection operations when set constraints are complicated. In this context, this article focuses on the Frank-Wolfe (FW) framework, which provides computational simplicity by avoiding expensive projection operations, for solving DSO with local set and coupled inequality constraints. By combining recursive momentum and weighted averaging, this article proposes a distributed stochastic FW primal-dual algorithm (DSFWPD), which is the first stochastic FW solver for DSO problems with coupled constraints. The proposed algorithm achieves zero constraint violation on average with a sublinear decay of the optimality gap over a directed and time-varying network. The efficacy of DSFWPD is demonstrated by several numerical experiments.

Pretrained foundation models offer substantial benefits for a wide range of downstream tasks, which can be one of the most potential techniques to access artificial general intelligence. However, scaling up foundation transformers for maximal task-agnostic knowledge has brought about computational challenges, especially on resource-limited devices such as mobiles. This work proposes the first Binary Pretrained Foundation Transformer (BiPFT) for natural language understanding (NLU) tasks, which remarkably saves 56 times operations and 28 times memory. In contrast to previous task-specific binary transformers, BiPFT exhibits a substantial enhancement in the learning capabilities of binary neural networks (BNNs), promoting BNNs into the era of pre-training. Benefiting from extensive pretraining data, we further propose a data-driven binarization method. Specifically, we first analyze the binarization error in self-attention operations and derive the polynomials of binarization error. To simulate full-precision self-attention, we define binarization error as binarization residual polynomials, and then introduce low-rank estimators to model these polynomials. Extensive experiments validate the effectiveness of BiPFTs, surpassing task-specific baseline by 15.4% average performance on the GLUE benchmark. BiPFT also demonstrates improved robustness to hyperparameter changes, improved optimization efficiency, and reduced reliance on downstream distillation, which consequently generalize on various NLU tasks and simplify the downstream pipeline of BNNs. Our code and pretrained models are publicly available at https://github.com/Xingrun-Xing/BiPFT.

Abstract Video Anomaly Detection (VAD) has been an active research field for several decades. However, most existing approaches merely extract a single type of feature from videos and define a single paradigm to indicate the extent of abnormalities. A coarse‐to‐fine three‐level prediction is built by integrating different levels of spatio‐temporal representations, better highlighting the difference between normal and abnormal behaviors. First, an object‐level trajectory prediction is proposed to model human historical position using a graph transformer network. Subsequently, skeleton‐level prediction is achieved by incorporating the positional information from the trajectory prediction. More importantly, based on the predicted skeleton, a skeleton‐guided pixel‐level region prediction is performed. A novel Skeleton Conditioned Generative Adversarial Network (SCGAN) is designed to explore the correlation between skeleton‐level and pixel‐level motion prediction. Benefiting from SCGAN, the prediction of human regions is contributed by both coarse‐grained and fine‐grained motion features. This three‐level prediction, namely Progressive Prediction Video Anomaly Detection (P 3 VAD), enlarges the prediction error on irregular motion patterns. Besides, a pixel‐level analysis method is proposed to achieve Background‐bias Elimination (BE) and denoise the predicted region. Experimental results validate the effectiveness of P 3 VAD on the four benchmark datasets (ShanghaiTech, CUHK Avenue, IITB‐Corridor, and ADOC).

3D reconstruction of Unmanned Aerial Vehicle (UAV) scenes is vital for agriculture, environmental protection, urban planning, and disaster response, to name a few. However, data acquisition can be constrained and hazardous under hostile environments, which limits the image data available in real-world applications. In this work, we propose a text-driven online editing framework for UAV scenes, which can generate novel views of existing scenes with abundant editing types. Compared with small single-object scenes, large-scale UAV scene editing suffers from several particular challenges: 1) broader capturing scope exhibits illumination variation and complicated objects that reduce the 3D scene consistency after editing; and 2) high-resolution 2D editing and 3D reconstruction can be computationally expensive with tremendous GPU memory. To tackle these issues, we first design a dual-branch compact NeRF structure to reduce memory usage and enhance accuracy for 3D reconstruction. We then introduce a sub-pixel sampling scheme to expedite the generation of low-resolution images for 2D editing, followed by a super-resolution module that restores the fine details of rendered images. Additionally, we develop a grouped content filtering mechanism to improve the 3D scene consistency of the model by matching the rendering images and text descriptions, which also significantly reduces memory usage during editing. Extensive experiments demonstrate that the proposed method can achieve various editing effects, including different seasons, weather conditions, times of the day, disaster scenarios, etc. Our technique is computationally efficient and conveniently expandable for large-scale UAV scenes, alleviating data scarcity in harsh scenarios.

The nonsmooth finite-sum minimization is a fundamental problem in machine learning. This article develops a distributed stochastic proximal-gradient algorithm with random reshuffling to solve the finite-sum minimization over time-varying multiagent networks. The objective function is a sum of differentiable convex functions and nonsmooth regularization. Each agent in the network updates local variables by local information exchange and cooperates to seek an optimal solution. We prove that local variable estimates generated by the proposed algorithm achieve consensus and are attracted to a neighborhood of the optimal solution with an O((1/T)+(1/√T)) convergence rate, where T is the total number of iterations. Finally, some comparative simulations are provided to verify the convergence performance of the proposed algorithm.

Deep learning models have been widely used for anomaly detection in surveillance videos. Typical models are equipped with the capability to reconstruct normal videos and evaluate the reconstruction errors on anomalous videos to indicate the extent of abnormalities. However, existing approaches suffer from two disadvantages. Firstly, they can only encode the movements of each identity independently, without considering the interactions among identities which may also indicate anomalies. Secondly, they leverage inflexible models whose structures are fixed under different scenes, this configuration disables the understanding of scenes. In this paper, we propose a Hierarchical Spatio-Temporal Graph Convolutional Neural Network (HSTGCNN) to address these problems, the HSTGCNN is composed of multiple branches that correspond to different levels of graph representations. High-level graph representations encode the trajectories of people and the interactions among multiple identities while low-level graph representations encode the local body postures of each person. Furthermore, we propose to weightedly combine multiple branches that are better at different scenes. An improvement over single-level graph representations is achieved in this way. An understanding of scenes is achieved and serves anomaly detection. High-level graph representations are assigned higher weights to encode moving speed and directions of people in low-resolution videos while low-level graph representations are assigned higher weights to encode human skeletons in high-resolution videos. Experimental results show that the proposed HSTGCNN significantly outperforms current state-of-the-art models on four benchmark datasets (UCSD Pedestrian, ShanghaiTech, CUHK Avenue and IITB-Corridor) by using much less learnable parameters.

In this article, the distributed cooperative control problem of redundant mobile manipulators is investigated. A novel method is proposed to solve the problem by integrating formation control with constrained optimization, which not only transports the object along a reference trajectory in a distributed manner but also obtains the dexterous joint postures and end-effector displacements under safety constraints for collision avoidance. For the constrained optimization, the cost function and safety constraints are designed to quantify the mobility and manipulability of mobile manipulators, and collision-free working ranges with the object and obstacles, respectively. A discontinuous projected primal-dual algorithm with damping terms is proposed to solve the constrained optimization problem, providing the joint postures and end-effector displacements, which minimize the cost function and satisfy safety constraints. For the formation control, a finite-time control law, guided by end-effector displacements from the primal-dual algorithm, is developed in order to transport the object by establishing a prescribed formation and moving its centroid to track the reference trajectory. The cooperative manipulation is therefore achieved by the proposed method, which is further validated through numerical simulations.

Lane change is a common-yet-challenging driving behavior for automated vehicles. To improve the safety and efficiency of automated vehicles, researchers have proposed various lane-change decision models. However, most of the existing models consider lane-change behavior as a one-player decision-making problem, ignoring the essential multi-agent properties when vehicles are driving in traffic. Such models lead to deficiencies in interaction and collaboration between vehicles, which results in hazardous driving behaviors and overall traffic inefficiency. In this paper, we revisit the lane-change problem and propose a bi-level lane-change behavior planning strategy, where the upper level is a novel multi-agent deep reinforcement learning (DRL) based lane-change decision model and the lower level is a negotiation based right-of-way assignment model. We promote the collaboration performance of the upper-level lane-change decision model from three crucial aspects. First, we formulate the lane-change decision problem with a novel multi-agent reinforcement learning model, which provides a more appropriate paradigm for collaboration than the single-agent model. Second, we encode the driving intentions of surrounding vehicles into the observation space, which can empower multiple vehicles to implicitly negotiate the right-of-way in decision-making and enable the model to determine the right-of-way in a collaborative manner. Third, an ingenious reward function is designed to allow the vehicles to consider not only ego benefits but also the impact of changing lanes on traffic, which will guide the multi-agent system to learn excellent coordination performance. With the upper-level lane-change decisions, the lower-level right-of-way assignment model is used to guarantee the safety of lane-change behaviors. The experiments show that the proposed approaches can lead to safe, efficient, and harmonious lane-change behaviors, which boosts the collaboration between vehicles and in turn improves the safety and efficiency of the overall traffic. Moreover, the proposed approaches promote the microscopic synchronization of vehicles, which can lead to the macroscopic synchronization of traffic flow.

This article proposes a distributed stochastic algorithm with variance reduction for general smooth non-convex finite-sum optimization, which has wide applications in signal processing and machine learning communities. In distributed setting, a large number of samples are allocated to multiple agents in the network. Each agent computes local stochastic gradient and communicates with its neighbors to seek for the global optimum. In this article, we develop a modified variance reduction technique to deal with the variance introduced by stochastic gradients. Combining gradient tracking and variance reduction techniques, this article proposes a distributed stochastic algorithm, gradient tracking algorithm with variance reduction (GT-VR), to solve large-scale non-convex finite-sum optimization over multiagent networks. A complete and rigorous proof shows that the GT-VR algorithm converges to the first-order stationary points with$O({1}/{k})$convergence rate. In addition, we provide the complexity analysis of the proposed algorithm. Compared with some existing first-order methods, the proposed algorithm has a lower$\mathcal {O}(PM\epsilon ^{-1})$gradient complexity under some mild condition. By comparing state-of-the-art algorithms and GT-VR in numerical simulations, we verify the efficiency of the proposed algorithm.

Human activity recognition (HAR) shows significant applicable values in health care. However, it has not been widely used due to some constraints, like privacy concerns of vision-based methods and the inconvenience of wearable devices-based methods. Recently, millimeter-wave (mmWave) based HAR has caught attention increasingly as it is able to provide a non-imaging, contactless and continuous approach for HAR. However, the existing mmWave-based HAR methods have limited ability and scalability to discriminate similar activities. To address existing problems, we present MM-HAT, an end-to-end Transformer network for mmWave point cloud based HAR. We design point-cloud-specific adaptations to Transformers that have gained success in natural language processing and vision. In addition, MM-HAT takes both mmWave point cloud data and target-data extracted from point clouds as input to mitigate the adverse effects of mmWave signals' vulnerability. We have collected a 7-activity mmWave dataset and carried out experiments on it. Evaluation results show that MM-HAT outperforms the existing methods by up to 34.76% (RadHAR) and 20.05% (MMPoint-GNN).

The problem of solving discrete-time Lyapunov equations (DTLEs) is investigated over multiagent network systems, where each agent has access to its local information and communicates with its neighbors. To obtain a solution to DTLE, a distributed algorithm with uncoordinated constant step sizes is proposed over time-varying topologies. The convergence properties and the range of constant step sizes of the proposed algorithm are analyzed. Moreover, a linear convergence rate is proved and the convergence performances over dynamic networks are verified by numerical simulations.

This article proposes a distributed optimization design to compute continuous-time algebraic Riccati inequalities (ARIs), where the information of matrices is distributed among agents. We propose a design procedure to tackle the nonlinearity, the inequality, and the coupled information structure of ARI; then, we design a distributed algorithm based on an optimization approach and analyze its convergence properties. The proposed algorithm is able to verify whether ARI is feasible in a distributed way and converges to a solution if ARI is feasible for any initial condition.

This article establishes an approach, based on distributed optimization, for solving continuous-time Lyapunov equations (CTLE) over multiagent networks. Each agent in the network knows partial information of the CTLE and has a dynamical system to estimate exact or least-squares solutions. The aim of agents is to find a solution to CTLE by sharing information with connected agents over a network. This article develops distributed algorithms with an exponential rate of convergence for CTLE via the convex optimization design. Finally, this article presents numerical simulations to show the efficacy of the main results.

This article focuses on the problem of a distributed computation of continuous-time algebraic Riccati equations (CARE), where information of matrices is split and known by multiple agents. This article proposes a distributed optimization design of the iterative refinement technique (IRM), a well-established centralized method for CARE. By assuming that each agent only knows partial information of CARE, we reformulate IRM for CARE as three classes of distributed optimization subproblems with different formulations and constraints. Then, we propose distributed algorithms for obtained distributed optimization subproblems and prove convergence properties of proposed algorithms. Numerical results show the efficacy of the proposed distributed IRM.

The distributed optimal consensus based on an event-triggered scheme for Euler-Lagrange (EL) multiagent systems is investigated in this article. The objective is to minimize the global cost function in a distributed manner while achieving consensus, where the local cost function of each agent is only known by itself. First, the distributed optimization algorithms based on the event-triggered scheme are proposed to achieve optimal consensus as well as reduce communication costs for the EL multiagent systems when the model parameters are available. Second, when the model parameters are unavailable, the tracking controllers are developed to solve the optimization problem for the EL multiagent systems. Then, the distributed optimization problem for EL systems can be transformed into the tracking problem for double-integrator multiagent systems. With the proposed algorithms, global optimization can be achieved with the exponential convergence rate. Finally, a simulation example is presented to illustrate the effectiveness of the proposed method.

This paper investigates the problem of solving a unique solution to discrete-time Lyapunov equations (DTLE) using multi-agent networks. We propose a distributed algorithm where each agent only uses partial information of the matrices. The agents of the algorithm reach a consensus by exchanging information with their neighbors over an undirected connected graph. We provide convergence analysis and the convergence rate estimate for the proposed algorithm. Finally, convergence performance is verified by numerical simulations.

This paper develops three novel hybrid stabilization techniques addressing fast energy equipartition for cyber-physical network systems, establishes an optimization-based network topology design framework to achieve cascading resilience and efficiency of geospatially distributed physical networks, and discusses the future application of the proposed approach to power network systems. Thus, the main contributions of this paper are three-fold. First, we present three hybrid distributed stabilization architectures for cyber-physical network systems to achieve the robust performance of geospatial physical networks by mimicking the dynamic behavior of thermodynamic systems. The proposed stabilization architectures are constructed in such a way that each stabilizer has a one-directional energy transfer from a plant to itself, and exchanges energy with its neighboring stabilizers. Second, to balance resilience to cascading failures and efficiency of energy flow in geospatially distributed physical networks, we propose an entropy metric-based multiobjective optimization framework for network topology design to characterize this resilience-efficiency trade-off design in networks. Moreover, we propose a novel cascade-connectivity swarm optimization algorithm which combines swarm intelligence and graph theory together to solve this multiobjective optimization problem. Finally, we apply our hybrid stabilization techniques and topology design algorithms to power network systems, and simulation studies are carried out to show the efficacy of the proposed approach.