Optimal transport (OT) theory has attracted much attention in machine learning and signal processing applications. OT defines a notion of distance between probability distributions of source and target data points. A crucial factor that influences OT-based distances is the ground metric of the embedding space in which the source and target data points lie. In this work, we propose to learn a suitable latent ground metric parameterized by a symmetric positive definite matrix. We use the rich Riemannian geometry of symmetric positive definite matrices to jointly learn the OT distance along with the ground metric. Empirical results illustrate the efficacy of the learned metric in OT-based domain adaptation.

Graph Neural Networks (GNNs) are proficient in graph representation learning and achieve promising performance on versatile tasks such as node classification and link prediction. Usually, a comprehensive hyperparameter tuning is essential for fully unlocking GNN's top performance, especially for complicated tasks such as node classification on large graphs and long-range graphs. This is usually associated with high computational and time costs and careful design of appropriate search spaces. This work introduces a graph-conditioned latent diffusion framework (GNN-Diff) to generate high-performing GNNs based on the model checkpoints of sub-optimal hyperparameters selected by a light-tuning coarse search. We validate our method through 166 experiments across four graph tasks: node classification on small, large, and long-range graphs, as well as link prediction. Our experiments involve 10 classic and state-of-the-art target models and 20 publicly available datasets. The results consistently demonstrate that GNN-Diff: (1) boosts the performance of GNNs with efficient hyperparameter tuning; and (2) presents high stability and generalizability on unseen data across multiple generation runs. The code is available at https://github.com/lequanlin/GNN-Diff.

Graph Neural Networks (GNNs) have emerged as fundamental tools for a wide range of prediction tasks on graph-structured data. Recent studies have drawn analogies between GNN feature propagation and diffusion processes, which can be interpreted as dynamical systems. In this paper, we delve deeper into this perspective by connecting the dynamics in GNNs to modern Koopman theory and its numerical method, Dynamic Mode Decomposition (DMD). We illustrate how DMD can estimate a low-rank, finite-dimensional linear operator based on multiple states of the system, effectively approximating potential nonlinear interactions between nodes in the graph. This approach allows us to capture complex dynamics within the graph accurately and efficiently. We theoretically establish a connection between the DMD-estimated operator and the original dynamic operator between system states. Building upon this foundation, we introduce a family of DMD-GNN models that effectively leverage the low-rank eigenfunctions provided by the DMD algorithm. We further discuss the potential of enhancing our approach by incorporating domain-specific constraints such as symmetry into the DMD computation, allowing the corresponding GNN models to respect known physical properties of the underlying system. Our work paves the path for applying advanced dynamical system analysis tools via GNNs. We validate our approach through extensive experiments on various learning tasks, including directed graphs, large-scale graphs, long-range interactions, and spatial-temporal graphs. We also empirically verify that our proposed models can serve as powerful encoders for link prediction tasks. The results demonstrate that our DMD-enhanced GNNs achieve state-of-the-art performance, highlighting the effectiveness of integrating DMD into GNN frameworks.

Traffic forecasting, a crucial application of spatio-temporal graph (STG) learning, has traditionally relied on deterministic models for accurate point estimations. Yet, these models fall short of quantifying future uncertainties. Recently, many probabilistic methods, especially variants of diffusion models, have been proposed to fill this gap. However, existing diffusion methods typically deal with individual sensors separately when generating future time series, resulting in limited usage of spatial information in the probabilistic learning process. In this work, we propose SpecSTG, a novel spectral diffusion framework, to better leverage spatial dependencies and systematic patterns inherent in traffic data. More specifically, our method generates the Fourier representation of future time series, transforming the learning process into the spectral domain enriched with spatial information. Additionally, our approach incorporates a fast spectral graph convolution designed for Fourier input, alleviating the computational burden associated with existing models. Compared with state-of-the-arts, SpecSTG achieves up to 8% improvements on point estimations and up to 0.78% improvements on quantifying future uncertainties. Furthermore, SpecSTG’s training and validation speed is 3.33× of the most efficient existing diffusion method for STG forecasting. The source code for SpecSTG is available at https://anonymous.4open.science/r/SpecSTG.

Supervised learning relies on high-quality labeled data, but obtaining such data through human annotation is both expensive and time-consuming. Recent work explores using large language models (LLMs) for annotation, but LLM-generated labels still fall short of human-level quality. To address this problem, we propose the Annotation with Critical Thinking (ACT) data pipeline, where LLMs serve not only as annotators but also as judges to critically identify potential errors. Human effort is then directed towards reviewing only the most "suspicious" cases, significantly improving the human annotation efficiency. Our major contributions are as follows: (1) ACT is applicable to a wide range of domains, including natural language processing (NLP), computer vision (CV), and multimodal understanding, by leveraging multimodal-LLMs (MLLMs). (2) Through empirical studies, we derive 7 insights on how to enhance annotation quality while efficiently reducing the human cost, and then translate these findings into user-friendly guidelines. (3) We theoretically analyze how to modify the loss function so that models trained on ACT data achieve similar performance to those trained on fully human-annotated data. Our experiments show that the performance gap can be reduced to less than 2% on most benchmark datasets while saving up to 90% of human costs.

Graph Neural Networks (GNNs) have become a cornerstone in multivariate time series forecasting by addressing the challenge of modeling inter-series dependencies often overlooked by traditional temporal approaches. However, real-world temporal dependencies (inter- and intra-dependencies) are inherently intertwined, making it difficult to treat them as separate processes. Recent pure graph paradigms attempt to capture these dependencies holistically by transforming time series into fully connected graphs. While effective, these methods suffer from prohibitive computational complexity O (( NT ) 2 ), limiting their scalability for large-scale data and long-term forecasting. To address these challenges, we propose UFGTime, a novel framework that leverages spectral signals to construct a "spectral-variate graph," embedding multivariate temporal dependencies in a compact spectral representation and modeling inter- and intra-signal connections through frequency similarities. Empowered by our proposed graph framelet message-passing function, UFGTime efficiently aggregates global information, avoids over-smoothing, and achieves near-linear complexity O ( kNT ). Extensive experiments on diverse datasets demonstrate that UFG-Time consistently outperforms state-of-the-art baselines, offering a scalable, accurate, and resource-efficient pure graph solution for multivariate time series forecasting.

Knowledge distillation (KD) has shown great potential for transferring knowledge from a complex teacher model to a simple student model in which the heavy learning task can be accomplished efficiently and without losing too much prediction accuracy. Recently, many attempts have been made by applying the KD mechanism to graph representation learning models such as graph neural networks (GNNs) to accelerate the model's inference speed via student models. However, many existing KD-based GNNs utilize multilayer perceptron (MLP) as a universal approximator in the student model to imitate the teacher model's process without considering the graph knowledge from the teacher model. In this work, we provide a KD-based framework on multiscaled GNNs, known as graph framelet, and prove that by adequately utilizing the graph knowledge in a multiscaled manner provided by graph framelet decomposition, the student model is capable of adapting both homophilic and heterophilic graphs and has the potential of alleviating the oversquashing issue with a simple yet effective graph surgery. Furthermore, we show how the graph knowledge supplied by the teacher is learned and digested by the student model via both algebra and geometry. Comprehensive experiments show that our proposed model can generate learning accuracy identical to or even surpass the teacher model while maintaining the high speed of inference.

Due to the depth degradation effect in residual connections, many efficient Vision Transformers models that rely on stacking layers for information exchange often fail to form sufficient information mixing, leading to unnatural visual perception. To address this issue, in this paper, we propose Aggregated Attention, a biomimetic design-based token mixer that simulates biological foveal vision and continuous eye movement while enabling each token on the feature map to have a global perception. Furthermore, we incorporate learnable tokens that interact with conventional queries and keys, which further diversifies the generation of affinity matrices beyond merely relying on the similarity between queries and keys. Our approach does not rely on stacking for information exchange, thus effectively avoiding depth degradation and achieving natural visual perception. Additionally, we propose Convolutional GLU, a channel mixer that bridges the gap between GLU and SE mechanism, which empowers each token to have channel attention based on its nearest neighbor image features, enhancing local modeling capability and model robustness. We combine aggregated attention and convolutional GLU to create a new visual backbone called TransNeXt. Extensive experiments demonstrate that our TransNeXt achieves state-of-the-art performance across multiple model sizes. At a resolution of 2242, TransNeXt-Tiny attains an ImageNet accuracy of 84.0%, surpassing ConvNeXt-B with 69% fewer parameters. Our TransNeXt-Base achieves an ImageNet accuracy of 86.2% and an ImageNet-A accuracy of 61. 6% at a resolution of 3842, a COCO object detection mAP of 57.1, and an ADE20K semantic segmentation mIoU of 54. 7.

Numerous recent research on graph neural networks (GNNs) has focused on formulating GNN architectures as an optimization problem with the smoothness assumption. However, in node classification tasks, the smoothing effect induced by GNNs tends to assimilate representations and over-homogenize labels of connected nodes, leading to adverse effects such as over-smoothing and misclassification. In this paper, we propose a novel bilevel optimization framework for GNNs inspired by the notion of Bregman distance. We demonstrate that the GNN layer proposed accordingly can effectively mitigate the over-smoothing issue by introducing a mechanism reminiscent of the “skip connection”. We validate our theoretical results through comprehensive empirical studies in which Bregman-enhanced GNNs outperform their original counterparts in both homophilic and heterophilic graphs. Furthermore, our experiments also show that Bregman GNNs can produce more robust learning accuracy even when the number of layers is high, suggesting the effectiveness of the proposed method in alleviating the over-smoothing issue.

Dimensionality reduction (DR) and manifold learning (ManL) have been applied extensively in many machine learning tasks, including computer vision, image analysis and pattern recognition just to name a few. However, the geometrical correctness of DR and ManL models learning results is largely neglected. In this work, we investigate this important aspect of some widely used DR and ManL methods through the lens of the chart map function, which is the essential part of the definition of a manifold. It turns out that the mapping functions induced by these methods do not have the injectivity (i.e. one-to-one mapping) between ambient space and latent space. This poses the distinguishability problem for down-stream tasks. Without injectivity, two distinct points on a manifold may be projected to the same point in the latent space learnt by DR or ManL methods, and hence the later process cannot separate them apart, resulting in inevitable errors. To address this problem, we provide a provably correct algorithm called fixed points Laplacian mapping (FPLM), which has the geometric guarantee to find a representation of a manifold with injectivity by using a simplical complex generated on manifold. We further discuss the property of our proposed method via various aspects, including its link to the popular graph neural networks (GNNs) and deep neural networks (DNNs). Its geometric correctness is demonstrated by extensive experimental results and theoretical proofs. Moreover, experiments also show our method is capable of evaluating the quality of simplex decomposition of the manifold and detecting manifold intrinsic dimensions for real-world datasets.

Three 5G enabling technologies, i.e, sparse code multiple access (SCMA), Polar codes and filtered OFDM (f-OFDM), are implemented into a massive MIMO system in the field trial conducted by HUAWEI and NTT DOCOMO to investigate their impacts on the system spectral efficiency. Taking massive MIMO with OFDMA and Turbo codes as the baseline, we observe about 30% downlink spectral efficiency improvement in the trial. Results from test field are summarized and analyzed in this paper, which verified that SCMA, Polar codes and f- OFDM are also feasible for massive MIMO systems in terms of spectral efficiency improvement.

Spectral efficiency is always a key factor to be improved and optimized along mobile communication networks evolving generation by generation. 5G enabling technologies must take spectral efficiency into consideration. In this paper, we show the performance of three key 5G technologies in sense of spectral efficiency improvement. Sparse code multiple access, polar codes, and filtered orthogonal frequency-division multiplexing are novel multiple access technology, channel coding scheme, and waveform, respectively. The combination of them is implemented in a 5G field trial testbed by NTT DOCOMO and Huawei for the first time. According to the field test results, we achieve over 100% spectral efficiency improvement comparison with baseline, where orthogonal frequency-division multiple access and turbo coding as LTE are used.

In this paper a crossed application and physical layer retransmission scheme is proposed for video streaming services in hybrid wireline and wireless networks. Each video packet carries the importance levels of succeeding packets at the application layer. In wireless transmission, by mapping the packet importance levels to different number of physical layer HARQ retransmissions, data packets are differentially protected which can greatly alleviate higher layer retransmissions and at the same time reduce the transmission delay for realtime services.