Graph Neural Networks (GNNs) have demonstrated impressive performance across diverse graph-based tasks by leveraging message passing to capture complex node relationships. However, on large-scale real-world graphs, GNNs face two major challenges: (1) GNNs struggle to ensure scalability and efficiency as repeated aggregation of large neighborhoods incurs significant computational overhead; (2) GNNs suffer from over-smoothing, where excessive propagation makes node representations indistinguishable, hindering model expressiveness. To tackle these, we propose ScaleGNN, which adaptively fuses multi-hop node features for scalable and effective graph learning. We first compute per-hop pure-neighbor matrices to isolate exclusive structural signals, then apply lightweight fusion to balance low- and high-order information, preserving both local detail and global correlations. To curb redundancy and over-smoothing, we introduce Local Contribution Score (LCS)–based masking to prune low-relevance high-order neighbors, and impose learnable sparsity to selectively integrate valuable multi-hop features. Extensive experiments on real-world datasets show that ScaleGNN consistently outperforms state-of-the-art GNNs in both predictive accuracy and computational efficiency. The source code is available at https://github.com/lx970414/ScaleGNN.

Rating is a typical user’s explicit feedback that visually reflects how much a user likes a related item. The (rating) matrix completion is essentially a rating prediction process, which is also a significant problem in recommender systems. Recently, graph neural networks (GNNs) have been widely used in matrix completion, which captures users’ preferences over items by formulating a rating matrix as a bipartite graph. However, existing methods are susceptible due to data sparsity and long-tail distribution in real-world scenarios. Moreover, the messaging mechanism of GNNs makes it difficult to capture high-order correlations and constraints between nodes, which are essentially useful in recommendation tasks. To tackle these challenges, we propose a M ulti-Channel H ypergraph C ontrastive L earning framework for matrix completion, named MHCL . Specifically, MHCL adaptively learns hypergraph structures to capture high-order correlations between nodes and jointly captures local and global collaborative relationships through attention-based cross-view aggregation. Additionally, to consider the magnitude and order information of ratings, we treat different rating subgraphs as different channels, encourage alignment between adjacent ratings, and further achieve the mutual enhancement between different ratings through multi-channel cross-rating contrastive learning. Extensive experiments on eight publicly available real-world datasets demonstrate that our proposed method significantly outperforms the current state-of-the-art approaches. The source code of our model is available at https://github.com/lx970414/MHCL .

Graph anomaly detection (GAD) is a critical task in graph machine learning, with the primary objective of identifying anomalous nodes that deviate significantly from the majority. This task is widely applied in various real-world scenarios, including fraud detection and social network analysis. However, existing GAD methods still face two major challenges: (1) They are often limited to detecting anomalies in single-type interaction graphs and struggle with multiple interaction types in multiplex heterogeneous graphs. (2) In unsupervised scenarios, selecting appropriate anomaly score thresholds remains a significant challenge for accurate anomaly detection. To address the above challenges, we propose a novel Unsupervised Multiplex Graph Anomaly Detection method, named UMGAD. We first learn multi-relational correlations among nodes in multiplex heterogeneous graphs and capture anomaly information during node attribute and structure reconstruction through graph-masked autoencoder (GMAE). Then, to further extract abnormal information, we generate attribute-level and subgraph-level augmented-view graphs, respectively, and perform attribute and structure reconstruction through GMAE. Finally, we learn to optimize node attributes and structural features through contrastive learning between original-view and augmented-view graphs to improve the model's ability to capture anomalies. Meanwhile, we propose a new anomaly score threshold selection strategy, which allows the model to be independent of ground truth information in real unsupervised scenarios. Extensive experiments on six datasets show that our UMGAD significantly outperforms state-of-the-art methods, achieving average improvements of 12.25% in AUC and 11.29% in Macro-F1 across all datasets. The source code of our model is available at https://github.com/lx970414/UMGAD.

Trajectory representation learning transforms the complex spatio-temporal features of trajectories into a dense, low-dimensional embedding, which supports various downstream analytics tasks such as trajectory classification, travel time estimation, and similar trajectory search. Existing trajectory representation learning methods treat trajectories merely as general point sequences and use sequence models to learn the correlations between points. However, the complex spatio-temporal features of trajectories are multi-scale, meaning they are not only reflected in the correlations between trajectory points but also in the correlations between trajectory segments. Moreover, most existing methods do not sufficiently capture the multi-faceted temporal features within trajectories. To fill these gaps, we propose a novel self-supervised Trajectory$R$epresentation$L$earning model with multi-scale spatio-temporal features exploration called TrajRL. Specifically, we utilize trajectory augmentation to generate two views to achieve self-supervised pre-training exploiting multiple self-supervisory signals. In each view, we can learn the multi-scale spatio-temporal correlations both within and between road segments and road segment sequences in trajectories through the proposed multi-scale trajectory encoder. Additionally, we perform multi-faceted temporal information encoding, especially leveraging time intervals to learn multi-scale context-aware time patterns within the trajectories. Extensive experiments demonstrate the superiority of our TrajRL as compared to state-of-the-art baselines on two real-world datasets across various downstream tasks. The source code of our model is available at https://github.com/Xfc30/TrajRL.

Since capturing underwater images without degradation is challenging, there are few real image datasets with paired ground truth for underwater image enhancement. In this article, we propose a generative adversarial network (UIGAN) for underwater imaging; the network can convert images and their corresponding depth maps captured in air into images in water. The underwater imaging mechanism relies on many intrinsic parameters in water, which are difficult to estimate without field calibration. As the strong modeling capability of deep neural networks, this article uses the deep learning model to extract parameters from the real underwater environment. Then the proposed UIGAN simulates the light propagation process (direct attenuation, backscattering, and forward scattering) in water by using three modules with different constraints. We can generate a large training dataset with paired images in air and real water environment. The generated UIGAN dataset serves as input to a forward-attention transfer underwater enhancement model (FATUECNN), and it can output the restored images with appearance like those captured in air. The proposed pipeline is verified both qualitatively and quantitatively by extensive experiments and comparison evaluation with the existing state-of-the-art methods. The source code and the pre-trained model are made publicly available.

Effective recommender systems play a crucial role in accurately capturing user and item attributes that mirror individual preferences. Some existing recommendation techniques have started to shift their focus towards modeling various types of interactive relations between users and items in real-world recommendation scenarios, such as clicks, marking favorites, and purchases on online shopping platforms. Nevertheless, these approaches still grapple with two significant challenges: (1) Insufficient modeling and exploitation of the impact of various behavior patterns formed by multiplex relations between users and items on representation learning, and (2) ignoring the effect of different relations within behavior patterns on the target relation in recommender system scenarios. In this work, we introduce a novel recommendation framework,Dual-ChannelMultiplexGraphNeuralNetwork (DCMGNN), which addresses the aforementioned challenges. It incorporates an explicit behavior pattern representation learner to capture the behavior patterns composed of multiplex user-item interactive relations, and includes a relation chain representation learner and a relation chain-aware encoder to discover the impact of various auxiliary relations on the target relation, the dependencies between different relations, and mine the appropriate order of relations in a behavior pattern. Extensive experiments on three real-world datasets demonstrate that our DCMGNN surpasses various state-of-the-art recommendation methods. It outperforms the best baselines by 10.06% and 12.15% on average across all datasets in terms of Recall@10 and NDCG@10 respectively. The source code of our paper is available athttps://github.com/lx970414/TKDE-DCMGNN.

Temporal motifs are recurring subgraph patterns in temporal graphs, and are present in various domains such as social networks, fraud detection, and biological networks. Despite their significance, counting temporal motifs efficiently remains a challenge, particularly on moderately sized datasets with millions of motif instances. To address this challenge, we propose a novel algorithm called Scalable Motif Counting with Time-aware Topology Constraint (MoTTo). MoTTo focuses on accurately counting temporal motifs with up to three nodes and three edges. It first utilizes a topology constraint-based pruning strategy to eliminate nodes that cannot participate in forming temporal motifs before the counting process. Then, it adopts a time-aware topology constraint-based pruning strategy to split large-scale datasets into independent partitions and filter out the unrelated ones, ensuring that the counting results remain unaffected. By investigating the second pruning strategy, we also find that MoTTo can be implemented in a multi-thread manner, further accelerating the counting process significantly. Experimental results on several real-world datasets of varying sizes demonstrate that MoTTo outperforms state-of-the-art methods in terms of efficiency, achieving up to a nine-fold improvement in total temporal motif counting. Specifically, the efficiency of counting triangular temporal motifs is enhanced by up to 31 times compared to state-of-the-art baselines.

Visual Semantic Embedding (VSE) is a prominent approach in image-text retrieval, aiming to learn a deep embedding space that aligns visual data with semantic text labels. However, current VSE methods oversimplify the retrieval task, treating it as a binary classification problem with triplet loss constraints. This ignores the semantic correlation between pairs of mismatched samples and fails to capture the similarity gradient between samples. In addition, hard constraints on negative samples with high semantic relevance can be detrimental to the model's representational capabilities. To address these limitations, we propose a novel training strategy that introduces dynamic soft labels without additional annotations. This captures the correlation between positive and negative sample pairs and guides feature representation learning using the Soft Negative Alignment Loss (SNAL). SNAL fully takes into account the influence by similar negative samples, enhancing the representation of cross-modal data. In addition, we propose the Stepwise Negative Decoupling Loss (SNDL) to increase the distance between positive and negative samples. Stepwise decoupling of negative samples can be adaptively distanced based on their semantic relevance to the anchor, resulting in a wider distribution of sample features in the common space. Experiments on Flickr30K and MS-COCO datasets validate the effectiveness of our dynamic soft labeling (DSL) methods, demonstrating the importance of considering complex relationships between sample pairs and the limitations of rigid negative sample categorization based on subjective annotations.

Heterogeneous graph convolutional networks have gained great popularity in tackling various network analytical tasks on heterogeneous graph data, ranging from link prediction to node classification. However, most existing works ignore the relation heterogeneity with multiplex networks between multi-typed nodes and the different importance of relations in meta-paths for node embedding, which can hardly capture the heterogeneous structure signals across different relations. To tackle this challenge, this work proposes a M ultiplex H eterogeneous G raph C onvolutional N etwork (MHGCN+) for multiplex heterogeneous network embedding. Our MHGCN+ can automatically learn the useful heterogeneous meta-path interactions of different lengths with different importance in multiplex heterogeneous networks through multi-layer convolution aggregation. Additionally, we effectively integrate both multi-relation structural signals and attribute semantics into the learned node embeddings with both unsupervised and semi-supervised learning paradigms. Extensive experiments on seven real-world datasets with various network analytical tasks demonstrate the significant superiority of MHGCN+ against state-of-the-art embedding baselines in terms of all evaluation metrics. The source code of our method is available at: https://github.com/FuChF/MHGCN-plus .

Trajectory-User Linking (TUL) is crucial for human mobility modeling by linking different trajectories to users with the exploration of complex mobility patterns. Existing works mainly rely on the recurrent neural framework to encode the temporal dependencies in trajectories, have fall short in capturing spatial-temporal global context for TUL prediction. To fill this gap, this work presents a new hierarchical spatio-temporal attention neural network, calledAttnTUL, to jointly encode the local trajectory transitional patterns and global spatial dependencies for TUL. Specifically, our first model component is built over the graph neural architecture to preserve the local and global context and enhance the representation paradigm of geographical regions and user trajectories. Additionally, a hierarchically structured attention network is designed to simultaneously encode the intra-trajectory and inter-trajectory dependencies, with the integration of the temporal attention mechanism and global elastic attentional encoder. Extensive experiments demonstrate the superiority of our AttnTUL method as compared to state-of-the-art baselines on various trajectory datasets. The source code of our model is available at https://github.com/Onedean/AttnTUL .

Next location recommendation plays an important role in various location-based services, yielding great value for both users and service providers. Existing methods usually model temporal dependencies with explicit time intervals or learn representation from customized point of interest (POI) graphs with rich context information to capture the sequential patterns among POIs. However, this problem is perceptibly complex, because various factors, e.g., users’ preferences, spatial locations, time contexts, activity category semantics, and temporal relations, need to be considered together, while most studies lack sufficient consideration of the collaborative signals. Toward this goal, we propose a novel M ulti-Level C ollaborative Neural N etwork for next location Rec ommendation (MCN4Rec). Specifically, we design a multi-level view representation learning with level-wise contrastive learning to collaboratively learn representation from local and global perspectives to capture complex heterogeneous relationships among user, POI, time, and activity categories. Then, a causal encoder-decoder is applied to the learned representations of check-in sequences to recommend the next location. Extensive experiments on four real-world check-in mobility datasets demonstrate that our model significantly outperforms the existing state-of-the-art baselines for the next location recommendation. Ablation study further validates the benefits of the collaboration of the designed sub-modules. The source code is available at https://github.com/quai-mengxiang/MCN4Rec .

Heterogeneous graph neural networks have gained great popularity in tackling various network analysis tasks on heterogeneous network data. However, most existing works mainly focus on general heterogeneous networks, and assume that there is only one type of edge between two nodes, while ignoring the multiplex characteristics between multi-typed nodes in multiplex heterogeneous networks and the different importance of multiplex structures among nodes for node embedding. In addition, the over-smoothing issue of graph neural networks limits existing models to only capturing local structure signals but hardly learning the global relevant information of the network. To tackle these challenges, this work proposes a model called Behavior Pattern based Heterogeneous Graph Neural Network (BPHGNN) for multiplex heterogeneous network embedding. Specifically, BPHGNN can collaboratively learn node representations across different multiplex structures among nodes with adaptive importance learning from local and global perspectives in multiplex heterogeneous networks through depth behavior pattern aggregation and breadth behavior pattern aggregation. Extensive experiments on six real-world networks with various network analytical tasks demonstrate the significant superiority of BPHGNN against state-of-the-art approaches in terms of various evaluation metrics.

Accurate citywide traffic inference is critical for improving intelligent transportation systems with smart city applications. However, this task is very challenging given the limited training data, due to the high cost of sensor installment and maintenance across the entire urban space. A more practical scenario to study the citywide traffic inference is effectively modeling the spatial and temporal traffic patterns with limited historical traffic observations. In this work, we propose a dynamic multi-view graph neural network for citywide traffic inference with the method CTVI+. Specifically, for the temporal dimension, we propose a temporal self-attention mechanism that is capable of learning the dynamics of traffic data with the time-evolving traffic volume variations. For spatial dimension, we build a multi-view graph neural network, employing the road-wise message passing scheme to capture the region dependencies. With the designed spatial-temporal learning paradigms, we enable our traffic inference model to encode the dynamism from both spatial and temporal traffic patterns, which is reflective of intra- and inter-road traffic correlations. In our evaluation, CTVI+ achieves consistent better performance compared with different baselines on real-world traffic volume datasets. Further ablation study validates the effectiveness of key components in CTVI+. We release the model implementation at https://github.com/dsj96/TKDD.

One fundamental problem in temporal graph anal-ysis is to count the occurrences of small connected subgraph patterns (i.e., motifs), which benefits a broad range of real-world applications, such as anomaly detection, structure prediction, and network representation learning. However, existing works focused on exacting temporal motif are not scalable to large-scale temporal graph data, due to their heavy computational costs or inherent inadequacy of parallelism. In this work, we propose a scalable parallel framework for exactly counting temporal motifs in large-scale temporal graphs. We first categorize the temporal motifs based on their distinct properties, and then design customized algorithms that offer efficient strategies to exactly count the motif instances of each category. Moreover, our compact data structures, namely triple and quadruple counters, enable our algorithms to directly identify the temporal motif instances of each category, according to edge information and relationship between edges, therefore significantly improving the counting efficiency. Based on the proposed counting algorithms, we design a hierarchical parallel framework that featuring both inter- and intra-node parallel strategies, and fully leverages the multi-threading capacity of modern CPU to concurrently count all temporal motifs. Extensive experiments on sixteen real-world temporal graph datasets demonstrate the superiority and capability of our proposed framework for temporal motif counting, achieving up to$538\times$speedup compared to the state-of-the-art methods. The source code of our method is available at: https://github.com/steven-ccq/FAST-temporal-motif.

Heterogeneous graph convolutional networks have gained great popularity in tackling various network analytical tasks on heterogeneous network data, ranging from link prediction to node classification. However, most existing works ignore the relation heterogeneity with multiplex network between multi-typed nodes and different importance of relations in meta-paths for node embedding, which can hardly capture the heterogeneous structure signals across different relations. To tackle this challenge, this work proposes a Multiplex Heterogeneous Graph Convolutional Network (MHGCN) for heterogeneous network embedding. Our MHGCN can automatically learn the useful heterogeneous meta-path interactions of different lengths in multiplex heterogeneous networks through multi-layer convolution aggregation. Additionally, we effectively integrate both multi-relation structural signals and attribute semantics into the learned node embeddings with both unsupervised and semi-supervised learning paradigms. Extensive experiments on five real-world datasets with various network analytical tasks demonstrate the significant superiority of MHGCN against state-of-the-art embedding baselines in terms of all evaluation metrics. The source code of our method is available at: https://github.com/NSSSJSS/MHGCN.

Abstract Recommending a limited number of Point-of-Interests (POIs) a user will visit next has become increasingly important to both users and POI holders for Location-Based Social Networks (LBSNs). However, POI recommendation is a challenging task since complex sequential patterns and rich contexts are contained in extremely sparse user check-in data. Recent studies show that embedding techniques effectively incorporate POI contextual information to alleviate the data sparsity issue, and Recurrent Neural Network (RNN) has been successfully employed for sequential prediction. Nevertheless, existing POI recommendation approaches are still limited in capturing user personalized preference due to separate embedding learning or network modeling. To this end, we propose a novel unified spatio-temporal neural network framework, named PPR, which leverages users’ check-in records and social ties to recommend personalized POIs for querying users by joint embedding and sequential modeling. Specifically, PPR first learns user and POI representations by joint modeling User-POI relation, sequential patterns, geographical influence, and social ties in a heterogeneous graph and then models user personalized sequential patterns using the designed spatio-temporal neural network based on LSTM model for the personalized POI recommendation. Furthermore, we extend PPR to an end-to-end recommendation model by jointly learning node representations and modeling user personalized sequential preference. Extensive experiments on three real-world datasets demonstrate that our model significantly outperforms state-of-the-art baselines for successive POI recommendation in terms of Accuracy, Precision, Recall and NDCG. The source code is available at: https://www.anonymous.4open.science/r/DSE-1BEC .

With the development of mobile position techniques, sensing the citywide traffic information has been well recognized as a crucial task for various urban computing applications, such as intelligent transportation system, location-based recommendation, and user mobility modeling. With the consideration of high cost for sensor installment and maintenance, the traffic monitoring spatial coverage is often very limited in practical urban sensing scenarios. The goal of this paper is to perform the traffic inference over road segments which lack of (with very limited) historical traffic observations. Towards this end, we propose a temporal multi-view graph convolutional network for C itywide T raffic V olume I nference (CTVI) which jointly captures the spatial-temporal dependencies across different time intervals and geographical locations. In our CTVI framework, we design our attentive multi-view graph neural architecture based on our generated spatial and feature affinity graphs, to perform the cross-layer message passing with the preservation of road segment-wise topological context. In addition, we develop a temporal self-attention module to encode the evolving traffic patterns over time, which incorporates the time-wise relation contextual signals into the main embedding space. Furthermore, we propose a joint learning objective function that consists of an unsupervised random walk enhancement and a semi-supervised spatio-temporal volume constraint to guide the learning of road segment representations for citywide traffic volume inference. Evaluation results on real-world traffic datasets demonstrate the superiority of our proposed CTVI framework as compared to state-of-the-art baselines.

Network embedding has emerged as a new learning paradigm to embed complex network into a low-dimensional vector space while preserving node proximities in both network structures and properties. It advances various network mining tasks, ranging from link prediction to node classification. However, most existing works primarily focus on static networks while many networks in real-life evolve over time with addition/deletion of links and nodes, naturally with associated attribute evolution. In this work, we present Motif-preserving Temporal Shift Network (MTSN), a novel dynamic network embedding framework that simultaneously models the local high-order structures and temporal evolution for dynamic attributed networks. Specifically, MTSN learns node representations by stacking the proposed TIME module to capture both local high-order structural proximities and node attributes by motif-preserving encoder and temporal dynamics by temporal shift operation in a dynamic attributed network. Finally, we perform extensive experiments on four real-world network datasets to demonstrate the superiority of MTSN against state-of-the-art network embedding baselines in terms of both effectiveness and efficiency. The source code of our method is available at: https://github.com/ZhijunLiu95/MTSN.

Network embedding, which aims at learning low-dimensional representations of nodes in a network, has drawn much attention for various network mining tasks, ranging from link prediction to node classification. In addition to network topological information, there also exist rich attributes associated with network structure, which exerts large effects on the network formation. Hence, many efforts have been devoted to tackling attributed network embedding tasks. However, they are also limited in their assumption of static network data as they do not account for evolving network structure as well as changes in the associated attributes. Furthermore, scalability is a key factor when performing representation learning on large-scale networks with huge number of nodes and edges. In this work, we address these challenges by developing the DRLAN-Dynamic Representation Learning framework for large-scale Attributed Networks. The DRLAN model generalizes the dynamic attributed network embedding from two perspectives: First, we develop an integrative learning framework with an offline batch embedding module to preserve both the node and attribute proximities, and online network embedding model that recursively updates learned representation vectors. Second, we design a recursive pre-projection mechanism to efficiently model the attribute correlations based on the associative property of matrices. Finally, we perform extensive experiments on three real-world network datasets to show the superiority of DRLAN against state-of-the-art network embedding techniques in terms of both effectiveness and efficiency. The source code is available at: https://github.com/ZhijunLiu95/DRLAN.

In recent years, heterogeneous network representation learning has attracted considerable attentions with the consideration of multiple node types. However, most of them ignore the rich set of network attributes (attributed network) and different types of relations (multiplex network), which can hardly recognize the multi-modal contextual signals across different interactions. While a handful of network embedding techniques are developed for attributed multiplex heterogeneous networks, they are significantly limited to the scalability issue on large-scale network data, due to their heavy cost both in computation and memory. In this work, we propose a Fast Attributed Multiplex heterogeneous network Embedding framework (FAME) for large-scale network data, by mapping the units from different modalities (i.e., network topological structures, various node features and relations) into the same latent space in a very efficient way. Our FAME is an integrative architecture with the scalable spectral transformation and sparse random projection, to automatically preserve both attribute semantics and multi-type interactions in the learned embeddings. Extensive experiments on four real-world datasets with various network analytical tasks, demonstrate that FAME achieves both effectiveness and significant efficiency over state-of-the-art baselines. The source code is available at: https://github.com/ZhijunLiu95/FAME.

Convolutional neural networks (CNN) have made great progress for synthetic aperture radar (SAR) images change detection. However, sampling locations of traditional convolutional kernels are fixed and cannot be changed according to the actual structure of the SAR images. Besides, objects may appear with different sizes in natural scenes, which requires the network to have stronger multi-scale representation ability. In this paper, a novel Deformable Residual Convolutional Neural Network (DRNet) is designed for SAR images change detection. First, the proposed DRNet introduces the deformable convolutional sampling locations, and the shape of convolutional kernel can be adaptively adjusted according to the actual structure of ground objects. To create the deformable sampling locations, 2-D offsets are calculated for each pixel according to the spatial information of the input images. Then the sampling location of pixels can adaptively reflect the spatial structure of the input images. Moreover, we proposed a novel pooling module replacing the vanilla pooling to utilize multi-scale information effectively, by constructing hierarchical residual-like connections within one pooling layer, which improve the multi-scale representation ability at a granular level. Experimental results on three real SAR datasets demonstrate the effectiveness of the proposed DR-Net.

In recent years, many deep architectures have been proposed for handwritten text recognition. However, most of the previous deep models need large scale training data and a long training time to obtain good results. In this paper, we propose a novel deep learning method based on “stretching” the projection matrices of stacked feature learning models. We call the proposed method “stretching deep architectures” (or SDA). In the implementation of SDA, stacked feature learning models are first learned layer by layer, and then the stretching technique is applied on the weight matrices between successive layers. As the feature learning models can be efficiently optimized and the stretching results can be easily computed, the training of SDA is very fast and no back propagation is needed. We have tested SDA on handwritten digits recognition, Arabic subword recognition and English letter recognition tasks. Extensive experiments demonstrate that SDA performs not only better than shallow feature learning models, but also state-of-the-art deep learning models.