The effectiveness of Transformer-based methods in tabular data processing has been extensively validated. However, most existing models primarily adopt Transformer-based encoder architectures, which overly emphasize feature correlations while neglecting the unique explicit information representation characteristics of tabular data. The significantly higher information density of tabular data compared to textual data introduces noise that degrades performance. Consequently, directly applying language models to tabular data often yields suboptimal results. To address this limitation, we propose the Residual Attention Transformer (RAT), a Transformer-based mechanism specifically designed for tabular data. In subsequent sections, the Residual Attention Transformer will be referred to as RAT. RAT introduces residual connections into the self-attention mechanism, effectively preserving raw information while capturing dependencies with higher-order features, thus enhancing the richness of information representation. Additionally, we design a simplified Transformer output head, termed Light-Transformer, which consists of a lightweight Transformer block specifically dedicated to the output stage. This design not only significantly reduces the model's parameter count, but also greatly improves training and inference efficiency. Extensive experiments were conducted on six public datasets to evaluate our model. The results show that the RAT model consistently outperforms other models in various tasks, confirming its effectiveness and superiority.

Intelligent financial forecasting models play a crucial role in warnings of market crashes, market recovery, portfolio optimization, and risk control. Among deep learning methods, graph neural networks have achieved remarkable results in time series prediction due to their ability to model complex relationship between nodes, but they still have limitation in capturing dynamic associations within attribute features, which restricts their ability to fully understand financial data. On the other hand, most deep learning methods are difficult to quantify the relationship between different financial data attributes (e.g., closing price, highest price, and lowest price), which prevents them from comprehensively analysing the attribute features, which in turn affects the prediction accuracy. To address this gap, this study proposes a new self-knowledge distillation model for multi-attribute financial time series data. This approach constructs an adaptive graph to represent different attribute features, performs self-distillation across mixed attributes, and integrates these techniques into a teacher network. Furthermore, this study introduces a novel multi-attribute feature attention fusion module to capture and integrate key features from multiple attributes, thereby enhancing the model’s predictive effectiveness and robustness. This research validates the superiority of the proposed method using time series data from three different financial markets: futures, cryptocurrency, and stock, demonstrating superior prediction performance. Given the cross-market and cross-asset correlations present in financial data, we have decided to open-source this dataset for time series prediction, data imputation, and correlation studies.

Emergency events like traffic accidents and natural disasters frequently cause severe disruptions to urban traffic patterns, posing challenges to conventional forecasting methods based on historical data. Recent progress has explored integrating auxiliary textual information from social media, news reports, and incident details to enhance forecasting models. However, these approaches often fail to effectively align semantic context from textual data with the spatio-temporal patterns in urban traffic flow. To bridge this disparity, we propose a novel framework called TPGM-LLM, which leverages text-prompt-guided multimodal large language models to dynamically assimilate real-time data, including incident reports, traffic sensors, and weather conditions. Specifically, the proposed framework integrates a pre-trained LLM-based text encoder to interpret descriptions of emergency events as semantic prompts. Furthermore, it incorporates a dynamic spatio-temporal hypergraph module that employs FastDTW to capture non-local dependencies among different road segments. Additionally, a multimodal feature extraction LLM is utilized to merge textual guidance with traffic dynamics in a hierarchical manner, enhancing the accuracy of long-term traffic forecasting. Experimental results on the Beijing Text-Traffic dataset (BjTT) demonstrate that the proposed model outperforms existing methods, highlighting its effectiveness in complex and dynamic traffic situations. Our code is available athttps://github.com/luyaxuan/TPGM-LLM

Temporal dynamic graphs (TDGs), representing the dynamic evolution of entities and their relationships over time with intricate temporal features, are widely used in various real-world domains. Existing methods typically rely on mainstream techniques such as transformers and graph neural networks (GNNs) to capture the spatiotemporal information of TDGs. However, despite their advanced capabilities, these methods often struggle with significant computational complexity and limited ability to capture temporal dynamic contextual relationships. Recently, a new model architecture called mamba has emerged, noted for its capability to capture complex dependencies in sequences while significantly reducing computational complexity. Building on this, we propose a novel method, TDG-mamba, which integrates mamba for TDG learning. TDG-mamba introduces deep semantic spatiotemporal embeddings into the mamba architecture through a specially designed spatiotemporal prior tokenization module (SPTM). Furthermore, to better leverage temporal information differences and enhance the modeling of dynamic changes in graph structures, we separately design a bidirectional mamba and a directed GNN for improved spatiotemporal embedding learning. Link prediction experiments on multiple public datasets demonstrate that our method delivers superior performance, with an average improvement of 5.11% over baseline methods across various settings.

Bound by social bonds, people often travel in groups, exhibiting mutual correlations in their mobility behaviors, known as mobility correlations. However, conventional graphs, which describe pairwise connections between vertices, fail to adequately capture the complex many-to-many connection characteristics of multiple individuals. To address this limitation, we propose a novel semisupervised hypergraph clustering (SSHC) model aimed at unraveling the intricate correlations among individuals before discerning their latent clusters. Each individual is characterized by features that encompass spatial, temporal, and attributive diversity. The high-order correlations concealed among multiple individuals are captured using a hypergraph framework featuring hyperedges. Building upon this foundation, we fuse individuals’ latent feature representations, extracted via an auto-encoder, with their structural representations derived from a hypergraph convolutional network. Through this fusion, we identify clusters of similar individuals exhibiting mobility correlations. Notably, our model incorporates a semisupervised learning scheme, leveraging priori patterns gleaned from a small pool of ground-truth data to better express latent patterns within unlabeled datasets. Experimental validations conducted in five public datasets corroborate the superiority of our proposed model over baseline methods in elucidating complex correlations within vast datasets. In addition, experiments conducted on Beijing Transit Smart Card Data showcase promising results in identifying correlated individuals with similar mobility patterns. The delineation of groups of individuals exhibiting mobility correlations is of paramount importance, particularly in accurately detecting their connection networks during public emergencies, such as the transmission of dangerous pathogens.

Radar Echo Extrapolation (REE) can facilitate accurate and expeditious nowcasting of precipitation, reducing the reliance on complex Numerical Weather Prediction (NWP) models. Spatial-temporal forecasting methods dominate this task because they can fully exploit the spatio-temporal dependencies and complex dynamic patterns inherent in radar echo data. However, they struggle with handling uncertainty and incorporating domain-specific knowledge, often resulting in blurry or unrealistic predictions. We propose a Hypergraph-enhanced Latent Diffusion Model (HyDiff) to address these limitations. The EchoDiff has been utilized to aid in accurate extrapolation. To accurately describe precipitation microphysics while adhering to the hydro-microphysical and multiscale coupling principles, the method integrates additional semantic information into the model. Specifically, the Differential Reflectivity Factor (ZDR) and Differential Propagation Phase Shift (KDP) are incorporated into the model as additional semantic information. Furthermore, we introduce a Hypergraph Neural Network (HGNN) into the extrapolation method to capture correlation information across regions. Experiments show that HyDiff effectively handles uncertainty, incorporates domain-specific prior knowledge, and generates forecasts with high operational utility.

As deep learning technology continues to advance in the field of meteorological forecasting, accurate radar echo extrapolation technology is crucial for two-hour precipitation prediction. Recent approaches commonly utilize 2D CNN and 3D CNN for feature extraction. However, the absence of high-level temporal sequence features prevents the model from acquiring sufficient information for accurate precipitation forecasting. To this end, we introduce a MIPRNet for precise precipitation forecasting. The Multi-Information Extractor (MIE), based on graph convolution and Fourier transforms, captures high-level complex temporal features of extrapolation evolution and frequency characteristics of precipitation dynamics. Meanwhile, MHPA, which utilizes the multi-head attention mechanism, aggregates and captures potential precipitation evolution patterns. In the extrapolation process, MIPRNet uses the potential evolution patterns of extrapolation obtained from the information extraction part to perform extrapolation. Experimental results on two radar meteorological datasets demonstrate that MIPRNet outperforms existing models in terms of multiple authoritative metrics.

Predicting air pollution plays a vital role in urban management and public health by providing early warnings on PM2.5, SO2, and NO2 concentrations, helping to mitigate the adverse effects of these pollutants. Traditional prediction methods, relying on physical and statistical models, often struggle to capture the complex spatio-temporal dependencies and dynamic characteristics of air pollution data. The application of deep learning methods, especially graph neural networks (GNNs), has shown promise in addressing these limitations. However, existing GNN-based methods ignore the integration of rich semantic information provided by high-resolution satellite data. To address this problem, we propose a Cross-Modality Dynamic Spatio-Temporal Graph Neural Network (CMDNet) for air pollution prediction. The model comprises two branches: a dynamic spatio-temporal graph neural network branch and a remote sensing image dynamic encoding network branch. The dynamic spatiotemporal graph neural network branch captures the spatiotemporal dependencies in air pollution data by constructing a dynamic graph structure. The remote sensing image dynamic encoding network branch extracts semantic information from high-resolution satellite data, which improves the model’s power to perceive air pollution conditions in different regions. Experiments on real-world datasets demonstrate that CMDNet achieves better air pollution prediction results than existing SOTA models, with maximum improvements of 2.4% (MAE), 1.8% (RMSE), 1.3% (CSI), 1.6% (FAR), and 1.4% (POD), providing more accurate prediction results.

Weather forecasting, compared to other multivariate time-series prediction tasks, exhibits notably non-uniform distribution of observation sites. The prevailing approaches primarily leverage graph convolutional neural networks to extract spatial features. However, some studies suggest that the poor performance on uneven graphs is primarily due to the fact that traditional graph neural networks (GNNs) are essentially low-pass filters, discarding information beyond low-frequency information on the graph. From another perspective, since the essence of graph convolution is the smoothing of node features, for uneven graphs, there are noticeable differences in the smoothing rates of node features, leading to the coexistence of overfitting and underfitting phenomena. This issue is further exacerbated in higher-order graph structures, such as hypergraphs, due to the irregular and complex nature of hyperedges. To address this issue, we propose a filtered hypergraph neural network. Building on the calculation of hypergraph node smoothing rates, we balance the low-pass and high-pass filter convolutions’ feature extraction through a dual-stream architecture. On uneven graphs, it can be observed that neglecting high-frequency information and concentrating solely on low-frequency information impede the learning of node representations, thereby significantly affecting the performance of downstream prediction tasks. We conducted multi-dimensional time-series prediction experiments using meteorological data, and the results demonstrate that our model performs with high accuracy in node regression tasks across multiple channels.

Satellite image sequence prediction is a fundamental yet challenging task in weather forecasting. Existing approaches often combine neural network models (e.g., RNN, CNN, transformer, and GAN) to capture spatial features and temporal state transitions. While expressive, these models frequently overlook higher order motion correlations, leading to ambiguities in motion estimation and loss of appearance details. To address this, we propose HyperMM, a novel method for satellite image sequence prediction using a hypergraph-enhanced motion matrix. Unlike prior methods that compute pairwise feature similarities between pixels or patches within the same module, HyperMM constructs an appearance-independent motion matrix enhanced by hypergraph neural networks (HGNNs) to model high-order spatial correlations among pixel groups more effectively. Furthermore, we introduce interframe dynamic attention in the trunk network to improve temporal feature extraction, and intraframe static attention to reduce appearance information loss. Experiments on the Fengyun-4B (FY-4B) satellite dataset show that HyperMM achieves state-of-the-art performance by retaining fine-grained appearance detail while having the superior ability to accurately predict motion trends.

Pedestrian trajectory prediction is a key component for various applications that involve human and vehicle interactions, such as autonomous driving, traffic management and smart city planning. Existing methods based on graph neural networks have limited ability to capture group interactions and precisely model complex associations among multi-agents. To solve these problems, we propose OST-HGCN, an optimized hypergraph convolutional network. It models multi-agent trajectory interactions from both temporal and spatial perspectives using hypergraph structures, and optimizes the spatio-temporal hypergraph structure to enable fine-grained analysis of multi-agent trajectory motion intentions and high-order interactions. We employ OST-HGCN to a CVAE-based prediction framework, and use the optimized hypergraph structure to predict multi-agent plausible trajectories. We conduct extensive experiments on four real trajectory prediction datasets of NBA, NFL, SDD and ETH-UCY, and verify the effectiveness of the proposed OST-HGCN.

Predicting and planning the future trajectories of various traffic participants is an important task with multiple applications, including autonomous vehicles, service robots, and intelligent transportation. However, the diversity of heterogeneous agents including pedestrians, bicycles, and vehicles in traffic scenarios presents substantial challenges to this task. Current models do not fully capture the implicit and explicit interaction relationships among these heterogeneous agents and often overlook the significance of extracting implicit correlations from agent features. To address these issues, we introduce a novel model for trajectory prediction: Heterogeneous Graph Structure Contrast Optimization (HGSCO). To accurately capture the interaction relationships among heterogeneous agents, HGSCO constructs semantic graph structures representing implicit relationships and meta-path graph structures representing explicit relationships. Then, HGSCO introduces a cross-view contrastive learning approach, which optimizes the heterogeneous graph structure by maximizing mutual information between the two types of graph structures. The model can provide precise interaction relationships among heterogeneous agents by effectively fusing these two graph representations with a gated fusion method. We utilized video data captured by camera sensors in complex environments with multiple agents to conduct experiments. Our proposed model achieved an 11.5% and 6.7% reduction in Average Displacement Error (ADE) across these datasets, respectively, and a reduction of 15.6% and 8.1% in Final Displacement Error (FDE). The results demonstrate that HGSCO significantly surpasses existing state-of-the-art methods regarding trajectory prediction accuracy.

The analysis of traffic situations under abnormal conditions is one of the bottleneck issues in Intelligent Transportation Systems (ITS). Influenced by the suddenness, randomness, and uncertainty, this issue is challenging to achieve through existing deep learning methods. It needs to be assisted by traffic simulation models for analysis. However, simulation models always require extensive scene modeling and calibration, making it difficult to meet the demands of natural human-machine interaction in the AIGC (Artificial Intelligence Generated Content) era, as well as the need for rapid and flexible implementation of situation analysis. With the accumulation of traffic data, the emergence of diffusion models offers a new entry point for the core method of data-driven analysis, namely Text-to-Traffic Generation (TTG). In this work, we explore how generative models combined with text describing the traffic system can be applied for traffic situation generation, and propose ChatTraffic, the first diffusion model for TTG. To guarantee the consistency between synthetic and real data, we augment a diffusion model with the Graph Convolutional Network (GCN) to extract spatial correlations of traffic data. In addition, we construct a large-scale dataset containing text-traffic pairs for TTG. We benchmarked ChatTraffic qualitatively and quantitatively on the released dataset. The experimental results indicate that ChatTraffic can rapidly and flexibly generate realistic traffic situations from text, which have practical significance in addressing bottlenecks in ITS. Our code and dataset are available athttps://github.com/ChyaZhang/ChatTraffic.

Multivariate Time Series Classification (MTSC) has important research significance and practical value. Deep learning models have achieved considerable success in addressing MTSC problems. However, a key challenge faced by existing classification models is how to effectively consider the correlations between time series instances and across channels simultaneously, as well as how to capture the dynamic of these inter-channel correlations over time. Current methods often fall short in these aspects: on one hand, they fail to fully account for the combined effects of inter-instance and inter-channel correlations; on the other hand, they largely overlook the dynamic nature of how inter-channel correlations change over time. To address these issues, we propose a novel graph neural network model, called Similarity-Aware Graph of Graphs neural networks (SAGoG), for multivariate time series classification. This model can comprehensively consider the dependencies between channel-level and instance-level time series, it dynamically learns dependency features through graph structure evolution and graph pooling layers. We conduct experiments on the UEA dataset to validate the SAGoG model, and the results demonstrate its outstanding performance in multivariate time series classification tasks.

Attribute graphs are a crucial data structure for graph communities. However, the presence of redundancy and noise in the attribute graph can impair the aggregation effect of integrating two different heterogeneous distributions of attribute and structural features, resulting in inconsistent and distorted data that ultimately compromises the accuracy and reliability of attribute graph learning. For instance, redundant or irrelevant attributes can result in overfitting, while noisy attributes can lead to underfitting. Similarly, redundant or noisy structural features can affect the accuracy of graph representations, making it challenging to distinguish between different nodes or communities. To address these issues, we propose the embedded fusion graph auto-encoder framework for self-supervised learning (SSL), which leverages multitask learning to fuse node features across different tasks to reduce redundancy. The embedding fusion graph auto-encoder (EFGAE) framework comprises two phases: pretraining (PT) and downstream task learning (DTL). During the PT phase, EFGAE uses a graph auto-encoder (GAE) based on adversarial contrastive learning to learn structural and attribute embeddings separately and then fuses these embeddings to obtain a representation of the entire graph. During the DTL phase, we introduce an adaptive graph convolutional network (AGCN), which is applied to graph neural network (GNN) classifiers to enhance recognition for downstream tasks. The experimental results demonstrate that our approach outperforms state-of-the-art (SOTA) techniques in terms of accuracy, generalization ability, and robustness.

In this paper, we propose an environment sensing-aided beam prediction model for smart factory that can be transferred from given environments to a new environment. In particular, we first design a pre-training model that predicts the optimal beam by sensing the present environmental information. When encountering a new environment, it generally requires collecting a large amount of new training data to retrain the model, whose cost severely impedes the application of the designed pre-training model. Therefore, we next design a transfer learning strategy that fine-tunes the pre-trained model by limited labeled data of the new environment. Simulation results show that when the pre-trained model is fine-tuned by 30% of labeled data from the new environment, the Top-10 beam prediction accuracy reaches 94%. Moreover, compared with the way to completely re-training the prediction model, the amount of training data and the time cost of the proposed transfer learning strategy reduce 70% and 75% respectively.

Millimeter-wave (mmWave) communications with abundant spectrum resources have become an enabling technology for high throughput, ultra-reliable, and low latency communications (URLLC). Since the mmWave signal is sensitive to blockage, accurate base station (BS) selection is the premise of achieving the URLLC. In this paper, we propose a multi-view images assisted proactive BS selection scheme that can predict the optimal BS for the user in the next frame. The proposed scheme utilizes vision sensing and thus does not require the entire pilot resources, such that the latency caused by seeding and receiving pilots reduces. In addition, we design a multitask learning strategy and a prior knowledge based fine tuning method to ensure the accuracy and reliability of BS selection. Simulation results in an outdoor environment demonstrate the superior performance of the proposed scheme in terms of both the accuracy and the robustness.

Spatial transcriptomics reveals the spatial distribution of genes in complex tissues, providing crucial insights into biological processes, disease mechanisms, and drug development. The prediction of gene expression based on cost-effective histology images is a promising yet challenging field of research. Existing methods for gene prediction from histology images exhibit two major limitations. First, they ignore the intricate relationship between cell morphological information and gene expression. Second, these methods do not fully utilize the different latent stages of features extracted from the images. To address these limitations, we propose a novel hypergraph neural network model, HGGEP, to predict gene expressions from histology images. HGGEP includes a gradient enhancement module to enhance the model's perception of cell morphological information. A lightweight backbone network extracts multiple latent stage features from the image, followed by attention mechanisms to refine the representation of features at each latent stage and capture their relations with nearby features. To explore higher-order associations among multiple latent stage features, we stack them and feed into the hypergraph to establish associations among features at different scales. Experimental results on multiple datasets from disease samples including cancers and tumor disease, demonstrate the superior performance of our HGGEP model than existing methods.

Abstract Referring image segmentation identifies the object masks from images with the guidance of input natural language expressions. Nowadays, many remarkable cross‐modal decoder are devoted to this task. But there are mainly two key challenges in these models. One is that these models usually lack to extract fine‐grained boundary information and gradient information of images. The other is that these models usually lack to explore language associations among image pixels. In this work, a Multi‐scale Gradient balanced Central Difference Convolution (MG‐CDC) and a Graph convolutional network‐based Language and Image Fusion (GLIF) for cross‐modal encoder, called Graph‐RefSeg, are designed. Specifically, in the shallow layer of the encoder, the MG‐CDC captures comprehensive fine‐grained image features. It could enhance the perception of target boundaries and provide effective guidance for deeper encoding layers. In each encoder layer, the GLIF is used for cross‐modal fusion. It could explore the correlation of every pixel and its corresponding language vectors by a graph neural network. Since the encoder achieves robust cross‐modal alignment and context mining, a light‐weight decoder could be used for segmentation prediction. Extensive experiments show that the proposed Graph‐RefSeg outperforms the state‐of‐the‐art methods on three public datasets. Code and models will be made publicly available at https://github.com/ZYQ111/Graph_refseg .

Abstract Trajectory prediction is essential for intelligent autonomous systems like autonomous driving, behavior analysis, and service robotics. Deep learning has emerged as the predominant technique due to its superior modeling capability for trajectory data. However, deep learning‐based models face challenges in effectively utilizing scene information and accurately modeling agent interactions, largely due to the complexity and uncertainty of real‐world scenarios. To mitigate these challenges, this study presents a novel multiagent trajectory prediction model, termed the global‐local scene‐enhanced social interaction graph network (GLSESIGN), which incorporates two pivotal strategies: global‐local scene information utilization and a social adaptive attention graph network. The model hierarchically learns scene information relevant to multiple intelligent agents, thereby enhancing the understanding of complex scenes. Additionally, it adaptively captures social interactions, improving adaptability to diverse interaction patterns through sparse graph structures. This model not only improves the understanding of complex scenes but also accurately predicts future trajectories of multiple intelligent agents by flexibly modeling intricate interactions. Experimental validation on public datasets substantiates the efficacy of the proposed model. This research offers a novel model to address the complexity and uncertainty in multiagent trajectory prediction, providing more accurate predictive support in practical application scenarios.

Abstract Frontal person images contain the richest detailed features of humans, which can effectively assist in behavioral recognition, virtual dress fitting and other applications. While many remarkable networks are devoted to the person image generation task, most of them need accurate target poses as the network inputs. However, the target pose annotation is difficult and time‐consuming. In this work, we proposed a first frontal person image generation network based on the proposed anchor pose set and the generative adversarial network. Specifically, our method first classify a rough frontal pose to the input human image based on the proposed anchor pose set, and regress all key points of the rough frontal pose to estimate an accurate frontal pose. Then, we consider the estimated frontal pose as the target pose, and construct a two‐stream generator based on the generative adversarial network to update the person's shape and appearance feature in a crossing way and generate a realistic frontal person image. Experiments on the challenging CMU Panoptic dataset show that our method can generate realistic frontal images from arbitrary‐view human images.

Multivariate time series forecasting plays an important role in many domain applications, such as air pollution forecasting and traffic forecasting. Modeling the complex dependencies among time series is a key challenging task in multivariate time series forecasting. Many previous works have used graph structures to learn inter-series correlations, which have achieved remarkable performance. However, graph networks can only capture spatio-temporal dependencies between pairs of nodes, which cannot handle high-order correlations among time series. We propose a Dynamic Hypergraph Structure Learning model (DHSL) to solve the above problems. We generate dynamic hypergraph structures from time series data using the K-Nearest Neighbors method. Then a dynamic hypergraph structure learning module is used to optimize the hypergraph structure to obtain more accurate high-order correlations among nodes. Finally, the hypergraph structures dynamically learned are used in the spatio-temporal hypergraph neural network. We conduct experiments on six real-world datasets. The prediction performance of our model surpasses existing graph network-based prediction models. The experimental results demonstrate the effectiveness and competitiveness of the DHSL model for multivariate time series forecasting.

Millimeter-wave (mmWave) communications with abundant spectrum resources have become an enabling technology for high throughput, ultra-reliable, and low latency communications (URLLC). Since the mmWave signal is sensitive to blockage, accurate base station (BS) selection and beam searching are the premises of achieving the URLLC. In this paper, we consider the mmWave communications systems where mobile users are served by the roadside unit (RSU). We propose a multi-camera view based proactive RSU selection and beam searching scheme that can predict the optimal RSU for the user in the next frame and search the corresponding beam pair. The proposed scheme utilizes vision sensing and reduces training resources. In addition, the visual information of multiple views makes the selection of the optimal RSU more accurate and reliable compared to the existing single view technologies. Simulation results in an outdoor environment show the superior performance of the proposed scheme in terms of predicting accuracy and achievable rate.

Community detection aims to identify dense subgroups of nodes within a network. However, in real-world networks, node attributes are often missing, making traditional methods less effective. In networks with missing attributes, the main challenge of community detection is to deal with the missing attribute information efficiently and use network structure information to make accurate predictions. This article proposes an innovative method called contrastive sampling-aggregating transformer (CSAT) for community detection in attribute-missing networks. CSAT incorporates the contrastive learning principle to capture hidden patterns among nodes and to aggregate information from different samples to create a more robust and accurate methodology for community detection. Specifically, CSAT utilizes a sampling and propagation strategy to obtain different samples and smooth attribute features of the network structure and leverages the Transformer architecture to model the pairwise relationships between nodes. Therefore, our method can address the attribute-missing issue by integrating the auxiliary information from both the network structure and other sources. Extensive experiments on several benchmark datasets demonstrate CSAT’s superior performance compared to the state-of-the-art methods for community detection.

Graph convolutional networks (GCNs) are widely used in social computation such as urban traffic prediction. However, when faced with city-level forecasting challenges, the graph-based deep learning methods struggle to process large-scale multivariate data effectively. To address the challenges of limited scalability, a traffic prediction framework based on a hypergraph message passing network (HMSG) is proposed in this article. The model represents the urban transportation network with hypergraph, where nodes denote transportation hubs and hyperedges represent their relationship at geographical and feature level. Compared with pairwise edges, hyperedges are more scalable and flexible, providing a more descriptive representation of traffic information. The HMSG algorithm updates node and hyperedge features in two steps, facilitating effective and efficient integration of hidden spatial features across layers. The proposed framework is evaluated on large-scale historical datasets and demonstrates its completion of city-scale traffic prediction tasks. The results also show that it matches the accuracy of existing traffic prediction methods on small-scale datasets. This validates the potential of the traffic prediction model based on the HMSG algorithm for intelligent transportation applications.

Accurate prediction of origin-destination (OD) demand is critical for service providers to efficiently allocate limited resources in regions with high travel demands. However, OD distributions pose significant challenges, characterized by high sparsity, complex spatial correlations within regions or chains, and potential repetition due to the recurrence of similar semantic contexts. These challenges impede traditional graph-based approaches, which connect two vertices through an edge, from performing effectively in OD prediction. Thus, we present a novel multichannel hypergraph convolutional neural network (MC-HGCN) to overcome the above challenges. The model innovatively extracts distinctive features from the channels of inflows, outflows, and OD flows, to conquer the high sparsity in OD matrices. High-order spatial proximity within regions and OD chains are then modeled by the three adjacency hypergraphs constructed for the above three channels. In each adjacency hypergraph, multiple neighboring stations are treated as vertices, while multiple OD pairs constitute hyperedges. These structures are learned by hypergraph convolutional networks for latent spatial correlations. On this basis, a semantic hypergraph is created for the OD channel to model OD distributions lacking spatial proximity but sharing semantic correlations. It utilizes hyperedges to represent semantic correlations among OD pairs whose origins and destinations both possess similar point-of-interest (POI) functions, before learned by a hypergraph convolutional network (HGCN). Both spatial and semantic correlations intrinsic to OD flows are accordingly captured and embedded into a gated recurrent unit (GRU) to unveil hidden spatiotemporal dependencies among OD distributions. These embedded correlations are ultimately integrated through a multichannel fusion module to enhance the prediction of OD flows, even for minor ones. Our model is validated through experiments on three public datasets, demonstrating its robust performances across long and short time steps. Findings may contribute theoretical insights for practical applications, such as coordinating traffic scheduling or route planning.

Recommender systems have become an indispensable tool in today's digital age, significantly enhancing user engagement on various online platforms by curating personalized item recommendations tailored to individual preferences. While the field has long been dominated by the collaborative filtering technique, which primarily leverages user–item interaction data, it often falls short in encapsulating the rich contextual intricacies and evolving dynamics inherent to these interactions. Recognizing this limitation, our research introduces the contextual semantic interaction graph embedding (CSI-GE) method. This advanced model incorporates a dynamic hop window within a multilayer graph convolutional network, ensuring a comprehensive extraction of both immediate and evolving contextual features. By amalgamating self-supervised contrastive learning, we achieve a refinement of user and item embeddings. Furthermore, our innovative variance–invariance–covariance (VIC) regularization-based loss function fortifies the robustness of these embeddings. Through rigorous testing, CSI-GE consistently outperformed contemporary methods, underscoring its superior accuracy and stability.

Weather forecasting is inextricably linked to human lives and represents a quintessential task of spatiotemporal modeling, necessitated by the spatial and temporal dependencies inherent in meteorological data. Recent studies have consistently shown the excellent performance of graph-based neural networks in accurately modeling spatiotemporal data across various applications. Yet, traditional graph neural networks (GNNs) are unable to handle the high-order diffusion and aggregation phenomena between meteorological data caused by advection. Moreover, the impacts of spatial correlation among multisource information and the presence of noise in meteorological data are often overlooked. This study proposes a novel approach for modeling the spatiotemporal dependencies in meteorological data using the multi-information spatiotemporal aggregation and estrangement hypergraph convolution network. This method employs a novel representation of meteorological data using hypergraphs to address the aforementioned challenges. Specifically, we construct adjacency and semantic hypergraphs to represent spatial correlations and then introduce aggregation and estrangement hypergraph convolution networks to effectively capture multi-information spatial correlations. A new reconstruction feature attention module has been developed to fuse aggregation and estrangement semantic spatial information across various subspaces. In addition, the hypergraph convolution is embedded within a recurrent neural network architecture to model the temporal correlations. Extensive experiments have been conducted on four weather datasets, and state-of-the-art performance has been achieved in comparison to mainstream baseline methods.

Precipitation nowcasting within 2 hours is an important and hard issue in weather research area. Benefiting from the outstanding nonlinear relationship modeling capability, methods based on deep learning have achieved significant success in the task of precipitation nowcasting compared to the others. However, existing deep learning based methods always disregard the intricate high-order correlations and lack substantial connections with the evolution of the precipitation system, which would lead blurred forecasts and implausible predictions. To address these issues, we proposed a new Precipitation Nowcasting Network within 2 hours model based on Hypergraph Neural Network (PN-HGNN). In this work, Hypergraph Neural Network is firstly adopted for extracting spatio-temporal dynamic echo features. Secondly, regulation evolution is in charge of capturing the memory features to guide the extrapolation. Finally, we design a dual branch module to extrapolate the radar echoes. The proposed model has been assessed on the dataset HKO-7. The experimental results demonstrate that PN-HGNN achieved better prediction performance than the six representative echo extrapolation models.

Crowd counting is very important in many fields such as public safety, urban planning, and is essential for the intelligent transportation systems. Due to the complexity and diversity of traffic scenes, point-level annotations for pedestrians would cost much human labor. Weakly supervised crowd counting methods are more suitable for these scenes, considering they only require count-level annotations. However, ignoring the uneven distribution of cross-distance crowd region density and multi-scale pedestrian head, existing weakly supervised methods can not achieve similar counting performance as fully supervised crowd counting methods. To solve these issues, we propose a novel multi-level dynamic graph convolutional networks for weakly supervised crowd counting. Within this network, a multi-level region dynamic graph convolutional module is designed to mine the cross-distance intrinsic relationship between crowd regions. A feature enhancement module is used to enhance crowd semantic information. In addition, we design a coarse grained multi-level feature fusion module to aggregate multi-scale pedestrian information. Experiments are conducted on five well-known benchmark crowd counting datasets, achieving state-of-the-art results compared to existing weakly supervised methods and competitive results compared to fully supervised methods.

Missing data in time series is a pervasive problem that serves as obstacles for subsequent traffic data analysis. Consequently, extensive research works have been conducted on traffic missing data imputation tasks. The state-of-the-art traffic data imputation models are mostly based on recurrent neural networks. However, these methods belong to autoregressive models which are highly susceptible to error propagation. The attention-based methods are non-autoregressive models that can avoid compounding errors and help achieve better imputation quality. Moreover, the attention-based methods in now widely applied and have achieved remarkable results, whereas their application on traffic data imputation is still limited. Thus, this paper proposes Self-Attention Graph Convolution Imputation Network (SAGCIN) for spatio-temporal traffic data. To ensure the accuracy of data imputation, it is necessary to fully capture the spatio-temporal contextual information of traffic data to impute missing values. To this end, the SAGCIN model incorporates self-attention mechanism with diffusion graph convolution network. The SAGCIN model consists of two spatio-temporal blocks with a spatio-temporal encoder and an imputation decoder. The encoder learns spatio-temporal representations specialized for traffic data imputation tasks. Based on the learned representation, the decoder performs two stages of imputation operator for missing data. A joint-optimization training approach of imputation and reconstruction is introduced for SAGCIN to perform missing value imputation for traffic data. Empirical results demonstrate that SAGCIN model outperforms state-of-the-art methods in imputation tasks on relevant real-world benchmarks.

Traffic prediction plays a significant role in Intelligent Transportation Systems (ITS). Although many datasets have been introduced to support the study of traffic prediction, most of them only provide time-series traffic data. However, urban transportation systems are always susceptible to various factors, including unusual weather and traffic accidents. Therefore, relying solely on historical data for traffic prediction greatly limits the accuracy of the prediction. In this paper, we introduce Beijing Text-Traffic (BjTT), a large-scale multimodal dataset for traffic prediction. BjTT comprises over 32,000 time-series traffic records, capturing velocity and congestion levels on more than 1,200 roads within the 5th ring area of Beijing. Meanwhile, each piece of traffic data is coupled with a text describing the traffic system (including time, location, and events). We detail the data collection and processing procedures and present a statistical analysis of the BjTT dataset. Furthermore, we conduct comprehensive experiments on the dataset with state-of-the-art traffic prediction methods and text-guided generative models, which reveal the unique characteristics of the BjTT. The dataset is available athttps://github.com/ChyaZhang/BjTT.

Most existing weakly supervised crowd counting methods utilize Convolutional Neural Networks (CNN) or Transformer to estimate the total number of individuals in an image. However, both CNN-based (grid-to-count paradigm) and Transformer-based (sequence-to-count paradigm) methods take images as inputs in a regular form. This approach treats all pixels equally but cannot address the uneven distribution problem within human crowds. This challenge would lead to a decline in the counting performance of the model. Compared with grid and sequence, the graph structure could better explore the relationship among features. In this article, we propose a new graph-based crowd counting method named CrowdGraph, which reinterprets the weakly supervised crowd counting problem from a graph-to-count perspective. In the proposed CrowdGraph, each image is constructed as a graph, and a graph-based network is designed to extract features at the graph level. CrowdGraph comprises three main components: a dynamic graph convolutional backbone, a multi-scale dilated graph convolution module, and a regression head. To the best of our knowledge, CrowdGraph is the first method that is completely formulated based on the Graph Neural Network (GNN) for the crowd counting task. Extensive experiments demonstrate that the proposed CrowdGraph outperforms pure CNN-based and pure Transformer-based weakly supervised methods comprehensively and achieves highly competitive counting performance.

Video object segmentation (VOS) exhibits heavy occlusions, large deformation, and severe motion blur. While many remarkable convolutional neural networks are devoted to the VOS task, they often mis-identify background noise as the target or output coarse object boundaries, due to the failure of mining detail information and high-order correlations of pixels within the whole video. In this work, we propose an edge attention gated graph convolutional network (GCN) for VOS. The seed point initialization and graph construction stages construct a spatio-temporal graph of the video by exploring the spatial intra-frame correlation and the temporal inter-frame correlation of superpixels. The node classification stage identifies foreground superpixels by using an edge attention gated GCN which mines higher-order correlations between superpixels and propagates features among different nodes. The segmentation optimization stage optimizes the classification of foreground superpixels and reduces segmentation errors by using a global appearance model which captures the long-term stable feature of objects. In summary, the key contribution of our framework is twofold: (a) the spatio-temporal graph representation can propagate the seed points of the first frame to subsequent frames and facilitate our framework for the semi-supervised VOS task; and (b) the edge attention gated GCN can learn the importance of each node with respect to both the neighboring nodes and the whole task with a small number of layers. Experiments on Davis 2016 and Davis 2017 datasets show that our framework achieves the excellent performance with only small training samples (45 video sequences).

Crowd counting provides an important foundation for public security and urban management. Due to the existence of small targets and large density variations in crowd images, crowd counting is a challenging task. Mainstream methods usually apply convolution neural networks (CNNs) to regress a density map, which requires annotations of individual persons and counts. Weakly-supervised methods can avoid detailed labeling and only require counts as annotations of images, but existing methods fail to achieve satisfactory performance because a global perspective field and multi-level information are usually ignored. We propose a weakly-supervised method, DTCC, which effectively combines multi-level dilated convolution and transformer methods to realize end-to-end crowd counting. Its main components include a recursive swin transformer and a multi-level dilated convolution regression head. The recursive swin transformer combines a pyramid visual transformer with a fine-tuned recursive pyramid structure to capture deep multi-level crowd features, including global features. The multi-level dilated convolution regression head includes multi-level dilated convolution and a linear regression head for the feature extraction module. This module can capture both low- and high-level features simultaneously to enhance the receptive field. In addition, two regression head fusion mechanisms realize dynamic and mean fusion counting. Experiments on four well-known benchmark crowd counting datasets (UCF_CC_50, ShanghaiTech, UCF_QNRF, and JHU-Crowd++) show that DTCC achieves results superior to other weakly-supervised methods and comparable to fully-supervised methods.

Social link is an important index to understand master students’ mental health and social ability in educational management. Extracting hidden social strength from students’ rich daily life behaviors has also become an attractive research hotspot. Devices with positioning functions record many students’ spatiotemporal behavior data, which can infer students’ social links. However, under the guidance of school regulations, students’ daily activities have a certain regularity and periodicity. Traditional methods usually compare the co-occurrence frequency of two users to infer social association but do not consider the location-intensive and time-sensitive in campus scenes. Aiming at the campus environment, a Spatiotemporal Entropy-Based Analyzing (S-EBA) model for inferring students’ social strength is proposed. The model is based on students’ multi-source heterogeneous behavioral data to calculate the frequency of co-occurrence under the influence of time intervals. Then, the three features of diversity, spatiotemporal hotspot and behavior similarity are introduced to calculate social strength. Experiments show that our method is superior to the traditional methods under many evaluating criteria. The inferred social strength is used as the weight of the edge to construct a social network further to analyze its important impact on students’ education management.

Abstract Crowd counting is a hot issue in visual data processing. It also plays an important role in the field of video surveillance, social security, and traffic control. However, most of the existing crowd counting methods always adopt a mount of training data or point‐level annotation to learn the mapping relationships between images and density maps, which would cost much human labor. In this paper, we propose a new weakly semi‐supervised crowd counting method which uses less count‐level data for data training. In particular, we extend the classical smoothness assumption and design a many‐to‐many Region Feature Smoothness Assumption to deal with the uneven density distribution problem within crowd region. Further, we adopt hypergraph representation to explore the complex high‐order relationship for different crowd regions. Besides, we design a multi‐scale dynamic hypergraph convolutional module and hyperedge contrastive loss. Extensive experiments have been conducted on five public datasets. The experimental results show that the proposed method outperforms the state‐of‐the‐art ones.

The exploration of self-supervised information mining of heterogeneous datasets has gained significant traction in recent years. Heterogeneous graph neural networks (HGNNs) have emerged as a highly promising method for handling heterogeneous information networks (HINs) due to their superior performance. These networks leverage aggregation functions to convert pairwise relations-based features from raw heterogeneous graphs into embedding vectors. However, real-world HINs contain valuable higher-order relations that are often overlooked but can provide complementary information. To address this issue, we propose a novel method calledSelf-supervisedNodes-HyperedgesEmbedding (SNHE), which leverages hypergraph structures to incorporate higher-order information into the embedding process of HINs. Our method decomposes the raw graph structure into snapshots based on various meta-paths, which are then transformed into hypergraphs to aggregate high-order information within the data and generate embedding representations. Given the complexity of HINs, we develop a dual self-supervised structure that maximizes mutual information in the enhanced graph data space, guides the overall model update, and reduces redundancy and noise. We evaluate our proposed method on various real-world datasets for node classification and clustering tasks, and compare it against state-of-the-art methods. The experimental results demonstrate the efficacy of our method. Our code is available athttps://github.com/limengran98/SNHE.

The traffic data corrupted by noise and missing entries often lead to the poor performance of Intelligent Transportation Systems (ITS), such as the bad congestion prediction and route guidance. How to efficiently impute the traffic data is an urgent problem. As a classic deep learning method, Generative Adversarial Network (GAN) achieves remarkable success in image recovery fields, which opens up a new way for the traffic data imputation. In this paper, we propose a novel spatio-temporal GAN model for the traffic data imputation (STGAN). Firstly, we design the generative loss and center loss, which not only minimizes the reconstructed errors of the imputed entries, but also ensures each imputed entry and its neighbors conform to the local spatio-temporal distribution. Then, the discriminator uses the convolution neural network classifier to judge whether the imputed matrix conforms to the global spatio-temporal distribution. As for the network architecture of the generator, we introduce the skip-connection to keep all well preserved data unchanged, and employ the dilated convolution to capture the spatio-temporal correlation in the traffic data. The experimental results show that our proposed method obviously outperforms other competitive traffic data imputation methods.

Complete and accurate traffic data is critical in urban traffic management, planning and operation. In fact, real-world traffic data contains missing values due to multiple factors, such as device outages and communication errors. For traffic data completion task, most of the existing methods are matrix/tensor completion methods, which usually enforce low rank constraint on traffic data matrix/tensor. But they neglect the graph structure of traffic data, resulting in low completion performance. Recently, graph convolutional networks have achieved remarkable results in traffic data forecasting due to their abilities of feature extraction and nonlinear fitting on arbitrarily graph-structured data. However, there are few studies based on graph neural networks for traffic data completion task. In this paper, we propose a traffic data completion model based on graph convolutional network model to impute missing values from the perspective of deep learning. This model utilizes graph convolution to model the local spatial dependency. As for global spatial dependency and temporal dependency, this model incorporates self-attention mechanism, which is applied in the spatial and temporal dimensions respectively. The experimental results on the two real-time datasets demonstrate that the proposed model outperforms the baseline methods significantly under arbitrarily missing scenarios.

Visual perception is an effective way to obtain the spatial characteristics of wireless channels and to reduce the overhead for communications system. A critical problem for the visual assistance is that the communications system needs to match the radio signal with the visual information of the corresponding user, i.e., to identify the visual user that corresponds to the target radio signal from all the environmental objects. In this paper, we propose a user matching method for environment with a variable number of objects. Specifically, we apply 3D detection to extract all the environmental objects from the images taken by multiple cameras. Then, we design a deep neural network (DNN) to estimate the location distribution of users by the images and beam pairs at multiple moments, and thereby identify the users from all the extracted environmental objects. Moreover, we present a resource allocation method based on the taken images to reduce the time and spectrum overhead compared to traditional resource allocation methods. Simulation results show that the proposed user matching method outperforms the existing methods, and the proposed resource allocation method can achieve 92% transmission rate of the traditional resource allocation method but with the time and spectrum overhead significantly reduced.

Graph convolutional networks (GCN) have been applied in the traffic flow forecasting tasks with the graph capability in describing the irregular topology structures of road networks. However, GCN based traffic flow forecasting methods often fail to simultaneously capture the short-term and long-term temporal relations carried by the traffic flow data, and also suffer the over-smoothing problem. To overcome the problems, we propose a hierarchical traffic flow forecasting network by merging newly designed the long-term temporal Transformer network (LTT) and the spatio-temporal graph convolutional networks (STGC). Specifically, LTT aims to learn the long-term temporal relations among the traffic flow data, while the STGC module aims to capture the short-term temporal relations and spatial relations among the traffic flow data, respectively, via cascading between the one-dimensional convolution and the graph convolution. In addition, an attention fusion mechanism is proposed to combine the long-term with the short-term temporal relations as the input of the graph convolution layer in STGC, in order to mitigate the over-smoothing problem of GCN. Experimental results on three public traffic flow datasets prove the effectiveness and robustness of the proposed method.

Graph neural networks (GNNs) have been widely used for graph structure learning and achieved excellent performance in tasks such as node classification and link prediction. Real-world graph networks imply complex and various semantic information and are often referred to as heterogeneous information networks (HINs). Previous GNNs have laboriously modeled heterogeneous graph networks with pairwise relations, in which the semantic information representation for learning is incomplete and severely hinders node embedded learning. Therefore, the conventional graph structure cannot satisfy the demand for information discovery in HINs. In this article, we propose an end-to-end hypergraph transformer neural network (HGTN) that exploits the communication abilities between different types of nodes and hyperedges to learn higher-order relations and discover semantic information. Specifically, attention mechanisms weigh the importance of semantic information hidden in original HINs to generate useful meta-paths. Meanwhile, our method develops a multi-scale attention module to aggregate node embeddings in higher-order neighborhoods. We evaluate the proposed model with node classification tasks on six datasets: DBLP, ACM, IBDM, Reuters, STUD-BJUT, and Citeseer. Experiments on a large number of benchmarks show the advantages of HGTN.

With the powerful learning ability of deep convolutional networks, deep clustering methods can extract the most discriminative information from individual data and produce more satisfactory clustering results. However, existing deep clustering methods usually ignore the relationship between the data. Fortunately, the graph convolutional network can handle such relationships, opening a new research direction for deep clustering. In this paper, we propose a cross-attention based deep clustering framework, named Cross-Attention Fusion based Enhanced Graph Convolutional Network (CaEGCN), which contains four main modules: the cross-attention fusion module which innovatively concatenates the Content Auto-encoder module (CAE) relating to the individual data and Graph Convolutional Auto-encoder module (GAE) relating to the relationship between the data in a layer-by-layer manner, and the self-supervised model that highlights the discriminative information for clustering tasks. While the cross-attention fusion module fuses two kinds of heterogeneous representation, the CAE module supplements the content information for the GAE module, which avoids the over-smoothing problem of GCN. In the GAE module, two novel loss functions are proposed that reconstruct the content and relationship between the data, respectively. Finally, the self-supervised module constrains the distributions of the middle layer representations of CAE and GAE to be consistent. Experimental results on different types of datasets prove the superiority and robustness of the proposed CaEGCN.

Passenger-flow anomaly detection and prediction are essential tasks for intelligent operation of the metro system. Accurate passenger-flow representation is the foundation of them. However, spatiotemporal dependencies, complex dynamic changes, and anomalies of passenger-flow data bring great challenges to data representation. Taking advantage of the time-varying characteristics of data, we propose a novel passenger-flow representation model based on low-rank dynamic mode decomposition (DMD), which also integrates the global low-rank nature and sparsity to explore the spatiotemporal consistency of data and depict abrupt data, respectively. The model can detect anomalies and predict short-term passenger flow conveniently and flexibly. For anomaly detection, we further introduce a strong temporal Toeplitz regularization to characterize the temporal periodic change of data, so as to more accurately detect anomalies. We conduct experiments with smart card transaction data from the Beijing metro system to assess the performance of the model in two use cases. In terms of anomaly detection, the experimental results demonstrate that our method can detect anomalies efficiently, especially for time sequence anomalies. As for short-term prediction, our model is superior to other methods in most cases.

Weakly supervised crowd counting involves the regression of the number of individuals present in an image, using only the total number as the label. However, this task is plagued by two primary challenges: the large variation of head size and uneven distribution of crowd density. To address these issues, we propose a novel Hypergraph Association Crowd Counting (HACC) framework. Our approach consists of a new multi-scale dilated pyramid module that can efficiently handle the large variation of head size. Further, we propose a novel hypergraph association module to solve the problem of uneven distribution of crowd density by encoding higher-order associations among features, which opens a new direction to solve this problem. Experimental results on multiple datasets demonstrate that our HACC model achieves new state-of-the-art results.

Clustering is a fundamental and hot issue in the unsupervised learning area. With the rapid development of deep learning and graph neural networks (GNNs) techniques, researchers have proposed a series of effective clustering methods. However, most existing approaches adopt a conventional graph to aggregate the neighborhood information, where only the pairwise relations are considered. Moreover, the redundancy/noise in the raw data samples may result in less accurate sample relations and inferior clustering results. In this paper, we proposed a new GNNs based clustering method, which adopts the hypergraph learning approach to explore the high-order relationship for accurate relation learning. Specifically, we first construct two hypergraph representations based on the topology feature and attribute feature from data samples. Then, a self-supervised structure is integrated to learn a cross-correlation matrix from the original hypergraph to act as a higher-order neighborhood with reduced redundancy and noise. Finally the embedding representation of the clustering space is learned in the graph convolution. The proposed method has been evaluated on six public datasets for clustering tasks. Experimental results show that our proposed method outperforms the state-of-the-art ones.

Traffic situation generation is of importance in the intelligent transportation field, evaluating and simulating the macroscopic traffic conditions. The government often uses the historical traffic data on the same weekday to analyze the future traffic situations, which works unfavorably due to some traffic-related information deficiency, such as weather, location, traffic accidents, social activities and so on. Therefore, how to accurately generate the traffic situation is a challenging problem. Fortunately, massive traffic-related information spread in social media often indicates the traffic situation variation trend, which provides the sufficient information for the traffic situation generation. In this paper, we propose a novel Text-to-Traffic generative adversarial network framework ($\text{T}^{2}$GAN), which fuses the traffic data and the semantic information collected from social media to generate the traffic situation. To reduce the huge gap between the above two modalities and improve the authenticity of the generated traffic situation, we raise a global-local loss. Additionally, we build a heterogeneous dataset containing the traffic-related text data collected from social media and the corresponding traffic passenger flow data. Experimental results show that the proposed methods are obviously better than many outstanding traffic situation generation methods based on neural networks.

Traffic prediction methods on a single-source data have achieved excellent results in recent years, especially the Graph Convolutional Networks (GCN) based models with spatio-temporal dependency. In reality, various modes of urban transportation operate simultaneously. They influence and complement each other in common space-time occasions, constituting the transportation system dynamically. Thus, traffic data from multiple sources is ostensibly heterogeneous, but internally correlated. The typical single data driven models are, however, not universally applicable for heterogeneous traffic data. To address this issue, we propose a Multi-task Hypergraph Convolutional Neural Network (MT-HGCN) for the multi-source traffic prediction problem. The framework consists of a main task and a related task. Both tasks are based on Hypergraph Convolutional Neural Networks (HGCN) and are devoted to two prediction problems. Furthermore, the tasks are bridged by a feature compress unit, which models the correlation and shares the latent feature to improve the performance of the main task. The node-level forecasting has been evaluated on historical datasets of Beijing to verify the effectiveness of the proposed method. Compared with the state-of-the-arts, the superior performance of the proposed method can be obtained.

Analyzing the traffic state of large citywide networks is an inherently difficult task. Various data issues, traffic signals, stops signs and other flow inhibitors of the network-level traffic state make the analysis more difficult than that under the small-scale local traffic state. To address this challenge, we propose a method based on spatio-temporal non-negative matrix factorization (ST-NMF), which is used for road network traffic pattern analysis. The method can be further extended to traffic data reconstruction and traffic prediction. In order to analyze traffic patterns, the proposed spatio-temporal non-negative matrix factorization model represents the network traffic as a linear combination of several basic patterns, which is also interpreted as the dynamics of spatial traffic characteristics over time in low-dimensional space. By the visual display of the spatial and temporal patterns and the assistance of clustering methods, the traffic pattern features are extracted. In the extended applications, data reconstruction relies on the sampling representation of missing data by ST-NMF, and data prediction is based on the prediction of the temporal patterns by ST-NMF. Through our method, we can not only obtain a high-quality data foundation, but also explore typical spatio-temporal patterns and general predictions of the future traffic state. The analysis results have important guiding significance on the management of intelligent transportation systems. Experiments on real-world traffic data are provided to verify the validity of our proposed approach.

Abstract Nowadays, face detection and head pose estimation have a lot of application such as face recognition, aiding in gaze estimation and modeling attention. For these two tasks, it is usually to design two different models. However, the head pose estimation model often depends on the region of interest (ROI) detected in advance, which means that a serial face detector is needed. Even the lightest face detector will slow down the whole forward inference time and cannot achieve real-time performance when detecting the head pose of multiple people. We can see that both face detection and head pose estimation need face features, so a shared face feature map can be used between them. In this paper, a multi-task learning model is proposed that can solve both problems simultaneously. We directly detect the location of the center point of the bounding box of face; at this location, we calculate the size of the bounding box of face and the head attitude. We evaluate our model’s performance on the AFLW. The proposed model has great competitiveness with the multi-stage face attribute analysis model, and our model can achieve real-time performance.

Metro passenger flow prediction is a strategically necessary demand in an intelligent transportation system to alleviate traffic pressure, coordinate operation schedules, and plan future constructions. Graph-based neural networks have been widely used in traffic flow prediction problems. Graph Convolutional Neural Networks (GCN) captures spatial features according to established connections but ignores the high-order relationships between stations and the travel patterns of passengers. In this paper, we utilize a novel representation to tackle this issue - hypergraph. A dynamic spatio-temporal hypergraph neural network to forecast passenger flow is proposed. In the prediction framework, the primary hypergraph is constructed from metro system topology and then extended with advanced hyperedges discovered from pedestrian travel patterns of multiple time spans. Furthermore, hypergraph convolution and spatio-temporal blocks are proposed to extract spatial and temporal features to achieve node-level prediction. Experiments on historical datasets of Beijing and Hangzhou validate the effectiveness of the proposed method, and superior performance of prediction accuracy is achieved compared with the state-of-the-arts.

Traffic flow data has three main characteristics: large amount of noise and incompleteness, temporal and spatial correlation, and dynamic sequential property. Problems of noise, loss and incompleteness could decrease the prediction performance and make it difficult for transportation system management. Inspired by recent work on low rank representation (LRR) and dynamic mode decomposition (DMD), we propose a Low Rank Dynamic Mode Decomposition (LRDMD) model which solves the aforementioned problems simultaneously. LRDMD predicts traffic flow by using a state transition matrix which characterizes the relationship between temporally neighboring fragments of traffic flow with low rank regularization. We conduct experiments of traffic flow prediction of different time intervals using loop coil detector data of Qingdao, and the results show that LRDMD outperforms state-of-the-art methods.

Public transportation agencies call for an intuitive, interactive, and reusable visualization tool to detect patterns of crime (i.e. pickpockets and gangs) or missing commuters on public transportation systems. Few existing visualization techniques have visually explored mobility correlations of targets and their companions, who are characterized in diverse mobility types, by using discrete travel hints extracted from a massive amount of data. To fill this gap, a visual analytical system is provided to conduct a group-based and individual-based exploration of mobility correlations of passengers of interest, based on an auto integration of multiple queries. How passengers differ from or correlate with each other are further examined based on their spatiotemporal distributions in trajectories and ODs. Real-world case studies, as well as user feedback made by 30 participants, demonstrate the effectiveness of the system in detecting specific targets and their companions featured in diverse mobility types, or in characterizing their spatiotemporal aggregation patterns for a further tracking on public transportation systems.

Travel smart cards record passengers' travel histories, which makes it possible to study personal traveling behaviors and passengers' mobility patterns. The existing researches on smart card data pay less attention to those who beg, steal or busk during traveling, and they are called irregular passengers in this paper. Moreover, the group consisted of irregular passengers is called irregular travel group. In this paper, we propose an approach to recognize irregular travel groups based on cascade clustering. Firstly, passengers' travel sequences of active state in hours are extracted as representations of travel patterns according to the records of smart cards in a continuous time period. The sequences are clustered with the K-means algorithm to detect irregular passengers. Then, travel similarities between the derived irregular passengers are measured and irregular travel groups are recognized based on DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm. The proposed method is applied to real Beijing subway smart card data, and the results are validated with the data derived from SINA Micro-blogs. Experimental results show that the detected irregular passengers' movement areas are consistent with the ground truth.

Data missing remains a difficult and important problem in the transportation information system, which seriously restricts the application of the intelligent transportation system (ITS), dominatingly on traffic monitoring, e.g., traffic data collection, traffic state estimation, and traffic control. Numerous traffic data imputation methods had been proposed in the last decade. However, lacking of sufficient temporal variation characteristic analysis as well as spatial correlation measurements leads to limited completion precision, and poses a major challenge for an ITS. Leveraging the low-rank nature and the spatial-temporal correlation of traffic network data, this paper proposes a novel approach to reconstruct the missing traffic data based on low-rank matrix factorization, which elaborates the potential implications of the traffic matrix by decomposed factor matrices. To further exploit the temporal evolvement characteristics and the spatial similarity of road links, we design a time-series constraint and an adaptive Laplacian regularization spatial constraint to explore the local relationship with road links. The experimental results on six real-world traffic data sets show that our approach outperforms the other methods and can successfully reconstruct the road traffic data precisely for various structural loss modes.

Intelligent Transportation Systems (ITS) plays a significant role in the traffic management, i.e. traffic jam prediction, route guidance. Due to the hardware failure or data transformation failure, some traffic observation data may be occasionally missed, which seriously affect intelligent transportation information service. So, the completion of traffic observation data has now become an issue that requires to be concerned and solved. By analyzing the traffic history data, we find that traffic data tend to have strong spatio-temporal correlation. Considering this feature, we propose a new low-rank representation based traffic data completion method. To further enhance the local correlation, we introduce an ordered regulation into our proposed method. We also give an efficient solution to our proposed methods. In order to verify the performance of our methods, some traffic data completion experiments are conducted on the Beijing metropolitan road speed dataset and the capital airport highway microwave dataset. Experimental results show that the proposed methods are superior to other state-of-the-art traffic data completion methods.