Accuracy and timeliness are indeed often conflicting goals in prediction tasks.Premature predictions may yield a higher rate of false alarms, whereas delaying predictions to gather more information can render them too late to be useful.In applications such as wildfires, crimes, and traffic jams, timely forecasting are vital for safeguarding human life and property.Consequently, finding a balance between accuracy and timeliness is crucial.In this paper, we propose an early spatio-temporal forecasting model based on Multi-Objective reinforcement learning that can either implement an optimal policy given a preference or infer the preference based on a small number of samples.The model addresses two primary challenges: 1) enhancing the accuracy of early forecasting and 2) providing the optimal policy for determining the most suitable prediction time for each area.Our method demonstrates superior performance on three large-scale real-world datasets, surpassing existing methods in early spatio-temporal forecasting tasks.

Given the increasing importance of predicting spatiotemporal events such as wildfire, crime, and traffic congestion, existing methods are faced with the challenge of balancing timeliness and accuracy. Late predictions may result in tremendous economic costs and human life loss, while inaccurate predictions are likely to cause unnecessary public resources and social anxiety. Therefore, balancing accuracy and timeliness is essential in general spatiotemporal event prediction problems. In this paper, we propose an Early Spatiotemporal Graph Convolutional Network (ESTGCN)1to adaptively determine the optimal prediction time, which makes a tradeoff between prediction accuracy and timeliness and addresses two major questions: 1) How can we determine optimal prediction time points for different areas, taking into account their unique characteristics and conditions? 2) How can we minimize the propagation of prediction errors throughout the forecast timeline? Extensive experiments on two large-scale real-world datasets demonstrate that our proposed approaches can give an optimal prediction time in advance for each area and outperform all baselines in early spatiotemporal prediction tasks.

The autonomous vehicles (AVs), as intelligent mobile robots, can undertake tasks to facilitate various computation-intensive services in intelligent transportation system (ITS). Due to hardware device failures or environmental identification errors, the AVs controlled by intelligent algorithms may cause accidents during driving. However, the existing studies in the post-accident stage lack the analysis of the impact degree of the accidents and the computing tasks undertaken by the AVs to determine the optimal maintenance strategy. In this article, we consider the accidents in a continuous period of time and design a digital twin (DT)-enabled post-accident maintenance scheme. Specifically, by considering the computing tasks undertaken by the AVs and the impact degree of the accidents, we first design a DT-enabled post-accident maintenance architecture. With the designed architecture, an optimal maintenance method under an incomplete information scenario is then proposed to help each accident AV decide its optimal maintenance strategy. Besides, based on the maintenance strategies of the AVs and the capacities of the maintenance service providers (MSPs), the two-way selection problem between the AVs and the MSPs in the continuous period of time is modeled as a dynamic matching game to obtain the optimal AV-MSP pairs. Simulation results demonstrate that the proposed scheme outperforms the benchmark schemes in terms of the maintenance rate of the accident AVs, the average utility of the MSPs, and the average social welfare.

With the growing number of cars in cities, smart parking is gradually becoming a strategic issue in building a smart city. As the precondition in smart parking, accurate parking prediction can reduce the time drivers spend searching for parking spaces and relieve traffic congestion. Meanwhile, VANET and the Internet-of-things (IoT) are the key elements of the current intelligent transportation system. With the IoT devices based on VANET becoming more extensively employed, a large amount of parking data is generated every day, and various methods are proposed for parking prediction, therefore, it is time to systematically summarize the parking prediction issues and the state-of-the-art prediction methods. In this survey, we first provide a comprehensive review of the existing methods used for parking prediction ranging from conventional statistical methods to the latest graph neural network methods. Then, we classify a variety of parking problems such as parking availability prediction, parking behavior prediction, and parking demand prediction. We also compile all the evaluation metrics, open data, and open-source code of the surveyed literature. Finally, we present the challenges and future directions of the parking prediction technique. As far as we know, this is the first survey exploring parking prediction methods, which will be of interest to both researchers and practitioners engaging in intelligent transportation systems (ITS) and smart cities.

Nowadays, people are accustomed to posting images and associated text for expressing their emotions on social networks. Accordingly, multimodal sentiment analysis has drawn increasingly more attention. Most of the existing image-text multimodal sentiment analysis methods simply predict the sentiment polarity. However, the same sentiment polarity may correspond to quite different emotions, such as happiness vs. excitement and disgust vs. sadness. Therefore, sentiment polarity is ambiguous and may not convey the accurate emotions that people want to express. Psychological research has shown that objects and words are emotional stimuli and that semantic concepts can affect the role of stimuli. Inspired by this observation, this paper presents a new MUlti-Level SEmantic Reasoning network (MULSER) for fine-grained image-text multimodal emotion classification, which not only investigates the semantic relationship among objects and words respectively, but also explores the semantic relationship between regional objects and global concepts. For image modality, we first build graphs to extract objects and global representation, and employ a graph attention module to perform bilevel semantic reasoning. Then, a joint visual graph is built to learn the regional-global semantic relations. For text modality, we build a word graph and further apply graph attention to reinforce the interdependencies among words in a sentence. Finally, a cross-modal attention fusion module is proposed to fuse semantic-enhanced visual and textual features, based on which informative multimodal representations are obtained for fine-grained emotion classification. The experimental results on public datasets demonstrate the superiority of the proposed model over the state-of-the-art methods.

Terahertz (THz) wireless communications are commonly regarded as one of the key technologies of 6G communication. Combined with THz, the newly exploited physical layer transmission dimension orbital angular momentum (OAM) that multiplexes a set of orthogonal modes on the same frequency channel, can unleash its potential in achieving high spectrum efficiency. Currently, most of the research on radio OAM communications focus on the point-to-point scenario in microwave and millimeter-wave (mmWave) bands. In this paper, we propose a uniform circular array (UCA)-based THz multi-user OAM (MU-OAM) communication system including downlink and uplink transmission schemes that simplifies the signal detection to the despiralization and amplitude detection (AD). A salient feature of the proposed MU-OAM communication scheme is lower computational complexity than the traditional MU-MIMO scheme without sacrifacing bit error rate (BER) and achievable sum rate performances, which is validated by mathematical analysis and numerical simulations.

With the advancement of smart cities and the development of heterogeneous vehicular networks (HetVNets), vehicles can collect data and generate valuable information to obtain profits, thus forming a new vehicular self-media paradigm in HetVNets. However, in the HetVNets with potential security risks, the vehicular self-media market lacks the consideration of the values of the data owned by the media data producers (MDPs) and the capabilities of the media data sellers (MDSs) to improve their utilities. To this end, we propose a value-based secure self-media data trading scheme in the HetVNets. Specifically, we first design a vehicular self-media trading mechanism based on smart contracts to provide participants with a safe and reliable transaction environment. Then, we model the interactions between the MDPs and the MDSs as a Stackelberg game by considering the values of various media data and the sales capabilities of different MDPs. After that, we design an iterative method to obtain the optimal game strategies for the MDPs and the MDSs to maximize their utilities. Compared with the traditional schemes, the simulation results show that our scheme can obtain the optimal strategies for the MDPs and the MDSs and bring them the highest utilities.

Many real-world ubiquitous applications, such as parking recommendations and air pollution monitoring, benefit significantly from accurate long-term spatio-temporal forecasting (LSTF). LSTF makes use of long-term dependency structure between the spatial and temporal domains, as well as the contextual information. Recent studies have revealed the potential of multi-graph neural networks (MGNNs) to improve prediction performance. However, existing MGNN methods do not work well when applied to LSTF due to several issues: the low level of generality, insufficient use of contextual information, and the imbalanced graph fusion approach. To address these issues, we construct new graph models to represent the contextual information of each node and exploit the long-term spatio-temporal data dependency structure. To aggregate the information across multiple graphs, we propose a new dynamic multi-graph fusion module to characterize the correlations of nodes within a graph and the nodes across graphs via the spatial attention and graph attention mechanisms. Furthermore, we introduce a trainable weight tensor to indicate the importance of each node in different graphs. Extensive experiments on two large-scale datasets demonstrate that our proposed approaches significantly improve the performance of existing graph neural network models in LSTF prediction tasks.

Traveling Salesman Problem (TSP) is a classic NP-hard problem in Combinatorial Optimization (CO), which has been widely studied. Traveling Officer Problem (TOP) derived from illegal parking in urban areas is a variant of TSP. Its solution aims to capture as many illegally parked vehicles as possible in a limited time. However, traditional methods of solving TSP cannot be applied to TOP because the illegally parked vehicle may leave before the officer arrives. Existing methods to solve TOP include heuristic search and deep learning algorithms such as ant colony optimization and feed-forward neural network. However, the performance based on capture rate and traveling distance of these algorithms is still comparably low. Hence, in this paper, we propose the heterogeneous pointer network to address this problem by modifying the encoder of the traditional pointer network to suit the spatial-temporal features of TOP. We conduct experiments using real-world datasets from Melbourne open data platform to show that our method achieves significant improvement and outperforms the existing algorithms based on capture rate and traveling distance.

Accurate long-term predictions help governments make decisions and residents travel, which is essential for the development of smart cities. Fortunately, due to the deployment of low-cost sensors, a large amount of time-series data such as parking availability data and air quality data has been stored, which makes it possible for long-term predictions. Many state-of-the-art studies based on multiple graphs have shown excellent performance in long-term prediction tasks. However, few previous studies employ multiple attention mechanisms to their models based on multi-graphs and thus fail to comprehensively capture the dynamic spatio-temporal correlations as well as the inner relationships among graphs. To this end, we propose a spatio-temporal multi-attention multi-graph convolutional network (STM2CN) framework for long-term prediction. We applied four different graphs to mine the potential contextual relationships and employed three attention mechanisms to capture the multiple graph and spatio-temporal correlations. Experiments on two large-scale real-world datasets demonstrate that the proposed STM2CN framework outperformed the state-of-the-art baselines.

The vehicular crowdsensing, which benefits from edge computing devices (ECDs) distributedly selecting autonomous vehicles (AVs) to complete the sensing tasks and collecting the sensing results, represents a practical and promising solution to facilitate the autonomous vehicular networks (AVNs). With frequent data transaction and rewards distribution in the crowdsensing process, how to design an integrated scheme which guarantees the privacy of AVs and enables the ECDs to earn rewards securely while minimizing the task execution cost (TEC) therefore becomes a challenge. To this end, in this article, we develop a blockchain-based collaborative crowdsensing (BCC) scheme to support secure and efficient vehicular crowdsensing in AVNs. In the BCC, by considering the potential attacks in the crowdsensing process, we first develop a secure crowdsensing environment by designing a blockchain-based transaction architecture to deal with privacy and security issues. With the designed architecture, we then propose a coalition game with a transferable reward to motivate AVs to cooperatively execute the crowdsensing tasks by jointly considering the requirements of the tasks and the available sensing resources of AVs. After that, based on the merge and split rules, a coalition formation algorithm is designed to help each ECD select a group of AVs to form the optimal crowdsensing coalition (OCC) with the target of minimizing the TEC. Finally, we evaluate the TEC of the task and the rewards of the ECDs by comparing the proposed scheme with other schemes. The results show that our scheme can lead to a lower TEC for completing crowdsensing tasks and bring higher rewards to ECDs than the conventional schemes.

With the popularity of image sensors in various mobile devices, image blurring caused by hand shaking or out of focus becomes ubiquitous, which deteriorates image quality and poses challenges for vision tasks, including object detection, image classification and image segmentation. Designing an efficient blur detection algorithm which can automatically detect and locate blurred regions becomes necessary. In this letter, we design an end-to-end convolution neural network called heterogeneous attention nested U-shaped network (HANUN) for blur detection. We introduce pyramid pooling into encoders to enhance the feature extraction at different scales and reduce the gradual information loss. Inspired by the nested network design, small U-shaped networks are embedded into our decoders to increase the network depth and promote feature fusion with different receptive field scales. In addition, we incorporate a channel attention mechanism in the proposed network to highlight the informative features for detecting the blurry regions. Experimental results show that HANUN outperforms other state-of-the-art algorithms for blur detection tasks on public datasets and real-world images.

Efficient path planning is the key enabling technology for the realization of intelligent transportation systems (ITS). However, due to poor real-time performance and lack of effective incentive methods, it is difficult for traditional path planning schemes to significantly improve the efficiency of traffic management. In addition, existing solutions that use driving distance and driving time as indicators cannot meet the personalized requirements of vehicle users. To this end, by considering the personalized requirements of vehicle users, we propose a digital-twin (DT) enabled path planning scheme to facilitate traffic management. To be specific, based on the collection of traffic data, we first establish a DT architecture for traffic scheduling to reduce the delay of path planning. Then, according to the traffic density of different road sections, we regard road sections as resources and set different rewards for different road sections to encourage vehicles to obey the scheduling instructions. In addition, by jointly considering the driving time and rewards, we further design personalized utility models to map the requirements of different vehicle users. After that, based on the personalized requirement of the vehicle user, we use a$Q$-learning algorithm to obtain the optimal path with the target of maximizing the user's utility. The simulation results show that the proposed scheme can bring higher utility to the vehicle users than the conventional schemes.

Incident scene text detection, as the most crucial step of an incident scene text recognition system, has received increasing research attention. In this paper, a two-layer mobile federated learning model (TMFL) is proposed to protect data privacy and improve training efficiency. Particularly, a fast scene text detector is proposed to detect the multi-directional and multi-scale text by using an asymmetric convolution based feature pyramid network (AC-FPN). Compared with the traditional feature pyramid, asymmetric convolutions can effectively extract rotation-invariant features to improve the model's robustness to directed text. Moreover, in order to achieve a balance between the detection accuracy and efficiency, we modify the lightweight backbone of mobilenetv3, and integrate it with the asymmetric convolution based feature pyramid. In addition, we evaluate the performance of our detector on three benchmark datasets, where the results show that both the accuracy and the speed can be improved. Our detector can achieve an F-measure of 87.8 on the ICDAR2013, 80.5 on the MSRA-TD500 and 84.1 on the ICDAR2015 dataset, running at 32.5 FPS.

In smart cities, on-street parking space prediction is the key yet difficult point in smart parking system. However, conventional prediction methods generally neglect spatial and temporal dependencies and cannot predict long-term parking events accurately. To this end, we propose a parking space prediction scheme based on the spatial-temporal graph convolution networks (STGCN). We first consider the instantaneous status of the parking to calculate the on-street parking occupancy rate (POR). Then, based on the POR, we exploit a time convolution module and a graph convolution module to extract spatial and temporal dependencies of the parking spaces, respectively. Next, we design the parameters of the STGCN to predict the POR of all the parking spaces based on the spatial and temporal dependencies. Finally, based on the real-world data sets, we compare the proposed scheme with the benchmark models. The experimental results show that the proposed scheme has the best performance in predicting the POR.

Recently, the abnormal intrusion detection has become an urgent problem in high-speed railway system. One way to solve this problem is the optical fiber distributed acoustic sensing (DAS) system that can monitor the intrusion events and provide early warning. However, most long-distance DAS systems are unable to distinguish signal types to improve the detection performance. Moreover, the traditional fiber optic sensing system is susceptible to interference from environmental factors, resulting in false detections and alarms. To this end, with the adoption of DAS system, we propose a railway intrusion detection system based on Faster R-CNN. In our system, we first design the DAS system to collect the optical fiber acoustic signals. Then, the collected signals are normalized in temporal and spatial dimensions and converted into Spatio-temporal images. After that, we design the Faster R-CNN algorithm to extract the Spatio-temporal features to detect and classify five types of abnormal intrusion events. The experimental results demonstrate that the average detection precision of our system for all abnormal intrusion events is above 89%. In addition, compared with the conventional methods, our system achieves the highest detection precision. Meanwhile, the system can distinguish the non-threatening background noise, which is of great help to reduce the system false positive rate.