It is significant to employ multiple autonomous underwater vehicles (AUVs) to execute the underwater target tracking task collaboratively. However, it's pretty challenging to meet various prerequisites utilizing traditional control methods. Therefore, we propose an effective two-stage learning from demonstrations training framework, FISHER, to highlight the adaptability of reinforcement learning (RL) methods in the multi-AUV underwater target tracking task, while addressing its limitations such as extensive requirements for environmental interactions and the challenges in designing reward functions. The first stage utilizes imitation learning (IL) to realize policy improvement and generate offline datasets. To be specific, we introduce multi-agent discriminator-actor-critic based on improvements of the generative adversarial IL algorithm and multi-agent IL optimization objective derived from the Nash equilibrium condition. Then in the second stage, we develop multi-agent independent generalized decision transformer, which analyzes the latent representation to match the future states of high-quality samples rather than reward function, attaining further enhanced policies capable of handling various scenarios. Besides, we propose a simulation to simulation demonstration generation procedure to facilitate the generation of expert demonstrations in underwater environments, which capitalizes on traditional control methods and can easily accomplish the domain transfer to obtain demonstrations. Extensive simulation experiments from multiple scenarios showcase that FISHER possesses strong stability, multi-task performance and capability of generalization.

Open-world contrastive learning aims to learn a compact representational space for known and novel classes. The task is currently dominated by Euclidean and spherical embeddings so that the decisions about class belongings are made using Euclidean distances, or spherical geodesic distances. However, we believe there exists an inherent hierarchical relationship among natural data, which previous methods have failed to capture. We propose Hyp-OC, a new hyperbolic-based model that use the distance in Poincaré ball to distinguish novel classes from unlabeled data and to cluster for open-world contrastive learning. The key to Hyp-OC lies in a vision transformer, which maps output embeddings to hyperbolic space. Through extensive experimentation, we validated the effectiveness of hyperbolic space embeddings in enhancing the performance of open-world contrastive learning. On the ImageNet dataset, Hyp-OC significantly outperforms the current best method by 5.2% and 3.6% on novel and overall classification accuracy, respectively.

In this paper, the scheduling of yard cranes (YCs) and external trucks (ETs) working in a U-shaped automated container terminal yard is proposed as a new problem. This problem arises due to the characteristic of the U-shaped layout, where the ETs enter the yard through the U-shaped lanes and interact with the YCs. To formulate this problem, a three-objective optimization model is established to simultaneously schedule YCs and ETs, considering their efficiencies. Its solution is based on the nondominated sorting genetic algorithm III (NSGA-III), which is improved during initialization, crossover and mutation to make it applicable to the problem. To calculate the objective values in each iteration of the NSGA-III, a method to make the equipment interactions equal at all times is proposed. As there are special requirements for the priority of the objectives, a new method of selecting the final solution is presented, which leads to a more suitable solution. The case study involving sensitivity analysis shows that appropriate equipment quantity arrangement and task assignments have a positive impact on improving the efficiency of yard operations. Experimental results demonstrate that the proposed model and algorithms perform better. In summary, the model and algorithms proposed in this paper can effectively solve the proposed problem, which can offer a variety of options for decision-makers in an actual terminal operation environment.Note to Practitioners—The YC scheduling of a U-shaped layout presents new characteristics, and the influence of ETs cannot be ignored, posing a novel problem. Furthermore, during the operation of container terminals, extended waiting time for ETs is often caused by considerations related to ship schedules, especially in the mixed stacking mode. This can frustrate cargo owners and waste valuable ET resources. Hence, this paper designs a joint scheduling strategy for YCs and ETs in U-shaped yard layouts. The proposed strategy combines optimizing YC operation plans with maximizing ET efficiency, while ensuring efficient yard operations.

Path planning is essential for autonomous underwater vehicles (AUVs) to perform tasks. Many existing single-objective path planning methods rely on prior knowledge of the underwater environment. However, extracting prior knowledge is challenging because the underwater environment is influenced by ocean current, complex terrain, and other factors. This paper proposes a two-stage reinforcement learning (RL) algorithm based on trajectory exploration and sequence modeling, called the Soft Actor Critic and Online Decision Transformer (SAC-ODT), which operates without prior knowledge. This algorithm utilizes the SAC strategy exploration capability to generate training data for ODT and uses the ODT strategy optimization capability to plan a smooth, energy-efficient, and safe path. In the first stage, a Multi-Reward Strategy Embedding (MRSE) method is designed to facilitate trajectory exploration with multiple strategies, enabling further training of an ODT with comprehensive decision-making capability. During the second stage, a Condition Prioritized Buffer Update and Sampling Strategy (CP-BUSS) is proposed to enhance the sensitivity of ODT to the reward function, enabling adaptation to various tasks while accelerating high-quality path learning. Experimental results demonstrate that compared to existing RL-based benchmarks, SAC-ODT reduces the path time and energy consumption by 2.7% and 2.5%, respectively, while improving path smoothness by 91.96%.

Tropical cyclones (TCs) are destructive weather systems, and it is crucial to accurately locating their positions, estimating their intensity, and predicting their trajectories to reduce their potential damage. Due to their excellent ability to extract deep spatial features from satellite images, machine learningbased methods can fully mine the relationships between image features and TC positions, which have become the mainstream approach in the field of TC center location. However, the existing methods mainly rely on the eye features identified from infrared channels of satellite images, which can be an invalid approach during the TC generation or extinction phase as the eye structures in these two phases are blurred. To address this problem, this study examines the relationship between the TC center and the low geopotential height center and proposes a non-eyed TC center location method by fusing the features from satellite images and geopotential height data. In the proposed method, a water vapor attention module is used to focus on regions with water vapor distribution patterns closely related to the TC center. Moreover, a multi-scale attention module is designed to mine and fuse knowledge from different spatial scales, aiming to extract the TC structural features fully. The performance of the proposed model is verified through experiments using Gridsat, IBTrACS and ECMWF HRES datasets. The results show that compared with the existing methods, the mean distance error of the TC’s center location is reduced by at least 11.3% compared with state-of-the-art prediction models.

Efficient oceanic mesoscale eddy (ME) trajectory prediction can help us better understand energy transport in the ocean while also ensuring maritime safety. However, the complex, nonlinear motion of MEs presents significant challenges. Existing data-driven methods are primarily focused on single-trajectory prediction, ignoring interactions between multiple MEs, which are the most important factors influencing trajectory evolution. This study addresses two major research limitations: a lack of methodologies that explicitly include multi-ME interactions, as well as a lack of quantitative assessments of their impact on prediction accuracy. To fill these gaps, this study proposes a spatial projection inference network for ME trajectory prediction (Eddy-SPIN). Eddy-SPIN uses a specially designed projection method based on ME intrinsic features to transform multiple 1-dimensional time-series trajectory data into 2-dimensional Gaussian distribution maps. Then, a spatiotemporal prediction network is designed to generate future Gaussian maps, which are then inverse-projected onto trajectories. Extensive evaluations using data from the South China Sea show that Eddy-SPIN reduces the 7-day cumulative mean geodesic distance error by at least 31.9% and 53.3% when compared to the state-of-the-art single-trajectory method and numerical model, respectively. Further, additional experiments conducted across four dynamically distinct oceanic regions consistently confirm the generalizability and robustness of Eddy-SPIN. The proposed Eddy-SPIN verifies the importance of interactions in the ME trajectory prediction problem and presents a novel predictive model that differs from previous works.

Deep reinforcement learning (DRL) algorithms have been proven to be effective in traffic signal timing scheme optimization and helpful for urban traffic congestion alleviation. Recently, the centralized control strategy has attracted great attention for coordinated signal control of multiple intersections overpassing the decentralized methods. However, this strategy has the problems of state space dimension disaster and credit assignment. Namely, the dimension of the global action space increases exponentially with an increase in the number of agents, and individual traffic signals cannot realize their contributions to the optimization of regional traffic. To solve the two aforementioned problems, this study proposes a centralized DRL algorithm for multiple signals using the sequential decision named the Sequential Light (SeqLight). First, a centralized DRL framework with a sequential-decision logic is constructed to decouple the joint actions of multiple agents, reducing the complexity of the action space to a polynomial level. Second, the multi-agent advantage value is decomposed into sequential advantage evaluations of every local agent, thus alleviating the credit assignment problem and improving the monotonic performance. Third, an adaptive optimization model for decision order is developed for traffic signal control, which coordinates the signal actions of local agents to learn the globally optimal policy. Finally, the proposed algorithm is verified by simulations using an actual road network. The simulation results show that compared to the other centralized RL method, the proposed method can reduce the average vehicle queue length by 12.64% and 14.20% under medium and high traffic demand conditions, respectively.

Panoramic map construction of polar sea ice can provide significant assistance for intelligent navigation and routing planning in polar regions. Traditional methods for panoramic observation and map generation suffer from issues such as limited parallax tolerance, poor stitching robustness, and low mapping accuracy. In this paper, these problems are addressed by a proposed online panoramic method based on multi-perspective image projection and camera pose rectification. The proposed method incorporates a modified inverse projection module that dynamically adapts to the ship’s attitude, thereby stably restoring the sea-ice images into bird’s-eye view (BEV) in real-time. In order to resolve the challenging sea-ice feature alignment under significant parallax, a planar feature registration method is proposed which robustly aligns the features between the projected images. Moreover, a camera pose rectification module is specifically designed for planar projection tasks, which virtually adjusts the camera’s extrinsic parameters for obtaining more precise and high-quality panoramic sea-ice maps. Finally, online construction of global sea-ice field maps is achieved by fusing the local maps during navigation. Extensive qualitative and quantitative experiments demonstrate that the proposed method outperforms other panoramic methods in terms of map accuracy and stitching quality. Additionally, the proposed method is more suitable for downstream tasks, including polar simultaneous localization and mapping (SLAM) and path planning.

Accurate and robust long-term global localization is a critical challenge for autonomous vehicles operating in complex urban transportation systems, where GPS signals are often unreliable and visual/inertial odometry suffers from inevitable error accumulation. Visual Place Recognition (VPR) offers a crucial solution by detecting loop closures to mitigate trajectory drift, but its performance severely degrades under complex urban traffic scenarios, such as drastic changes in viewpoint, illumination, and weather. To address these limitations, we propose the 3D Geometric feature Enhanced VPR (GE-VPR), a novel framework that improves the robustness of vehicular global localization. As a purely vision-based system, GE-VPR reconstruct 3D point clouds through dense simultaneous localization and mapping. A 3D geometric feature extraction network is designed to obtain the stable structural features of the point clouds, and a 2D-3D hybrid network is then developed to further augment these 3D features with 2D semantics. Additionally, a descriptor refinement strategy is proposed to fine-tune the raw 2D descriptors by aggregating the most relevant hybrid structural features, thus effectively fusing rich 2D appearance, color, and semantic information with stable 3D geometry. Extensive experiments on challenging urban autonomous driving datasets demonstrate that GE-VPR significantly improves vehicle localization accuracy and robustness. The overall recognition recall is increased by more than 5%, and the positioning accuracy is significantly improved in practical scenarios, demonstrating its potential as an effective solution to improve the safety and reliability of vehicular localization in autonomous navigation systems.

A tropical cyclone (TC) is a type of catastrophic weather encountered in the tropical or subtropical ocean, and it is of great significance to accurately estimate its intensity. Many estimation methods based on statistics have been proposed, but these methods have obvious problems, such as poor robustness and low accuracy. Therefore, in this study, a TC intensity estimation method is proposed based on satellite image data using the Xception network as a backbone. The main idea of the proposed method is to estimate the TC maximum wind speed by image feature extraction. First, a Laplacian pyramid image fusion method for the infrared (IR) and water vapor (WV) channels of satellite images is adopted to enhance the total amount of information in the basic input data of the model. Second, an optimization strategy for the depth and width of the Xception network model is proposed with the objective of reducing parameter redundancy and improving the estimation accuracy. Third, a multiscale feature extraction module and a multilayer feature fusion module are designed to realize the fusion of different features. In addition, a dual attention module is introduced to allow the model to focus on the key regions of cyclone images. Finally, the proposed network is evaluated on the HURSAT, FY-2, and Gridsat datasets. The results show that, on average, the maximum cyclone wind speed estimation errors, the mean absolute error (MAE), and root mean square error (RMSE), are 8.0% and 11.4% lower than the state-of-the-art models on the three datasets.

Ice floes and icebergs pose serious risks to ship navigation in the Arctic. It is important to identify the sea ice distribution around ships, and subsequently generate a sea ice map and optimize an efficient and safe navigation route. Existing sea ice classification methods that use the shipboard camera images have low accuracy for the distal ice field. This study establishes a multi-task sea ice classification method with dynamic segmentation, which can extract unique features of different regions and intra-region feature correlation to improve the overall classification accuracy. First, a segmentation network based on U-Net is established to divide the image into two parts: an upper part containing the sky and the distal ice field, and a lower part containing the proximal ice field. Second, a ResU-Net is utilized in each part to learn local features, and a multi-task network for sea ice classification is built to improve the feature extraction quality, especially in the distal region. Meanwhile, the spatial attention and channel attention modules are introduced to eliminate the least significant features and enhance the learning efficiency in the feature aggregation regions. Last, the case analysis and performance comparison are carried out using actual data collected by the Chinese icebreaker Xuelong. The results show that the classification accuracy, weighted mean dice coefficient, F1-score and mean intersection over union of the proposed model are 94.7%, 0.937, 0.947 and 0.904, respectively, which outperform the frequently-used models such as PSPNet and Deeplab v3+ while using a significantly lower number of parameters.

Deep Reinforcement Learning has shown some promise in dynamic traffic signal control by adapting to real-time traffic conditions. However, multi-intersection control presents challenges, primarily due to the need for efficient information exchange across increasing intersections, and the importance of spatiotemporal dynamics in traffic flows. Traditional methods often focus solely on spatial or temporal aspects, leading to suboptimal control strategies. This paper introduces a novel Multi-Agent Incentive Communication Deep Reinforcement Learning (MICDRL) method, designed for collaborative control across multiple intersections. MICDRL features an incentive communication mechanism, allowing agents to generate customized messages that influence other agents’ policies, thereby enhancing coordination and achieving globally optimal decisions. A key feature of MICDRL is its reliance on local information for message generation, effectively reducing communication overhead while ensuring collaboration. Additionally, MICDRL integrates a teammate module that leverages temporal data for predicting other agents’ actions, crucial for understanding collective dynamics and spatial environment characteristics. Empirical results show that MICDRL outperforms several state-of-the-art methods in metrics like queue length and throughput. Furthermore, we introduce a tailored three-layer Internet-of-Things architecture to enhance data collection and transmission.

Customized bus (CB) is a kind of demand-responsive and one-stop transit service for commuters with similar travel demands. In actual applications, the bus companies identify commuter demands including origin-destination (OD) flow and departure time using online surveys, and manually plan the CB routes, which is inefficient and costly. This study aims to identify spatial patterns of travel demands using a clustering algorithm based on vehicle trajectory data, and automatically extract the potential CB route from each cluster. The clustering algorithms are based on similarity measurements. However, the existing measurements cannot be used to synchronously identify the multiple characteristics of vehicle trajectories, such as OD location, direction, and multiple sub-sequences. To address this issue, we propose a comprehensive similarity (CS) to simultaneously evaluate all the characteristics. Subsequently, we utilize the density-based spatial clustering of applications with noise (DBSCAN) to divide the trajectories into several groups, and determine the trade-off parameters in the DBSCAN using vlseKriterijumska optimizacija i kompromisno resenje (VIKOR), which means a multi-criteria optimization and compromise solution. Then, a case study is conducted based on one-week taxi trajectory data collected in Suzhou, China. The extracted results are compared to the existing routes designed by bus companies, providing a coverage rate of 66.7%. Finally, the advantages of the proposed method were discussed from the two perspectives of clustering algorithm and trajectory similarity measurement. The proposed method can help traffic management authorities and bus companies to plan and design new CB routes.

Traffic signal control is a critical method that ensures the efficiency of traffic flow in cities across the world. There are massive studies that focus on generating optimal signal timing plans. Most of the these studies are model-based, where the signal plan is determined by optimization models with fixed parameters. Reinforcement learning (RL) is a model-free method that learns the optimal control policy over time which avoids limitations of traditional methods. In the past a few years, deep RL methods become particularly attractive since deep neural networks can provide scalable learning capability given the complex traffic condition in real life. In this manuscript, we develop a deep actor-critic method that can provide efficient traffic signal plans. First, a novel deep neural network model is applied to mine the recent traffic condition information as a sequence of temporally sequential image representations of the intersection, which improves from relying solely on limited traffic information such as queue length; then the deep neural network model is integrated with an actor-critic model which avoids drawbacks from the value-based or the policy-based methods. Simulation experiments illustrate that the deep actor-critic method outperforms the classic model-based method and several existing deep reinforcement learning methods according to different measures such as queue length, average delay time, and throughput.

The demand for transport between terminals within port areas, known as inter terminal transportation (ITT), is increasing. This paper proposes a dynamic rolling horizon scheduling strategy for ITT using a fleet of waterborne Autonomous Guided Vessels (waterborne AGVs). The strategy is dynamic in that it can handle dynamically arriving ITT requests. Every certain period of time, transport schedules are updated according to the current vessel states, dynamic waterway transport network, and ITT requests over a future time horizon. Specifically, the dynamic scheduling problem is mathematically modeled in a rolling horizon fashion considering time windows of ITT requests, capacity limits of waterborne AGVs and load/unload service times at terminals. Considering the computational complexity for possible large scale ITT scenarios, we further propose an efficient solution approach based on improved insertion, tabu search and restart heuristics. Initial routes are first constructed by inserting new ITT requests into the previously computed routes in the rolling horizon framework. Tabu search with two types of neighborhoods are then designed to improve the initial routes. Moreover, a select-remove-insert restart procedure is activated to diversify the search space whenever necessary. A waterborne ITT network in the port of Rotterdam is considered. Comprehensive simulations based on realistic ITT dataset are run to demonstrate the effectiveness of the proposed dynamic scheduling strategy. This work could be readily used to build towards a fully autonomous waterborne ITT system. Insights that support long term strategical decisions, such as the fleet size, could also be gained from the simulations.

Image retrieval with text feedback is an emerging research topic with the objective of integrating inputs from multiple modalities as queries. In this paper, queries contain a reference image plus text feedback that describes modifications between this image and the desired image. The existing work for this task mainly focuses on designing a new fusion network to compose the image and text. Still, little research pays attention to the modality gap caused by the inconsistent distribution of features from different modalities, which dramatically influences the feature fusion and similarity learning between queries and the desired image. We propose a Distribution-Aligned Text-based Image Retrieval (DATIR) model, which consists of attention mutual information maximization and hierarchical mutual information maximization, to bridge this gap by increasing non-linear statistic dependencies between representations of different modalities. More specifically, attention mutual information maximization narrows the modality gap between different input modalities by maximizing mutual information between the text representation and its semantically consistent representation captured from the reference image and the desired image by the difference transformer. For hierarchical mutual information maximization, it aligns distributions of features from the image modality and the fusion modality by estimating mutual information between a single-layer representation in the fusion network and the multi-level representations in the desired image encoder. Extensive experiments on three large-scale benchmark datasets demonstrate that we can bridge the modality gap between different modalities and achieve state-of-the-art retrieval performance.

Traffic flow forecasting is an important problem for the successful deployment of intelligent transportation systems, which has been studied for more than two decades. In recent years, deep learning methods are emerging to serve as the benchmark tool for traffic flow forecasting due to its superior prediction performance. However, most studies are based on simple deep learning methods that can not capture inter- and intra-day traffic patterns as well as the correlation between contextual factors like the weather and the traffic flow. In this paper, we propose a novel deep-learning-based method for daily traffic flow forecasting where incorporating contextual factors and traffic flow patterns can be critical. First, a particular convolutional neural network (CNN) is deployed to extract inter- and intra- day traffic flow patterns. Then extracted features are fed into long short-term memory (LSTM) units to learn the intra-day temporal evolution of traffic flow. Finally, contextual information of historical days is integrated to enhance the prediction performance. Through a real-data case study, we show that the proposed approach achieves over 90% prediction accuracy which greatly outperforms existing benchmark methods and its forecasting performance is robust under various scenarios.

Traffic signal control under oversaturated conditions presents a major challenge in metropolitan transportation networks. Previous works have demonstrated the ability of back-pressure methods to maximize network throughput and guarantee network stability. However, most of these methods are implemented adaptively. At present, fixed phase sequences are still widely used in traffic signal control systems. Herein, we propose a new back-pressure-based signal optimization method that combines fixed phase sequences with spatial model predictive control. First, a spatial prediction model for traffic flow was constructed to analyze the movement of vehicles between a central intersection and four peripheral intersections. Then, a multi-objective optimization model of traffic signal timing was developed with the purpose to reduce the risk of spillover and to balance the distribution of vehicles across the whole network. Next, a method based on the Multi-Objective Particle Swarm Optimization algorithm and Technique for Order Preference by Similarity to an Ideal Solution principle was used to achieve the Pareto frontier and the optimal solution. Finally, traffic simulations were performed in Paramics to assess the performance of the proposed method. The results of the simulations suggest the performance of the proposed method surpasses fixed-time control and cycle-based back-pressure schemes under oversaturated conditions.

This paper makes a basic assumption that energy conservation exists, between psychological potential and a vehicle's kinetic energy, in the driver's psychological field based on the driver's mental activities. A virtual spring is used to describe the storage and release of psychological potential energy. Under the aforementioned conditions, we established a macroscopic traffic flow model with conservation law. Each parameter in the new model is physically meaningful and explicit. Additionally, the model can fit field data consistently well, both in free-flow and congested situations. The results of this paper prove the rationality of the energy conservation concept in traffic flow, which improves the understanding of traffic flow and provides a new theoretical foundation.