With the rapid development of Internet of Things (IoT) technologies, autonomous driving has become an integral part of the IoT ecosystem, where millimeter-wave radar plays a crucial role in ensuring robust perception under challenging weather and lighting conditions. However, its sparse and noisy data often require enhancement using high-end sensors such as LiDAR or RTK-GNSS, which are not common in commercial vehicles. This paper introduces mmEMP+, a self-supervised learning technique that leverages pervasive visual and inertial (VI) measurements to enhance radar sensing data. Using VI data to improve radar sensing introduces several challenges. First, moving objects in a scene are inaccurately reconstructed by VI structure-from-motion, which consequently fails to enhance radar sensing. Second, multipath effects generate spurious radar points that can distort the representation of the environment. Finally, the temporal misalignment between the camera, IMU, and mmWave radar results in mismatched data association, thereby degrading system performance. To address these issues, mmEMP+ first proposes a dynamic 3D reconstruction method to recover the positions of moving features accurately. Then, we develop a spatial-stability checking method to filter out spurious radar points. Finally, mmEMP+ devises a tightly coupled sensor fusion method to calibrate the multi-sensor temporal offset. Experiments on a real-world dataset show that mmEMP+ achieves performance comparable to high-channel LiDAR-supervised methods while using only low-cost sensors. We further validate its effectiveness in IoT-relevant applications such as object detection, localization, and mapping.

This article introduces a radar perception learning framework guided by data collected from commonly equipped visual-inertial (VI) sensor suites on smart vehicles. Unlike existing approaches that rely on dense point clouds from 3D LiDARs, which are costly and not widely deployed, this method leverages the broader availability of VI data. However, visual images alone lack the ability to capture the three-dimensional motion of moving targets, which limits their effectiveness in supervising motion-related tasks. To overcome this limitation, the framework integrates multiple perception tasks such as odometry estimation, motion segmentation, and scene flow prediction into a unified learning process. The first component is an odometry estimation module that combines deterministic ego-motion models with data-driven learning results. This fusion helps accurately infer the scene flow of static background points while minimizing drift. The second component is a supervision signal extraction module that aligns optical and millimeter-wave radar measurements to guide the learning of radar scene flow and rigid transformations. This module improves the reliability of dynamic point supervision through joint constraints across sensing modalities. The third component introduces a feature-selection module designed for cross-modal learning. It enhances the accuracy of motion segmentation and enforces consistency between odometry and scene flow, resulting in more coherent radar perception outputs. Experimental evaluations show that this framework achieves superior performance in challenging conditions such as smoke-obscured environments. It surpasses state-of-the-art (SOTA) methods that depend on high-cost LiDAR systems. The implementation of VISC+ will be open-source athttps://github.com/weini-Eve/VISC

Vessel trajectory prediction is fundamental to intelligent maritime systems. Within this domain, short-term prediction of rapid behavioral changes in complex maritime environments has established multimodal trajectory prediction (MTP) as a promising research area. However, existing vessel MTP methods suffer from limited scenario applicability and insufficient explainability. To address these challenges, we propose a unified MTP framework incorporating explainable navigation intentions, which we classify into sustained and transient categories. Our method constructs sustained intention trees from historical trajectories and models dynamic transient intentions using a Conditional Variational Autoencoder (CVAE), while using a non-local attention mechanism to maintain global scenario consistency. Experiments on real Automatic Identification System (AIS) datasets demonstrates our method’s broad applicability across diverse scenarios, achieving significant improvements in both ADE and FDE. Furthermore, our method improves explainability by explicitly revealing the navigational intentions underlying each predicted trajectory.

This article investigates the multitarget dynamic encirclement control for multiple autonomous surface vehicles (ASVs). The ASVs are subjected to unavailable velocity state information, finite communication resources, saturated actuators, model uncertainties, and environmental disturbances. The control objective is to enable an ASV team to perform encirclement maneuvers while tracking the convex combination of the states of multiple targets. Specifically, an adaptive fuzzy state observer (AFSO) is developed to estimate the unavailable velocity state information. Next, the system is divided into two regions to avoid singularities. A time-varying terminal sliding mode controller and a time-varying auxiliary sliding mode controller are designed for each region. By applying finite-time control theory, the multitarget dynamic encirclement can be achieved within a finite time. Moreover, a dynamic event-triggered mechanism (DETM) is proposed to optimize the use of communication resources. Finally, the validity of the obtained theory results is demonstrated through the Lyapunov analysis and numerical simulation.

This work proposes a mmWave radar’s scene flow estimation framework supervised by data from a widespread visual-inertial (VI) sensor suite, allowing crowdsourced training data from smart vehicles. Current scene flow estimation methods for mmWave radar are typically supervised by dense point clouds from 3D LiDARs, which are expensive and not widely available in smart vehicles. While VI data are more accessible, visual images alone cannot capture the 3D motions of moving objects, making it difficult to supervise their scene flow. Moreover, the temporal drift of VI rigid transformation also degenerates the scene flow estimation of static points. To address these challenges, we propose a drift-free rigid transformation estimator that fuses kinematic model-based ego-motions with neural network-learned results. It provides strong supervision signals to radar-based rigid transformation and infers the scene flow of static points. Then, we develop an optical-mmWave supervision extraction module that extracts the supervision signals of radar rigid transformation and scene flow. It strengthens the supervision by learning the scene flow of dynamic points with the joint constraints of optical and mmWave radar measurements. Extensive experiments demonstrate that, in smoke-filled environments, our method even outperforms state-of-the-art (SOTA) approaches using costly LiDARs.

With the advancement of satellite communication technology, the maritime Internet of Things (IoT) has made significant progress. As a result, vast amounts of Automatic Identification System (AIS) data from global vessels are transmitted to various maritime stakeholders through Maritime IoT systems. AIS data contains a large amount of dynamic and static information that requires effective and intuitive visualization for comprehensive analysis. However, two major deficiencies challenge current visualization models: a lack of consideration for interactions between distant pixels and low efficiency. To address these issues, we developed a large-scale vessel trajectories visualization algorithm, called the Non-local Kernel Density Estimation (NLKDE) algorithm, which incorporates a non-local convolution process. It accurately calculates the density distribution of vessel trajectories by considering correlations between distant pixels. Additionally, we implemented the NLKDE algorithm under a Graphics Processing Unit (GPU) framework to enable parallel computing and improve operational efficiency. Comprehensive experiments using multiple vessel trajectory datasets show that the NLKDE algorithm excels in vessel trajectory density visualization tasks, and the GPU-accelerated framework significantly shortens the execution time to achieve real-time results. From both theoretical and practical perspectives, GPU-accelerated NLKDE provides technical support for real-time monitoring of vessel dynamics in complex water areas and contributes to constructing maritime intelligent transportation systems. The code for this paper can be accessed at: https://github.com/maohliang/GPU-NLKDE.

Large Language Models (LLMs), with their strong generalization and inference capabilities, have been increasingly leveraged to address the challenges of handling corner cases in autonomous driving (AD). However, a critical unresolved issue remains: the lack of a comprehensive understanding and formal assessment of LLMs’ driving theory knowledge and practical skills. To address this issue, we propose the first dedicated driving theory test framework and benchmark for LLMs. That is a crucial yet unexplored area in the literature, particularly for safety-critical applications in autonomous driving and driver assistance. Our framework systematically evaluates LLMs’ competence in driving theory and hazard perception, akin to the official UK driving theory test, ensuring their qualification for critical driving-related tasks. To facilitate rigorous benchmarking, we construct a comprehensive dataset comprising over 700 multiple-choice questions (MCQs) and 54 hazard perception video tests sourced from the official UK driving theory examination. Additionally, we incorporate two standardized MCQ sets from the UK’s Driver and Vehicle Standards Agency (DVSA). For these two types of theoretical test items, we design tailored assessment methodologies and evaluation metrics, including accuracy, recall, precision, F1-score, real-time performance, and computational efficiency. The experimental results reveal that among all LLMs tested, only GPT-4o achieved an accuracy of 88. 21% in the MCQs test, successfully passing this component. However, in hazard perception testing, none of the evaluated models met the passing criteria under the given settings, highlighting the substantial improvements required before these models can be practically deployed for real-world driving applications. Our key insight is that the specific test questions LLMs fail to answer correctly directly reflect their deficiencies in understanding and flexibly applying traffic regulations, as well as in analyzing and responding to complex driving scenarios. This provides clear directions for future improvements.

Complementary to prevalent LiDAR and camera systems, millimeter-wave (mmWave) radar is robust to adverse weather conditions like fog, rainstorms, and blizzards but offers sparse point clouds. Current techniques enhance the point cloud by the supervision of LiDAR’s data. However, high-performance LiDAR is notably expensive and is not commonly available on vehicles. This paper presents mmEMP, a supervised learning approach that enhances radar point clouds using a low-cost camera and an inertial measurement unit (IMU), enabling crowd-sourcing training data from commercial vehicles. Bringing the visual-inertial (VI) supervision is challenging due to the spatial agnostic of dynamic objects. Moreover, spurious radar points from the curse of RF multipath make robots misunderstand the scene. mmEMP first devises a dynamic 3D reconstruction algorithm that restores the 3D positions of dynamic features. Then, we design a neural network that densifies radar data and eliminates spurious radar points. We build a new dataset in the real world. Extensive experiments show that mmEMP achieves competitive performance compared with the SOTA approach training by LiDAR’s data. In addition, we use the enhanced point cloud to perform object detection, localization, and mapping to demonstrate mmEMP’s effectiveness.

Constructing reliable trajectories for ship targets in busy waterways involves associating tracks from various sensor sources. This poses challenges due to the incoherence in sensor data precision and acquisition frequency, the high density of targets, and the intertwined correlation scenarios involving trajectory interruptions and multi-source (also known as multi-sensor) observations. Utilizing the intuitive idea that a ship's motion pattern is influenced by its geographic location, we integrate geographic and dynamic information to enhance trajectory accuracy in dense scenes. The proposed approach employs self-supervised contrastive learning to extract relevant features, accompanied by a dual-level augmentation strategy tailored to two association scenarios. Additionally, a specialized feature enhancement module is introduced to enable a comprehensive representation of trajectories. When tested on a large dataset and compared with established methods, our method demonstrates superior capabilities in extracting trajectory features and excels in a variety of association contexts.

An emerging evacuation path planning technique that uses Low Power Wide Area Networks (LPWAN) to enable real-time danger prediction and user-oriented path planning can ensure the safe and timely navigation of evacuees in complex scenarios such as cruise ships. However, most existing LPWAN-based evacuation models assume pedestrians’ walking speed remains constant and ignore crowd congestion in corridors before exits, which is not appropriate for rocking ships. To overcome these issues, this paper proposes a congestion-relived guiding framework with dedicated path planning for emergency evacuation on passenger ships. The basic idea is to averagely minimize the total evacuation time while meeting the deadline for ship capsizing under all circumstances by selecting uncrowded paths for each passenger individually. First, we use probability distributions rather than constant numbers to represent walking time (also called delay) along passageways. A worst-case delay bound with a high level of trustworthiness is also estimated for each passageway under the boundary condition of ship capsizing. Next, we predict the congestion of corridors by modeling the spatiotemporal movement of passengers, and then distribute evacuation loads evenly among corridors to alleviate the congestion. The total expected evacuation time of all corridors is finally minimized based on the delay probability distribution and estimated congestion, and the deadline for ship evacuation under all circumstances is met with the worst-case delay bound. Simulation results show that our approach significantly reduces the total escaping time of crowd evacuation by 45% and 34% while improving the navigation success ratio by more than 20% and 80% compared with the state-of-the-art emergency evacuation systems, namely the look-up table guiding scheme and the group-based guiding evacuation scheme, respectively.

Vessel trajectory clustering, which aims to find similar trajectory patterns, has been widely leveraged in maritime applications. Most traditional methods use predefined rules and thresholds to identify discrete vessel behaviors. They aim for high-quality clustering and conduct clustering on entire sequences, whether the original trajectory or its sub-trajectories, overlooking the behavioral significance and evolution characteristics. To resolve this problem, we propose a Predictive Clustering of Hierarchical Vessel Behavior (PC-HiV). PC-HiV first utilizes hierarchical representations to transform every trajectory into a behavioral sequence. It then predicts evolution at each timestamp of the sequence based on the representations. By applying predictive clustering and latent encoding, PC-HiV improves clustering and predictions simultaneously. Experiments conducted on real AIS datasets demonstrate that PC-HiV effectively captures behavioral evolution discrepancies between different vessel types (tramp vs. liner) and near emission control area boundaries. Additionally, the results show that PC-HiV outperforms NN-Kmeans and Robust DAA by 3.9% and 6.4% in terms of purity scores, thereby proving the superiority of the proposed PC-HiV over existing models.

Maritime Situational Awareness (MSA) is a critical component of intelligent maritime traffic surveillance. However, it becomes increasingly challenging to gain MSA accurately given the growing complexity of ship traffic patterns due to multi-ship interactions possibly involving classical manned ships and emerging autonomous ships. This study proposes a new traffic partitioning methodology to realise the optimal maritime traffic partition in complex waters. The methodology combines conflict criticality and spatial distance to generate conflict-connected and spatially compact traffic clusters, thereby improving the interpretability of traffic patterns and supporting ship anti-collision risk management. First, a composite similarity measure is designed using a probabilistic conflict detection approach and a newly formulated maritime traffic route network learned through maritime knowledge mining. Then, an extended graph-based clustering framework is used to produce balanced traffic clusters with high intra-connections but low inter-connections. The proposed methodology is thoroughly demonstrated and tested using Automatic Identification System (AIS) trajectory data in the Ningbo-Zhoushan Port. The experimental results show that the proposed methodology 1) has effective performance in decomposing the traffic complexity, 2) can assist in identifying high-risk/density traffic clusters, and 3) is sufficiently generic to handle various traffic scenarios in complex geographical waters. Therefore, this study makes significant contributions to intelligent maritime surveillance and provides a theoretical foundation for promoting maritime anti-collision risk management for the future mixed traffic of both manned and autonomous ships.

Security check-in life-support areas, e.g., bridge and engine room are crucial for cruise ships due to numerous and diverse passenger identities. Instead of conventional security check approaches, such as facial recognition and fingerprint identification, device-free approaches enabled by WiFi-based gait recognition have attracted considerable attention owing to their low cost, nonintrusiveness, and privacy protection. Despite the excellent performance of existing indoor methods, they cannot be trivially extended to cruise ships because of the unique characteristics of hull deformation caused by vibrating engines and waves. This stems from the flexible structure of cruise ships, which introduces additional noise to the WiFi signals. To address this challenge, we propose WiCrew, a device-free gait recognition system that detects crew identity anomalies in cruise ships. WiCrew consists of two components: 1) a spatial separation algorithm that separates the signal components from ship vibration and human activity and 2) a speed-independent adversarial learning framework that identifies the ship’s crew using human gaits at an arbitrary walking speed. Extensive experiments on a cruise ship demonstrate the effectiveness of WiCrew. While the crew members walk at speed of 0.7 to 1.8 m/s, the average recognition accuracy reaches 82%, which is similar to vision-based approaches.

Being able to track passengers’ movement without invasion of their privacy plays an important role in cruise ships; it enables crucial location-based services, such as maritime search and rescue, tourist services, and epidemic prevention. The past few years have witnessed commodity WiFi holding great potential that provides such services available thanks to its ubiquitous in indoor scenarios. However, existing WiFi-based tracking methods suffer from huge performance degradation in sailing ships due to their complex metal structures and dynamic hull deformation caused by engines and waves/payloads pressure. In this article, we present CRLoc, a deep learning-based passive human tracking system that can overcome the practical limitations of traditional WiFi-based localization approaches applied in a multipath-rich and mobile ship environment, and provide decimeter-level tracking accuracy in cruise ships. Specifically, we make two contributions, i.e., we propose a super-resolution parameter estimation algorithm that better characterizes ship indoor environments, and a deep neural network-based end-to-end solution to remove the impact of noise, interference, and mobility in ships. The real-world implementation and extensive experiments in several passenger ships demonstrate that CRLoc tracks human motions with a median error of 92 cm, better than state-of-the-art localization methods. To our knowledge, this is one of the first WiFi-based passive human motion tracking system in a cruise ship environment.

Accurate and pervasive device-free indoor localization with meter-level resolution is critical for large cruise and passenger ships due to safety-critical rescue and evacuation requirements when accidents occur. However, existing localization techniques would severely suffer on ships because of their unique mobility characteristics. In this paper, we take the first attempt to build a ubiquitous passive localization system using WiFi fingerprints for the mobile ship environment. By conducting extensive experiments and measurements during several cruise trips, we identified a major influence factor on the fingerprints in the mobile environment: varying the ship speeds may significantly change the patterns of fingerprints at runtime. Since it may be too expensive to identify the fingerprints associated with different speeds, we propose an efficient localization method, namely SWIM, which calibrates the fingerprints from only a single-speed scenario to multiple-speed scenarios using a signal reconstruction analysis. SWIM is designed to learn the predictive fingerprint variation introduced by environmental speed changes and reconstruct the original fingerprints to adapt to the runtime speed scenarios. We have implemented and extensively evaluated SWIM on actual cruise ships. Experimental results demonstrate that SWIM improves localization accuracy from 63.2 to 82.9 percent, while reducing the overall system deployment cost by 87 percent.

Device-free indoor localization may play a critical role in improving passengers' safety in large vessels, particularly for scenarios without equipped radios. However, due to dynamic internal and external influences from the sailing ship such as changing sailing speed, the existing localization systems suffer huge accuracy degradation in a mobile ship environment. The challenges are mainly due to rich and arbitrary ship motions and the resulting complicated impacts on the indoor wireless channels. To address the challenges, in this article, we first propose a ship motion descriptor to extract discriminative latent representation from complex ship motions by leveraging deep-learning techniques. Based on this representation, we then design a novel fingerprint roaming model, i.e., MoLoc, to automatically learn the predictive fingerprint variation pattern and transfer the online fingerprint measurement to adapt to dynamic ship motions in real time. Furthermore, an unsupervised learning strategy is proposed to train the fingerprint roaming model using unlabeled onboard collected data which do not incur any labor costs. We have implemented and extensively evaluated MoLoc on real-world cruise ships, where experimental results demonstrate that MoLoc improves localization accuracy from 63.2% to 92.8% compared to the state-of-the-art localization methods, including Pilot, LiFS, SpotFi, and AutoFi while achieving a mean error of 0.68 m.

In this paper, we develop an analytical model for IEEE 802.11 MAC protocol with arbitrary buffer size in unsaturated conditions. The model comprises of a generalized Markov chain model and an M/G/1/K queuing model. We give the thorough and accurate queuing analysis of IEEE 802.11 MAC layer. It's shown that for practical 802.11 WLAN (i.e., buffer size larger than one and number of stations comparatively large) under the optimal offered load, the total throughput is maximized, the packet blocking probability (due to limited buffer size) and the average queuing delay tends to zero, the average MAC service delay as well as its standard deviation is much lower than that in saturated conditions. The simulations show the analytical model is highly accurate.

The wireless nature of the medium combined with energy constraints pose big challenges on the design of energy efficient and reliable protocols for wireless sensor networks (WSN). In this paper, the virtual MIMO scheme is incorporated with the multi-hop networking for energy saving in wireless sensor network (WSN). An energy consumption model is developed to investigate the energy saving performance by incorporating the virtual MlMO scheme with the multi-hop networking. Then, an optimization model is developed to find the optimal number of virtual nodes and the optimal number of hops. Based on the model, a virtual MlMO protocol based on clustering is developed. Simulation results show that the protocol can save energy significantly.