With the rapid development of Internet of Things (IoT) technologies, autonomous driving has become an integral part of the IoT ecosystem, where millimeter-wave radar plays a crucial role in ensuring robust perception under challenging weather and lighting conditions. However, its sparse and noisy data often require enhancement using high-end sensors such as LiDAR or RTK-GNSS, which are not common in commercial vehicles. This paper introduces mmEMP+, a self-supervised learning technique that leverages pervasive visual and inertial (VI) measurements to enhance radar sensing data. Using VI data to improve radar sensing introduces several challenges. First, moving objects in a scene are inaccurately reconstructed by VI structure-from-motion, which consequently fails to enhance radar sensing. Second, multipath effects generate spurious radar points that can distort the representation of the environment. Finally, the temporal misalignment between the camera, IMU, and mmWave radar results in mismatched data association, thereby degrading system performance. To address these issues, mmEMP+ first proposes a dynamic 3D reconstruction method to recover the positions of moving features accurately. Then, we develop a spatial-stability checking method to filter out spurious radar points. Finally, mmEMP+ devises a tightly coupled sensor fusion method to calibrate the multi-sensor temporal offset. Experiments on a real-world dataset show that mmEMP+ achieves performance comparable to high-channel LiDAR-supervised methods while using only low-cost sensors. We further validate its effectiveness in IoT-relevant applications such as object detection, localization, and mapping.

This article introduces a radar perception learning framework guided by data collected from commonly equipped visual-inertial (VI) sensor suites on smart vehicles. Unlike existing approaches that rely on dense point clouds from 3D LiDARs, which are costly and not widely deployed, this method leverages the broader availability of VI data. However, visual images alone lack the ability to capture the three-dimensional motion of moving targets, which limits their effectiveness in supervising motion-related tasks. To overcome this limitation, the framework integrates multiple perception tasks such as odometry estimation, motion segmentation, and scene flow prediction into a unified learning process. The first component is an odometry estimation module that combines deterministic ego-motion models with data-driven learning results. This fusion helps accurately infer the scene flow of static background points while minimizing drift. The second component is a supervision signal extraction module that aligns optical and millimeter-wave radar measurements to guide the learning of radar scene flow and rigid transformations. This module improves the reliability of dynamic point supervision through joint constraints across sensing modalities. The third component introduces a feature-selection module designed for cross-modal learning. It enhances the accuracy of motion segmentation and enforces consistency between odometry and scene flow, resulting in more coherent radar perception outputs. Experimental evaluations show that this framework achieves superior performance in challenging conditions such as smoke-obscured environments. It surpasses state-of-the-art (SOTA) methods that depend on high-cost LiDAR systems. The implementation of VISC+ will be open-source athttps://github.com/weini-Eve/VISC

This work proposes a mmWave radar’s scene flow estimation framework supervised by data from a widespread visual-inertial (VI) sensor suite, allowing crowdsourced training data from smart vehicles. Current scene flow estimation methods for mmWave radar are typically supervised by dense point clouds from 3D LiDARs, which are expensive and not widely available in smart vehicles. While VI data are more accessible, visual images alone cannot capture the 3D motions of moving objects, making it difficult to supervise their scene flow. Moreover, the temporal drift of VI rigid transformation also degenerates the scene flow estimation of static points. To address these challenges, we propose a drift-free rigid transformation estimator that fuses kinematic model-based ego-motions with neural network-learned results. It provides strong supervision signals to radar-based rigid transformation and infers the scene flow of static points. Then, we develop an optical-mmWave supervision extraction module that extracts the supervision signals of radar rigid transformation and scene flow. It strengthens the supervision by learning the scene flow of dynamic points with the joint constraints of optical and mmWave radar measurements. Extensive experiments demonstrate that, in smoke-filled environments, our method even outperforms state-of-the-art (SOTA) approaches using costly LiDARs.

Large Language Models (LLMs), with their strong generalization and inference capabilities, have been increasingly leveraged to address the challenges of handling corner cases in autonomous driving (AD). However, a critical unresolved issue remains: the lack of a comprehensive understanding and formal assessment of LLMs’ driving theory knowledge and practical skills. To address this issue, we propose the first dedicated driving theory test framework and benchmark for LLMs. That is a crucial yet unexplored area in the literature, particularly for safety-critical applications in autonomous driving and driver assistance. Our framework systematically evaluates LLMs’ competence in driving theory and hazard perception, akin to the official UK driving theory test, ensuring their qualification for critical driving-related tasks. To facilitate rigorous benchmarking, we construct a comprehensive dataset comprising over 700 multiple-choice questions (MCQs) and 54 hazard perception video tests sourced from the official UK driving theory examination. Additionally, we incorporate two standardized MCQ sets from the UK’s Driver and Vehicle Standards Agency (DVSA). For these two types of theoretical test items, we design tailored assessment methodologies and evaluation metrics, including accuracy, recall, precision, F1-score, real-time performance, and computational efficiency. The experimental results reveal that among all LLMs tested, only GPT-4o achieved an accuracy of 88. 21% in the MCQs test, successfully passing this component. However, in hazard perception testing, none of the evaluated models met the passing criteria under the given settings, highlighting the substantial improvements required before these models can be practically deployed for real-world driving applications. Our key insight is that the specific test questions LLMs fail to answer correctly directly reflect their deficiencies in understanding and flexibly applying traffic regulations, as well as in analyzing and responding to complex driving scenarios. This provides clear directions for future improvements.

An emerging evacuation path planning technique that uses Low Power Wide Area Networks (LPWAN) to enable real-time danger prediction and user-oriented path planning can ensure the safe and timely navigation of evacuees in complex scenarios such as cruise ships. However, most existing LPWAN-based evacuation models assume pedestrians’ walking speed remains constant and ignore crowd congestion in corridors before exits, which is not appropriate for rocking ships. To overcome these issues, this paper proposes a congestion-relived guiding framework with dedicated path planning for emergency evacuation on passenger ships. The basic idea is to averagely minimize the total evacuation time while meeting the deadline for ship capsizing under all circumstances by selecting uncrowded paths for each passenger individually. First, we use probability distributions rather than constant numbers to represent walking time (also called delay) along passageways. A worst-case delay bound with a high level of trustworthiness is also estimated for each passageway under the boundary condition of ship capsizing. Next, we predict the congestion of corridors by modeling the spatiotemporal movement of passengers, and then distribute evacuation loads evenly among corridors to alleviate the congestion. The total expected evacuation time of all corridors is finally minimized based on the delay probability distribution and estimated congestion, and the deadline for ship evacuation under all circumstances is met with the worst-case delay bound. Simulation results show that our approach significantly reduces the total escaping time of crowd evacuation by 45% and 34% while improving the navigation success ratio by more than 20% and 80% compared with the state-of-the-art emergency evacuation systems, namely the look-up table guiding scheme and the group-based guiding evacuation scheme, respectively.

Security check-in life-support areas, e.g., bridge and engine room are crucial for cruise ships due to numerous and diverse passenger identities. Instead of conventional security check approaches, such as facial recognition and fingerprint identification, device-free approaches enabled by WiFi-based gait recognition have attracted considerable attention owing to their low cost, nonintrusiveness, and privacy protection. Despite the excellent performance of existing indoor methods, they cannot be trivially extended to cruise ships because of the unique characteristics of hull deformation caused by vibrating engines and waves. This stems from the flexible structure of cruise ships, which introduces additional noise to the WiFi signals. To address this challenge, we propose WiCrew, a device-free gait recognition system that detects crew identity anomalies in cruise ships. WiCrew consists of two components: 1) a spatial separation algorithm that separates the signal components from ship vibration and human activity and 2) a speed-independent adversarial learning framework that identifies the ship’s crew using human gaits at an arbitrary walking speed. Extensive experiments on a cruise ship demonstrate the effectiveness of WiCrew. While the crew members walk at speed of 0.7 to 1.8 m/s, the average recognition accuracy reaches 82%, which is similar to vision-based approaches.

Being able to track passengers’ movement without invasion of their privacy plays an important role in cruise ships; it enables crucial location-based services, such as maritime search and rescue, tourist services, and epidemic prevention. The past few years have witnessed commodity WiFi holding great potential that provides such services available thanks to its ubiquitous in indoor scenarios. However, existing WiFi-based tracking methods suffer from huge performance degradation in sailing ships due to their complex metal structures and dynamic hull deformation caused by engines and waves/payloads pressure. In this article, we present CRLoc, a deep learning-based passive human tracking system that can overcome the practical limitations of traditional WiFi-based localization approaches applied in a multipath-rich and mobile ship environment, and provide decimeter-level tracking accuracy in cruise ships. Specifically, we make two contributions, i.e., we propose a super-resolution parameter estimation algorithm that better characterizes ship indoor environments, and a deep neural network-based end-to-end solution to remove the impact of noise, interference, and mobility in ships. The real-world implementation and extensive experiments in several passenger ships demonstrate that CRLoc tracks human motions with a median error of 92 cm, better than state-of-the-art localization methods. To our knowledge, this is one of the first WiFi-based passive human motion tracking system in a cruise ship environment.

Accurate and pervasive device-free indoor localization with meter-level resolution is critical for large cruise and passenger ships due to safety-critical rescue and evacuation requirements when accidents occur. However, existing localization techniques would severely suffer on ships because of their unique mobility characteristics. In this paper, we take the first attempt to build a ubiquitous passive localization system using WiFi fingerprints for the mobile ship environment. By conducting extensive experiments and measurements during several cruise trips, we identified a major influence factor on the fingerprints in the mobile environment: varying the ship speeds may significantly change the patterns of fingerprints at runtime. Since it may be too expensive to identify the fingerprints associated with different speeds, we propose an efficient localization method, namely SWIM, which calibrates the fingerprints from only a single-speed scenario to multiple-speed scenarios using a signal reconstruction analysis. SWIM is designed to learn the predictive fingerprint variation introduced by environmental speed changes and reconstruct the original fingerprints to adapt to the runtime speed scenarios. We have implemented and extensively evaluated SWIM on actual cruise ships. Experimental results demonstrate that SWIM improves localization accuracy from 63.2 to 82.9 percent, while reducing the overall system deployment cost by 87 percent.

Device-free indoor localization may play a critical role in improving passengers' safety in large vessels, particularly for scenarios without equipped radios. However, due to dynamic internal and external influences from the sailing ship such as changing sailing speed, the existing localization systems suffer huge accuracy degradation in a mobile ship environment. The challenges are mainly due to rich and arbitrary ship motions and the resulting complicated impacts on the indoor wireless channels. To address the challenges, in this article, we first propose a ship motion descriptor to extract discriminative latent representation from complex ship motions by leveraging deep-learning techniques. Based on this representation, we then design a novel fingerprint roaming model, i.e., MoLoc, to automatically learn the predictive fingerprint variation pattern and transfer the online fingerprint measurement to adapt to dynamic ship motions in real time. Furthermore, an unsupervised learning strategy is proposed to train the fingerprint roaming model using unlabeled onboard collected data which do not incur any labor costs. We have implemented and extensively evaluated MoLoc on real-world cruise ships, where experimental results demonstrate that MoLoc improves localization accuracy from 63.2% to 92.8% compared to the state-of-the-art localization methods, including Pilot, LiFS, SpotFi, and AutoFi while achieving a mean error of 0.68 m.