This paper presents a novel autonomous navigation framework that integrates Large Language Models (LLMs) with multimodal sensor fusion to enable dynamic obstacle avoidance and human-aware path planning in diverse environments. The proposed system leverages an FPGA-accelerated fusion pipeline, combining LiDAR and vision data for real-time perception. A Hungarian algorithm-based object matching technique ensures robust tracking, while a bird's-eye view (BEV) representation enhances spatial reasoning and occlusion handling. The fused sensory inputs are processed by a fine-tuned LLM, which contextualizes pedestrian behavior and environmental constraints to generate adaptive, human-centric navigation strategies. Unlike traditional rule-based methods, LLMs provide generalization capabilities to novel scenarios, significantly improving interaction with vulnerable pedestrians such as children, elderly individuals, and wheelchair users. Extensive evaluations in both simulated and real-world scenarios confirm the system's ability to reduce collisions and enhance navigation efficiency in high-density environments. By bridging semantic reasoning and robotic control, this work lays the foundation for next-generation intelligent navigation systems that are both safety-aware and scalable across autonomous platforms.

The increasing need for pedestrian tracking within public and commercial domains necessitates innovative solutions that respect privacy concerns. Traditional surveillance methods employing RGB video footage for the monitoring of pedestrian movements have raised significant privacy issues, attributed to their capability to easily identify individuals. This paper shifts focus towards the utilization of 3D point cloud data as a less intrusive alternative for pedestrian monitoring, which inherently minimizes the risk of individual identification. Notwithstanding, traditional tracking methodologies frequently encounter difficulties in environments characterized by high pedestrian density, where occlusions caused by individuals obscuring one another from the sensors’ perspective are commonplace. This research introduces an advanced approach leveraging 3D point cloud data to address the challenges inherent in pedestrian tracking. A particular obstacle addressed is the frequent obstruction caused by pedestrians, who may be accompanied by objects such as baby strollers and/or luggage, thus complicating their detection and tracking when positioned behind others or in close proximity to the sensors. Such scenarios significantly undermine the efficacy of pedestrian segmentation and tracking, particularly in the context of Kalman-Filter-based methodologies. To mitigate these challenges, the proposed solution incorporates a method to spatially enhance the missing segments within the framework of Kalman-Filter-based multi-object tracking. The effectiveness of this approach has been rigorously assessed through the application of 3D point cloud data derived from diverse real-world settings, including laboratory environments, shopping mall entrances, and a densely populated urban area in Kita-ku, Osaka City, during a festival event.

Tracking pedestrian flow in large public areas is vital, yet ensuring privacy is paramount. Traditional visual-based tracking systems are raising concerns for potentially obtaining persistent and permanent identifiers that can compromise individual identities. Moreover, in areas such as the vicinity of restrooms, any form of data acquisition capturing human behavior should be refrained from, making it also crucial to appropriately address and complement these blind spots for a comprehensive analysis of pedestrian movement in the entire area. In this paper, we present our pedestrian tracking algorithm using distributed 3D LiDARs (Light Detection and Ranging), which capture pedestrians as 3D point clouds, omitting identifiable features. Our system bridges blind spots by leveraging historical movement data and 3D point cloud features, complemented by a generative diffusion model to predict trajectories in unseen areas. In a large-scale testbed with 70 LiDARs, the system achieved a 0.98 F-measure, highlighting its potential as a leading privacy-preserving tracking solution.

The growing demand for intelligent environments unleashes an extraordinary cycle of privacy-aware applications that makes individuals' life more comfortable and safe. Examples of these applications include pedestrian tracking systems in large areas. Although the ubiquity of camera-based systems, they are not a preferable solution due to the vulnerability of leaking the privacy of pedestrians. In this paper, we introduce a novel privacy-preserving system for pedestrian tracking in smart environments using multiple distributed LiDARs of non-overlapping views. The system is designed to leverage LiDAR devices to track pedestrians in partially covered areas due to practical constraints, e.g., occlusion or cost. Therefore, the system uses the point cloud captured by different LiDARs to extract discriminative features that are used to train a metric learning model for pedestrian matching purposes. To boost the system's robustness, we leverage a probabilistic approach to model and adapt the dynamic mobility patterns of individuals and thus connect their sub-trajectories. We deployed the system in a large-scale testbed with 70 colorless LiDARs and conducted three different experiments. The evaluation result at the entrance hall confirms the system's ability to accurately track the pedestrians with a 0.98 F-measure even with zero-covered areas. This result highlights the promise of the proposed system as the next generation of privacy-preserving tracking means in smart environments.