This work presents a novel data-driven path planning algorithm named Instruction-Guided Probabilistic Roadmap (IG-PRM). Despite the recent development and widespread use of mobile robot navigation, the safe and effective travels of mobile robots still require significant engineering effort to take into account the constraints of robots and their tasks. With IG-PRM, we aim to address this problem by allowing robot operators to specify such constraints through natural language instructions, such as “aim for wider paths” or “mind small gaps”. The key idea is to convert such instructions into embedding vectors using large-language models (LLMs) and use the vectors as a condition to predict instruction-guided cost maps from occupancy maps. By constructing a roadmap based on the predicted costs, we can find instruction-guided paths via the standard shortest path search. Experimental results demonstrate the effectiveness of our approach on both synthetic and real-world indoor navigation environments.

Semantic segmentation plays a crucial role in scene understanding, particularly in the context of autonomous driving. Traditional methods mainly rely on single-modal RGB images, which perform well only under good lighting conditions during the daytime and poorly in challenging lighting conditions like nighttime. Recent research has started integrating thermal images generated by thermal cameras as an additional modality into deep neural networks, known as RGB-T semantic segmentation. However, existing approaches are limited by the limited receptive field of CNNs and often rely on complex fusion modules to improve accuracy, neglecting the computational time of the model. To address these issues, we propose a real-time multimodal fusion network (TSTR) for RGB-T semantic segmentation. Specifically, taking advantage of the global self-attention mechanism and long-range dependency modeling of transformers, we design a dual-branch encoder based on transformers to extract RGB and thermal features separately. To ensure real-time performance, we introduce more efficient self-attention blocks and employ a lightweight MLP decoder to aggregate information from different levels for dense predictions. Additionally, we employ a simple element-wise addition operation for multimodal feature fusion. Extensive quantitative and qualitative experiments demonstrate that the proposed network outperforms several state-of-the-art real-time RGB-T semantic segmentation methods. It achieves an mIoU of 56.2% on benchmark datasets, running in real-time at a speed of 59 FPS with a 480x640 input on an NVIDIA GeForce GTX 2080 Ti GPU. This method is significantly faster than existing networks while achieving better segmentation accuracy.

IoT networks can be viewed as collections of Internet-enabled physical devices and objects, embedded with sensor, actuator, computation, storage and communication components, that are capable of connecting and exchanging data to one another. In recent years, organizations have allowed more and more IoT devices to be connected to their networks, thereby increasing their risks of and exposure to security vulnerabilities and threats. Therefore, it is important for such organizations to be able to identify which devices are connected to their network and which ones are legitimate and pose no risk. Leveraging network traffic to identify devices through supervised learning has recently been gaining popularity, where feature information is first extracted by intercepting device traffic and then exploited to provide device classification. The main limitation of prior works is that they can only identify previously seen types of devices, and any newly added device types are treated as abnormal types. In the real world, hundreds of millions of new IoT devices are produced each year, and the lack of a large amount of training data makes a system based solely on supervised learning unrealistic. In this paper, we propose a hybrid supervised and unsupervised learning method that enables secondary classification of unseen device types. Our technique combines deep neural networks with clustering to enable both seen and unseen device classification, and employs autoencoder technique to reduce dimensionality of datasets, thereby providing a good balance between classification accuracy and overhead.