Integrating Large Language Models (LLMs) into robotic systems has revolutionised embodied artificial intelligence, enabling advanced decision-making and adaptability. However, ensuring reliability — encompassing both security against adversarial attacks and safety in complex environments — remains a critical challenge. To address this, we propose a unified framework that mitigates prompt injection attacks while enforcing operational safety through robust validation mechanisms. Our approach combines prompt assembling, state management, and safety validation, evaluated using both performance and security metrics. Experiments show a 30.8% improvement under injection attacks and up to a 325% improvement in complex environment settings under adversarial conditions compared to baseline scenarios. This work bridges the gap between safety and security in LLM-based robotic systems, offering actionable insights for deploying reliable LLM-integrated mobile robots in real-world settings. The framework is open-sourced with simulation and physical deployment demos at https://llmeyesim.vercel.app/ .

The increasing presence of autonomous vehicles on public roads has highlighted the limitations of traditional incident reporting systems, which rely on human-generated tables and descriptions. To address this, we propose a generative reporting framework that integrates a large language model (LLM) with semantic scene generation models. This framework utilizes perception snapshots and self-diagnostic data to generate detailed incident reports, addressing environmental blind spots. Our 3D scene completion network, combining diffusion and state-space models, reconstructs blind zones undetected by exterior sensors, achieving IoU scores of 41.92 on SSCBench-KITTI360 and 44.13 on SemanticKITTI. Public road experiments validate the system's ability to improve incident report quality while maintaining performance.

In recent years, Large Language Models (LLMs) have demonstrated remarkable capabilities in understanding and solving mathematical problems, leading to advancements in various fields. We propose an LLM-embodied path planning framework for mobile agents, focusing on solving high-level coverage path planning issues and low-level control. Our proposed multi-layer architecture uses prompted LLMs in the path planning phase and integrates them with the mobile agents' low-level actuators. To evaluate the performance of various LLMs, we propose a coverage-weighted path planning metric to assess the performance of the embodied models. Our experiments show that the proposed framework improves LLMs' spatial inference abilities. We demonstrate that the proposed multi-layer framework significantly enhances the efficiency and accuracy of these tasks by leveraging the natural language understanding and generative capabilities of LLMs. Experiments conducted in our Eye Sim simulation demonstrate that this framework enhances LLMs' 2D plane reasoning abilities and enables the completion of coverage path planning tasks. We also tested three LLM kernels: gpt-4o, gemini-1.5-flash, and claude-3.5-sonnet. The experimental results show that claude-3.5 can complete the coverage planning task in different scenarios, and its indicators are better than those of the other models. We have made our experimental simulation platform, Eye Sim, freely available at https://roblab.org/eyesim/.

Over the last year, significant advancements have been made in the realms of large language models (LLMs) and multi-modal large language models (MLLMs), particularly in their application to autonomous driving. These models have showcased remarkable abilities in processing and interacting with complex information. In autonomous driving, LLMs and MLLMs are extensively used, requiring access to sensitive vehicle data such as precise locations, images, and road conditions. This data is transmitted to an LLM-based inference cloud for advanced analysis. However, concerns arise regarding data security, as the protection against data and privacy breaches primarily depends on the LLM’s inherent security measures, without additional scrutiny or evaluation of the LLM’s inference outputs. Despite its importance, the security aspect of LLMs in autonomous driving remains underexplored. Addressing this gap, our research introduces a novel security framework for autonomous vehicles, utilizing a multi-agent LLM approach. This framework is designed to safeguard sensitive information associated with autonomous vehicles from potential leaks, while also ensuring that LLM outputs adhere to driving regulations and align with human values. It includes mechanisms to filter out irrelevant queries and verify the safety and reliability of LLM outputs. Utilizing this framework, we evaluated the security, privacy, and cost aspects of eleven large language model-driven autonomous driving cues. Additionally, we performed QA tests on these driving prompts, which successfully demonstrated the framework’s efficacy.

This paper presents an approach to specifying a modularised safety system which comprehensively addresses the safety requirements for highly autonomous (SAE Level 3+) road vehicles featuring advanced sensing and automated navigation. As these requirements are often overlooked in similar autonomous driving system proposals, we present a method of hazard and risk analysis which investigates hardware failures, environmental perception limitations, human interaction and functional requirements for artificial intelligence. We then define a system design which implements the required safeguards and examines the application on two electric autonomous vehicle testbeds: a race car and a shuttle bus. The close-coupling of a safety-oriented architecture and multi-regime Hazard and Risk Assessment process was tested to measure the system's ability to detect and react to pedestrian stimuli, resulting in accurate detections and reactions, thereby confirming its ability to design safety systems for autonomous research vehicles in a scalable and easily assured fashion.

Software frameworks for autonomous vehicles are required to interface and process data from several different sensors on board the vehicle, in addition to performing navigational processes such as path planning and lane keeping. These can include a combination of cameras, LIDARs, GPS, IMU, and odometric sensors to achieve positioning and localisation for the vehicle and can be challenging to integrate. In this paper, we present a unified software framework that combines sensor and navigational processing for autonomous driving. Our framework is modular and scalable whereby the use of protocol buffers enables segregating each sensor and navigation subroutine individual classes, which can then be independently modified or tested. It is redesigned to replace the existing software on our Formula SAE vehicle, which we use for testing autonomous driving. Our testing results verify the suitability of our framework to be used for fully autonomous drives.

During the virtual development and experimental testing of advanced construction machinery, automation approaches for automated task execution can prove very valuable. In this paper, modeling and automation approaches for a hydraulic mini excavator are developed. In particular, a physical model for detailed system analysis and a simplified Hammerstein model for controller tuning are developed and validated with measurement data from the mini excavator. For attitude estimation of the excavator, inertial measurement units and extended Kalman filters are used in a sensor fusion framework. The control concept for automation is based on a virtual driver consisting of a state machine for task coordination as well as offset-free model predictive controllers (MPCs) for decentralized and robust tracking control of all motion axes. The constrained MPC optimization problems are solved in real time by means of the accelerated proximal gradient method. Experimental results from the mini excavator prove the developed control approach to be valuable for virtual development and automated testing during the commissioning of hydraulic machinery.

This article describes WAMbot, a multi-robot system designed to navigate, explore and map large-scale urban environments while performing visual object recognition. The system can be easily adapted and deployed into a variety of scenarios such as search and rescue or surveillance. The original WAMbot system was developed for the Multi Autonomous Ground-robotic International Challenge in 2010 (MAGIC 2010). It successfully passed a competitive down-selection process and placed 4th in the final challenge. Our ongoing work has been to modularize the WAMbot system further, and standardize it using the open source Robot Operating System (ROS) software framework. We describe the core components, or stacks, in the new WAMbot system including: 1) a large-scale mapping stack capable of building 500×500m global maps from over 20 robots in real-time, 2) an efficient, frontier-based multi-robot exploration stack, 3) a navigation stack including a centralized global planner with decentralized local navigation and 4) an intuitive graph-based visual object recognition pipeline.

We propose a novel approach for transferring evolved control systems from a simulated environment to a real robot. Multiple dynamic simulation systems are simultaneously employed to provide a valid range of simulation variance that can be exploited to generate robust controllers in a purely virtual environment. These controllers can then be directly transferred to a physical robot.

A key requirement for any mobile robot is achieving safe motions. This is especially difficult in a large dynamic environment where hostile objects must be avoided with incomplete information. This paper presents a novel hierarchical architecture that Team MAGICian developed, which integrates multi-vehicle tactical planning, path planning, collision avoidance, trajectory generation and dynamic vehicle control. The system is designed for a fleet of six Unmanned Ground Vehicles (UGVs) executing an Intelligence, Surveillance and Reconnaissance (ISR) mission in a large urban environment for the Multi Autonomous Ground-robotic International Challenge (MAGIC 2010).

This paper presents different approaches to fully autonomous exploration of unknown terrain for individual as well as groups of mobile robots. All algorithms have been tested on a simulation environment and verified on the real robot platform WAMbot, the West Australian entry for the MAGIC2010 robot challenge, based on Pioneer 3 AT robots, augmented by various sensor systems.

In ambulance location models, fleet size and ambulance location sites are two critical factors that emergency medical service (EMS) managers can control to ensure efficient delivery of the system. The ambulance relocation and dispatch policies that are studied in dynamic ambulance relocation models also significantly contribute to improving the response time of EMS. In this paper, we review dynamic ambulance relocation models from the perspective of dispatch policies. The connection between the reviewed ambulance dispatch policies and real-life policies is highlighted. Our ambulance model is based on the modified maximal covering location problem (MCLP). It is used to examine the commonly used dispatch policy and the proposed method of free-ambulance exploitation to further improve urgent call response time. Simulation results show that the proposed method can reduce the response time of urgent calls, especially during low-ambulance-supply period. We also compared the performance of EMS with and without reroute-enabled dispatch.

The rough-fuzzy hybridization scheme has become of research interest in a variety of areas over the past decade. The present paper proposes a general framework for adaptive T-S type rough-fuzzy inference systems (ARFIS) for many practical applications. Rough set theory is utilized to reduce the number of attributes and to obtain a minimal set of decision rules based on input-output data sets. A T-S type fuzzy inference system is constructed by the automatic generation of membership functions and rules by the fuzzy c-means clustering algorithm and the rough set approach, respectively. The generated T-S type rough-fuzzy inference system is then adjusted by the least squares fit and the conjugate gradient descent algorithm towards better performance with a validity checking for the generated minimal set of rules. The proposed framework of ARFIS is able to reduce the number of rules which increases exponentially when more input variables are involved and also to assess the validity of the minimized decision rules. The performance of the proposed framework of ARFIS is compared with other existing approaches in a variety of application areas and shown to be very competitive.

Evolutionary algorithms have often been applied to evolve controllers for robot locomotion. In an attempt to imitate biological systems, most previous approaches have utilized neural networks and central pattern generators to construct the controllers. In contrast to the conventional approach, control points for Hermite splines are evolved, rather than neuron weightings. A spline based control system has the advantages that it is simple to implement, requires little processing power, and the complexity of the controller can easily be altered. Initially limiting the splines control point parameters allows for faster evolution of the initial gait, and by progressively adding extra parameters the initial gait can be refined to produce an optimized final gait. This provides the robot designer with an early approximate gait for the robot, in less time than required to evolve a full control sequence by other means.

EyeSim is a simulation system for multiple interacting mobile robots. Environment modeling is 2 1/2 D, while 3D images are generated matching each robots camera view of the scene. The simulation system is implemented as a library, which is being linked to the robot application program. The system integrates the robot console with text and graphics I/O, sensors, and actuators at the level of a velocity/position-control driving interface.

We present the tool Retro (Register-Transfer Object Hardware Simulator) for simulating digital hardware systems at register-transfer level. The tool has been implemented in Java and is used for educational purposes. It provides simulation primitives for combinatorial gates, multiplexers, function units, clocks, registers, memory and I/O units. A system is constructed by picking and placing components from the menu to the canvas and interconnecting them. Simulation is then performed either in a stepwise or continuous fashion. Simulated circuits can be either synchronous or asynchronous and can range from simple examples to complete CPUs with programs in ROM and data in RAM.

Parallaxis-III is an architecture-independent data parallel programming language based on Modula-2. It has been designed for teaching data parallel concepts and is in use at a large number of institutions. Compilers exist for data parallel systems, as well as for a sequential simulation system. A data parallel graphics debugger allows efficient source level analysis for parallel programs.

Parallaxis is a programming language for massively parallel single instruction-multiple data (SIMD) systems, based on Modula-2. There are only a small number of additional constructs to handle parallel data (vectors) and data exchange among processors or between the front-end and back-end. Parallaxis helps to solve parallel problems in a natural way and does not require special skills. The major language constructs are described and a number of sample programs are given together with their simulated processor element PE-load and efficiency values. Parallaxis is available as a simulation system which is chiefly used in universities for instructional purposes. However, a compiler for the massively parallel MasPar computer system has been finished, and a compiler for the Connection Machine is being developed.>

No abstract available.