Long-term monitoring and exploration of extreme environments, such as underwater storage facilities, is costly, labor-intensive, and hazardous. Automating this process with low-cost, collaborative robots can greatly improve efficiency. These robots capture images from different positions, which must be processed simultaneously to create a spatio-temporal model of the facility. In this paper, we propose a novel approach that integrates data simulation, a multi-modal deep learning network for coordinate prediction, and image reassembly to address the challenges posed by environmental disturbances causing drift and rotation in the robots' positions and orientations. Our approach enhances the precision of alignment in noisy environments by integrating visual information from snapshots, global positional context from masks, and noisy coordinates. We validate our method through extensive experiments using synthetic data that simulate real-world robotic operations in underwater settings. The results demonstrate very high coordinate prediction accuracy and plausible image assembly, indicating the real-world applicability of our approach. The assembled images provide clear and coherent views of the underwater environment for effective monitoring and inspection, showcasing the potential for broader use in extreme settings, further contributing to improved safety, efficiency, and cost reduction in hazardous field monitoring.

Aquatic robots require an accurate and reliable localization system to navigate autonomously and perform practical missions. Kalman filters (KFs) and their variants are typically used in aquatic robots to combine sensor data. The two critical drawbacks of KFs are the requirement for skilled tuning of several filter parameters and the fact that changes to how the Inertial Measurement Unit (IMU) is oriented necessitate modifying the filter. To overcome those problems, this paper presents a novel method of fusing sensor data from a Doppler Velocity Log (DVL) and IMU using an adaptive nonlinear estimator to provide dead reckoning localization for a small autonomous surface vehicle. The proposed method has only one insensitive tuning parameter and is agnostic to the configuration of the IMU. The system was validated using a small ASV in a 2.4×3.6×2.4 m water tank, with a motion capture system as ground truth, and was evaluated against a state-of-the-art method based on KFs. Experiments showed that the average drift error of the nonlinear filter was 0.16 m (s.d. 0.06 m) compared to 0.15 m (s.d. 0.05 m) for the state of the art, meaning that the benefits in terms of tuning and flexible configuration do not come at the expense of performance.

We address the unique challenge of vibration suppression for a flexible long-reach robotic manipulator system, namely, the through-wall deployment (TWD) system that is used in nuclear environments. This paper proposes a novel dynamics-based trajectory optimization approach, which minimizes both the acceleration and the jerk at the manipulator’s joints, as well as the vibrations of the flexible long-reach boom where the manipulator’s base is mounted. Firstly, we create an integrated model for the system dynamics based on the knowledge of the robotic manipulator and the acceleration data from the vibration tests. We then develop an original procedure for generating the high-order polynomial trajectory that guarantees the zero-boundary condition for a flexible number of optimization parameters and waypoints. Following the simulation of a multi-objective optimization scheme, the optimized trajectory is experimentally validated on the practical TWD system with around 28% vibration reduction on average compared to the benchmark. Importantly, this reduction is achieved without compromising on the average speed of motion. The methodology is transferable to a wider range of flexible robotic manipulator systems with similar characteristics.

Driven by advancements in Industry 4.0, the Internet of Things (IoT), digital twins (DT), and cyber–physical systems (CPS), there is a growing interest in the digitalizing of asset integrity management. CPS, in particular, is a pivotal technology for the development of intelligent and interconnected systems. The design of a scalable, low-latency communication network with efficient data management is crucial for connecting physical and digital twins in heterogeneous robot fleets. This paper introduces a generalised cyber–physical architecture aimed at governing an autonomous multi-robot ecosystem via a scalable communication network. The objective is to ensure accurate and near-real-time perception of the remote environment by digital twins during robot missions. Our approach integrates techniques such as downsampling, compression, and dynamic bandwidth management to facilitate effective communication and cooperative inspection missions. This allow for efficient bi-directional data exchange between digital and physical twins, thereby enhancing the overall performance of the system. This study contributes to the ongoing research on the deployment of cyber–physical systems for heterogeneous multi-robot fleets in remote inspection missions. The feasibility of the approach has been demonstrated through simulations in a representative environment. In these experiments, a fleet of robots is used to map an unknown building and generate a common 3D probabilistic voxel-grid map, while evaluating and managing bandwidth requirements. This study represents a step forward towards the practical implementation of continuous remote inspection with multi-robot systems through cyber–physical infrastructure. It offers potential improvements in scalability, interoperability, and performance for industrial asset monitoring.

This article is concerned with the problem of planning optimal maneuver trajectories and guiding the mobile robot toward target positions in uncertain environments for exploration purposes. A hierarchical deep learning-based control framework is proposed which consists of an upper level motion planning layer and a lower level waypoint tracking layer. In the motion planning phase, a recurrent deep neural network (RDNN)-based algorithm is adopted to predict the optimal maneuver profiles for the mobile robot. This approach is built upon a recently proposed idea of using deep neural networks (DNNs) to approximate the optimal motion trajectories, which has been validated that a fast approximation performance can be achieved. To further enhance the network prediction performance, a recurrent network model capable of fully exploiting the inherent relationship between preoptimized system state and control pairs is advocated. In the lower level, a deep reinforcement learning (DRL)-based collision-free control algorithm is established to achieve the waypoint tracking task in an uncertain environment (e.g., the existence of unexpected obstacles). Since this approach allows the control policy to directly learn from human demonstration data, the time required by the training process can be significantly reduced. Moreover, a noisy prioritized experience replay (PER) algorithm is proposed to improve the exploring rate of control policy. The effectiveness of applying the proposed deep learning-based control is validated by executing a number of simulation and experimental case studies. The simulation result shows that the proposed DRL method outperforms the vanilla PER algorithm in terms of training speed. Experimental videos are also uploaded, and the corresponding results confirm that the proposed strategy is able to fulfill the autonomous exploration mission with improved motion planning performance, enhanced collision avoidance ability, and less training time.

The development of mobile robot platforms for inspection has gained traction in recent years. However, conventional mobile robots are unable to address the challenge of operating in extreme environments where the robot is required to traverse narrow gaps in highly cluttered areas with restricted access, typically through narrow ports. This article presents MIRRAX, a robot designed to meet these challenges by way of its reconfigurable capability. Controllers for the robot are detailed, along with an analysis on the controllability of the robot given the use of mecanum wheels in a variable configuration. Characterization on the robot's performance identified suitable configurations for operating in narrow environments. The experimental validation of the robot's controllability shows good agreement with the theoretical analysis and the capability to address the challenges of accessing entry ports as small as 150-mm diameter, as well as navigating through cluttered environments. This article also presents results from a deployment in a Magnox facility at the Sellafield nuclear site in the U.K.—the first robot to ever do so, for remote inspection and mapping.

ACEFusion is the first 3D reconstruction system able to capture the geometry and semantics of dynamic scenes using an RGB-D camera in real-time on a robotic computing platform. Harnessing the hardware accelerators of an Nvidia Jetson AGX Xavier, the system uses heterogeneous computing to achieve 30 FPS under a 30W power budget. Using a data-parallel design, we perform most image computation on the dedicated hardware accelerators, freeing the general purpose cores and GPU to process 3D geometry. To further increase efficiency, we employ a hybrid geometry representation with octrees for static-semantic reconstruction and surfels for dynamic reconstruction. ACEFusion achieves competitive results on standard benchmarks while efficiently performing a more complex overall task than existing SLAM techniques. Figure. 1 shows the output of our system on a dynamic sequence.

Reconfigurable mobile robots are well suited for inspection tasks in legacy nuclear facilities where access is restricted and the environment is often cluttered. A reconfig-urable snake robot, MIRRAX, has previously been developed to investigate such facilities. The joints used for the robot's reconfiguration introduce additional constraints on the robot's control, such as balance, on top of the existing actuator and collision constraints. This paper presents a set-point controller for MIRRAX using vector-field inequalities to enforce hard constraints on the robot's balance, actuator limits, and collision avoidance in a single quadratic programming formulation. The controller has been evaluated in simulation and early experiments in some scenarios. The results show that the controller generates feasible control inputs that enable the robot to retain its balance while moving with less oscillation and operating within the actuation and collision constraints.

There is a growing interest in the use of robotics and AI in the nuclear industry, however it is important to ensure these systems are ethically grounded, trustworthy and safe. An emerging technique to address these concerns is the use of explainability. In this paper we present the results of an interview study with nuclear industry experts to explore the use of explainable intelligent systems within the field. We interviewed 16 participants with varying backgrounds of expertise, and presented two potential use cases for evaluation; a navigation scenario and a task scheduling scenario. Through an inductive thematic analysis we identified the aspects of a deployment that experts want to know from explainable systems and we outline how these associate with the folk conceptual theory of explanation, a framework in which people explain behaviours. We established that an intelligent system should explain its reasons for an action, its expectations of itself, changes in the environment that impact decision making, probabilities and the elements within them, safety implications and mitigation strategies, robot health and component failures during decision making in nuclear deployments. We determine that these factors could be explained with cause, reason, and enabling factor explanations.

Digital twins tend to be on the way of becoming the future of robots, artificial intelligence, and IoT devices, especially in industrial applications. Creating a digital twin solution will help to solve challenges faced in managing the tasks in their operating environment since it enables an integrated solution for the framework that seamlessly connects to their physical counterparts with the latest internet technologies. In this study, we aim to develop a synchronous, situational-aware, bi-directional/multi-directional digital twin platform that allows us to perceive real-time/simultaneous flow of sensory environmental data and remotely operate robots through digital twins for continuous inspection. To achieve this aim, we will investigate the interoperability issues in robot teleoperation with 3D mapping of a remote unknown environment case scenario.

In recent years, the inspection of extreme environments using mobile robots has gained traction, as robots are able to mitigate the risk placed on humans and at times achieve what humans are unable to. In some scenarios, the robot is required to operate in cluttered environments with highly restricted access through 150 mm diameter ports. The MIRRAX robot has been designed to meet these challenges with the capability of reconfiguring itself to both access environments and navigate through tightly spaced obstacles. The joints used for reconfiguration of the robot introduce additional challenges for path planning due to the significant changes that can occur between adjacent poses. This paper presents a global path planner for MIRRAX. A Voronoi diagram is first used to generate a sparse graph to represent the topology of the environment, which allows for fast, coarse path planning. The coarse path is then refined via a heuristic pose fitting routine to ensure that the path is both collision-free and reduce unnecessary joint angle changes. The planner has been evaluated in simulation, demonstrating the feasibility of generating collision-free paths through narrow pathways for a reconfigurable robot.

A long short-term memory neural network is used to provide a system model that captures the temporal-dynamics of a holonomic, fully-actuated aquatic surface vehicle. As is true in many fields, new developments in robotics often are made in simulation first before being applied to real systems. To simulate an aquatic or aerial robot, a dynamic system model of the robot is required. The more representative the dynamic model is of the real robot, the smaller the simulation-to-reality gap becomes. The performance of the neural network is compared against a classical parametric model, where coefficients of the parametric model were identified using the same data that was used to train the neural network. The results show that the neural network consistently outperforms the classical parametric model and significantly reduces the error between real velocities and estimated velocities. The neural network also demonstrated the ability to capture complex hydrodynamic effects that were not captured in the parametric model. In addition to the performance improvements, the neural network method can be easily adapted to similarly actuated aquatic vehicles by simply retraining, whereas the classical approach would require manual selection of new equation terms. The neural network model that was created has been used in a vehicle simulation and is presently being used as a research tool.

Progress in the last decade has brought about significant improvements in the accuracy and speed of SLAM systems, broadening their mapping capabilities. Despite these advancements, long-term operation remains a major challenge, primarily due to the wide spectrum of perturbations robotic systems may encounter.Increasing the robustness of SLAM algorithms is an ongoing effort, however it usually addresses a specific perturbation. Generalisation of robustness across a large variety of challenging scenarios is not well-studied nor understood. This paper presents a systematic evaluation of the robustness of open-source state-of-the-art SLAM algorithms with respect to challenging conditions such as fast motion, non-uniform illumination, and dynamic scenes. The experiments are performed with perturbations present both independently of each other, as well as in combination in long-term deployment settings in unconstrained environments (lifelong operation).The detailed results (approx. 20,000 experiments) along with comprehensive documentation of the benchmarking tool for integrating new datasets and evaluating SLAM algorithms not studied in this work are available at https://robustslam.github.io/evaluation.

This paper proposes an intuitive human-swarm interaction framework inspired by our childhood memory in which we interacted with living ants by changing their positions and environments as if we were omnipotent relative to the ants. In virtual reality, analogously, we can be a super-powered virtual giant who can supervise a swarm of robots in a vast and remote environment by flying over or resizing the world, and coordinate them by picking and placing a robot or creating virtual walls. This work implements this idea by using Virtual Reality along with Leap Motion, which is then validated by proof-of-concept experiments using real and virtual mobile robots in mixed reality. We conduct a usability analysis to quantify the effectiveness of the overall system as well as the individual interfaces proposed in this work. The results reveal that the proposed method is intuitive and feasible for interaction with swarm robots, but may require appropriate training for the new end-user interface device.

Abstract Flocking is a social animals’ common behaviour observed in nature. It has a great potential for real-world applications such as exploration in agri-robotics using low-cost robotic solutions. In this paper, an extended model of a self-organised flocking mechanism using heterogeneous swarm system is proposed. The proposed model for swarm robotic systems is a combination of a collective motion mechanism with obstacle avoidance functions, which ensures a collision-free flocking trajectory for the followers. An optimal control model for the leader is also developed to steer the swarm to a desired goal location. Compared to the conventional methods, by using the proposed model, the swarm network has less requirement for power and storage. The feasibility of the proposed self-organised flocking algorithm is validated by realistic robotic simulation software.

When exploring an unknown environment, a mobile robot must decide where to observe next. It must do this whilst minimising the risk of failure, by only exploring areas that it expects to be safe. In this context, safety refers to the robot remaining in regions where critical environment features (e.g. terrain steepness, radiation levels) are within ranges the robot is able to tolerate. More specifically, we consider a setting where a robot explores an environment modelled with a Markov decision process, subject to bounds on the values of one or more environment features which can only be sensed at runtime. We use a Gaussian process to predict the value of the environment feature in unvisited regions, and propose an estimated Markov decision process, a model that integrates the Gaussian process predictions with the environment model transition probabilities. Building on this model, we propose an exploration algorithm that, contrary to previous approaches, considers probabilistic transitions and explicitly reasons about the uncertainty over the Gaussian process predictions. Furthermore, our approach increases the speed of exploration by selecting locations to visit further away from the currently explored area. We evaluate our approach on a real-world gamma radiation dataset, tackling the challenge of a nuclear material inspection robot exploring an a priori unknown area.

This work presents a practical method of obtaining a dynamic system model for small omnidirectional aquatic vehicles. The models produced can be used to improve vehicle localisation, aid in the design or tuning of control systems and facilitate the development of simulated environments. The use of a dynamic model for onboard real-time velocity prediction is of particular importance for aquatic vehicles because, unlike ground vehicles, fast and direct measurement of velocity using encoders is not possible. Previous work on model identification of aquatic vehicles has focused on large vessels that are typically underactuated and have low controllability in the sway direction. In this paper it is demonstrated that the procedure for identifying the model coefficients can be performed quickly, without specialist equipment and using only onboard sensors. This is of key importance because the dynamic model coefficients will change with the payload. Two different thrust allocation schemes are tested, one of which is a known method and another is proposed here. Validation tests are performed and the models are shown to be suitable for their intended applications. Significant reduction in model error is demonstrated using the novel thrust allocation method that is designed to avoid deadbands in the thruster responses.

As the SLAM research area matures and the number of SLAM systems available increases, the need for frameworks that can objectively evaluate them against prior work grows. This new version of SLAMBench moves beyond traditional visual SLAM, and provides new support for scene understanding and non-rigid environments (dynamic SLAM). More concretely for dynamic SLAM, SLAMBench 3.0 includes the first publicly available implementation of DynamicFusion, along with an evaluation infrastructure. In addition, we include two SLAM systems (one dense, one sparse) augmented with convolutional neural networks for scene understanding, together with datasets and appropriate metrics. Through a series of use-cases, we demonstrate the newly incorporated algorithms, visulation aids and metrics (6 new metrics, 4 new datasets and 5 new algorithms).

In this paper, we proposed a pheromone-based aggregation method based on the state-of-the-art BEECLUST algorithm. We investigated the impact of pheromone-based communication on the efficiency of robotic swarms to locate and aggregate at areas with a given cue. In particular, we evaluated the impact of the pheromone evaporation and diffusion on the time required for the swarm to aggregate. In a series of simulated and real-world evaluation trials, we demonstrated that augmenting the BEECLUST method with artificial pheromone resulted in faster aggregation times.

This paper presents the motion planning framework for a hexapod, based on advanced motions, for accessing challenging spaces, namely narrow pathways and large holes, both of which are surrounded by walls. The advanced motions, wall and chimney walking, utilise environment surfaces that are perpendicular to the ground plane to support the robot motion. Such techniques have not yet been studied in the literature. The hierarchical planning framework proposed here is an extension to existing approaches which have only considered ground walking where foothold contacts are confined to the ground plane. During the pre-processing phase of the 2.5D grid map, the motion primitives employed are assessed for each cell and stacked to the graph if valid. The A* algorithm is then used to find a path to the goal position. Following that, the path is post-processed to smoothen the motions and generate a continuous path. Footholds are then selected along the path. The framework has been evaluated in simulation on the custom-designed Corin hexapod. The resulting path enables access to areas that are previously thought to be inaccessible and reduces the travelling distance compared to previous studies.

Limitations with the on-board computational resources installed on untethered robots such as humanoids and mobile robots in general affects significantly the performance and capabilities of these machines. An approach to address this issue is to make use of the cloud robotics concept and take advantage of the extensive computational resources of the cloud. XBotCloud is a recently developed component of the XBot framework. It tackles the above challenges by introducing the tools and mechanisms to enable users and robots to exploit the computational resources of the cloud allowing the execution of services with low, soft or hard Real-Time execution/communication performance. The latter is ensured thanks to the functionality provided by the XBotCore Real-Time cross-robot software component of the XBot framework. XBotCloud addresses also one of the main challenges related with cloud robotics: security. To avoid remote attacks it takes advantage of the Amazon Web Services (AWS)Cloud Security and it uses an internal VPN Network to handle the connectivity between the robot and the cloud server. The full implementation of the framework is presented and its functionality is demonstrated in realistic tasks involving pipelines that mix the execution of cloud services with moderate execution time constraints and Real-Time modules running on the robot local control unit. XBotCloud performances and cross-robot flexibility are experimentally validated on two different robotic platforms, the WALK-MAN humanoid and the CENTAURO upper body/full-body.

Characterisation of nuclear environments is critical for long term operation and decommissioning. Laser Induced Breakdown Spectroscopy (LIBS) is an example of a scientific instrument that could be deployed to aid in characterisation of unknown environments. LIBS consists of a high intensity pulsed laser being focussed down onto a target to create a plasma, and optical emission from the plasma is then used to determine elemental composition of unknown materials. For robots deployed with these instruments in extreme environments, mission time can be limited by hazards present such as radiation. Once deployed a robot must be able to collect the best data possible whilst maximising operational runtime. We present a data quality based probabilistic approach to robot pose estimation to maximise data quality, by considering optimum sensor placement whilst avoiding harmful environmental features such as radiation for a fixed-focus laser diagnostic such as LIBS. This approach is able to determine optimum robot poses for arbitrary targets in 3D for arbitrary diagnostic mounting with respect to the robot. The approach is able to avoid obstacles and avoid occlusion of the target by said obstacles. This can be used as part of autonomous investigation and characterisation performed by mobile robots in hazardous environments.