Underwater autonomous robotic operations require online localization and 3D mapping. Because of the absence of absolute positioning underwater, these tasks strongly rely on embedded sensors, including proprioceptive or navigation sensors - which can be fused for an odometry, - and exteroceptive sensors. One of the most popular exteroceptive sensors for underwater is the imaging sonar, which emits a large fan-shaped acoustic signal and estimates the position of the surrounding obstacles from a measure of the reflected signal. This paper addresses underwater online localization and 3D mapping using a forward looking, wide-aperture imaging sonar and vehicle's intrinsic navigation estimates. We introduce 3DSSDF (3D Sonar Reconstruction Using Signed Distance Functions), a new localization and 3D mapping algorithm based on signed distance functions, which is evaluated in simulation and on real data, in man-made and natural environments. Comparisons to reference trajectories and maps demonstrate that, in our tests, 3DSSDF efficiently corrects navigation drift and that trajectory and map accuracy is always below 1 m and below 1% of the distanced travelled, which can be sufficient for the safe inspection of natural or artificial underwater structures.

Simulations are highly valuable in marine robotics, offering a cost-effective and controlled environment for testing in the challenging conditions of underwater and surface operations. Given the high costs and logistical difficulties of real-world trials, simulators capable of capturing the operational conditions of subsea environments have become key in developing and refining algorithms for remotely-operated and autonomous underwater vehicles. This paper highlights recent enhancements to the Stonefish simulator, an advanced open-source platform supporting development and testing of marine robotics solutions. Key updates include a suite of additional sensors, such as an event-based camera, a thermal camera, and an optical flow camera, as well as, visual light communication, support for tethered operations, improved thruster modelling, more flexible hydrodynamics, and enhanced sonar accuracy. These developments and an automated annotation tool significantly bolster Stonefish's role in marine robotics research, especially in the field of machine learning, where training data with a known ground truth is hard or impossible to collect. https://github.com/patrykcieslak/stonefish

Autonomous motion planning is critical for efficient and safe underwater manipulation in dynamic marine environments. Current motion planning methods often fail to effectively utilize prior motion experiences and adapt to real-time uncertainties inherent in underwater settings. In this paper, we introduce an Adaptive Heuristic Motion Planner framework that integrates a Heuristic Motion Space (HMS) with Bayesian Networks to enhance motion planning for autonomous under-water manipulation. Our approach employs the Probabilistic Roadmap (PRM) algorithm within HMS to optimize paths by minimizing a composite cost function that accounts for distance, uncertainty, energy consumption, and execution time. By leveraging HMS, our framework significantly reduces the search space, thereby boosting computational performance and enabling real-time planning capabilities. Bayesian Networks are utilized to dynamically update uncertainty estimates based on real-time sensor data and environmental conditions, thereby refining the joint probability of path success. Through extensive simulations and real-world test scenarios, we showcase the advantages of our method in terms of enhanced performance and robustness. This probabilistic approach significantly advances the capability of autonomous underwater robots, ensuring optimized motion planning in the face of dynamic marine challenges.

This study introduces MarineGym, a high-performance reinforcement learning platform tailored for underwater robotics. It aims to address the limitations of existing underwater simulation environments in terms of reinforcement learning compatibility, training efficiency, and standardized benchmarking. MarineGym integrates a proposed GPU-accelerated hydrodynamic plugin based on Isaac Sim, achieving a rollout speed of 250,000 frames per second on a single NVIDIA RTX 3060 GPU. It also provides five models of unmanned underwater vehicles, multiple propulsion systems, and a set of predefined tasks covering core underwater control challenges. Additionally, the domain randomization toolkit allows flexible adjustments of the simulation and task parameters during training to improve the Sim2Real transfer. Further benchmark experiments demonstrate that MarineGym improves training efficiency over existing platforms and supports robust policy adaptation under various perturbations in the marine environment. We expect this platform to drive further advancements in RL research for underwater robotics. For more details about MarineGym and its applications, please visit our project page: https://marine-gym.com/.

Accurate quadcopter navigation under windy conditions remains challenging for traditional control methods, especially in the presence of unpredictable wind gusts and strict navigational constraints. This paper evaluates Deep Reinforcement Learning (DRL) based controllers under such conditions, analysing the impact of wind domain randomisation, multi-goal training, enhanced state representations with explicit wind information, and the use of temporal data to capture affecting dynamics over time. Experiments in the AirSim simulator across four trajectories — evaluated under both no-wind and windy conditions — demonstrate that DRL-based controllers outperform classical methods, particularly under stochastic wind disturbances. Moreover, we show that training a DRL agent with domain randomisation improves robustness against wind but reduces efficiency in no-wind scenarios. However, incorporating wind information into the agent’s state space enhances robustness without sacrificing performance in wind-free settings. Furthermore, training with stricter waypoint constraints emerges as the most effective strategy, leading to precise trajectories and improved generalisation to wind disturbances. To further interpret the learned policies, we apply Shapley Additive explanations analysis, revealing how different training configurations influence the agent’s feature importance. These findings underscore the potential of DRL-based neural controllers for resilient autonomous aerial systems, highlighting the importance of structured training strategies, informed state representations, and explainability for real-world deployment.

This study presents a pioneering effort to replicate human neuromechanics experiments within a virtual environment utilising a digital human model. By employing MyoSuite, a state-of-the-art human motion simulation platform enhanced by Reinforcement Learning (RL), multiple types of impedance identification experiments of human elbows were replicated on a digital musculoskeletal model. We compared the motor control capability of an RL agent with that of an actual human elbow in terms of the impedance identified through torque perturbation. The findings reveal that the RL agent exhibits higher elbow impedance to stabilise the target elbow motion under perturbation than a human does. It is likely due to the shorter reaction time and superior sensory capabilities of the RL agent. This study serves as a preliminary exploration into the potential of human digital twins for neuromechanics experiments. An RL-controlled digital twin with the musculoskeletal structure of the human body is expected to be useful in validating rehabilitation techniques before experiments on real human subjects.

In robotics, occupancy grids serve as required repositories of information about the environment in numerous applications. One such critical application is Simultaneous Localization and Mapping (SLAM), where robots dynamically scan and explore their surroundings while in motion. In the context of extended-duration missions, it becomes imperative to confront the complexities linked to the expansion of occupancy grids as well as handling loop closure detection. These challenges primarily revolve around two key aspects: enabling the seamless expansion of the map on multiple occasions, thus avoiding the need to map smaller regions in numerous separate missions, and ensuring real-time updates to the map to sustain the robot’s knowledge base and enhance its responsiveness. To address these challenges, we introduce an innovative map called Frustum Accelerated GPU-Based Grid Map (FRAGG-Map). This map adopts a highly parallelizable 3D grid structure and leverages the power of CUDA kernels to facilitate efficient insertion of point-clouds and enables real-time updates of the map. FRAGG-Map identifies the portions of the map that require updates and utilises the GPU to update them, significantly enhancing computational performance. Our results show that FRAGG-Map can run 31 times faster than OctoMap, significantly outperforming state-of-the-art methods.

Open ad hoc teamwork is the problem of training a single agent to efficiently collaborate with an unknown group of teammates whose composition may change over time. A variable team composition creates challenges for the agent, such as the requirement to adapt to new team dynamics and dealing with changing state vector sizes. These challenges are aggravated in real-world applications in which the controlled agent only has a partial view of the environment. In this work, we develop a class of solutions for open ad hoc teamwork under full and partial observability. We start by developing a solution for the fully observable case that leverages graph neural network architectures to obtain an optimal policy based on reinforcement learning. We then extend this solution to partially observable scenarios by proposing different methodologies that maintain belief estimates over the latent environment states and team composition. These belief estimates are combined with our solution for the fully observable case to compute an agent's optimal policy under partial observability in open ad hoc teamwork. Empirical results demonstrate that our solution can learn efficient policies in open ad hoc teamwork in fully and partially observable cases. Further analysis demonstrates that our methods' success is a result of effectively learning the effects of teammates' actions while also inferring the inherent state of the environment under partial observability.

This paper presents the design of a novel expert system for robotic manipulators performing sanding tasks on work surfaces. The expert system adjusts the velocity of the robotic manipulator based on the observed surface quality. These observation are obtained by an analysis of the raw force data provided by a force-torque sensor at the end-effector level. The expert system consists of two governing control laws that act in parallel, a variable velocity generation law and a pose regulation-based law. The variable velocity law regulates the velocity of the manipulator along a set path, in the tangent direction, based on an analysis of the frequency and amplitude of the force signal generated during the sanding process. The pose regulation-based law drives the manipulator in the bi-normal and rotational direction, ensuring the manipulators remain on the sanding path with the desired orientation. The proposed strategy is experimentally evaluated using the UR5e collaborative robotic manipulator sanding wood and metal panels. The obtained results show that such an approach is beneficial to ensure accurate contact between the sanding tool and the working environment, robust path tracking, and smart sanding.