Multi-task missions for unmanned aerial vehicles (UAVs) involving inspection and landing tasks are challenging for novice pilots due to the difficulties associated with depth perception and the control interface. We propose a shared autonomy system, alongside supplementary information displays, to assist pilots to successfully complete multi-task missions without any pilot training. Our approach comprises of three modules: (1) a perception module that encodes visual information onto a latent representation, (2) a policy module that augments pilot’s actions, and (3) an information augmentation module that provides additional information to the pilot. The policy module is trained in simulation with simulated users and transferred to the real world without modification in a user study ( \(n=29\) ), alongside alternative supplementary information schemes including learnt red/green light feedback cues and an augmented reality display. The pilot’s intent is unknown to the policy module and is inferred from the pilot’s input and UAV’s states. The assistant increased task success rate for the landing and inspection tasks from 16.67% and 54.29%, respectively, to 95.59% and 96.22%. With the assistant, inexperienced pilots achieved similar performance to experienced pilots. Red/green light feedback cues reduced the required time by 19.53% and trajectory length by 17.86% for the inspection task, where participants rated it as their preferred condition due to the intuitive interface and providing reassurance. This work demonstrates that simple user models can train shared autonomy systems in simulation, and transfer to physical tasks to estimate user intent and provide effective assistance and information to the pilot.

This paper extends our previous work on a pneumatic bending module and presents two more modules for rotational and translational motions. In these modules, antagonistic chambers enveloped by rigid shells are adopted to realize bidirectional actuation, and they are characterized by safe actuation, enhanced torque/force output, independent stiffness tuning, and real-time position control. Due to their mechanical modularity, they can be conveniently assembled into robotic systems with multiple degrees of freedom (DoFs) according to different requirements. A complete workflow is presented including the module design, fabrication, theoretical modelling, controller design, and experimental validation. A reconfigurable robotic arm with high dexterity is also assembled using these modules, demonstrating the effectiveness of the proposed modules to develop robotic systems for safe, forceful, and precise tasks.

This paper describes a new type of bending module inspired, in part, by the musculoskeletal structure of the lobster leg joint. The bending module proposed combines enhanced torque output, reconfigurability in assembling, safe compliant actuation, and accurate control on its mechanical performance. In this module, antagonistic soft chambers are enveloped by exoskeleton shells, and the bending angle and the stiffness can be independently adjusted by controlling the input pressure in the two chambers. Theoretical models are developed to characterize the relationships between the input pressure, bending angle, and stiffness, and a controller for angle control and stiffness tuning is constructed with experimental validation. The fabricated module can reach the maximum torque output of 109.7 N•mm under 40 kPa and the stiffness range from 40 to 220 N•mm / rad, demonstrating its capacity to fulfill both safe interactions and forceful tasks.

There are many potential applications that require flying robots to navigate through tunnel-like environments, such as inspections of small railway culverts and mineral mappings of mining tunnels. Nevertheless, those environments present many challenges for quadrotors to navigate through. The aerodynamic disturbances created from the fluid interaction between the propellers' downwash and the surrounding surfaces of the environment, as well as longitudinal wind gusts, add hardship in stabilising the vehicle while the restricted narrow space increases the risk of collision. Furthermore, poor visibility and dust blown by the downwash make vision-based localisation extremely difficult. This paper presents a cross-sectional localisation system using Hough Scan Matching and a simple kinematic Kalman filter. Using the estimated state information, an integral backstepping controller is implemented which enables quadrotors to robustly fly in tunnel-like confined environments. A semi-autonomous system is proposed with self-stabilisation in the vertical and lateral axes while a pilot provides commands in the longitudinal direction. The results of a series of experiments in a simulated tunnel show that the proposed system successfully hovered itself and tracked various trajectories in a cross-sectional area without the aid of any external sensing or computing system.

Relative localization is a core problem for swarm robotics since each swarm node must determine where neighboring robots are located to accomplish cooperative tasks such as formation control. In this paper we present a system that performs relative localization between a stationary marker and a flying robot in real-time. Relative distances are obtained using ultra-wide band radio, while a low-cost webcam provide angle measurements. To achieve the latter, we employed the Camshift algorithm and a Kalman filter. We tested our system outdoors during daylight using centimeter-accuracy GPS measurements as ground truth. Three data sets have been collected from a series of experiments and it shows that errors in estimated relative positions are between +/-0.190 m on the x-East axis and +/-0.291 m on the z-North axis at 95% confidence level.

In this paper we present a new 3D line alignment technique that can be used in limited resourced MAVs for performing loop closures as well as mutual localisation between MAVs. We identify pairs of 3D line matches from RGB-D frames and find the optimal transformation which aligns these two line sets onto each other in order to find the corresponding relative pose. The alignment of the two 3D line sets are achieved by converting each line match into two matching point pairs and then subjecting them to a least squares minimization process. As we only maintain line features extracted from key frames, our method does not require large memory, high processing power nor a high communication bandwidth between robots. We validate our 3D line alignment technique by performing loop closures and mutual localisation on real-world datasets.

We present a computationally efficient RGB-D based pose estimation solution for less computationally resourced MAVs, which are ideally suited as members in a swarm. Our approach applies the sufficient statistics derived for a least-squares problem to our problem context. RANSAC-based outlier detection in aligning corresponding feature points is a time consuming operation in visual pose estimation. The additive nature of the used sufficient statistics significantly reduces the computation time of the RANSAC procedure since the pose estimation in each test loop can be computed by reusing previously computed sufficient statistics. This eliminates the need for recomputing estimates from scratch each time. A simpler hypotheses testing method gave similar performance in terms of speed but less accurate than our proposed method. We further increase the efficiency by reducing the problem size to four dimensions using attitude data from an Attitude and Heading Reference System (AHRS). Using a real-world dataset, we show that our algorithm saves up to 94% of computation time for the RANSAC-based procedure in pose estimation while improving the accuracy.

This paper presents recent advances in multiagent sensing and operation in dynamic environments. Technology trends point towards a fusion of wireless sensor networks with robotic swarms of mobile robots. In this paper, we discuss the coordination and collaboration between networked robotic systems, featuring algorithms for cooperative operations such as unmanned aerial vehicles (UAVs) swarming. We have developed cooperative actions of groups of agents such as probabilistic pursuit-evasion game for search and rescue operations, protection of resources, and security applications. We have demonstrated a hierarchical system architecture which provides wide-range sensing capabilities to unmanned vehicles through spatially deployed wireless sensor networks, highlighting the potential collaboration between wireless sensor networks and unmanned vehicles. This paper also includes a short review of our current research efforts in heterogeneous sensor networks, which is being evolved into mobile sensor networks with swarm mobility. In a very essential way, this represents the fusion of mobility of ensembles with the network embedded systems, the robotic sensor web.

Proposes an impedance control method with impedance modulation strategy for biped robot locomotion. The constrained leg as well as the free leg is controlled by the impedance control. To reduce the magnitude of an impact with the ground and guarantee a stable footing when a foot contacts with the ground, the paper proposes to increase the damping of the leg drastically and to modify the reference trajectory of the leg. Computer simulations with a 3-DOF environment model with a combination of a nonlinear and linear compliant contact model show that the proposed controller is superior to computed-torque controllers in the impact regulation and stability robustness in locomotion.