Aerial manipulation for safe physical interaction with their environments is gaining significant momentum in robotics research. In this paper, we present a disturbance-observer-based safety-critical control for a fully actuated aerial manipulator interacting with both static and dynamic structures. Our approach centers on a safety filter that dynamically adjusts the desired trajectory of the vehicle's pose, accounting for the aerial manipulator's dynamics, the disturbance observer's structure, and motor thrust limits. We provide rigorous proof that the proposed safety filter ensures the forward invariance of the safety set—representing motor thrust limits—even in the presence of disturbance estimation errors. To demonstrate the superiority of our method over existing control strategies for aerial physical interaction, we perform comparative experiments involving complex tasks, such as pushing against a static structure and pulling a plug firmly attached to an electric socket. Furthermore, to highlight its repeatability in scenarios with sudden dynamic changes, we perform repeated tests of pushing a movable cart and extracting a plug from a socket. These experiments confirm that our method not only outperforms existing methods but also excels in handling tasks with rapid dynamic variations.

Vision sensors are extensively used for localizing a robot's pose, particularly in environments where global localization tools such as GPS or motion capture systems are unavailable. In many visual navigation systems, localization is achieved by detecting and tracking visual features or landmarks, which provide information about the sensor's relative pose. For reliable feature tracking and accurate pose estimation, it is crucial to maintain visibility of a sufficient number of features. This requirement can sometimes conflict with the robot's overall task objective. In this paper, we approach it as a constrained control problem. By leveraging the invariance properties of visibility constraints within the robot's kinematic model, we propose a real-time safety filter based on quadratic programming. This filter takes a reference velocity command as input and produces a modified velocity that minimally deviates from the reference while ensuring the information score from the currently visible features remains above a user-specified threshold. Numerical simulations demonstrate that the proposed safety filter preserves the invariance condition and ensures the visibility of more features than the required minimum. We also validated its real-world performance by integrating it into a visual simultaneous localization and mapping (SLAM) algorithm, where it maintained high estimation quality in challenging environments, outperforming a simple tracking controller.

Multi-task reinforcement learning (MTRL) offers a promising approach to improve sample efficiency and generalization by training agents across multiple tasks, enabling knowledge sharing between them. However, applying MTRL to robotics remains challenging due to the high cost of collecting diverse task data. To address this, we propose MT-Lévy, a novel exploration strategy that enhances sample efficiency in MTRL environments by combining behavior sharing across tasks with temporally extended exploration inspired by Lévy flight [1]. MT-Lévy leverages policies trained on related tasks to guide exploration towards key states, while dynamically adjusting exploration levels based on task success ratios. This approach enables more efficient state-space coverage, even in complex robotics environments. Empirical results demonstrate that MT-Lévy significantly improves exploration and sample efficiency, supported by quantitative and qualitative analyses. Ablation studies further highlight the contribution of each component, showing that combining behavior sharing with adaptive exploration strategies can significantly improve the practicality of MTRL in robotics applications.

Unsupervised skill discovery in reinforcement learning (RL) aims to learn diverse behaviors without relying on external rewards. However, current methods often overlook the periodic nature of learned skills, focusing instead on increasing the mutual dependency between states and skills or maximizing the distance traveled in latent space. Considering that many robotic tasks—particularly those involving locomotion—require periodic behaviors across varying timescales, the ability to discover diverse periodic skills is essential. Motivated by this, we propose Periodic Skill Discovery (PSD), a framework that discovers periodic behaviors in an unsupervised manner. The key idea of PSD is to train an encoder that maps states to a circular latent space, thereby naturally encoding periodicity in the latent representation. By capturing temporal distance, PSD can effectively learn skills with diverse periods in complex robotic tasks, even with pixel-based observations. We further show that these learned skills achieve high performance on downstream tasks such as hurdling. Moreover, integrating PSD with an existing skill discovery method offers more diverse behaviors, thus broadening the agent’s repertoire. Our code and demos are available at https://jonghaepark.github.io/psd

Aerial physical interaction (APhI) with a multirotor-based platform such as pushing a heavy object demands generation of a sufficiently large interaction force while maintaining the stability. Such requirement can cause rotor saturation, because the rotor thrust enlarged for interaction force may leave a reduced margin for attitude stabilization. We first design an H-shaped coaxial tiltrotor that can generate a sufficiently large interaction force than a conventional multirotor. We then propose an overall framework composed of high-level robust controller and low-level control allocation for the coaxial tiltrotor to ensure robustness against uncertain motion of the unknown interacting object and to overcome the saturation issue. To guarantee the robustness at all time, we design a controller based on a nonlinear disturbance observer (DOB). Then, we formulate a problem of computing low-level actuator inputs avoiding rotor saturation as a tractable nonlinear optimization problem, which can be solved real-time. The proposed framework is validated in extensive real-world experiments where the 3.3 kg tiltrotor successfully pushes a cart weighing up to 60 kg. An ablation study with the tiltrotor shows effectiveness of the proposed control allocation law in avoiding rotor saturation. Furthermore, a comparative experiment with a conventional multirotor shows failure in the same setting, which validates the use of the coaxial tiltrotor. An experimental video can be found athttps://youtu.be/Gdmcmoz_UjUNote to Practitioners—The motivation of this work is to enable heavy object manipulation, especially pushing operation, using a multirotor platform. To push a heavy object, which has not been treated in existing works, is challenging due to the presence of a large unknown interaction wrench and rotor saturation issue. To resolve this problem, we first build an H-shaped coaxial tiltrotor that can effectively utilize rotor thrusts to generate horizontal interaction force. Then, a robust controller is designed to address uncertainty in the interaction wrench for the tiltrotor. Finally, we propose an optimization-based control allocation to avoid rotor saturation. Real-world experiments including heavy object pushing and ablation study with respect to the control allocation law are conducted to show the effectiveness of the proposed framework. We validate the use of the coaxial tiltrotor in additional comparative experiments with a conventional multirotor which fails in pushing a 60 kg cart. The proposed framework can be utilized in various applications such as disaster relief operations, emergency rescue and aerial delivery.

Maintaining the visibility of the target is one of the major objectives of aerial tracking missions. This paper proposes a target-visible trajectory planning pipeline using quadratic programming (QP). Our approach can handle various tracking settings, including 1) single- and dual-target following and 2) both static and dynamic environments, unlike other works that focus on a single specific setup. In contrast to other studies that fully trust the predicted trajectory of the target and consider only the visibility of the target’s center, our pipeline considers error in target path prediction and the entire body of the target to maintain the target visibility robustly. First, a prediction module uses a sample-check strategy to quickly calculate the reachable areas of moving objects, which represent the areas their bodies can reach, considering obstacles. Subsequently, the planning module formulates a single QP problem, considering path homotopy, to generate a tracking trajectory that maximizes the visibility of the target’s reachable area among obstacles. The performance of the planner is validated in multiple scenarios, through high-fidelity simulations and real-world experiments.

In autonomous driving, a prior LiDAR map(PLM) is used as a powerful tool for correcting SLAM drift, but finding robust and accurate correspondences between cross modal sensors is a challenging problem. To address this cross-modality issue, this paper proposes a real-time plane-based stereo localization system with a PLM. In the proposed system, drift in visual pose estimation is eliminated through plane-based joint optimization and the registration module. Two types of planes are employed in this system: surfel, ensuring accuracy in a narrow domain, and global plane, providing robustness in a wide area. For accurate and robust matching between the visual map point and PLM, surfels and global planes in PLM are utilized collaboratively based on point-to-PLM distance. To reduce the computational cost of the registration module, a plane-to-plane drift estimation module is proposed. The performance of the proposed system is extensively validated across synthetic simulation and real-world indoor and outdoor datasets. We validate the effectiveness of each module through ablation studies and also assess the robustness against error that may exist in PLM and initial pose. In most of the validation, the proposed system shows more accurate and robust performance compared to the state-of-the-art methods.

Generative modeling of complex behaviors from labeled datasets has been a longstanding problem in decision-making. Unlike language or image generation, decision-making requires modeling actions – continuous-valued vectors that are multimodal in their distribution, potentially drawn from uncurated sources, where generation errors can compound in sequential prediction. A recent class of models called Behavior Transformers (BeT) addresses this by discretizing actions using k-means clustering to capture different modes. However, k-means struggles to scale for high-dimensional action spaces or long sequences, and lacks gradient information, and thus BeT suffers in modeling long-range actions. In this work, we present Vector-Quantized Behavior Transformer (VQ-BeT), a versatile model for behavior generation that handles multimodal action prediction, conditional generation, and partial observations. VQ-BeT augments BeT by tokenizing continuous actions with a hierarchical vector quantization module. Across seven environments including simulated manipulation, autonomous driving, and robotics, VQ-BeT improves on state-of-the-art models such as BeT and Diffusion Policies. Importantly, we demonstrate VQ-BeT’s improved ability to capture behavior modes while accelerating inference speed 5× over Diffusion Policies. Videos can be found https://sjlee.cc/vq-bet/

Safety verification in motion planning is known to be computationally burdensome, despite its importance in robotics. In this paper, we investigate the behavior of safe receding horizon motion planners when the update interval becomes infinitesimal. By requiring the trajectory parameters to evolve continuously in time, the trajectory optimization problem is reformulated into a time-derivative form, whose decision variables are their rate of change. This results in a quadratic programming problem which directly provides safe input, and can be regarded as a real-time safety filter. The input expressivity is also enhanced by leveraging the differentiable structure of the parameter space. The proposed safety filter is experimentally validated using a wheeled ground robot in obstacle-cluttered environments. The result shows that the safety filter is capable of generating safe inputs in real-time, while addressing hundreds of constraints simultaneously.

Aerial unperching of multirotors has received little attention as opposed to perching that has been investigated to elongate operation time. This study presents a new aerial robot capable of both perching and unperching autonomously on/from a ferromagnetic surface during flight, and a switching controller to avoid rotor saturation and mitigate overshoot during transition between free-flight and perching. To enable stable perching and unperching maneuvers on/from a vertical surface, a lightweight (≈ 1 kg), fully actuated tiltrotor that can hover at 90◦pitch angle is first developed. We design a perching/unperching module composed of a single servomotor and a magnet, which is then mounted on the tiltrotor. A switching controller including exclusive control modes for transitions between free-flight and perching is proposed. Lastly, we propose a simple yet effective strategy to ensure robust perching in the presence of measurement and control errors and avoid collisions with the perching site immediately after unperching. We validate the proposed framework in experiments where the tiltrotor successfully performs perching and unperching on/from a vertical surface during flight. We further show effectiveness of the proposed transition mode in the switching controller by ablation studies where large overshoot and even collision with a perching site occur. To the best of the authors’ knowledge, this work presents the first autonomous aerial unperching framework using a fully actuated tiltrotor.

Autonomous aerial manipulation requires the capability to handle inevitable dynamic changes during physical interaction. Previously, very few studies have addressed the stability and transient performance of the scenarios involving abrupt changes in dynamics. This paper proposes a hybrid controller enhancing transient performance for an aerial manipulator extracting an object wedged in a static structure. This task incurs a significant jump in the interaction force on the end-effector so that the analysis using the concept of hybrid dynamical systems is required. To demonstrate the dynamic characteristics of the object-extracting aerial manipulator, we derive the dynamic equations for two flight modes, i.e., free-flight and object-extracting, and the rule of state jumps. Also, we design control strategies which enhance the transient performance during flight mode transition. Then, the stability of the proposed control law is proven, and the overshoot reduction after the object extraction is analyzed. To show the improved performance, we conduct plug-pulling experiments with a quadrotor-based aerial manipulator using the proposed controller and two different existing controllers. The comparative results confirm that our controller enables the aerial manipulator to maintain its stability after the flight mode transition and shows the best transient performance in overshoot minimization among three controllers.Note to Practitioners—The motivation for this article is the desire to prevent unexpected collisions between obstacles and an aerial manipulator after the vehicle extracts a wedged object from a static structure. To resolve this problem, we present a hybrid control method for such tasks while avoiding an excessive overshoot after the extraction. This method can be utilized in tasks involving abrupt changes in the dynamic model such as retrieving a device attached to a tall structure, reclaiming an object in disaster recovery, or pulling a plug out of a socket. Unlike existing methods, the proposed method simultaneously considers the stabilityandthe initial overshoot right after pulling the object out of the structure, by employing a disturbance observer (DOB) in a hybrid control structure. It can be utilized for the aerial manipulator to avoid collision in a narrow space or to keep it in a safe operation envelope even though the task requires producing a relatively large pulling force.

While reinforcement learning (RL) has achieved great success in acquiring complex skills solely from environmental interactions, it assumes that resets to the initial state are readily available at the end of each episode. Such an assumption hinders the autonomous learning of embodied agents due to the time-consuming and cumbersome workarounds for resetting in the physical world. Hence, there has been a growing interest in autonomous RL (ARL) methods that are capable of learning from non-episodic interactions. However, existing works on ARL are limited by their reliance on prior data and are unable to learn in environments where task-relevant interactions are sparse. In contrast, we propose a demonstration-free ARL algorithm via Implicit and Bi-directional Curriculum (IBC). With an auxiliary agent that is conditionally activated upon learning progress and a bidirectional goal curriculum based on optimal transport, our method outperforms previous methods, even the ones that leverage demonstrations.

As previous representations for reinforcement learning cannot effectively incorporate a human-intuitive understanding of the 3D environment, they usually suffer from sub-optimal performances. In this paper, we present Semantic-aware Neural Radiance Fields for Reinforcement Learning (SNeRL), which jointly optimizes semantic-aware neural radiance fields (NeRF) with a convolutional encoder to learn 3D-aware neural implicit representation from multi-view images. We introduce 3D semantic and distilled feature fields in parallel to the RGB radiance fields in NeRF to learn semantic and object-centric representation for reinforcement learning. SNeRL outperforms not only previous pixel-based representations but also recent 3D-aware representations both in model-free and model-based reinforcement learning.

This study aims to design a motion/force controller for an aerial manipulator which guarantees the tracking of time-varying motion/force trajectories as well as the stability during the transition between free and contact motions. To this end, we model the force exerted on the end-effector as the Kelvin-Voigt linear model and estimate its parameters by recursive least-squares estimator. Then, the gains of the disturbance-observer (DOB)-based motion/force controller are calculated based on the stability conditions considering both the model uncertainties in the dynamic equation and switching between the free and contact motions. To validate the proposed controller, we conducted the time-varying motion/force tracking experiments with different approach speeds and orientations of the surface. The results show that our controller enables the aerial manipulator to track the time-varying motion/force trajectories.

The motion planning problem for multiple unstop-pable agents is of interest in many robotics applications, for example, autonomous traffic management for multiple fixed-wing aircraft. Unfortunately, many of the existing algorithms cannot provide safety for such agents, because they require the agents to be able to brake to a complete stop for safety and feasibility insurance. In this paper, we present a distributed multi-agent motion planner that guarantees collision avoidance and persistent feasibility, which can be applied to a team of homogeneous mobile vehicles that cannot stop. The planner is built on top of the idea that a collision-free trajectory in form of a loop can safely accommodate multiple unstoppable agents, while avoiding collisions among them and static obstacles. At every time step, in a distributed manner, the agents generate trajectory-manipulating actions that preserve the loop structure. Then, a deconfliction process selects a conflict-free subset of the generated actions, which are applied at the next time step. Through simulation using an unstoppable Dubins car model, we show that the proposed motion planner is able to provide persistent safety guarantees for such agents in obstacle-cluttered space in real-time.

This study presents a globally defined dynamics for a conventional multirotor equipped with a single$n\mathbf{-DOF}$manipulator using modified Lagrangian dynamics. This enables the reformulation of entire dynamics directly on$\text{SO}(3)$without exploiting any local coordinates, and thus problems such as the singularity of Euler angles can be avoided. Since skew-symmetric property of Coriolis matrix$C$and inertia matrix facilitates stability analysis, we propose a method to compute$C$which guarantees the skew-symmetric property by considering$C$as a summation of two sub-matrices. Then, a geometric tracking controller is designed based on decoupled dynamics applying passive decomposition. The proposed controller guarantees almost global region of attraction. We validate our method via consecutive aerial flipping experiments.

Existence of symmetric objects, whose observation at different viewpoints can be identical, can deteriorate the performance of simultaneous localization and mapping (SLAM). This work proposes a system for robustly optimizing the pose of cameras and objects even in the presence of symmetric objects. We classify objects into three categories depending on their symmetry characteristics, which is efficient and effective in that it allows to deal with general objects and the objects in the same category can be associated with the same type of ambiguity. Then we extract only the unambiguous parameters corresponding to each category and use them in data association and joint optimization of the camera and object pose. The proposed approach provides significant robustness to the SLAM performance by removing the ambiguous parameters and utilizing as much useful geometric information as possible. Comparison with baseline algorithms confirms the superior performance of the proposed system in terms of object tracking and pose estimation, even in challenging scenarios where the baseline fails.

This paper presents a decentralized multi-agent trajectory planning (MATP) algorithm that guarantees to generate a safe, deadlock-free trajectory in an obstacle-rich environment under a limited communication range. The proposed algorithm utilizes a grid-based multi-agent path planning (MAPP) algorithm for deadlock resolution, and we introduce the subgoal optimization method to make the agent converge to the waypoint generated from the MAPP without deadlock. In addition, the proposed algorithm ensures the feasibility of the optimization problem and collision avoidance by adopting a linear safe corridor (LSC). We verify that the proposed algorithm does not cause a deadlock in both random forests and dense mazes regardless of communication range, and it outperforms our previous work in flight time and distance. We validate the proposed algorithm through a hardware demonstration with ten quadrotors.

Monocular depth estimation plays a critical role in various computer vision and robotics applications such as localization, mapping, and 3D object detection. Recently, learning-based algorithms achieve huge success in depth estimation by training models with a large amount of data in a supervised manner. However, it is challenging to acquire dense ground truth depth labels for supervised training, and the unsupervised depth estimation using monocular sequences emerges as a promising alternative. Unfortunately, most studies on unsupervised depth estimation explore loss functions or occlusion masks, and there is little change in model architecture in that ConvNet-based encoder-decoder structure becomes a de-facto standard for depth estimation. In this paper, we employ a convolution-free Swin Transformer as an image feature extractor so that the network can capture both local geometric features and global semantic features for depth estimation. Also, we propose a Densely Cascaded Multi-scale Network (DCMNet) that connects every feature map directly with another from different scales via a top-down cascade pathway. This densely cascaded connectivity reinforces the interconnection between decoding layers and produces high-quality multi-scale depth outputs. The experiments on two different datasets, KITTI and Make3D, demonstrate that our proposed method outperforms existing state-of-the-art unsupervised algorithms.

Thanks to the development of 2D keypoint detectors, monocular 3D human pose estimation (HPE) via 2D-to-3D uplifting approaches have achieved remarkable improvements. Still, monocular 3D HPE is a challenging problem due to the inherent depth ambiguities and occlusions. To handle this problem, many previous works exploit temporal information to mitigate such difficulties. However, there are many real-world applications where frame sequences are not accessible. This paper focuses on reconstructing a 3D pose from a single 2D keypoint detection. Rather than exploiting temporal information, we alleviate the depth ambiguity by generating multiple 3D pose candidates which can be mapped to an identical 2D keypoint. We build a novel diffusion-based framework to effectively sample diverse 3D poses from an off-the-shelf 2D detector. By considering the correlation between human joints by replacing the conventional denoising U-Net with graph convolutional network, our approach accomplishes further performance improvements. We evaluate our method on the widely adopted Human3.6M and HumanEva-I datasets. Comprehensive experiments are conducted to prove the efficacy of the proposed method, and they confirm that our model outperforms state-of-the-art multi-hypothesis 3D HPE methods.

This study presents a new hardware design and control of a minimally actuated 5 control degrees of freedom (CDoF) quadrotor-based tiltrotor. The proposed tiltrotor possesses several characteristics distinct from those found in existing works, including: 1) minimal number of actuators for 5 CDoF, 2) large margin to generate interaction force during aerial physical interaction (APhI), and 3) no mechanical obstruction in thrust direction rotation. Thanks to these properties, the proposed tiltrotor is suitable for perching-enabled APhI since it can hover parallel to an arbitrarily oriented surface and can freely adjust its thrust direction. To fully control the 5-CDoF of the designed tiltrotor, we construct an asymptotically stabilizing controller with stability analysis. The proposed tiltrotor design and controller are validated in experiments where the first two experiments of x, y position tracking and pitch tracking show controllability of the added CDoF compared to a conventional quadrotor. Finally, the last experiment of perching and cart pushing demonstrates the proposed tiltrotor's applicability to perching-enabled APhI.

Recent curriculum Reinforcement Learning (RL) has shown notable progress in solving complex tasks by proposing sequences of surrogate tasks. However, the previous approaches often face challenges when they generate curriculum goals in a high-dimensional space. Thus, they usually rely on manually specified goal spaces. To alleviate this limitation and improve the scalability of the curriculum, we propose a novel curriculum method that automatically defines the semantic goal space which contains vital information for the curriculum process, and suggests curriculum goals over it. To define the semantic goal space, our method discretizes continuous observations via vector quantized-variational autoencoders (VQ-VAE) and restores the temporal relations between the discretized observations by a graph. Concurrently, ours suggests uncertainty and temporal distance-aware curriculum goals that converges to the final goals over the automatically composed goal space. We demonstrate that the proposed method allows efficient explorations in an uninformed environment with raw goal examples only. Also, ours outperforms the state-of-the-art curriculum RL methods on data efficiency and performance, in various goal-reaching tasks even with ego-centric visual inputs.

Reinforcement learning (RL) often faces the challenges of uninformed search problems where the agent should explore without access to the domain knowledge such as characteristics of the environment or external rewards. To tackle these challenges, this work proposes a new approach for curriculum RL called $\textbf{D}$iversify for $\textbf{D}$isagreement \& $\textbf{C}$onquer ($\textbf{D2C}$). Unlike previous curriculum learning methods, D2C requires only a few examples of desired outcomes and works in any environment, regardless of its geometry or the distribution of the desired outcome examples. The proposed method performs diversification of the goal-conditional classifiers to identify similarities between visited and desired outcome states and ensures that the classifiers disagree on states from out-of-distribution, which enables quantifying the unexplored region and designing an arbitrary goal-conditioned intrinsic reward signal in a simple and intuitive way. The proposed method then employs bipartite matching to define a curriculum learning objective that produces a sequence of well-adjusted intermediate goals, which enable the agent to automatically explore and conquer the unexplored region. We present experimental results demonstrating that D2C outperforms prior curriculum RL methods in both quantitative and qualitative aspects, even with the arbitrarily distributed desired outcome examples.

This paper proposes a new approach to scale-aware monocular visual odometry (VO) and extrinsic calibration using constraints on camera motion by vehicle kinematics. Main idea is to utilize the Ackermann steering model to observe absolute metric scale in turning motion. To describe motion of the camera attached to the vehicle, we first estimate unknown camera-vehicle relative pose by the proposed extrinsic calibration method. To stably observe scale, we detect turn regions and design an observer to estimate the absolute scale as a function of the camera rotation and direction of translational motion during turning. Using the observed scale, we propose an absolute scale recovery to estimate the unknown scale between turns. Because the proposed scale observer becomes singular near zero rotation, we conduct sensitivity analysis on the scale observer, and investigate appropriate conditions for stable scale estimation. For quantitative evaluation of the extrinsic calibration and the absolute scale recovery, we randomly generate synthetic driving datasets with various noise conditions, and evaluate the performance of each module statistically by Monte-Carlo simulations. To evaluate the overall performance, we implement our method and state-of-the-art monocular and stereo VO methods in the public outdoor driving KITTI dataset, and our method shows competitive scale recovery performance with no external sensor and no assumption on surroundings such as planar ground landmarks. To show promising applicability, we collect real-world driving datasets in two multi-floor underground parking lots, and demonstrate the accurate absolute scale recovery performance of our method in indoor driving situations.

This article presents an online distributed trajectory planning algorithm for a quadrotor swarm in a maze-like dynamic environment. We utilize a dynamic linear safe corridor to construct the feasible collision constraints that can ensure interagent collision avoidance and consider the uncertainty of moving obstacles. We introduce mode-based subgoal planning to resolve deadlock faster in a complex environment using only previously shared information. For dynamic obstacle avoidance, we adopt heuristic methods such as collision alert propagation and escape point planning to deal with the situation where dynamic obstacles approach the agents clustered in a narrow corridor. We prove that the proposed algorithm guarantees the feasibility of the optimization problem for every replanning step. In an obstacle-free space, the proposed method can compute the trajectories for 60 agents on average 7.66 ms per agent with an Intel i7 laptop and shows the perfect success rate. Also, our method shows 64.5$\%$shorter flight time than buffered Voronoi cell and 34.6$\%$shorter than with our previous work. We conduct the simulation in a random forest and maze with four dynamic obstacles, and the proposed algorithm shows the highest success rate and shortest flight time compared to state-of-the-art baseline algorithms. In particular, the proposed algorithm shows over 97$\%$success rate when the velocity of moving obstacles is below the agent's maximum speed. We validate the safety and robustness of the proposed algorithm through a hardware demonstration with ten quadrotors and two pedestrians in a maze-like environment.

Safety guarantee prior to the deployment of robots can be difficult due to unexpected disturbances in runtime. This article presents a real-time receding-horizon robust trajectory planning algorithm for nonlinear closed-loop systems, which guarantees the safety of the system under unknown but bounded disturbances. We characterize the forward reachable sets (FRSs) of the system based on the Hamilton–Jacobi reachability analysis as a means for safety verification. For the online computation of the FRSs, we approximate nonlinear systems as LTV systems with linearization errors and compute ellipsoids that encompass the FRSs in continuous time. Using the proposed ellipsoidal approximation of the FRSs, we formulate a computationally tractable robust planning problem that can be solved online. Consequently, the proposed method enables real-time replanning of a reference trajectory with safety guarantees even when the system encounters unexpected disturbances in runtime. The flight experiment of obstacle avoidance in a windy environment validates the proposed robust planning algorithm.

Offline reinforcement learning (Offline RL) suffers from the innate distributional shift as it cannot interact with the physical environment during training. To alleviate such limitation, state-based offline RL leverages a learned dynamics model from the logged experience and augments the predicted state transition to extend the data distribution. For exploiting such benefit also on the image-based RL, we firstly propose a generative model, S2P (State2Pixel), which synthesizes the raw pixel of the agent from its corresponding state. It enables bridging the gap between the state and the image domain in RL algorithms, and virtually exploring unseen image distribution via model-based transition in the state space. Through experiments, we confirm that our S2P-based image synthesis not only improves the image-based offline RL performance but also shows powerful generalization capability on unseen tasks.

Hierarchical Reinforcement Learning (HRL) has made notable progress in complex control tasks by leveraging temporal abstraction. However, previous HRL algorithms often suffer from serious data inefficiency as environments get large. The extended components, $i.e.$, goal space and length of episodes, impose a burden on either one or both high-level and low-level policies since both levels share the total horizon of the episode. In this paper, we present a method of Decoupling Horizons Using a Graph in Hierarchical Reinforcement Learning (DHRL) which can alleviate this problem by decoupling the horizons of high-level and low-level policies and bridging the gap between the length of both horizons using a graph. DHRL provides a freely stretchable high-level action interval, which facilitates longer temporal abstraction and faster training in complex tasks. Our method outperforms state-of-the-art HRL algorithms in typical HRL environments. Moreover, DHRL achieves long and complex locomotion and manipulation tasks.

We propose a new linear RGB-D simultaneous localization and mapping (SLAM) formulation by utilizing planar features of the structured environments. The key idea is to understand a given structured scene and exploit its structural regularities such as the Manhattan world. This understanding allows us to decouple the camera rotation by tracking structural regularities, which makes SLAM problems free from being highly nonlinear. Additionally, it provides a simple yet effective cue for representing planar features, which leads to a linear SLAM formulation. Given an accurate camera rotation, we jointly estimate the camera translation and planar landmarks in the global planar map using a linear Kalman filter. Our linear SLAM method, called L-SLAM, can understand not only the Manhattan world but the more general scenario of the Atlanta world, which consists of a vertical direction and a set of horizontal directions orthogonal to the vertical direction. To this end, we introduce a novel tracking-by-detection scheme that infers the underlying scene structure by Atlanta representation. With efficient Atlanta representation, we formulate a unified linear SLAM framework for structured environments. We evaluate L-SLAM on a synthetic dataset and RGB-D benchmarks, demonstrating comparable performance to other state-of-the-art SLAM methods without using expensive nonlinear optimization. We assess the accuracy of L-SLAM on a practical application of augmented reality.

This paper presents an approach that generates the overall trajectory of mobile manipulators for a complex mission consisting of several sub-tasks. Parametric dynamic movement primitives (PDMPs) can quickly generalize the online motion of robot manipulation by learning multiple demonstrations in offline. However, regarding complex missions consisting of multiple sub-tasks, a large number of demonstrations are required for full generalization, which is impractical. In this paper, we propose a framework that reduces the number of demonstrations for a complex mission. In the proposed method, complex demonstrations are segmented into multiple unit motions representing sub-tasks, and one PDMP is formed per each segment, resulting in multiple PDMPs. The phase decision process determines which sub-task and associated PDMPs to be executed online, allowing multiple PDMPs to be autonomously configured within an integrated framework. In order to generalize the execution time and regional goal in each phase, the Gaussian process regression (GPR) is applied. Simulation results from two different scenarios confirm that the proposed framework not only effectively reduces the number of demonstrations but also improves generalization performance. The actual experiments also demonstrate that the mobile manipulators effectively perform complex missions through the proposed framework. Note to Practitioners—This paper presents an approach of learning from demonstration (LfD) to generalize complex movements of robots. Parametric dynamic movement primitives (PDMPs) compute styles of movements from multiple demonstrations. However, the complexity of the PDMP increases as the mission involves more sub-tasks. In this paper, we resolve this issue by segmenting the complex mission into multiple sub-tasks and configuring multiple PDMPs. This work effectively reduces the number of required demonstrations for PDMPs, moderates the complexity of the algorithm. Also, the proposed approach allows flexible sub-task sequencing. It enables the mission in an unlearned sequence or a new combination of sub-tasks. The proposed approach is validated in both simulation and experimental results. Our approach is applicable for complex missions whose sub-tasks are clearly identified

This paper proposes an automatic and accuracy- enhanced extrinsic calibration method for 3D LiDARs with a range offset correction, which needs only an arbitrarily-shaped single planar board. One of the most exhaustive parts of existing LiDAR calibration procedures is to manually find target objects from massive point clouds. To obviate user interventions, we propose an automated planar board detection from LiDAR range images. To extract a target completely, we suppress outliers and restore rejected inliers of the target board by introducing a target completion method. We empirically find that range measurements of various LiDARs are mainly skewed by constant offset values. To compensate for this, we suggest a range offset model for each laser channel in calibration procedures. The relative pose between LiDARs and range offsets are jointly estimated by minimizing bi-directional point- to-board distances within the iterative re-weighted least squares (IRLS) framework. To verify the suggested range offset model, we obtain and analyze extensive real-world measurements. By conducting experiments using the various sensor configurations and shapes of boards, we quantitatively and qualitatively confirm accuracy and versatility of the proposed method by comparing with the state-of-the-art LiDAR calibration methods. All the source code and data used in the paper are available at : https://github.com/JunhaAgu/AutoL2LCalib.

Domain randomization (DR) is a powerful tool to make a policy robust to the uncertainty of dynamics caused by unobservable environmental parameters. Conventional DR has adopted model-free reinforcement learning as a policy optimizer. However, the model-free methods in DR demand high time-complexity due to the randomization process where the environment is extremely changed. In this paper, we introduce model-based dynamics and policy learning for efficient DR. A Bayesian model of locally linear embedding is designed to fit the stochastic dynamics in DR. By virtue of locally linear dynamics, model-based optimal control is substituted for the policy optimization. Unlike previous works, our proposed Bayesian model with a MNIW prior allows the locally linear embedding to capture the dynamics in DR as a stochastic model. We show that a training method that combines variational and adversarial approaches is adequate for Bayesian embedding. Finally, a model-based controller is designed on our Bayesian locally linear embedding, and it shows better performance in DR environments compared with the non-Bayesian model of locally linear embedding.

Estimating a depth map from a single RGB image has been investigated widely for localization, mapping, and 3- dimensional object detection. Recent studies on a single-view depth estimation are mostly based on deep Convolutional neural Networks (ConvNets) which require a large amount of training data paired with densely annotated labels. Depth annotation tasks are both expensive and inefficient, so it is inevitable to leverage RGB images which can be collected very easily to boost the performance of ConvNets without depth labels. However, most self-supervised learning algorithms are focused on capturing the semantic information of images to improve the performance in classification or object detection, not in depth estimation. In this paper, we show that existing self- supervised methods do not perform well on depth estimation and propose a gradient-based self-supervised learning algorithm with momentum contrastive loss to help ConvNets extract the geometric information with unlabeled images. As a result, the network can estimate the depth map accurately with a relatively small amount of annotated data. To show that our method is independent of the model structure, we evaluate our method with two different monocular depth estimation algorithms. Our method outperforms the previous state-of-the- art self-supervised learning algorithms and shows the efficiency of labeled data in triple compared to random initialization on the NYU Depth v2 dataset.

In this paper, an autonomous aerial manipulation task of pulling a plug out of an electric socket is conducted, where maintaining the stability and robustness is challenging due to sudden disappearance of a large interaction force. The abrupt change in the dynamical model before and after the separation of the plug can cause destabilization or mission failure. To accomplish aerial plug-pulling, we employ the concept of hybrid automata to divide the task into three operative modes, i.e, wire-pulling, stabilizing, and free-flight. Also, a strategy for trajectory generation and a design of disturbance-observer-based controllers for each operative mode are presented. Furthermore, the theory of hybrid automata is used to prove the stability and robustness during the mode transition. We validate the proposed trajectory generation and control method by an actual wire-pulling experiment with a multirotor-based aerial manipulator.

Motion planners for mobile robots in unknown environments face the challenge of simultaneously maintaining both robustness against unmodeled uncertainties and persistent feasibility of the trajectory-finding problem. That is, while dealing with uncertainties, a motion planner must update its trajectory, adapting to the newly revealed environment in real-time; failing to do so may involve unsafe circumstances. Many existing planning algorithms guarantee these by maintaining the clearance needed to perform an emergency brake, which is itself a robust and persistently feasible maneuver. However, such maneuvers are not applicable for systems in which braking is impossible or risky, such as fixed-wing aircraft. To that end, we propose a real-time robust planner that recursively guarantees persistent feasibility without any need of braking. The planner ensures robustness against bounded uncertainties and persistent feasibility by constructing a loop of sequentially composed funnels, starting from the receding horizon local trajectory’s forward reachable set. We implement the proposed algorithm for a robotic car tracking a speed-fixed reference trajectory. The experiment results show that the proposed algorithm can be run at faster than 16 Hz, while successfully keeping the system away from entering any dead end, to maintain safety and feasibility.

Visual navigation has been widely used for state estimation of micro aerial vehicles (MAVs). For stable visual navigation, MAVs should generate perception-aware paths which guarantee enough visible landmarks. Many previous works on perception-aware path planning focused on sampling-based planners. However, they may suffer from sample inefficiency, which leads to computational burden for finding a global optimal path. To address this issue, we suggest a perception-aware path planner which utilizes topological information of environments. Since the topological class of a path and visible landmarks during traveling the path are closely related, the proposed algorithm checks distinctive topological classes to choose the class with abundant visual information. Topological graph is extracted from the generalized Voronoi diagram of the environment and initial paths with different topological classes are found. To evaluate the perception quality of the classes, we divide the initial path into discrete segments where the points in each segment share similar visual information. The optimal class with high perception quality is selected, and a graph-based planner is utilized to generate path within the class. With simulations and real-world experiments, we confirmed that the proposed method could guarantee accurate visual navigation compared with the perception-agnostic method while showing improved computational efficiency than the sampling-based perception-aware planner.

Automation of excavation tasks requires real-time trajectory planning satisfying various constraints. To guarantee both constraint feasibility and real-time trajectory re-plannability, we present an integrated framework for real-time optimization-based trajectory planning of a hydraulic excavator. The proposed framework is composed of two main modules: a global planner and a real-time local planner. The global planner computes the entire global trajectory considering excavation volume and energy minimization while the local counterpart tracks the global trajectory in a receding horizon manner, satisfying dynamic feasibility, physical constraints, and disturbance-awareness. We validate the proposed planning algorithm in a simulation environment where two types of operations are conducted in the presence of emulated disturbance from hydraulic friction and soil-bucket interaction: shallow and deep excavation. The optimized global trajectories are obtained in an order of a second, which is tracked by the local planner at faster than 30 Hz. To the best of our knowledge, this work presents the first real-time motion planning framework that satisfies constraints of a hydraulic excavator, such as force/torque, power, cylinder displacement, and flow rate limits.

Autonomous aerial videography is a challenging task, which involves collision avoidance against obstacles and visibility guaranteed target tracking in unstructured environments. In this paper, we organize a two micro aerial vehicle (MAV) team, which consists of a target agent responsible for a specific mission and a camera agent for filming the target agent. Especially, this paper focuses on trajectory planning of the camera agent to chase without occlusion of target agent. Our trajectory planner module includes two phases of guaranteeing target visibility. In the first phase, we generate homotopic safe flight corridor (SFC) to attain target-visible regions. In the subsequent phase, we generate a safe and smooth trajectory with the continuous visibility constraint based on the SFC, using quadratic programming (QP). Regardless of complexity of map, our planner converts an overall problem to a single QP and generates a steady flight trajectory without undesirable fluctuating motion, while guaranteeing all-time visibility. We validate our approach in Gazebo simulations and a real-world experiment.

Multirobot simultaneous localization and mapping (SLAM) requires technical ingredients such as systematic construction of multiple SLAM systems and collection of information from each robot. In particular, map fusion is an essential process of multirobot SLAM that combines multiple local maps estimated by team robots into a global map. Fusion of multiple local maps is usually based on interloop detection that recognizes the same scene visited by multiple robots, or robot rendezvous where a member(s) of a robot team is observed in another member's images. This article proposes a collaborative monocular SLAM including a map fusion algorithm that utilizes rendezvous, which can happen when multirobot team members operate in close proximity. Unlike existing rendezvous-based approaches that require additional sensors, the proposed system uses a monocular camera only. Our system can recognize robot rendezvous using nonstatic features (NSFs) without fiducial markers to identify team robots. NSFs, which are abandoned as outliers in typical SLAM systems for not supporting ego-motion, can include relative bearing measurements between robots in a rendezvous situation. The proposed pipeline consists of the following: first, a feature identification module that extracts the relative bearing measurements between robots from NSFs consisting of anonymous bearing vectors with false positives, and second, a map fusion module that integrates the map from the observer robot with the maps from the observed robots using identified relative measurements. The feature identification module can operate quickly using the proposed alternating minimization algorithm formulated by two subproblems with closed-form solutions. The experimental results confirm that our collaborative monocular SLAM system recognizes rendezvous rapidly and robustly, and fuses local maps of team robots into a global map accurately.

This work suggests an integrated approach for a drone (or multirotor) to perform an autonomous videography task in a 3-D obstacle environment by following a moving object. The proposed system includes 1) a target motion prediction module which can be applied to dense environments and 2) a hierarchical chasing planner. Leveraging covariant optimization, the prediction module estimates the future motion of the target assuming it efforts to avoid the obstacles. The other module, chasing planner, is in a bi-level structure composed of preplanner and smooth planner. In the first phase, we exploit a graph-search method to plan a chasing corridor which incorporates safety and visibility of target. In the subsequent phase, we generate a smooth and dynamically feasible trajectory within the corridor using quadratic programming (QP). We validate our approach with multiple complex scenarios and actual experiments. The source code and the experiment video can be found in https://github.com/icsl-Jeon/traj_gen_vis and https://www.youtube.com/watch?v=_JSwXBwYRl8.

Detection of moving objects is an essential capability in dealing with dynamic environments. Most moving object detection algorithms have been designed for color images without depth. For robotic navigation where real-time RGBD data is often readily available, utilization of the depth information would be beneficial for obstacle recognition. Here, we propose a simple moving object detection algorithm that uses RGB-D images. The proposed algorithm does not require estimating a background model. Instead, it uses an occlusion model which enables us to estimate the camera pose on a background confused with moving objects that dominate the scene. The proposed algorithm allows to separate the moving object detection and visual odometry (VO) so that an arbitrary robust VO method can be employed in a dynamic situation with a combination of moving object detection, whereas other VO algorithms for a dynamic environment are inseparable. In this paper, we use dense visual odometry (DVO) as a VO method with a bi-square regression weight. Experimental results show the segmentation accuracy and the performance improvement of DVO in the situations. We validate our algorithm in public datasets and our dataset which also publicly accessible.

Existing studies for environment interaction with an aerial robot have been focused on interaction with static surroundings. However, to fully explore the concept of an aerial manipulation, interaction with moving structures should also be considered. In this paper, a multirotor-based aerial manipulator opening a daily-life moving structure, a hinged door, is presented. In order to address the constrained motion of the structure and to avoid collisions during operation, model predictive control (MPC) is applied to the derived coupled system dynamics between the aerial manipulator and the door involving state constraints. By implementing a constrained version of differential dynamic programming (DDP), MPC can generate position setpoints to the disturbance observer (DOB)-based robust controller in real-time, which is validated by our experimental results.

This paper presents a new efficient algorithm which guarantees a solution for a class of multi-agent trajectory planning problems in obstacle-dense environments. Our algorithm combines the advantages of both grid-based and optimization-based approaches, and generates safe, dynamically feasible trajectories without suffering from an erroneous optimization setup such as imposing infeasible collision constraints. We adopt a sequential optimization method with dummy agents to improve the scalability of the algorithm, and utilize the convex hull property of Bernstein and relative Bernstein polynomial to replace non-convex collision avoidance constraints to convex ones. The proposed method can compute the trajectory for 64 agents on average 6.36 seconds with Intel Core i7-7700 @ 3.60GHz CPU and 16G RAM, and it reduces more than 50% of the objective cost compared to our previous work. We validate the proposed algorithm through simulation and flight tests.

Planning a trajectory with guaranteed safety is a core part for a risk-free flight of a multirotor. If a trajectory planner only aims to ensure safety, it may generate trajectories which overly bypass risky regions and prevent the system from achieving specific missions. This work presents a robust trajectory planning algorithm which simultaneously guarantees the safety and reachability to the target state in the presence of unknown disturbances. We first characterize how the forward and backward reachable sets (FRSs and BRSs) are constructed by using Hamilton-Jacobi reachability analysis. Based on the analysis, we present analytic expressions for the reachable sets and then propose minimal ellipsoids which closely approximate the reachable sets. In the planning process, we optimize the reference trajectory to connect the FRSs and BRSs, while avoiding obstacles. By combining the FRSs and BRSs, we can guarantee that any state inside of the initial set reaches the target set. We validate the proposed algorithm through a simulation of traversing a narrow gap.

In micro aerial vehicle (MAV) operations, the success of a mission is highly dependent on navigation performance, which has raised recent interests on navigation-aware path planning. One of the challenges lies in that optimal motions for successful navigation and the designated mission are often different in unknown, unstructured environments, and only sub-optimality may be obtained in each aspect. We aim to organize a two-MAV team that can effectively execute the mission and simultaneously guarantee navigation quality, which consists of a main-agent responsible for mission and a sub-agent for navigation of the team. Especially, this paper focuses on path planning of the sub-agent to provide navigational assistance to the main-agent using a monocular camera. We adopt a graph-based receding horizon planner to find a dynamically feasible path in order for the sub-agent to help the main-agent's navigation. In this process, we present a metric for evaluating the localization performance utilizing the distribution of the features projected to the image plane. We also design a map management strategy and pose-estimation support mechanism in a monocular camera setup, and validate their effectiveness in two scenarios.

This work investigates an efficient trajectory generation for chasing a dynamic target, which incorporates the detectability objective. The proposed method actively guides the motion of a cinematographer drone so that the color of a target is well-distinguished against the colors of the background in the view of the drone. For the objective, we define a measure of color detectability given a chasing path. After computing a discrete path optimized for the metric, we generate a dynamically feasible trajectory. The whole pipeline can be updated on-the- fly to respond to the motion of the target. For the efficient discrete path generation, we construct a directed acyclic graph (DAG) for which a topological sorting can be determined analytically without the depth-first search. The smooth path is obtained in quadratic programming (QP) framework. We validate the enhanced performance of state-of-the-art object detection and tracking algorithms when the camera drone executes the trajectory obtained from the proposed method.

We propose an efficient edge-based stereo visual odometry (VO) using multiple quadtrees created according to image gradient orientations. To characterize edges, we classify them into eight orientation groups according to their image gradient directions. Using the edge groups, we construct eight quadtrees and set overlapping areas belonging to adjacent quadtrees for robust and efficient matching. For further acceleration, previously visited tree nodes are stored and reused at the next iteration to warm-start. We propose an edge culling method to extract prominent edgelets and prune redundant edges. The camera motion is estimated by minimizing point-to-edge distances within a re-weighted iterative closest points (ICP) framework, and simultaneously, 3-D structures are recovered by static and temporal stereo settings. To analyze the effects of the proposed methods, we conduct extensive simulations with various settings. Quantitative results on public datasets confirm that our approach has competitive performance with state-of-the-art stereo methods. In addition, we demonstrate the practical values of our system in author-collected modern building scenes with curved edges only.

In this article, we introduce a translational force control method with disturbance observer (DOB)-based force disturbance cancellation for precise 3-D acceleration control of a multi-rotor unmanned aerial vehicle (UAV). The acceleration control of the multi-rotor requires conversion of the desired acceleration signal to the desired roll, pitch, and total thrust. However, because the attitude dynamics and the thrust dynamics are different, simple kinematic signal conversion without consideration of those difference can cause serious performance degradation in acceleration tracking. Unlike most existing translational force control techniques that are based on such simple inversion, our new method allows controlling the acceleration of the multi-rotor more precisely by considering the dynamics of the multi-rotor during the kinematic inversion. By combining the DOB with the translational force system that includes the improved conversion technique, we achieve robustness with respect to the external force disturbances that hinder the accurate acceleration control. μ-analysis is performed to ensure the robust stability of the overall closed-loop system, considering the combined effect of various possible model uncertainties. Both simulation and experiment are conducted to validate the proposed technique, which confirms the satisfactory performance to track the desired acceleration of the multi-rotor.

This study addresses an operation of unmanned aerial vehicles (UAVs) in a network environment where there is time-varying network delay. The network delay entails undesirable effects on the stability of the UAV control system due to delayed state feedback and outdated control input. Although several networked control algorithms have been proposed to deal with the network delay, most existing studies have assumed that the plant dynamics is known and simple, or the network delay is constant. These assumptions are improper to multirotor-type UAVs because of their nonlinearity and time-sensitive characteristics. To deal with these problems, we propose a networked control system using model predictive control (MPC) designed under the consideration of multirotor characteristics. We also apply a Gaussian process (GP) to learn an unknown nonlinear model, which increases the accuracy of path planning and state estimation. Flight experiments show that the proposed algorithm successfully compensates the network delay and Gaussian process learning improves the UAV's path tracking performance.

In this paper, we introduce a cargo transportation method with a new type of multi-rotor UAV platform known as T3-multirotor, to achieve stable and constant flight performance regardless of the type of cargo attached to the fuselage. The T3-multirotor, which consists of the `Thrust Generating Part' and the `Fuselage Part', can directly control the relative attitude between the two parts using the novel servomechanism. By utilizing the servomechanism with the proposed relative attitude control strategy, the T3-multirotor with cargo attached to the fuselage part can behave as a multi-rotor with only the moment of inertia of the thrust generating part during entire transportation. This allows the T3-multirotor to achieve the reliable performance in the event of any cargo being attached to the fuselage, achieving stable platform motion control. Detailed hardware description and dynamic analysis of T3-Multirotor is performed in this paper, and the validity of the proposed control strategy is also analyzed. The feasibility of the proposed control strategy is verified through experimental results with analysis.

This paper presents planning and control algorithms for a multi-rotor with a suspended load. The suspended load cannot be controlled easily by the multi-rotor due to severe dynamic coupling between them. Difficulties are exacerbated by under-actuated, highly nonlinear nature of multi-rotor dynamics. Although many studies have been proposed to plan trajectories and control this system, there exist only a few reports on real-time trajectory generation. With this in mind, we propose a planning method which is capable of generating collision-free trajectories real-time and applicable to a high-dimensional nonlinear system. Using a differential flatness property, the system can be linearized entirely with elaborately chosen flat outputs. Convexification of non-convex constraints is carried out, and concave obstacle-avoidance constraints are converted to convex ones. After that, a convex optimization problem is solved to generate an optimal trajectory, but semi-feasible trajectory which considers only some parts of the initial state. We apply model predictive control with a sequential linear quadratic solver to compute a feasible collision-free trajectory and to control the system. Performance of the algorithm is validated by flight experiment.

This work deals with a moving target chasing mission of an aerial vehicle equipped with a vision sensor in a cluttered environment. In contrast to obstacle-free or sparse environments, the chaser should be able to handle collision and occlusion together with flight efficiency. In order to tackle these challenges in real-time, we introduce a metric for target visibility and propose a hierarchical chasing planner. In the first phase, we generate a sequence of waypoints and chasing corridors which ensure safety and optimize visibility. In the following phase, the corridors and waypoints are utilized as constraints and objective respectively in quadratic programming from which we complete a dynamically feasible trajectory for chasing. The proposed algorithm is tested in multiple dense environments. The simulator AutoChaser with full code implementation & GUI can be found in https://github.com/icsl-Jeon/traj_gen_vis and video is available at https://youtu.be/-2d3uDlYR_M.

Applying deep reinforcement learning to physical systems, as opposed to learning in simulation, presents additional challenges in terms of sample efficiency and safety. Collecting large amounts of hardware demonstration data is time-consuming and the exploratory behavior of reinforcement learning algorithms may lead the system into dangerous states, especially during the early stages of training. To address these challenges, we apply transfer learning to reuse a previously learned policy instead of learning from scratch. In this paper, we propose a method where given a source policy, policy adaptation is performed via transfer learning to produce a target policy suitable for real-world deployment. For policy adaptation, alignment-based transfer learning is applied to trajectories generated by the source policy and their corresponding safe target trajectories. We apply this method to manipulators and show that the proposed method is applicable to both inter-task and inter-robot transfer whilst considering safety. We also show that the resulting target policy is robust and can be further improved with reinforcement learning.

This paper presents a motion planning approach for an aerial pick-and-place task where an aerial manipulator is supposed to pick up or place an object at locations specified as way-points. In particular, we focus on situations where such way-point constraints are imposed on certain partial state variables, rather than on full state variables. Our proposed framework, based on rapidly exploring random trees star (RRT*) in a bidirectional manner, enables an aerial manipulator to find an optimal trajectory that satisfies way-point constraints with only partial specifications. Here, we suggest an extra merging process to integrate the trees, each originated from the start and goal point. In the merging process, we search various candidate points satisfying a given condition that partially constrains state variables, and select a way-point with full specifications optimal in the perspective of the entire trajectory. Simulation and experiment results are included to validate the proposed framework.

Visual servoing, i.e. control with visual information, is a valuable capability in many robotic applications. In particular, position based visual servoing (PBVS) estimates position information from the observed image to generate visual servo control. However, the estimation of the position of an unknown target using monocular images is still difficult due to the complexity of the image information. For the target estimation problem, we propose to integrate three complementary techniques for monocular visual servoing. First, to estimate the probability of a target's existence, the learning model with spatial features from convolution neural network is proposed. Second, the extended Kalman filter based on epipolar geometry estimates the 3D position of the target; moreover, from this 3D position, the perception model is trained online by self-generated virtual ground-truth. Finally, visual servo control is generated, and the resulting movement helps to construct epipolar geometry. Finally. the experimental validation is performed in a challenging setting involving occlusion and target's shape change.

In the paper, we propose a robust real-time visual odometry in dynamic environments via rigid-motion model updated by scene flow. The proposed algorithm consists of spatial motion segmentation and temporal motion tracking. The spatial segmentation first generates several motion hypotheses by using a grid-based scene flow and clusters the extracted motion hypotheses, separating objects that move independently of one another. Further, we use a dual-mode motion model to consistently distinguish between the static and dynamic parts in the temporal motion tracking stage. Finally, the proposed algorithm estimates the pose of a camera by taking advantage of the region classified as static parts. In order to evaluate the performance of visual odometry under the existence of dynamic rigid objects, we use self-collected dataset containing RGB-D images and motion capture data for ground-truth. We compare our algorithm with state-of-the-art visual odometry algorithms. The validation results suggest that the proposed algorithm can estimate the pose of a camera robustly and accurately in dynamic environments.

This paper presents a new trajectory planning method for multiple quadrotors in obstacle-dense environments. We suggest a relative safe flight corridor (RSFC) to model safe region between a pair of agents, and it is used to generate linear constraints for inter-collision avoidance by utilizing the convex hull property of relative Bernstein polynomial. Our approach employs a graph-based multi-agent pathfinding algorithm to generate an initial trajectory, which is used to construct a safe flight corridor (SFC) and RSFC. We express the trajectory as a piecewise Bernstein polynomial and formulate the trajectory planning problem into one quadratic programming problem using linear constraints from SFC and RSFC. The proposed method can compute collision-free trajectory for 16 agents within a second and for 64 agents less than a minute, and it is validated both through simulation and indoor flight test.

Ensuring safety in trajectory planning of multirotor systems is an essential element for risk-free operation. Even if the generated trajectory is known to be safe in the planning phase, unknown disturbance during an actual operation can lead to a dangerous situation. This paper proposes safety-guaranteed receding horizon planning against unknown, but bounded, disturbances. We first characterize forward reachable set (FRS) of the system, the set of states after a certain duration considering all possible disturbances, using Hamilton-Jacobi (HJ) reachability analysis. To compute the FRSs in real-time, we conservatively approximate the true FRS and perform ellipsoidal parameterization on the FRSs. Using the FRSs, we can plan a robust trajectory that avoids risky regions and rapidly re-plan the trajectory when the system encounters sudden disturbance. The proposed method is validated through an experiment of avoiding obstacles in a wind.

We propose a novel approach to estimate the three degrees of freedom (DoF) drift-free rotational motion of an RGB-D camera from only a single line and plane in the Manhattan world (MW). Previous approaches exploit the surface normal vectors and vanishing points to achieve accurate 3-DoF rotation estimation. However, they require multiple orthogonal planes or many consistent lines to be visible throughout the entire rotation estimation process; otherwise, these approaches fail. To overcome these limitations, we present a new method that estimates absolute camera orientation from only a single line and a single plane in RANSAC, which corresponds to the theoretical minimal sampling for 3-DoF rotation estimation. Once we find an initial rotation estimate, we refine the camera orientation by minimizing the average orthogonal distance from the endpoints of the lines parallel to the MW axes. We demonstrate the effectiveness of the proposed algorithm through an extensive evaluation on a variety of RGB-D datasets and compare with other state-of-the-art methods.

We present a low-drift visual odometry algorithm that separately estimates rotational and translational motion from lines, planes, and points found in RGB-D images. Previous methods estimate drift-free rotational motion from structural regularities to reduce drift in the rotation estimate, which is the primary source of positioning inaccuracy in visual odometry. However, multiple orthogonal planes are required to be visible throughout the entire motion estimation process; otherwise, these VO approaches fail. We propose a new approach to estimate drift-free rotational motion jointly from both lines and planes by exploiting environmental regularities. We track the spatial regularities with an efficient SO(3)-manifold constrained mean shift algorithm. Once the drift-free rotation is found, we recover the translational motion from all tracked points with and without depth by minimizing the de-rotated reprojection error. We compare the proposed algorithm to other state-of-the-art visual odometry methods on a variety of RGB-D datasets (including especially challenging pure rotations) and demonstrate improved accuracy and lower drift error.

This paper presents a new design of multirotor, named as `Tilting Thruster Type' (T3)-multirotor. The new platform is equipped with mechanically separated thrusters, which can take any fuselage posture within a specified range regardless of any direction of translational acceleration. A specially designed servo-linkage mechanism is employed for relative attitude control between the thruster and the fuselage. Mathematical modeling and analysis of the new platform are conducted to explore the control method of the dynamically complex system. For demonstrating the potential of the new T3-multirotor, an autonomous level flight is performed where the fuselage maintains zero roll and pitch angle during the entire flight. Both simulation and experimental results are provided with detailed analysis.

This paper investigates a multi-rotor with a suspended load in perspectives of 1) real-time path planning, 2) obstacle avoidance, and 3) transportation of a suspended object. A suspended load cannot be controlled with conventional controllers designed for nominal multi-rotors due to the dynamic coupling between the multi-rotor and load. Although several control and planning algorithms have been proposed based on elaborately derived dynamic equations, most existing studies separate control and path planning problems by following predefined trajectories after trajectory generation. Moreover, many state-of-the-art trajectory generation algorithms cannot work real-time for a system with high degrees of freedom, which makes it not suitable to operate the system in dynamic environments where obstacles appear abruptly or move unexpectedly. With this in mind, we apply Model Predictive Control (MPC) with Sequential Linear Quadratic (SLQ) solver to compute feasible and optimal trajectory real-time and to operate a multi-rotor with a suspended load in dynamic environments. We design an obstacle-avoidance algorithm suitable for the current platform flying in cluttered environments. Flight experiments shows that the proposed algorithm successfully controls the multi-rotor and allows to avoid obstacles simultaneously.

This paper proposes an edge-based robust RGB-D visual odometry (VO) using 2-D edge divergence minimization. Our approach focuses on enabling the VO to operate in more general environments subject to low texture and changing brightness, by employing image edge regions and their image gradient vectors within the iterative closest points (ICP) framework. For more robust and stable ICP-based optimization, we propose a robust edge matching criterion with image gradient vectors. In addition, to reduce a bad effect of outlier residuals, we propose an improved edge registration problem of 2-D edge divergence minimization in the manner of an iterative re-weight least squares (IRLS) motion estimation. To accelerate the proposed approach, a pixel sub-sampling method is employed. We evaluate estimation performance of our method in changing brightness conditions and low-textured scenes. Our approach shows more robust motion estimation than state-of-the-art methods while maintaining comparable accuracy in challenging image sequences at real-time (25 Hz) operation.

This paper presents an approach to learn and generalize robotic skills from a demonstration using deep reinforcement learning (deep RL). Dynamic Movement Primitives (DMPs) formulate a nonlinear differential equation and produce the observed movement from a demonstration. However, it is hard to generate new behaviors from using DMPs. Thus, we apply DMPs framework into deep RL as an initial setting for learning the robotic skills. First, we build a network to represent this differential equation, and learn and generalize the movements by optimizing the shape of DMPs with respect to the rewards up to the end of each sequence of movement primitives. In order to do this, we consider a deterministic actor-critic algorithm for deep RL and we also apply a hierarchical strategy. This drastically reduces the search space for a robot by decomposing the task, which allows to solve the sparse reward problem from a complex task. In order to integrate DMPs with hierarchical deep RL, the differential equation is considered as temporal abstraction of option. The overall structure is mainly composed of two controllers: meta-controller and sub-controller. The meta-controller learns a policy over intrinsic goals and a sub-controller learns a policy over actions to accomplish the given goals. We demonstrate our approach on a 6 degree-of-freedom (DOF) arm with a I-DOF gripper and evaluate our approach through a pick-and-place task.

This paper describes a vision-based target tracking method for a skid-steer vehicle. With the development of deep reinforcement learning, many researchers have tried to generate an end-to-end policy to control the mobile robot from a raw pixel image data. However, the action in most research only concerns high-level decisions such as go straight, turn left and right. High-level decisions alone are not sufficient to precisely control platforms such as a skid-steer vehicle due to the lack of steering mechanism. Thus, unlike existing work, we aim to control the motor command for the wheels directly. To this end, we employ guided policy search (GPS) based on the general kinematic slip model for the skid-type robot. Furthermore, to prohibit the target from getting out of the camera field of view (FOV) in the training phase, we update local policy optimization with a FOV constraint and perform a pre-training to make the initial policy more efficient. Our method allows the skid-type robot to automatically acquire the vision-based tracking policy while local policies satisfy the FOV constraint during the training phase. We evaluate our method through both simulation and experiment with a skid-steer mobile robot. Finally, we test the performance of learned policy with a moving target in a new environment.

Flapping-wing micro air vehicles (FWMAVs) become promising research platforms due to their advantages such as various maneuverability, and concealment. However, unsteady flow at low Reynolds number around the wings makes their dynamics time-varying and highly non-linear. It makes autonomous flight of FWMAV as a big challenge. In this paper, we suggest a model-based control strategy for FWMAV using learning architecture. For this task, we construct a ground station for logging flight data and control inputs, and train dynamics with a neural network. Then, we apply model predictive control (MPC) to the trained model. We validate our method by hardware experiments.

This paper presents planning and control synthesis for multiple aerial manipulators to transport a common object. Each aerial manipulator that consists of a hexacopter and a two-degree-of-freedom robotic arm is controlled by an augmented adaptive sliding mode controller based on a closed-chain robot dynamics. We propose a motion planning algorithm by exploiting rapidly exploring random tree star (RRT*) and dynamic movement primitives (DMPs). The desired path for each aerial manipulator is obtained by using RRT* with Bezier curve, which is designed to handle environmental obstacles, such as buildings or equipments. During aerial transportation, to avoid unknown obstacle, DMPs modify the trajectory based on the virtual leader- follower structure. By the combination of RRT* and DMPs, the cooperative aerial manipulators can carry a common object to keep reducing the interaction force between multiple robots while avoiding an obstacle in the unstructured environment. To validate the proposed planning and control synthesis, two experiments with multiple custom-made aerial manipulators are presented, which involve user-guided trajectory and RRT*-planned trajectory tracking in unstructured environments.

We present an illumination-robust visual localization algorithm for Astrobee, a free-flying robot designed to autonomously navigate on the International Space Station (ISS). Astrobee localizes with a monocular camera and a pre-built sparse map composed of natural visual features. Astrobee must perform tasks not only during the day, but also at night when the ISS lights are dimmed. However, the localization performance degrades when the observed lighting conditions differ from the conditions when the sparse map was built. We investigate and quantify the effect of lighting variations on visual feature-based localization systems, and discover that maps built in darker conditions can also be effective in bright conditions, but the reverse is not true. We extend Astrobee's localization algorithm to make it more robust to changing-light environments on the ISS by automatically recognizing the current illumination level, and selecting an appropriate map and camera exposure time. We extensively evaluate the proposed algorithm through experiments on Astrobee.

This paper presents a motion planning approach for cooperative transportation using aerial robots. We describe a framework based on Parametric Dynamic Movement Primitives (PDMPs) for coordinating multiple aerial robots and their manipulators quickly in an environment cluttered with obstacles. In order to emulate the optimal motion, we combine PDMPs and Rapidly Exploring Randomized Trees star (RRT*) by using the results of RRT* as demonstrations for PDMPs. For efficient description of the motions corresponding to the environment, we utilize Gaussian Process Regression (GPR) to acquire of the explicit relationship between environmental parameters and style parameters of PDMPs which decide the motions. Simulation and experiment results are attached to validate the proposed framework.

This paper presents a practical swing-angle estimation method for slung load operations of the multi-rotor unmanned aerial vehicle (UAV), which is essential to maintain the safety during the operation. In order not to rely on extra sensors for monitoring the swing angle, the proposed method in this paper offers an autonomous swing-angle estimation using only an inertial measurement unit (IMU) and a single load cell attached to the slung load. The disturbance observer (DOB) derived external force estimation is performed to estimate the swing angle. The unique structure of the proposed DOB-based disturbance force estimation technique utilizes the IMU data only. Both simulation and actual experiment are performed to validate the feasibility of the proposed algorithm.

This paper concentrates on design of a vision-based guidance command for aerial manipulation of a cylindrical object, using a stochastic model predictive approach. We first develop an image-based cylinder detection algorithm that utilizes a geometric characteristic of perspectively projected circles in 3D space. To enforce the object to be located inside sight of a camera, we formulate a visual servoing problem as a stochastic model predictive control (MPC) framework. By regarding x and y axes rotational velocities as stochastic variables, we guarantee the visibility of the camera considering underactuation of the system. We also provide experimental results that validate effectiveness of the proposed algorithm.

Hovering flight is one of the valuable capabilities for unmanned aerial vehicles (UAVs) to perform various missions. It is easily attainable by rotary-wing aircrafts such as a helicopter and multirotor. This flight maneuver, however, remains a significant challenge to a flapping wing micro air vehicle (FWMAV). This paper aims to regulate attitude angles of a FWMAV and maintain an altitude during hovering. For this task, we model dynamic equations of FWMAV to analyze the stability of longitudinal dynamics under the influence of tail wings. In order to examine the effect of the flapping wings on the fuselage, we carry out measurements on the power train. We develop a FWMAV with two pairs of main flapping wings and tail wings, to achieve stable hovering flight. For autonomous control, PID controllers are designed and onboard sensors are used; 9-degree of freedom (DoF) IMU is for attitude regulation and barometer is for an altitude control. For estimating an altitude, we also apply processing filter. Experimental results show that our strategy can accomplish and sustain autonomous hovering of a FWMAV.

Aerial manipulation tasks necessitate a reliable trajectory planning algorithms to perform complicated tasks. This paper provides a method of developing the locally optimal trajectory for aerial manipulation in constrained environments. We first show differential flatness of the aerial manipulation system when the inertial effect due to equipped robotic arm is compensated with the aid of a robust controller. To find the locally optimal path, we parameterize flat outputs as polynomials of time, and formulate a sequential quadratic programming (SQP) problem. Given a convex mesh representation of environment, we obtain a collision-free trajectory in almost real time, by imposing constraints based on a signed distance metric. We also conduct an experiment of operating an object located in a confined space, which validates effectiveness of the proposed algorithm.

In the paper, we propose a rigid motion segmentation algorithm with the grid-based optical flow. The algorithm selects several adjacent points among grid-based optical flows to estimate motion hypothesis based on a so-called entropy and generates motion hypotheses between two images, thus separates objects which move independently of each other. The grid-based entropy is accumulated as a new motion hypothesis generated and the high value of entropy means that the motion has been estimated inaccurately in the corresponding grid. The motion hypothesis is estimated by three-dimensional rigid transformation and classified by the open-source implementation of density-based spatial clustering of applications with noise (DBSCAN). For the evaluation of the proposed algorithm, we use a self-made dataset captured by ASUS Xtion Pro live RGB-D camera. Our algorithm implemented in the unoptimized MATLAB code spends 170 ms of average computational time per frame, showing the potential for the application to the robust real-time visual odometry.

In some of recent advances in automation of construction equipment, much research has been conducted on the control of hydraulic excavators in both industry and academia for the benefit of safety and efficiency. However, most relevant works have employed model-based control approaches that require a mathematical representation of the target plant. For hydraulic excavators, obtaining a useful dynamic model for control can be challenging due to the nonlinearity of the hydraulic servo system. With this in mind, this paper investigates the feasibility of an online learning control framework based on echo-state networks (ESNs) to the position control of hydraulic excavators. While ESNs are a class of recurrent neural networks, the training of ESNs corresponds to solving a linear regression problem, thus making it suitable for online implementation. By exploiting the dynamic properties of ESNs, the deployed control framework uses the input and output signals of the plant to learn an inverse model, which is then used to simultaneously generate control inputs to track the desired trajectory. Experiments conducted on a 21-ton class hydraulic excavator show the promising results in that accurate tracking is achieved even in the absence of a dynamic model.

This paper presents the planning and control synthesis of cooperative aerial manipulators to carry an unknown object together. The online parameter estimation algorithm is designed to estimate the unknown physical parameters of the common payload such as mass and moment of inertia, without the need of multiaxis force/torque sensors. Based on the augmented adaptive sliding mode controller with the estimated physical parameters, the desired trajectory of each aerial manipulator is generated to track the desired trajectory of corresponding end effector. In order to carry an unknown object safely considering the actuation limit of the hexacopter, we use the task priority to satisfy the unilateral constraints determined by the allowable flight envelope. To validate our approach, the experimental result on a successful transportation by using multiple custom-made aerial manipulators is shown. This result suggests that the proposed approach can be utilized for safe cooperative aerial transportation.

This paper suggests a mapless indoor localization using wifi received signal strength (RSS) of a smartphone, collected by multiple people. A new trajectory learning algorithm by combining a dynamic time warping and a machine learning technique is proposed in order to generate an alternative map. Moreover, we combine particle filter and Gaussian process (GP) for the position estimation, because it can use the alternative map as the probabilistic function (the prior), and can use probabilistic relationship (the likelihood) between wifi RSSs and location. Field experimental results confirm the usefulness of our algorithm when the map is not available and robustness against outliers, in that the accuracy of the proposed localization is similar to that using the true map information.

In this paper, we design an autonomous flight controller for height regulation and bang-bang controller for directional control of a light-weight flapping-wing micro air vehicle (FWMAV) with limited payload. We also present autonomous vision-based target tracking for a FWMAV equipped with a low-cost and light-weight first person view (FPV) camera. We construct a ground station, integrated with control and vision algorithms, which performs the image processing and the computation of control inputs based on the acquired state variables from motion capture system. In addition, we employ a vision algorithm for a low-quality camera to detect a static target with the discussions on the techniques to improve the reliability of visual detection. Experimental results show satisfactory flight performance, achieving the height regulation and directional control, and autonomous vision-based target tracking.

In this paper, we describe a method for using Bayesian inference to create probabilistic point correspondences in video sequences. Using state prior distribution information from the onboard Kalman filter found in most unmanned aerial vehicles, we derive an accurate normal approximation for the movement of points in image space. It is then a direct application of Bayesian inference to find the probability that a feature point found in the current image is the same feature point found in a previous image. The biggest advantage of this method is that it does not require the use of feature descriptors, which can be computationally expensive to compute. The algorithm is tested in two scenarios. The first uses data and images collected from a sample flight to evaluate velocity estimation performance, which is critical for achieving good control. When compared to using a standard matching algorithm based on ORB (oriented FAST and rotate BRIEF) descriptors, our algorithm provides both improved efficiency and accuracy. We then implement the algorithm for online testing as well as develop a simple visual odometry method based on probabilistic correspondences for full state estimation. Flight tests demonstrate fully autonomous indoor hover using only the sensors embedded in the onboard smartphone and sonar.

This paper is about opening and closing an unknown drawer using an aerial manipulator. To accommodate practical applications, it is assumed that the direction of motion and mechanical properties of the drawer are not given beforehand. A multirotor combined with a robotic arm is used for the manipulation task. Typical drawers are allowed to move in only one direction, which constrains the motion of the aerial manipulator while operating a drawer. To analyze this interaction, the dynamic characteristics of the aerial manipulator are modeled. Also, configuration of the aerial manipulator for exerting the desired force to a drawer is presented. To handle the uncertainties associated with the mechanism of a drawer, strategies exploiting velocity of the end effector are employed. The proposed approach is validated with experiments including opening and closing a common drawer, which is detected by a camera mounted in the palm of the end effector.

Sensitivity to illumination conditions poses a challenge when utilizing visual odometry (VO) in various applications. To make VO robust with respect to illumination conditions, they need to be considered explicitly. In this paper, we propose a direct visual odometry method which can handle illumination changes by considering an affine illumination model to compensate abrupt, local light variations during direct motion estimation process. The core of our proposed method is to estimate the relative camera pose and the parameters of the illumination changes by minimizing the sum of squared photometric error with efficient second-order minimization. We evaluate the performance of the proposed algorithm on synthetic and real RGB-D datasets with ground-truth. Our result implies that the proposed method successfully estimates 6-DoF pose under significant illumination changes whereas existing direct visual odometry methods either fail or lose accuracy.

This paper presents planning and control of multiple aerial manipulators for cooperative transportation. Individual aerial manipulators which consist of a hexacopter and 2-DOF robotic arm are controlled by an augmented adaptive sliding mode controller based on a closed-chain robot dynamics. The desired path for each aerial manipulator is obtained by using RRT* to transport an object to the desired position. It also considers the constraints about the grasping point at the end effector. To validate the proposed planning and control algorithm, an experimental result with multiple custom-made aerial manipulators is presented, which involves two aerial manipulators tracking the user-guided command and planned trajectory.

This paper is about fully-distributed support vector machine (SVM) learning over wireless sensor networks. With the concept of the geometric SVM, we propose to gossip the set of extreme points of the convex hull of local data set with neighboring nodes. It has the advantages of a simple communication mechanism and finite-time convergence to a common global solution. Furthermore, we analyze the scalability with respect to the amount of exchanged information and convergence time, with a specific emphasis on the small-world phenomenon. First, with the proposed naive convex hull algorithm, the message length remains bounded as the number of nodes increases. Second, by utilizing a small-world network, we have an opportunity to drastically improve the convergence performance with only a small increase in power consumption. These properties offer a great advantage when dealing with a large-scale network. Simulation and experimental results support the feasibility and effectiveness of the proposed gossip-based process and the analysis.

In this paper, a new motor control input and controller for small-scale electrically powered multirotor aerial vehicles is proposed. The proposed scheme is based on controlling aerodynamic power as opposed to the rotor speed of each motor-rotor system. Electrical properties of the brushless direct current motor are used to both estimate and control the mechanical power of the motor system which is coupled with aerodynamic power using momentum theory analysis. In comparison to current state-of-the-art motor control for multirotor aerial vehicles, the proposed approach is robust to unmodelled aerodynamic effects such as wind disturbances and ground effects. Theory and experimental results are presented to illustrate the performance of the proposed motor control.

Real-time approach for monocular visual simultaneous localization and mapping (SLAM) within a large-scale environment is proposed. From a monocular video sequence, the proposed method continuously computes the current 6-DOF camera pose and 3D landmarks position. The proposed method successfully builds consistent maps from challenging outdoor sequences using a monocular camera as the only sensor, while existing approaches have utilized additional structural information such as camera height from the ground. By using a binary descriptor and metric-topological mapping, the system demonstrates real-time performance on a large-scale outdoor environment without utilizing GPUs or reducing input image size. The effectiveness of the proposed method is demonstrated on various challenging video sequences including the KITTI dataset and indoor video captured on a micro aerial vehicle.

In this paper, we suggest a new 6-DoF velocity estimation algorithm using RGB and depth images. Autonomous control of mobile robots requires their velocity information. There exist numerous researches on estimating and measuring the velocity. However, more investigations are needed related to vision sensors and depth image. In this work, we propose an algorithm for velocity estimation with an RGB-D sensor based on image jacobian matrix usually used in image-based visual servoing. We validate the performance of the proposed estimation algorithm in various environments with the RGB-D benchmark dataset. The velocity estimation results show the high quality of estimated 6-DoF velocity compared to the ground truth velocity.

Online least square support vector regression (LS-SVR) is an extension from a standard SVR for fast learning. In this paper, we consider a combination of the online LS-SVR and semi-supervised learning in order to boost estimation accuracy. The semi-supervised learning is useful for real-time and complex estimation applications because it uses a small amount of the labeled data but sufficient unlabeled data that can be easily obtained. The algorithms are evaluated for two experiments, i.e. state estimation of a robot arm and forecasting of chaotic time-series. The experimental results show that the proposed algorithm yields more accurate estimation than the compared online LS-SVR.

This paper considers a boundary tracking problem using mobile sensor networks, in which we design controllers for the mobile sensors to obtain the boundary of physical events. We set the boundary estimation problem as a classification problem of the region in which the physical events occurs, and employ support vector learning (SVL). By using the hyper-dimensional radius function obtained from SVL, we build the hyper-potential field to generate a velocity vector field which is globally attractive to a desired closed path with circulation at the desired speed. We also study stabilizing the collective configuration of the multiple mobile sensors. To coordinate the mobile sensors in the formation that encloses the boundary, we define virtual phases of mobile sensors and compute the desired speed of each mobile sensors minimizing the level of synchrony of the virtual phases. Both a simulation and an experiment is performed and the results demonstrate that this study provides good performance of the collective boundary tracking.

This paper presents aerial manipulation using a quadrotor with a two-DOF robot arm. By considering a quadrotor and robot arm as a combined system, the kinematic and dynamic models are developed, and an adaptive sliding mode controller is designed. With the controller, an autonomous flight experiment is conducted including picking up and delivering an object, which requires accurate control of a quadrotor and robot arm. Overall result shows that the proposed approach demonstrates satisfactory performance as a potential platform which can be utilized in various applications such as inspection, manipulation, or transportation in remote places.

Efficient environmental monitoring is an important application area of wireless sensor networks or unmanned ground/flying robots. In this paper, predictive modeling of environmental information and its usage for informative path planning is considered. In order to represent information gathered by observations, we use radial basis function (RBF) based approximation with Kalman filter. Also, we employ recurrent neural networks (RNNs) to include dynamic inference of the information process, and train it with echo state network (ESN) approach. Moreover, we introduce a technical approach to informative path planning using the predictive information model. A hybrid decision policy is employed as an illustrative method for reducing overall variance of the model prediction and both a simulation and an experiment are performed to validate the proposed framework.

This paper presents a back stepping controller based on SE(3) to track the desired trajectory for a quad rotor unmanned aerial vehicle. The controller consists of two parts: 1) a position control part and 2) an attitude control part. The position controller is used to track the desired Cartesian coordinates using position and velocity errors, while the attitude controller uses the rotation matrix error and body angular velocity error to stabilize attitude dynamics expressed on SO(3). Simulation results illustrate the more stable tracking performance of the proposed controller in noisy environments in comparison with geometric controller. Experimental results on a micro quad rotor show the satisfactory performance of the proposed controller.

In tracking a mobile target using wireless sensor networks (WSN), efficient sleep scheduling is needed to reduce energy consumption without severely deteriorating the tracking performance. Conventional prediction-based sleep scheduling techniques require position, velocity and even acceleration of the target in order to wake up the nodes according to the next predicted location of the target. In these data, effects of sensor faults and uncertainties are included, which can degrade the overall performance. In order to address this issue, we propose a new predictive target detection algorithm which uses only local measurement, eliminating the need for communication among nodes. The proposed prediction algorithm provides a prediction of the number of detections that will occur starting from the moment when a target enters the sensor coverage area until it leaves. This algorithm is evaluated via experiments including several movement scenarios of the target. The results show that the algorithm accurately reflects the remaining number of target detections until the target leaves. In addition, the prediction algorithm is applied to sleep scheduling and compared with the circle-based sleep scheduling. Our scheduling strategy improves energy efficiency with a very small, negligible loss in the tracking performance.

In this paper, we present a method for using linear matrix inequalities (LMIs) to synthesize controller gains for a quadrotor system. The controller is based on approximate feedback linearization and is structured to allow for tuning similar to proportional-integral-derivative (PID) controllers. The synthesis procedure generates suboptimal gains with respect to mixed${\cal H}_{2}$and${\cal H}_{\infty}$performance cost functions and a pole placement region constraint. The basic procedure is extended to account for dynamic external disturbances, inexact nonlinearity cancellation, multiplicative actuator uncertainty, and saturated integrators in the control loop. The controller is tested in a real-world flight using 10 Hz position updates with 2 cm standard deviation noise to approximate GPS or vision-based control scenarios.

Target localization, whose goal is to estimate the location of an unknown target, is one of the key issues in applications of wireless sensor networks (WSNs). With recent advances in fabrication technology, deployments of large-scale WSNs have become economically feasible. However, there exist issues such as limited communication and the curse of dimensionality in applying machine-learning algorithms such as support vector regression (SVR) on large-scale WSNs. Here, in order to overcome such issues, we propose an ensemble implementation of SVR for the problem of target localization. The convergence property of the localization algorithm using the ensemble SVR is verified, and the robustness of the proposed scheme against measurement noise is analyzed. Furthermore, experimental results confirm that the estimation performance of the proposed method is more accurate and robust to measurement noise than the conventional SVR predictor.

In this paper, we propose to use distributed support vector machine (SVM) training to solve a multi-target tracking problem in wireless sensor networks. We employ gossip-based incremental SVM to obtain the discriminant function. By gossiping the support vectors with neighboring sensor nodes, the local SVM training results can achieve the agreement of the sub-optimal discriminant planes. After training the local SVM at each node, we can calculate the posterior probability of the existence of the targets using Platt's method. By maximum a posterior (MAP), the target trajectories are estimated. In order to validate the proposed tracking framework in wireless sensor networks, we perform two different target-tracking experiments. The experimental results demonstrate that the proposed procedure provides a good estimator, and supports the feasibility of applying the distributed SVM training to the target tracking problems.

In this paper we describe a vision-based algorithm to control a vertical-takeoff-and-landing unmanned aerial vehicle while tracking and landing on a moving platform. Specifically, we use image-based visual servoing (IBVS) to track the platform in two-dimensional image space and generate a velocity reference command used as the input to an adaptive sliding mode controller. Compared with other vision-based control algorithms that reconstruct a full three-dimensional representation of the target, which requires precise depth estimation, IBVS is computationally cheaper since it is less sensitive to the depth estimation allowing for a faster method to obtain this estimate. To enhance velocity tracking of the sliding mode controller, an adaptive rule is described to account for the ground effect experienced during the maneuver. Finally, the IBVS algorithm integrated with the adaptive sliding mode controller for tracking and landing is validated in an experimental setup using a quadrotor.

This paper considers autonomous onboard hovering flight control of a micro quadrotor using a strapdown optical flow sensor which is conventionally used for desktop mice. The vehicle considered in this paper can carry only a few dozen grams of payload, therefore conventional camera-based optical flow methods are not applicable. We present autonomous hovering flight control of the micro quadrotor using a single-chip optical flow sensor, implemented on an 8-bit microprocessor without external positioning sensors. Detailed description of all the system components is provided along with evaluation of the accuracy. Experimental results from flight tests are validated with the ground-truth data provided by a high-accuracy motion capture system.

Object localization using wireless sensor networks (WSN) often requires data from many sensor nodes and different types of sensors for position estimation. This incurs a heavy communication load, which can cause packet loss, communication delay and much energy consumption, deteriorating the performance of object localization. Here we employ an event-driven Gaussian process in order to learn the position of an unknown object using WSN with multiple types of sensors. In the event-driven framework, each sensor node transmits data only when decision criteria are satisfied. We consider the error-bounded algorithm as the decision criteria based on the measurement history of each sensor node. The overall communication between sensor nodes is reduced, thus increasing energy-efficiency of the network and relieving the concentration of communication traffic at the base node. Experiments to track the position of a mobile robot are conducted using a multi-sensor WSN, and the comparison is made between the event-driven framework and the conventional approach in which sensors transmit data at a constant sampling rate. Experimental results demonstrate the efficiency and accuracy of the proposed event-driven Gaussian process approach.

This paper considers a target localization problem whose goal is to estimate the location of an unknown object. It is one of the key issues in applications of wireless sensor networks (WSNs). With recent advances in fabrication technology, deployment of a large WSNs has become economically feasible. On the other hand, this has caused the curse of dimensionality in applying learning algorithms such as support vector regression (SVR). To handle this, we use an ensemble implementation of SVRs for target localization and validate it experimentally. This paper draws a comparison between the conventional SVR method and the proposed method in terms of the accuracy and robustness. Experimental results show that the prediction performance of the proposed method is more accurate and robust to the measurement noise than conventional SVR predictor.

This correspondence paper presents the validation of a formation flight control technique with obstacle avoidance capability based on nonlinear model predictive algorithms. Control architectures for multi-agent systems employed in this correspondence paper can be categorized as centralized, sequential decentralized, and fully decentralized methods. Centralized methods generally have better performance than decentralized methods. However, it is well known that the performance of the centralized methods for formation flight degrades when there exists communication failure among the vehicles, and they require more computation time than the decentralized method. This correspondence paper evaluates the control performance and the computation time reduction of the sequential decentralized and fully decentralized methods in comparison with the centralized method and shows that the fully decentralized method can be made effective against short term communication failure. The control inputs for formation flight are computed by nonlinear model predictive control (NMPC). The control input saturation and state constraints are incorporated as inequality constraints using Karush Kuhn Tucker conditions in the NMPC framework, and the collision avoidance can be considered in real time. The proposed schemes are validated by numerical simulations, which include the process and measurement noise for more realistic situations.

This paper studies an active steering problem of unmanned ground vehicles (UGVs) when avoiding obstacles during sensor based navigation in unknown environments. The overall problem is treated using the nonlinear model predictive framework, in which the sensor information of a limited sensing range is incorporated online. Results show that the introduction of the modified parallax effectively reflects the threat of obstacles and consequently achieves safe navigation in unknown environments satisfying dynamic constraints.

In this paper, we present an efficient method for finding collision-free trajectory for multiple unmanned aerial vehicles (UAVs) with kinematic constraints and for their rendezvous to form a formation. First, we construct a visibility graph that supports a minimum turning radius constraint when constructing the graph, so that additional smoothing process is not necessary. Second, we modify the standard A* to consider velocity conditions for rendezvous and collision avoidance with obstacles or other UAVs. Permitting velocity decrease only when it is required that the robot slow down the speed, unnecessary node expansions are avoided. This multi-vehicle problem is solved in a decoupled manner. In order to show the effectiveness of this approach, we present simulation results of rendezvous and independent flight for multiple UAVs.

Learning algorithms have enjoyed numerous successes in robotic control tasks. In problems with time-varying dynamics, online learning methods have also proved to be a powerful tool for automatically tracking and/or adapting to the changing circumstances. However, for safety-critical ap- plications such as airplane flight, the adoption of these algorithms has been significantly hampered by their lack of safety, such as "stability," guarantees. Rather than trying to show difficult, a priori, stability guar- antees for specific learning methods, in this paper we propose a method for "monitoring" the controllers suggested by the learning algorithm on- line, and rejecting controllers leading to instability. We prove that even if an arbitrary online learning method is used with our algorithm to control a linear dynamical system, the resulting system is stable.

Autonomous helicopter flight represents a challenging control problem, with complex, noisy, dynamics. In this paper, we describe a successful application of reinforcement learning to autonomous helicopter flight. We first fit a stochastic, nonlinear model of the helicopter dynamics. We then use the model to learn to hover in place, and to fly a number of maneuvers taken from an RC helicopter competition.

This paper presents a flight management system (FMS) implemented as on-board intelligence for rotorcraft-based unmanned aerial vehicles (RUAV's), in order to gradually refine given abstract mission commands into real-time control signals for each vehicle. A strategy planner uses the probabilistic decision making algorithms to determine suboptimal action at each time step. A graphical interface on ground station enables human intervention. We derive nonlinear dynamics model upon which we design a tracking control layer using nonlinear model predictive control and integrate with a trajectory generator for logistical action planning. The proposed structure has been implemented on Berkeley RUAVs and validated in probabilistic pursuit-evasion games to show the possibility of intelligent flying robots.

We consider the problem of having a team of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) pursue a second team of evaders while concurrently building a map in an unknown environment. We cast the problem in a probabilistic game theoretical framework, and consider two computationally feasible greedy pursuit policies: local-mar and global-max. To implement this scenario on real UAVs and UGVs, we propose a distributed hierarchical hybrid system architecture which emphasizes the autonomy of each agent, yet allows for coordinated team efforts. We describe the implementation of the architecture on a fleet of UAVs and UGVs, detailing components such as high-level pursuit policy computation, map building and interagent communication, and low-level navigation, sensing, and control. We present both simulation and experimental results of real pursuit-evasion games involving our fleet of UAVs and UGVs, and evaluate the pursuit policies relating expected capture times to the speed and intelligence of the evaders and the sensing capabilities of the pursuers.