This paper builds upon the well-established ML-based grasping technique, known as the Grasp-Rectangle (GR) method. The original GR method made two simplifying assumptions: it was designed exclusively for two-finger grippers, and it assumed that the gripper would approach objects solely from a top-down perspective on a horizontal surface. We have extended the GR method, for a multi-finger hand beyond these assumptions to (1) enable grasping from top and side views and (2) engage multiple points of contact, enhancing the algorithm’s overall performance. Our approach leverages geometric cues extracted from object images to calculate the optimal grasp pose and contact points, thereby enhancing grasp reliability. Extensive testing was conducted using a 7DOF robotic arm equipped with a 7-DOF 3-finger gripper. We achieved an accuracy of 98.6% on the Cornell Grasping Dataset with a processing time of 120 milliseconds. Furthermore, when assessing object grasping from both top and side perspectives, our algorithm delivered successful grasps at rates of 95% and 96%, respectively. These findings are rooted in a comprehensive series of tests performed across a diverse array of objects.

In the pursuit of object retrieval using an aerial manipulator, developing robust visual servoing techniques in the presence of projection and motion model uncertainties is paramount. This paper proposes a novel approach to conducting image-space planning within the controllable-space of the aerial manipulator. Our new strategy resolves the inherent challenge of adhering to a piecewise linear camera trajectory which is infeasible for an aerial manipulator due to the platform’s underactuation and presence of secondary tasks for visual servoing. Through this approach, we introduce center of gravity alignment and camera orientation potential fields without relying on specific degrees of freedom from the arm. Moreover, we introduce a new approach that utilizes an image-resolution scaling technique involving an adaptive virtual camera focal length, leading to a numerically well-conditioned image Jacobian. Our proposed framework maintains robustness to the uncertainty in the intrinsic parameters of the camera. We substantiate the efficacy of our methodology through experiments conducted in a realistic physics-based simulation environment.

Unmanned Aerial Manipulators (UAM) are gaining attention within the unmanned aerial systems research community. They can be used for aerial manipulation tasks such as object retrieval from confined and hard-to-reach spaces. The coupled dynamics between the arm and the base and also the limited flight time would require the development and implementation of optimal motion planning and robust flight control strategies. In this paper, we propose a novel integrated planning and control strategy for object retrieval. A new kinodynamic version of the Rapidly-exploring Random Tree (RRT*), called Lazy-Steering-RRT*, is developed for planning UAM’s motion from its start to a pre-grasp state, while keeping the motion of the arm to a minimum. This planning can be carried out on the fly by using Machine-Learning-based techniques to construct the edges in the search tree in a time-efficient way. This facilitates re-planning, as the environment is gradually sensed by limited-range sensors onboard. Once the UAM reaches the pre-grasp state at the end of the motion cycle, an RRT approach is then utilized, where motions of the base and the arm are coordinated for reaching and grasping the object. A novel partitioned control approach, composed of model predictive and PID controllers, is utilized for the UAM to track the planned trajectory. The overall motion planning and control algorithms have been implemented in simulation using Multibody Physics Engines and a number of representative simulation runs are presented. Our results show that the approach is effective in successfully executing the object retrieval task even in confined spaces.Note to Practitioners—This research was motivated by the problem of retrieving an object from a confined space such as that in a collapsed building using an unmanned aerial manipulator. Our approach has the following advantages over existing methods: a) Planning and re-planning can be carried out on the fly, b) No prior information about the environment is needed, c) High-frequency energy-optimal control of the UAM is carried out, and d) the planner and the controller are coupled. In this paper, we suggest some novel approaches on both planning and control aspects of the navigation and also on the integration of these two modules to provide a navigation package that meets all the requirements in the retrieval task. Furthermore, our algorithm is computationally-light, therefore, well suited for on the fly planning and control.

Rotary-wing flying machines draw attention within the UAV community for their in-place hovering capability, and recently, holonomic motion over fixed-wings. In this paper, we investigate about the power-optimality in a mono-spinner, i.e., a class of rotary-wing UAVs with one rotor only, whose main body has a streamlined shape for producing additional lift when counter-spinning the rotor. We provide a detailed dynamic model of our mono-spinner. Two configurations are studied: (1) a symmetric configuration, in which the rotor is aligned with the fuselage's COM, and (2) an asymmetric configuration, in which the rotor is located with an offset from the fuselage's COM. While the former can generate an in-place hovering flight condition, the latter can achieve trajectory tracking in 3D space by resolving the yaw and precession rates. Furthermore, it is shown that by introducing a tilting angle between the rotor and the fuselage, within the asymmetric design, one can further minimize the power consumption without compromising the overall stability. It is shown that an energy optimal solution can be achieved through the proper aerodynamic design of the mono-spinner for the first time.

We present an integrated grasping system for a mobile manipulator to grasp an unknown object of interest (OI) in an unknown environment. The system autonomously scans its environment, models the OI, plans and executes a grasp, while taking into account base pose uncertainty. Due to inherent line of sight limitations in sensing, a single scan of the OI often does not reveal enough information to complete grasp analysis; as a result, our system autonomously builds a model of an object via multiple scans from different locations until a grasp can be performed. A volumetric next-best-view (NBV) algorithm is used to model an arbitrary object and terminates modeling when force closure for the gripper is satisfied. Two experiments are presented: i) modeling and registration error is reduced by selecting viewpoints with more scan overlap, and ii) model reconstruction and grasps are successfully achieved while experiencing base pose uncertainty.

The seminal work cited in [1],[2] showed, for the first time, that flight stability of quadcopters would be possible in case of one or even multiple rotor failures. However, the quadcopter can remain airborne only by going through a spinning maneuver about an axis, fixed w.r.t the vehicle (i.e., resolved yaw). Furthermore, positional control can be achieved by periodically tilting this axis. This paper builds upon this concept with two major improvements: (1) introducing a precise aerodynamic model of propellers that takes the flapping torque due to unbalanced lifting force in the advancing and retreating blades subjected to freestream, into account, and (2) adding to the stability and flight efficiency of the quadcopter by introducing symmetric fixed tilting angles to the trust vectors. In our previous work [3], it was shown how the flight stability and energy efficiency can be improved by introducing fixed tilting angles in the thrust vectors. For controlled crash landing in case of one rotor failure, where a resolved yaw maneuver would be inevitable, introducing a titling angle in rotors can generate a reasonable resolved-rate-yaw spinning speed to keep the quadcopter airborne at a lower rotational speed of the blades by taking advantage of the freestream generated by spinning. This tilting angle would also lead to passive stability in yaw motion of the quadcopter before the failure. Our hypothesis was successfully tested via simulations.

During a robot to human object handover task, several intended or unintended events may occur with the object - it may be pulled, pushed, bumped or simply held - by the human receiver. We show that it is possible to differentiate between these events solely via tactile sensors. Training data from tactile sensors were recorded during interaction of human subjects with the object held by a 3-finger robotic hand. A Bag of Words approach was used to automatically extract effective features from the tactile data. A Support Vector Machine was used to distinguish between the four events with over 95 percent average accuracy.

A special type of rotary-wing Unmanned Aerial Vehicles (UAV), called Quadcopter have prevailed to the civilian use for the past decade. They have gained significant amount of attention within the UAV community for their redundancy and ease of control, despite the fact that they fall under an under-actuated system category. They come in a variety of configurations. The “+” and “x” configurations were introduced first. Literature pertinent to these two configurations is vast. However, in this paper, we define 6 additional possible configurations for a Quadcopter that can be built under either “+” or “x” setup. These configurations can be achieved by changing the angle that the axis of rotation for rotors make with the main body, i.e., fuselage. This would also change the location of the COM with respect to the propellers which can add to the overall stability. A comprehensive dynamic model for all these configurations is developed for the first time. The overall stability for these configurations are addressed. In particular, it is shown that one configuration can lead to the most statically-stable platform by adopting damping motion in Roll/Pitch/Yaw, which is described for the first time to the best of our knowledge.

We incorporate a randomized kinodynamic path planning approach with image-based control of a robotic arm equipped with an in-hand camera. The proposed approach yields continuously differentiable camera trajectories by taking camera dynamics into account, while accounting for a critical set of image and physical constraints at the planning stage. The proposed planner explores the camera state space for permissible trajectories by iteratively extending a search tree in this space and simultaneously tracking these trajectories in the robot configuration space. The planned camera trajectories are projected into the image space to obtain desired feature trajectories which are then tracked using an image-based visual servoing scheme. We validate the effectiveness of the proposed framework in incorporating the aforementioned constraints through a number of visual servoing experiments on a six-degree-of-freedom robotic arm. We also provide empirical results that demonstrate its performance in the presence of uncertainties, and accordingly suggest additional planning strategies to increase robustness with respect to possible deviations from planned trajectories.

We address the problem of incorporating path planning with image-based control of a wheeled mobile manipulator (WMM) performing visually-guided tasks in complex environments. The WMM consists of a wheeled (non-holonomic) mobile platform and an on-board robotic arm equipped with a camera mounted at its end-effector. The visually-guided task is to move the WMM from an initial to a desired location while respecting image and physical constraints. We propose a kinodynamic planning approach that explores the camera state space for permissible trajectories by iteratively extending a search tree in this space and simultaneously tracking these trajectories in the WMM configuration space. We utilize weighted pseudo-inverse Jacobian solutions combined with a null space optimization technique to effectively coordinate the motion of the mobile platform and the arm. We also present the preliminary results obtained by executing the planned trajectories on a real WMM system via a decoupled control scheme where the on-board arm is servo controlled along the planned feature trajectories while the mobile platform is simultaneously controlled along its trajectory using a state feedback tracking method.

In this paper we incorporate a randomized kinodynamic path planning approach with image-based control for a robotic arm equipped with an in-hand camera in a servoing task. The proposed approach yields C2-smooth camera trajectories by taking camera dynamics into account while accounting for a critical set of image and physical constraints. The proposed planner explores the camera state space (i.e., a space of camera poses and velocities) for permissible trajectories by iteratively extending a search tree in this space and simultaneously tracking these trajectories in the robot configuration space (i.e., joint space). The planned camera trajectories are then projected into the image space to obtain desired feature trajectories. In the execution stage an image-based visual servoing scheme is then adopted to track the feature trajectories. The effectiveness of the proposed approach has been experimentally demonstrated on a robotic arm with an in-hand camera executing servoing tasks in complex environments.

We incorporate sampling-based global path planning with Visual Servoing (VS) for a robotic arm equipped with an in-hand camera. The path planning accounts for a number of constraints: 1) maintaining continuous visibility of the target within the camera's field of view, 2) avoiding visual occlusion of target features caused by the workspace obstacles, robot's body, or the target itself, 3) avoiding collision with physical obstacles or self collision, and 4) joint limits. Incorporating these constraints enhances the applicability of VS to significantly more complex environments/tasks, thereby making the resulting VS much more robust. The proposed planner explores the camera space, i.e. 3D Cartesian space, for permissible camera paths satisfying the aforementioned constraints by iteratively extending a search tree in camera space and simultaneously tracking these paths in the robot's joint space using a local planner. The planned camera path is then projected into the image space and tracked using an image-based visual servoing scheme. The validity and effectiveness of the proposed approach in accomplishing VS tasks in complex environments are demonstrated through a number of simulations on a 6-dof robot arm moving among obstacles.

In this paper the design and development of a crawling robot for inspection of live water pipes are addressed. The mechanical design of the robot is described in detail. The governing dynamics equations of the robot moving against water flow as well as gravity in a straight pipe are also derived. Specifically, the hydrodynamic forces exerted on the robot when moving in a live pressurized pipe are taken into account. Two fuzzy-logic based control strategies are adopted. The first one is to maintain a constant translational speed in robot's motion when subjected to flow disturbances that are numerically modeled using step changes in flow velocity within a human-in-the-loop real-time simulation environment, and the second is to steer the real robot inside the pipe while following a numerically modeled time-varying velocity set point with no fluid present in the pipe. The controller parameters were tuned based on data obtained from a human-in-the-loop control system via an artificial neural network.

In this paper the design and development of a crawling robot for inspection of live water pipes is addressed. The mechanical design of the robot is described briefly. A fuzzy-logic based control strategy is adopted to maintain a constant translational speed in robotpsilas motion when subjected to flow disturbances that are numerically modeled using step changes in flow velocity within a simulation environment. The controller has been synthesized in real-time through a human-in-the-loop setting within a virtual reality environment.

A new hybrid motion planning technique based on Harmonic Functions (HF) and Probabilistic Roadmaps (PRM) is presented. The proposed approach consists of incrementally building a Probabilistic Roadmap using information obtained about the workspace topology through the Fluid Dynamic (FD) paradigm based on HFs. The crux of our approach is to identify narrow passages using FD paradigm and pass the information obtained over to a PRM method to build a roadmap to capture the connectivity of free configuration space (C-space) especially in narrow regions. As an extension to our recent works on using Harmonic Function-based Probabilistic Roadmaps (HFPRM) for robotic navigation [1], we propose an Incremental HFPRM (IHFPRM) technique which is more general and can be applied to virtually any type of robot. Simulation results presented in this paper show that the combination of the HF and the PRM works better than each individual in terms of finding a collision free path in environments where narrow passages exist. This technique can be extended to the sensor-based motion planning of robots (mobile and/or articulated) which is the long-term objective in carrying out this research.

This paper introduces a novel visual servoing structure for 3D robot positioning under an eye-to-hand camera configuration. The proposed algorithm is based on image-based visual servoing (IBVS) using only one camera in conjunction with a flat mirror. A landmark mounted on the robot along with its mirror reflection, when viewed by the camera, provides enough information for 3D reasoning based on a 2D image. The governing equations describing the relationship between the robot's velocity and rate of change in image features are fully described. Furthermore, a methodology for on-line estimation of the image Jacobian with no camera calibration is developed. Simulation and experimental results illustrate the robustness of the proposed visual servoing structure.

This paper presents a new hybrid motion planning technique based on harmonic functions (HF) and probabilistic roadmaps (PRM). The proposed harmonic function based probabilistic roadmap (HFPRM) method comprises three phases: in phase one, the Laplace's equation, pertinent to potential flow, in an environment cluttered with obstacles is solved. In phase two, a probabilistic roadmap with a novel sampling scheme is constructed based on information obtained about the environment topology through the HF technique developed in phase one. The roadmap is then searched for the shortest path in phase three. Simulation results presented in this paper show that the combination of the HF and the PRM works better than each individual in terms of finding a collision free path in environments where narrow passages exist. The proposed HFPRM method can be extended to sensor-based motion planning problem in environments not known a priori.

A novel rendezvous-guidance algorithm is proposed in this paper for the robotic interception of moving objects. The object is assumed to be moving on an a priori unknown trajectory, though, its instantaneous state is visually observed and fed to the proposed algorithm. The output of the guidance algorithm is an instantaneous end-effector velocity command with the largest possible closing velocity component while not violating the rendezvous criteria and the dynamic limitations of the actuators of the robot. The proposed method reduces the interception-times achieved via conventional tracking methods as demonstrated by simulation examples.

This paper addresses a general class of problems for sensor-based path planning and exploration for robots moving in unknown environments. The robot is assumed to be equipped with a large number of proximity sensors mounted on its body (a "skin" sensor). Robot's own motion is used to sense the free space, analogous to a blind person "groping and exploring" using his sense of touch. This sensed free space can be memorized and used to further plan the robot's motion. Using this key idea, we propose a general framework for such skin sensor based motion planning that is valid for robots with large degrees of freedom. The specific motion-planning algorithm developed uses a variant of the probabilistic roadmap method. A novel "metric" based on the notion of C-zone map for robot's movement that results in efficient exploration of configuration space is proposed and implemented. Planar simulations demonstrate our results.

We propose a mode-sequential task sharing-for co-operating robot manipulators carrying out a pick-and-place task sharing in a common workspace. As the name implies, in this mode, each individual robot completes part of the same task. The first manipulator picks up the part(s) and directly passes it over to the second manipulator (like a baton being passed from one runner to the other in a relay race), which completes the task by placing the part at its desired goal location. The point at which the transition, i.e., passing over the part, occurs is the rendezvous point. We analyse this approach to minimize the total task time subject to dynamic constraints of the robots. A key step is to determine the optimal rendezvous point (ORP) that results in the optimal task time. We present an algorithm to determine the ORP and show that our approach results in a speed up by a factor of more than two over the conventional single manipulator case.

An Ideal Proportional Navigation Guidance (IPNG) technique is applied to the problem of intercepting a fast maneuvering object in real time using an industrial robotic manipulator. A hybrid method is utilized, in which IPNG is combined with a conventional tracking method to ensure smooth interception. The method is suited for the interception of both fast maneuvering as well as slow maneuvering targets, and does not require long term target trajectory prediction. Experimental results are reported.

Presents an approach to online, robot-motion planning for moving-object interception. The proposed approach utilizes a navigation-guidance-based technique, that is robust and computationally efficient for the interception of fast-maneuvering objects. Navigation-based techniques were originally developed for the control of missiles tracking free-flying targets. Unlike a missile, however, the end-effector of a robotic arm is connected to the ground, via a number of links and joints, subject to kinematic and dynamic constraints. Also, unlike a missile, the velocity of the robot and the moving object must be matched for a smooth grasp, thus, a hybrid interception scheme, which combines a navigation-based interception technique with a conventional trajectory tracking method is proposed herein for intercepting fast-maneuvering objects. The implementation of the proposed technique is illustrated via numerous simulation examples.

This paper presents a novel approach to online robot-motion planning for moving-object interception. A hybrid scheme, which combines a navigation-based technique with a conventional trajectory tracking method, is proposed for intercepting fast-manoeuvring objects. The implementation of the proposed technique is discussed via a simulation example.