In this paper, a novel prototype for harvesting table-top grown strawberries is presented, that is minimalist in its footprint interacting with the fruit. In our methodology, a smooth trapper manipulates the stem into a precise groove location at which a distant laser beam is focused. The tool reaches temperatures as high as 188°Celsius and as such killing germs and preventing the spread of local plant diseases. The burnt stem wound preserves water content and in turn the fruit shelf life. Cycle and cut times achieved are 5.56 and 2.88 seconds respectively in successful in-door harvesting demonstration. Extensive experiments are performed to optimize the laser spot diameter and lateral speed against the cutting time.

Tractors and heavy machinery have been used for decades to improve the quality and overall agriculture production. Moreover, agriculture is becoming a trend domain for robotics, and as a consequence, the efforts towards automatizing agricultural task increases year by year. However, for autonomous applications, accident prevention is of prior importance for warrantying human safety during operation in any scenario. This paper rephrases human safety as a classification problem using a custom distance criterion where each detected human gets a risk level classification. We propose the use of a neural network trained to detect and classify humans in the scene according to these criteria. The proposed approach learns from real-world data corresponding to an open-field scenario and is assessed with a custom risk assessment method.

In this paper, a novel prototype for hanging produce harvesting is presented, that is productive, versatile, and robust. In our methodology, the robot-mounted tube approaches, and eventually surrounds the produce of interest at the entry side, that can be as small as the produce diameter, plus a small margin. The stem is then cut by a laser beam, with the optics set up for a distant focal point. Such arrangement allows for minimal hardware at the produce-entry side and in turn, the interaction, and possible disturbance to the local environment. This is essential for fruit reachability and avoiding it dislocation. Experiments has been conducted to drive the laser power to time of cut relation, as well as successful demonstration to sample harvest strawberry.

We propose an approach for robot-supervised learning that automates label generation for semantic segmentation with Convolutional Neural Networks (CNNs) for crop row detection in a field. Using a training robot equipped with RTK GNSS and RGB camera, we train a neural network that can later be used for pure vision-based navigation. We test our approach on an agri-robot in a strawberry field and successfully train crop row segmentation without any hand-drawn image labels. Our main finding is that the resulting segmentation output of the CNN shows better performance than the noisy labels it was trained on. Finally, we conduct open-loop field trials with our agri-robot and show that row-following based on the segmentation result is accurate enough for closed-loop guidance. We conclude that training with noisy segmentation labels is a promising approach for learning vision-based crop row following.

Safety integration components for robotic applications are a mandatory feature for any autonomous mobile application, including human avoidance behaviors. This paper proposes a novel parametrizable scene risk evaluator for open-field applications that use humans motion predictions and pre-defined hazard zones to estimate a braking factor. Parameters optimization uses simulated data. The evaluation is carried out by simulated and real-time scenarios, showing the impact of human predictions in favor of risk reductions on agricultural applications.

Selectively picking a target fruit surrounded by obstacles is one of the major challenges for fruit harvesting robots. Different from traditional obstacle avoidance methods, this paper presents an active obstacle separation strategy that combines push and drag motions. The separation motion and trajectory are generated based on the 3D visual perception of the obstacle information around the target. A linear push is used to clear the obstacles from the area below the target, while a zig-zag push that contains several linear motions is proposed to push aside more dense obstacles. The zig-zag push can generate multi-directional pushes and the side-to-side motion can break the static contact force between the target and obstacles, thus helping the gripper to receive a target in more complex situations. Moreover, we propose a novel drag operation to address the issue of mis-capturing obstacles located above the target, in which the gripper drags the target to a place with fewer obstacles and then pushes back to move the obstacles aside for further detachment. Furthermore, an image processing pipeline consisting of color thresholding, object detection using deep learning and point cloud operation, is developed to implement the proposed method on a harvesting robot. Field tests show that the proposed method can improve the picking performance substantially. This method helps to enable complex clusters of fruits to be harvested with a higher success rate than conventional methods.

This work proposes a new strategy for autonomous navigation of mobile robots in sugarcane plantations based on thermal imaging. Unlike ordinary agricultural fields, sugarcane farms are generally vast and accommodates numerous arrangements of row crop tunnels, which are very tall, dense and hard-to-access. Moreover, sugarcane crops lie in harsh regions, which hinder the logistics for employing staff and heavy machinery for mapping, monitoring, and sampling. One solution for this problem is TIBA (Tankette for Intelligent BioEnergy Agriculture), a low-cost skid-steering mobile robot capable of infiltrating the crop tunnels with several sensing/sampling systems. The project concept is to reduce the product cost for making the deployment of a robot swarm feasible over a larger area. A prototype was built and tested in a bioenergy farm in order to improve the understanding of the environment and bring about the challenges for the next development steps. The major problem is the navigation through the crop tunnels, since most of the developed systems are suitable for open field operations and employ laser scanners and/or GPS/IMU, which in general are expensive technologies. In this context, we propose a low-cost solution based on infrared (IR) thermal imaging. IR cameras are simple and inexpensive devices, which do not pose risks to the user health, unlike laser-based sensors. This idea was highly motivated by the data collected in the field, which have shown a significant temperature difference between the ground and the crop. From the image analysis, it is possible to clearly visualize a distinguishable corridor and, consequently, generate a straight path for the robot to follow by using computationally efficient approaches. A rigorous analysis of the collected thermal data, numerical simulations and preliminary experiments in the real environment were included to illustrate the efficiency and feasibility of the proposed navigation methodology.

This paper presents a novel cable-driven gripper with perception capabilities for autonomous harvesting of strawberries. Experiments show that the gripper allows for more accurate and faster picking of strawberries compared to existing systems. The gripper consists of four functional parts for sensing, picking, transmission, and storing. It has six fingers that open to form a closed space to swallow a target strawberry and push other surrounding berries away from the target. Equipped with three IR sensors, the gripper controls a manipulator arm to correct for positional error, and can thus pick strawberries that are not exactly localized by the vision algorithm, improving the robustness. Experiments show that the gripper is gentle on the berries as it merely cuts the stem and there is no physical interaction with the berries during the cutting process. We show that the gripper has close-to-perfect successful picking rate when addressing isolated strawberries. By including internal perception, we get high positional error tolerance, and avoid using slow, high-level closed-loop control. Moreover, the gripper can store several berries, which reduces the overall travel distance for the manipulator, and decreases the time needed to pick a single strawberry substantially. The experiments show that the gripper design decreased picking execution time noticeably compared to results found in literature.

This paper presents a novel agricultural robot for greenhouse applications. In many greenhouses, including the greenhouse used in this work, sets of pipes run along the floor between plant rows. These pipes are components of the greenhouse heating system, and doubles as rails for trolleys used by workers. A flat surface separates the start of each rail set at the greenhouse headland. If a robot is to autonomously drive along plant rows, and also be able to move from one set of rails to the next, it must be able to locomote both on rails and on flat surfaces. This puts requirements on mechanical design and navigation, as the robot must cope with two very different operational environments. The robot presented in this paper has been designed to overcome these challenges and allows for autonomous operation both in open environments and on rails by using only low-cost sensors. The robot is assembled using a modular system created by the authors and tested in a greenhouse during ordinary operation. Using the robot, we map the environment and automatically determine the starting point of each rail in the map. We also show how we are able to identify rails and estimate the robots pose relative to theses using only a low-cost 3D camera. When a rail is located, the robot makes the transition from floor to rail and travels along the row of plants before it moves to the next rail set which it has identified in the map. The robot is used for UV treatment of cucumber plants.

This paper presents a novel control architecture for controlling a moving remote center of motion in addition to the end-effector motion during robotic surgery. In minimally invasive surgery, it is common to require that the point at which the robot enters the body, called the incision point or the trocar, does not allow for any lateral motion. It is generally considered that no motion should be applied to this point in order to avoid inflicting damage to the patient's skin. However, in surgery, the patient's body may be moving, for example due to breathing or the beating of the heart. In order to compensate for this motion-or if we for some other reason want to leverage the possible motion of the incision point to improve performance in any other way-we derive a new framework which allows us to actively control the motion both at the incision point and the end effector. The novelty of the approach lies in the possibility of controlling both the incision point and the end effector to follow a trajectory, and that we find a Jacobian matrix that satisfies the velocity constraints in both the end-effector and the incision point frames. This allows us to formulate a framework that is not only suited for control, but also for analyzing the condition number of the Jacobian and avoid any singular configurations that may arise either as a result of the constrained motion or the manipulator geometry. The approach is verified experimentally on a redundant industrial manipulator.

This paper presents a new set of approaches for teleoperation of mobile manipulators with on-board cameras. Mobile manipulators consist of a robotic arm which provides for interaction and manipulation, and a mobile base which extends the workspace of the arm. While the position of the onboard camera is determined by the base motion, the principal control objective is the motion of the manipulator arm. This calls for intelligent control allocation between the base and the manipulator arm in order to obtain intuitive control of both the camera and the arm. We implement virtual mass-spring-damper forces between the end-effector and the camera so that the camera follows the end-effector with an overdamped characteristics. The operator therefore only needs to control the end-effector motion, while the vehicle with the camera will follow naturally. The operator is thus able to control the more than six degrees of freedom of the vehicle and manipulator through a standard haptic device. The control allocation problem, i.e., whether the vehicle or manipulator arm actuation is applied, is then performed automatically so that the operator can concentrate on the manipulator motion.

This paper presents a novel control architecture for enhanced operator awareness and improved safety in telesurgical systems. We introduce an intermediate layer between the master and slave sides which allows us to modify the slave motion for safety and the force feedback for operator awareness. The intermediate layer is then designed with a performance objective for the specific task at hand. The control scheme is validated via experiments using a suture and an industrial manipulator. More specifically we show that operator awareness should be implemented as an integrated part of the safety level to maintain safety during surgery.

This is a correction to [Duindam and Stramigioli, “Singularity-free dynamic equations of open-chain mechanisms with general holonomic and nonholonomic joints,” IEEE Trans. Robot., vol. 24, no. 3, pp. 527-526, Jun. 2008] where the singularity-free dynamic equations of mechanical systems with Euclidean or non-Euclidean configuration spaces are presented. We present the correct explicit expressions of the equations presented in the above referenced paper.

In this paper, we present the experimental results of a new spray paint algorithm presented in previous publications. Both theory and simulations indicate that the proposed method allows a robotic manipulator to paint a given surface using substantially lower joint torques than with conventional approaches. In this paper, we confirm this by implementing the algorithm on an ABB robot and we find that the joint torques needed to follow the trajectory are substantially lower than for the conventional approach. The approach presented is based on the observation that a small error in the orientation of the end effector does not affect the quality of the paint job. It is far more important to maintain constant velocity for the entire trajectory. We thus propose to allow a small error in the specification of the end-effector orientation, and we show how this allows us to obtain a higher constant speed throughout the trajectory. In addition, to improve the uniformity of the paint coating we are also able perform the paint job in less time.

Robotic manipulators on non-inertial platforms, such as ships, have to endure large inertial forces due to the non-inertial motion of the platform. When the non-inertial platform's motion is known, motion planning and control algorithms can eliminate these perturbations-in fact, in some situations the motion planning algorithms can even leverage the inertial forces to more cheaply move to a target point. However, for many non-inertial platforms, the motion is unknown. In this paper we investigate how prediction errors and the choice of the prediction horizon affect the motion planning and control of robots mounted on a non-inertial base with a particular focus on seaborne platforms. We study the following three aspects: (i) We study prediction of ship motion and how prediction errors affect the motion planning and control of the manipulator. (ii) We evaluate the relationship between prediction accuracy and control. In particular, we study what prediction horizon length is useful for motion planning and control. We also consider how uncertainties in the ship motion predictions map to uncertainties in the future state of the robot and how to include the variance in the cost function to increase the optimal horizon length. (iii) Finally, we study a receding horizon approach, which re-solves the optimal control problem on-line over a horizon as determined to be meaningful from (ii). Several simulations are presented and, to our knowledge, for the first time experiments of ship-manipulator systems based on real ship motion data are presented.

In this paper, we present a method for increasing the speed at which a standard industrial manipulator can paint a surface. The approach is based on the observation that a small error in the orientation of the end effector does not affect the quality of the paint job. It is far more important to maintain constant velocity throughout the trajectory. We consider the freedom in the end-effector orientation as functional redundancy and represent the restriction on the orientation error as barrier functions or linear matrix inequalities. In doing this, we cast the problem of finding the optimal orientation at every time step into a convex optimization problem that can be solved very efficiently and in real time. We show that the approach allows the end effector to maintain a higher constant velocity throughout the trajectory guaranteeing uniform paint coating and substantially reducing the time needed to paint the object.

Robotic manipulators on ships and platforms suffer from large inertial forces due to the non-inertial motion of the ship or platform. When operating in high sea state, operation of such manipulators can be made more efficient and robust if these non-inertial effects are taken into account in the motion planning and control systems. Motivated by this application, we present a rigorous and singularity-free formulation of the dynamics of a robotic manipulator mounted on a non-inertial base. We extend the classical dynamics equations for a serial manipulator to include the 6-DoF motion of the non-inertial base. Then, we show two examples of a 1-DoF and a 4-DoF manipulator to illustrate how these non-inertial effects can be taken into account in the motion planning.

In a wide range of applications the orientation of a rigid body does not need to be restricted to one given orientation, but can be given as a continuous set of frames. We address the problem of defining such sets and to find simple tests to verify if an orientation lies within a given set. The unit quaternion is used to represent the orientation of the rigid body and we develop three different sets of orientations that can easily be described by simple constraints in quaternion space. The three sets discussed can also be described as convex cones in Ropf3defined by different norms. By describing the sets as convex cones and using certain properties of dual cones, we are able find simpler representations for the set of orientations and computationally faster and more accurate tests to verify if a quaternion lies within the given set.

In this paper we show how a continuous set of orientations can be represented as a positive definiteness test on a given matrix. When this continuous set is restricted by the maximum allowed orientation error in some or all directions it is shown that the requirement for an orientation to satisfy these restrictions is equivalent to positive definiteness for a certain matrix. The problem of finding the optimal orientation that satisfies these restrictions is hence transformed into an optimisation problem on the Riemannian manifold of linearly constrained symmetric positive definite matrices. Thus, the problem of finding the optimal orientation can be solved as a standard optimisation problem with the constraints written in the form of linear matrix inequalities or barrier functions. Linear matrix inequalities have been extensively studied in the optimisation communities and good and efficient algorithms are available.