Underwater vehicle manipulator systems (UVMS) are increasingly popular platforms for performing subsea operations that require precision manipulation. While there is high demand for fully autonomous or even semi-autonomous systems, most UVMS still require human support teams. Developing new hardware and algorithms for autonomous underwater manipulation is challenging. Simulations do not capture the full complexity of the underwater environment, and deploying a UVMS at sea for testing/validation is resource-intensive and expensive. In this paper, we present a physical testbed for underwater manipulation that bridges the gap between simulation and full field trials. The underWater Arm-Vehicle Emulator (WAVE) is a 10-degree of freedom system designed to replicate an inspection-class UVMS. WAVE includes an underwater perception sensor and has 2 operating modes: rigid or passive-mode. In passive-mode, the ROV body can pitch similar to how a dynamically-coupled underactuated UVMS without pitch control would rotate during manipulation tasks. To validate the overall design and passive pitch concept, we evaluated the testbed during underwater experiments in energetic conditions at a wave basin. To support continued research and development in underwater robotics, we make the design open-access and freely available to the community.

We present and evaluate a novel suction-based gripper designed for fruit picking. This work is motivated by common problems observed in field trials of robotic harvesting: Calibration/perception errors, workspace obstacles, fruit swinging/moving when contacted, and varying stem and branch stiffnesses. The gripper consists of three suction-cups located on the palm, along with in-hand perception. To evaluate the gripper, we developed a physical proxy that approximates a realistic apple-stem-branch dynamic system. We performed 756 apple picks on the proxy with varying branch stiffness, stem strength and gripper pose (yaw, roll and offset w.r.t. the apple). Our results show that grasping performance improves when the gripper yaw w.r.t. the apple has two suction cups on the bottom of the apple and one suction cup on top. Even with ±15mm offset, at least two suction cups engaged with the apple 80% of the time, regardless of branch stiffness. Moreover, the gripper withstands ±20mm offset when it approaches the apple near its equator.

Robotic tree pruning requires highly precise manipulator control in order to accurately align a cutting implement with the desired pruning point at the correct angle. Simultaneously, the robot must avoid applying excessive force to rigid parts of the environment such as trees, support posts, and wires. In this paper, we propose a hybrid control system that uses a learned vision-based controller to initially align the cutter with the desired pruning point, taking in images of the environment and outputting control actions. This controller is trained entirely in simulation, but transfers easily to real trees via a neural network which transforms raw images into a simplified, segmented representation. Once contact is established, the system hands over control to an interaction controller that guides the cutter pivot point to the branch while minimizing interaction forces. With this simple, yet novel, approach we demonstrate an improvement of over 30 percentage points in accuracy over a baseline controller that uses camera depth data.

Apple picking is a challenging manipulation task, but it is difficult to test solutions due to the limited window of time that apples are in season. Previous methods have built simulations of apple trees, but simulations rarely capture soft contact and deformation well, both of which are common in fruit picking. In this paper we present and validate a physical proxy that replicates the mechanics of a real world apple pick. This proxy, in conjunction with a novel hand with multiple sensors, enables large-scale capture of sensor data for data collection and testing. To validate our approach, we train a Long Short-Term Memory network to classify a pick as successful or failed based on sensor feedback from the robot hand. We show that a network trained on the proxy performs as well, or even better, than a network trained solely on real apple trees, with accuracies up to 90%. We determine which sensors are most important for pick classification and also demonstrate that our proxy preserves the most important sensor feature data for pick classification. For our hand, the most informative sensor was the finger's servomotor effort.

Machine vision is a critical subsystem for enabling robots to be able to perform a variety of tasks in orchard environments. However, orchards are highly visually complex environments, and computer vision algorithms operating in them must be able to contend with variable lighting conditions and background noise. Past work on enabling deep learning algorithms to operate in these environments has typically required large amounts of hand-labeled data to train a deep neural network or physically controlling the conditions under which the environment is perceived. In this paper, we train a neural network system in simulation only using simulated RGB data and optical flow. This resulting neural network is able to perform foreground segmentation of branches in a busy orchard environment without additional real-world training or using any special setup or equipment beyond a standard camera. Our results show that our system is highly accurate and, when compared to a network using manually labeled RGBD data, achieves significantly more consistent and robust performance across environments that differ from the training set.

Studies have shown that picking techniques play an important role in determining fruit quality at harvest (e.g. bruising, stem retention, etc). When picking fruit such as apples and pears, professional pickers use active perception, incorporating both visual and tactile input about fruit orientation, stem location, and the fruit’s immediate surroundings. This combination of tactile, visual, and force feedback is what enables human workers to execute dynamic movements that quickly and efficiently remove fruit from the tree without damage. However, much of the prior work on robotic fruit picking has formulated the harvesting problem as a position-control problem, using visual feedback for closed-loop end-effector placement while disregarding feedback on physical contact. As a first step towards more intelligent fruit picking — combining proprioception, localized sensing, and observed forces — we have developed a custom end-effector with multiple in-hand sensors, including tactile sensors on the fingertips. This paper presents the mechatronic design of the device as well as results from multiple outdoor picking trials with a Honeycrisp apple tree. Preliminary results show that, with multi-modal sensing, fruit slip, fruit separation from the tree, and fruit release from the hand can be detected.

Soft robotics promises developments in the research areas of safety, bio-mimicry, manipulation, human-robot interaction, and alternative locomotion techniques. The research presented here is directed towards developing an improved, low-cost, and open-source method for soft robotic control using electrorheological fluids in compact, 3D-printed electroactive hydraulic valves. We construct high-pressure electrorheological valves and deformable actuators using only commercially available materials and accessible fabrication methods. The printed valves were characterized with industrial-grade electrorheological fluid (RheOil 3.0), but the design is generalizable to other electrorheological fluids. Valve performance was shown to be an improvement over comparable work with demonstrated higher yield pressures at lower voltages (up to 230 kPa), larger flow rates (up to 15 ml/min) and lower response times (1 to 3 seconds, depending on design). The resulting valve and actuator systems enable future novel applications of electrorheological fluid-based control and hydraulics in soft robotics and other disciplines.

Dormant pruning is a major cost component of fresh market tree fruit production, nearly equal in scale to harvesting the fruit. However, relatively little focus has been given to the problem of pruning trees autonomously. In this paper, we introduce a robotic system consisting of an industrial manipulator, an eye-in-hand RGB-D camera configuration, and a custom pneumatic cutter. The system is capable of planning and executing a sequence of cuts while making minimal assumptions about the environment. We leverage a novel planning framework designed for high-throughput operation which builds upon previous work to reduce motion planning time and sequence cut points intelligently. In end-to-end experiments with a set of ten different branch configurations, the system achieved a high success rate in plan execution and a 1.5x speedup in throughput versus a baseline planner, representing a significant step towards the goal of practical implementation of robotic pruning.

There are no mechanical harvesters for the fresh market apple industry commercially available. The absence of automated harvesting technology is a critical problem because of rising production costs and increasing uncertainty about future labor availability. This paper presents the preliminary design of a robotic apple harvester. The approach adopted was to develop a low-cost, `undersensed' system for modern orchard systems with fruiting wall architectures. A machine vision system fuses Circular Hough Transform and blob analysis to detect clustered and occluded fruit. The design includes a custom, six degree of freedom manipulator with an underactuated, passively compliant end-effector. After fruit localization, the system makes a linear approach to the apple and replicates the human picking process. Integrated testing of the robotic harvesting system has been completed in a laboratory environment with a replica apple tree for proof-of-concept demonstration. Experimental results show that the system picked 95 of the 100 fruit attempted with average localization and picking times of 1.2 and 6.8 seconds, respectively, per fruit. Additional work planned in preparation for field evaluation in a commercial orchard is also described.