Effective pollination is a key challenge for indoor farming, since bees struggle to navigate without the sun. While a variety of robotic system solutions have been proposed, it remains difficult to autonomously check that a flower has been sufficiently pollinated to produce high-quality fruit, which is especially critical for self-pollinating crops such as strawberries. To this end, this work proposes a novel robotic system for indoor farming. The proposed hardware combines a 7 -degree-of-freedom (DOF) manipulator arm with a custom end-effector, comprised of an endoscope camera, a 2-DOF microscope subsystem, and a custom vibrating pollination tool; this is paired with algorithms to detect and estimate the pose of strawberry flowers, navigate to each flower, pollinate using the tool, and inspect with the microscope. The key novelty is vibrating the flower from below while simultaneously inspecting with a microscope from above. Each subsystem is validated via extensive experiments.

The small scale of urban farms and the commercial availability of low-cost robots (such as the FarmBot) that automate simple tending tasks enable an accessible platform for plant phenotyping. We have used a FarmBot with a custom camera end-effector to estimate strawberry plant flower pose (for robotic pollination) from acquired 3D point cloud models. We describe a novel algorithm that translates individual occupancy grids along orthogonal axes of a point cloud to obtain 2D images corresponding to the six viewpoints. For each image, 2D object detection models for flowers are used to identify 2D bounding boxes which can be converted into the 3D space to extract flower point clouds. Pose estimation is performed by fitting three shapes (superellipsoids, paraboloids and planes) to the flower point clouds and compared with manually labeled ground truth. Our method successfully finds approximately 80% of flowers scanned using our customized FarmBot platform and has a mean flower pose error of 7.7 degrees, which is sufficient for robotic pollination and rivals previous results. All code will be made available at https://github.com/harshmuriki/flowerPose.git.

Global blackberry demand has been surging due to their antioxidant and nutritional value in a traditional diet. However, blackberries have extreme fragility (resulting in up to 85% of harvest batches sustaining damage) and near-ripe and ripe blackberries are difficult to distinguish in normal lighting conditions. These challenges in maintaining the blackberry supply motivate the development of an autonomous robotic solution to harvest fully ripe blackberries with minimal damage. The present paper details the mechanical design, methodology, analysis, and experimental results of a compliant robotic gripper created for this purpose. The gripper has a compact form factor and retractable fingers with specialized TPU finger pads for gentle picks, a near-infrared (NIR) reflectance-based probe for detecting full ripeness and a standardized harvesting sequence for effectively picking berries. In an outdoor harvesting experiment, the gripper attempted picking 26 berries without ripeness sensing, with 65.4% (17) being successfully picked and 38.5% (10) sustaining damage. The movements of the robot arm in the harvesting sequence were accordingly adjusted and finalized for following in-lab experiments, in which the gripper was also outfitted with ripeness sensing. Out of 40 berries, 62.5% (25) were successfully picked, with 0% of them sustaining damage. The ripeness probe classified 56 ripe and 11 near-ripe berries, with 89% (50) of the ripe and 64% (7) of the near-ripe berries being correctly classified. In a second in-lab experiment, 16 of 20 berries were successfully picked, with 2 sustaining damage.

Growing global demand for food, coupled with continuing labor shortages, motivate the need for automated agricultural harvesting. While some specialty crops (e.g., apples, peaches, blueberries) can be harvested via existing harvesting modalities, fruits such as blackberries and raspberries require delicate handling to mitigate fruit damage that could significantly impact marketability. This motivates the development of soft robotic solutions that enable efficient, delicate harvesting. This paper presents the design, fabrication, and feasibility testing of a tendon-driven soft gripping system focused on blackberries, which are a fragile fruit susceptible to post-harvest damage. The gripper is low-cost and small form factor, allowing for the integration of a micro-servo for tendon retraction, a near-infrared (NIR) based blackberry ripeness sensor utilizing the reflectance modality for identifying fully ripe blackberries, and an endoscopic camera for visual servoing. The gripper was used to harvest 139 berries with manual positioning in two separate field tests. Field testing found an average retention force of 2.06 N and 6.08 N for ripe and unripe blackberries, respectively. Sensor tests identified an average reflectance of 16.78 and 21.70 for ripe and unripe blackberries, respectively, indicating a clear distinction between the two ripeness levels. Finally, the soft robotic gripper was integrated onto a UR5 robot arm and successfully harvested fifteen artificial blackberries in a lab setting using visual servoing.

Control of wire-borne underactuated brachiating robots requires a robust feedback control design that can deal with dynamic uncertainties, actuator constraints and unmeasurable states. In this paper, we develop a robust feedback control for brachiating on flexible cables, building on previous work on optimal trajectory generation and time-varying LQR controller design. We propose a novel simplified model for approximation of the flexible cable dynamics, which enables inclusion of parametric model uncertainties in the system. We then use semidefinite programming (SDP) and sum-of-squares (SOS) optimization to synthesize a time-varying feedback control with formal robustness guarantees to account for model uncertainties and unmeasurable states in the system. Through simulation, hardware experiments and comparison with a time-varying LQR controller, it is shown that the proposed robust controller results in relatively large robust backward reachable sets and is able to reliably track a pre-generated optimal trajectory and achieve the desired brachiating motion in the presence of parametric model uncertainties, actuator limits, and unobservable states.

This paper describes the dynamic modeling and locomotion control of a two-link underactuated brachiating robot traversing a flexible cable. A multi-body system comprised of a two-link robot, a flexible cable, and coupling soft junctions is modeled dynamically. This model is used to formulate an energy-minimizing optimal control strategy that includes the effects of cable vibration induced by robot locomotion. Optimized trajectories and control torque profiles are obtained via multiple-shooting and parametric trajectory approaches. Simulation results show that these optimal torque profiles result in energy-efficient continuous brachiation over a flexible cable. Additional studies examine how the optimal torque profiles change depending on the robot's initial position along a catenary cable.

A novel brachiating robot design is presented for the purpose of traversing elevated wire networks. The robot is capable of moving along a single wire and between parallel wires, thereby enabling traversal of a two-dimensional space. Several novel features distinguish this design compared to previous brachiating robots. These include the ability to transition to and from both “rope” and “ladder” brachiation modes through an integrated wrist, a locking hand design for minimal power consumption, and distributed electronics packages that communicate wirelessly. A payload mounting point is installed, offering space for a variety of remote sensing packages. Experimental results using a prototype robot demonstrate that the system can reliably brachiate along a single wire, and can also reliably perform a swing-up maneuver after failed swing attempts or when transitioning between the rope and ladder locomotion modes. Energy expenditure for a single swing is quantified using experimental data. Overall, the proposed robot design is shown to provide a promising platform for traversal of wire networks in two dimensions.

We propose a novel implementation of visual servoing whereby a human operator can guide a robot relative to the coordinate frame of an eye-in-hand camera. Among other applications, this can allow the operator to work in the image space of the eye-in-hand camera. This is achieved using a gamepad, a time-of-flight camera (an active sensor that creates depth data), and recursive least-squares update with Gauss-Newton control. Contributions of this paper include the use of a person to cause the control action in a visual-servoing system, and the introduction of uncalibrated position-based visual servoing. The system's efficacy is evaluated via trials involving human operators in different scenarios.

Real-world systems are comprised of interdependent components creating integrated systems. These systems are developed by multidisciplinary teams. The goal of this project is the development of a comprehensive undergraduate course in robotics that encompasses various fields that are integral to robotic systems: Computer Science, Electrical and Computer Engineering, and Mechanical Engineering. A main pedagogical goal of the course is to teach group dynamics and the skills necessary for interaction with people in different disciplines in multidisciplinary teams. Descriptions of the course and the hands-on lab assignments are presented along with course assessment.