Autonomous navigation is crucial for various robotics applications in agriculture. However, many existing methods depend on RTK-GPS devices, which can be susceptible to loss of radio signal or intermittent reception of corrections from the internet. Consequently, research has increasingly focused on using RGB cameras for crop-row detection, though challenges persist when dealing with grown plants. This paper introduces a LiDAR-based navigation system that can achieve crop-agnostic over-canopy autonomous navigation in row-crop fields, even when the canopy fully blocks the inter-row spacing. Our algorithm can detect crop rows across diverse scenarios, encompassing various crop types, growth stages, illumination conditions, the presence of weeds, curved rows, and discontinuities. Without utilizing a global localization method (i.e., based on GPS), our navigation system can perform autonomous navigation in these challenging scenarios, detect the end of the crop rows, and navigate to the next crop row autonomously, providing a crop-agnostic approach to navigate an entire field. The proposed navigation system has undergone tests in various simulated and real agricultural fields, achieving an average cross-track error of 3.55 cm without human intervention. The system has been deployed on a customized UGV robot, which can be reconfigured depending on the field conditions.

Sim2Real transfer, particularly for manipulation policies relying on RGB images, remains a critical challenge in robotics due to the significant domain shift between syn-thetic and real-world visual data. In this paper, we propose SplatSim, a novel framework that leverages Gaussian Splatting as the primary rendering primitive to reduce the Sim2Real gap for RGB-based manipulation policies. By replacing traditional mesh representations with Gaussian Splats in simulators, SplatSim produces highly photorealistic synthetic data while maintaining the scalability and cost-efficiency of simulation. We demonstrate the effectiveness of our framework by training manipulation policies within SplatSim and deploying them in the real world in a zero-shot manner, achieving an average success rate of 86.25%, compared to 97.5% for policies trained on real-world data. Videos can be found on our project page: https://splatsim.github.io

We outline a neural network-based pipeline for perception, control and planning of a 7 DoF robot for tasks that involve reaching into a dormant grapevine canopy. The proposed system consists of a 6 DoF industrial robot arm and a linear slider that can actuate on an entire grape vine. Our approach uses Convolutional Neural Networks to detect buds in dormant grape vines and a Reinforcement Learning based control strategy to reach desired cut-point locations for pruning tasks. Within this framework, three methodologies are developed and compared to reach the desired locations: the learned policy-based approach (RL), a hybrid method that uses the learned policy and an inverse kinematics solver (RL+IK), and lastly a classical approach commonly used in robotics. We first tested and validated the suitability of the proposed learning methodology in a simulated environment that resembled laboratory conditions. A reaching accuracy of up to 61.90% and 85.71% for the RL and RL+IK approaches respectively was obtained for a vine that the agent observed while learning. When testing in a new vine, the accuracy was up to 66.66% and 76.19% for RL and RL+IK, respectively. The same methods were then deployed on a real system in an end to end procedure: autonomously scan the vine using a vision system, create its model and finally use the learned policy to reach cutting points. The reaching accuracy obtained in these tests was 73.08%.

Object detection and semantic segmentation are two of the most widely adopted deep learning algorithms in agricultural applications. One of the major sources of variability in image quality acquired outdoors for such tasks is changing lighting conditions that can alter the appearance of the objects or the contents of the entire image. While transfer learning and data augmentation reduce the need for large amount of data to train deep neural networks to some extent, the large variety of cultivars and the lack of shared datasets in agriculture makes wide-scale field deployments difficult. In this paper, we present an active lighting-based camera system that generates robust and uniform images in any lighting conditions. We provide extensive validation experiments to evaluate the consistency in the quality of the images. Metrics for assessing image uniformity like Structural Similarity (SSIM) index and the Peak Signal to Noise Ratio (PSNR) ranged from 83.78 to 93.79 and from 25.30 to 31.78 respectively, showing stability over a day with changing sunlight. The validation stage also showed that the generated images effectively reduce the amount of training samples for object detection using deep neural networks. The camera system was then deployed in real field experiments for counting buds in dormant vines and shoots in early season grape vines as well as for counting apples in orchards. The mean absolute errors obtained when compared to ground truth were 5%, 2.55% and 8.57%, respectively.