A major component of an efficient farming strategy is the precise detection and characterization of plant deficiencies followed by the proper deployment of fertilizers. Through the thoughtful utilization of modern computer vision techniques, it is possible to achieve positive financial and environmental results for these tasks. This work introduces an automation framework that attempts to address the three main drawbacks of existing approaches: 1) lack of generality (methods are tuned for specific data sets); 2) difficulty to apply in variable field conditions; and 3) lack of tool sophistication that limits their applicability. The cultivation of corn lies in the core of the American and global economy with 81.7 million acres harvested only in the USA for the year 2018. The ubiquity of its cultivation makes it an ideal candidate to highlight the large economic benefits from even a small improvement in nutrient deficiency detection. The proposed methodology utilizes drone collected images to detect nitrogen (N) deficiencies in maize fields and assess their severity using low-cost RGB sensors. The proposed methodology is twofold. A low complexity recommendation scheme identifies candidate plants exhibiting N deficiency and, with minimal interaction, assists the annotator in the creation of a training data set that is then used to train an object detection deep neural network. Results on data from experimental fields support the merits of the proposed methodology with mean average precision for the detection of N-deficient leaves reaching 82.3%.Note to Practitioners—The motivation behind this article is the problem of inefficient fertilizer application in corn fields throughout the cultivation season. Current widely spread techniques to counter plant malnutrition suggest the application of excessive amounts of nitrogen fertilizer prior to seeding or the uniform application during the plant growth. These practices result in financial losses and have severe environmental consequences, e.g., the dead zone in the Gulf of Mexico. We propose an automation framework that automatically detects corn nitrogen deficiencies in the field during the plants’ growth, and to achieve our goal, we employ low-cost robotic platforms and RGB sensors. The framework that we developed is able to detect the characteristic pattern of nitrogen deficiency on corn leaves and provide an estimation of the in-field spatial variability of the deficiency.

Financial and social elements of modern societies are closely connected to the cultivation of corn. Due to its massive production, deficiencies during the cultivation process directly translate to major financial losses. Since proper surveillance in a large scale is still very challenging, the companies that specialize in optimizing crop yield are trying to address the problem at its root by developing hybrid plants able to resist the harsh conditions of the field. The selection of the best hybrid is not easy and every year hundreds of test plants with different phenotypic characteristics are planted while their performance is quantified by inconsistent and rough measurements gathered by humans. We propose a pipeline that takes advantage of the structure from motion technology to create a detailed 3D point cloud of a few plants and segment it into the basic elements of the scene; the ground, the plants, the plant stems, and the plant leaves. The focus is on the segmentation process through which several phenotypic characteristics of individual plants can be extracted. As an example, we show the results for the plant counting and plant height estimation processes where we achieve an accuracy of 88.1% and 89.2%.

The recent activity in the area of autonomous vehicle navigation has initiated a series of reactions that stirred the automobile industry, pushing for the fast commercialization of this technology which, until recently, seemed futuristic. The LiDAR sensor is able to provide a detailed understanding of the environment surrounding the vehicle making it useful in a plethora of autonomous driving scenarios. Segmenting the 3D point cloud that is provided by modern LiDAR sensors, is the first important step towards the situational assessment pipeline that aims for the safety of the passengers. This step needs to provide accurate segmentation of the ground surface and the obstacles in the vehicle's path, and to process each point cloud in real time. The proposed pipeline aims to solve the problem of 3D point cloud segmentation for data received from a LiDAR in a fast and low complexity manner that targets real world applications. The two-step algorithm first extracts the ground surface in an iterative fashion using deterministically assigned seed points, and then clusters the remaining non-ground points taking advantage of the structure of the LiDAR point cloud. Our proposed algorithms outperform similar approaches in running time, while producing similar results and support the validity of this pipeline as a segmentation tool for real world applications.

Financial and social elements of modern societies are closely connected to the cultivation of corn. Due to the massive production of corn, deficiencies during the cultivation process directly translate to major financial losses. The early detection and treatment of crops deficiencies is thus a task of great significance. Towards an automated health condition assessment, this study introduces a scheme for the computation of plant health indices. Based on the 3D reconstruction of small batches of corn plants, an alternative to existing cumbersome Leaf Area Index (LAI) estimation methodologies is presented. The use of 3D models provides an elevated information content, when compared to planar methods, mainly due to the reduced loss attributed to leaf occlusions. High resolution images of corn stalks are collected and used to obtain 3D models of plants of interest. Based on the extracted 3D point clouds, an accurate calculation of the Leaf Area Index (LAI) of the plants is performed. An experimental validation (using artificially made corn plants used as ground truth of the LAI estimation), emulating real world scenarios, supports the efficacy of the proposed methodology. The conclusions of this work, suggest a fully automated scheme for information gathering in modern farms capable of replacing current labor intensive procedures, thus greatly impacting the timely detection of crop deficiencies.

The continuously growing need for increasing the production of food and reducing the degradation of water supplies, has led to the development of several precision agriculture systems over the past decade so as to meet the needs of modern societies. The present study describes a methodology for the detection and characterization of Nitrogen (N) deficiencies in corn fields. Current methods of field surveillance are either completed manually or with the assistance of satellite imaging, which offer infrequent and costly information to the farmers about the state of their fields. The proposed methodology promotes the use of small-scale Unmanned Aerial Vehicles (UAVs) and Computer Vision algorithms that operate with information in the visual (RGB) spectrum. Through this implementation, a lower cost solution for identifying N deficiencies is promoted. We provide extensive results on the use of commercial RGB sensors for delivering the essential information to farmers regarding the condition of their field, targeting the reduction of N fertilizers and the increase of the crop performance. Data is first collected by a UAV that hovers over a stressed area and collects high resolution RGB images at a low altitude. A recommendation algorithm identifies potential segments of the images that are candidates exhibiting N deficiency. Based on the feedback from experts in the area a training set is constructed utilizing the initial suggestions of the recommendation algorithm. Supervised learning methods are then used to characterize crop leaves that exhibit signs of N deficiency. The performance of 84.2% strongly supports the potential of this scheme to identify N-deficient leaves even in the case of images where the unhealthy leaves are heavily occluded by other healthy or stressed leaves.

Object segmentation and classification is an important and difficult task in robotic vision. The task is complicated even further when the different objects are partially or completely occluded. Allowing a robot to take measurements from varying points of view can help in alleviating or completely removing occlusions. A robot equipped with an RGB-D sensor has the capability of searching for new and better points of view to facilitate object recognition. In this work, a motion control algorithm is designed and implemented on a mobile robot to facilitate object classification in RGB-D data of clustered objects.

In this paper, we introduce a new covariance based feature descriptor to be used on “colored” point clouds gathered by a mobile robot equipped with an RGB-D camera. Although many recent descriptors provide adequate results, there is not yet a clear consensus on how to best tackle “colored” point clouds. We present the notion of a covariance on RGB-D data. Covariances have not only been proven to be successful in image processing, but in other domains as well. Their main advantage is that they provide a compact and flexible description of point clouds. Our work is a first step towards demonstrating the usability of the concept of covariances in conjunction with RGB-D data. Experiments performed on an RGB-D database and compared to previous results show the increased performance of our method.