The determination of a crop's growth-stage is critical information for precision agriculture. Estimates of the growth-stage are used to guide irrigation and the application of agrochemicals. Of particular importance is the use of fertilizers, however, growth-stage estimates may also suggest further investigation of potential crop infections and infestations. Traditionally, the growth-stage is based upon a manual random sample of a very small number of plants that are then analyzed to produce an estimate for the entire crop (up to thousands of acres). In order to increase the sample size (and thus accuracy) and to enable precision agriculture to address non-uniform crop development across a field, we present an analysis methodology that facilitates the automated growth-stage analysis of dense point clouds that are derived from drone imagery. Our method utilizes a standard camera drone and does not use specialized sensors or geo-spatial tagging. We propose a multi-stage unsupervised method, which provides information about the individual plant locations in a field plot with a high probability. The method also produces a measure of individual plant heights, which along with their location are critical for later growth-stage estimation and necessary for robotic precision application. We confirm our method's efficacy with experimental results on corn fields in Minnesota.

The segmentation and tracking of visual patterns, particularly those patterns related to natural imagery, have sparked renewed interest in the computer vision and image processing communities. Applications in robotics, automated systems, geographical information systems, etc. require efficient and accurate methods for processing visual data. Prior work in textural analysis has led to systems with promising accuracy, but poor efficiency. Work on fractal snakes provided both accuracy and efficiency, but at the loss of orientation with respect to the texture. In many applications, resolving orientation is an important piece of information. We have built upon our work in fractal snakes to expand our snake models from a purely isotropic measure of surface roughness to an orientation sensitive model: the Anisotropic Fractal Snake.

Studies have shown that visually-impaired people, particularly those with recent impairment, suffer from reduced social interaction. Technological innovations for these people have overlooked social interaction in favor of environmental mapping and mobility. Our work develops an Enhanced Social Awareness network. This network assists visually-impaired people in identifying friends and provides them with the cues necessary to initiate social contact. We experimentally verify our work and discuss the future implications and needs of such networks.

The natural environments that robotic applications often encounter can present difficult problems for image-based task execution. Prior efforts have used both grayscale and color as statistical appearance descriptors in these applications. In the case of natural environments, the statistical measures of luminosity and chromaticity are often ineffective due to relatively constant shades and colors of soil, flora, and fauna. Texture can provide an alternative/additional appearance descriptor in many of these environments; however the common approaches to textural segmentation are computationally intensive and cannot be used for real-time robotic visual servoing. We present a technique for textural segmentation and tracking that can discriminate between natural textures that are otherwise similar in color and brightness. The technique builds upon earlier work in fractal imaging and in statistical deformable models (a.k.a. snakes) to provide a simple and efficient method for extracting target shape and location from an initial textural example. We then give results from the application of this technique on standard texture test patterns. We then demonstrate the effectiveness of the method on natural imagery. Finally, we show how the technique can be applied to challenging robotic applications.

Several robotic and computer vision applications depend upon the efficient determination of polygonal self- and mutual-intersection checking. The commonly used algorithms for intersection checking rely upon static geometric primitives, such as lines and vertices. When these geometric primitives are dynamic, that is moving or changing shape, these algorithms become inefficient due to repeated actions that do not utilize topological features of the primitives. In this paper we present a novel algorithm for line segment intersection checking that builds a query structure and then updates the structure using previously computed topological data. We exploit the fact that the amount of model deformation is limited during any single iteration, yielding a relatively small bookkeeping cost to maintain the query structure. The result is an algorithm whose asymptotic runtime complexity in the expected case is better than competing methods. We then suggest an extension of this work into higher dimensions (polytope intersection for 3D and higher).

Statistical active contour models (aka statistical pressure snakes) have attractive properties for use in mobile manipulation platforms as both a method for use in visual servoing and as a natural component of a human-computer interface. Unfortunately, the constantly changing illumination expected in outdoor environments presents problems for statistical pressure snakes and for their image gradient-based predecessors. This paper introduces a new color-based variant of statistical pressure snakes that gives superior performance under dynamic lighting conditions and improves upon the previously published results of attempts to incorporate color imagery into active deformable models.

The famous expectation maximization technique suffers two major drawbacks. First, the number of components has to be specified apriori. Also, the expectation maximization is sensitive to initialization. In this paper, we present a new stochastic technique for estimating the mixture parameters. Parzen Window is used to estimate a discrete estimate of the PDF of the given data. Stochastic learning automata is then used to select the mixture parameters that minimize the distance between the discrete estimate of the PDF and the estimate of the expectation maximization. The validity of the proposed approach is verified using bivariate simulation data.

In this paper, a new non-parametric generalized formulation to statistical pressure snakes is presented. We discuss the shortcomings of the traditional pressure snakes. We then introduce a new generic pressure model that alleviates these shortcomings, based on the Bayesian decision theory. Non-parametric techniques are used to obtain the statistical models that drive the snake. We discuss the advantages of using the proposed non-parametric model compared to other parametric techniques. Multi-colored-target tracking is used to demonstrate the performance of the proposed approach. Experimental results show enhanced, real-time performance.

A fundamental problem in computer vision is the issue of shape ambiguity. Simply stated, a silhouette cannot uniquely identify an object or an object's classification since many unique objects can present identical occluding contours. This problem has no solution in the general case for a monocular vision system. This paper presents a method for disambiguating objects during silhouette matching using a visual servoing system. This method identifies the camera motion(s) that gives disambiguating views of the objects. These motions are identified through a new technique called contour migration. The occluding contour's shape is used to identify objects or object classes that are potential matches for that shape. A contour migration is then determined that disambiguates the possible matches by purposive viewpoint adjustment. The technique is demonstrated using an example set of objects.

The classical active contour model has two basic internal forces: tension and curvature. These forces are included to provide cohe sion, equal control point spacing, and locally smooth shape. These classical internal forces have undesirable attributes that are in conflict with these original desired characteristics. Tension evenly spaces the control points, but also causes the models to collapse in weak image gradients. Curvature produces locally smooth curvature, but it does so by forcing the model toward a straight line. The paper returns to the original active contour model motivations to reformulate these internal forces. The desired properties are achieved without the introduction of unwanted model behavior A new spacing force and a new constant change in curvature force are introduced and their performance characteristics are discussed. The paper includes experimental results that demonstrate the efficacy and performance of the proposed reformulations.

This paper presents research on a new computer vision approach to an intelligent transportation system domain: bicycle counting. Traditionally, bicycles have not received the attention of transportation departments for a variety of reasons. As a result, effective and portable automatic methods for counting bicycles have not been developed. This paper presents a method for identifying bicycles in natural scenes taken from either time-lapse or full-motion video streams. The method is based on a blend of two separate techniques, the Hough transform and the directed Hausdorff distance. Experimental results demonstrate that neither technique individually achieves the performance of the hybrid technique.

The paper describes a new approach to creating a map of the environment for a mobile robot by identifying large-scale indoor features directly from sonar observations. Current research in creating environmental maps for mobile robots either use grid-based representations, or use small-scale features to construct larger entities for topological maps, or use combinations of these approaches. These methods do not directly identify features on the scale of corridors, alcoves, and intersections that have high semantic content for people and provide a compact representation of the topology of the environment. We present a simple rule-based technique to differentiate and identify a set of large-scale indoor features. The rule-based approach is used to demonstrate the potential of our method. The experimental results show that processor intensive techniques such as pattern recognition/signal processing, neural networks, or clustering algorithms may not be required for successful, direct large-scale feature identification.

Grasping is one of the most fundamental and challenging tasks in robotics. Applications range from space missions (e.g., collection of rock samples) to industrial automation. In this work, we use a camera mounted on the end-effector of a manipulator to grasp an unknown object in the workspace. A novel deformable contour model is used to determine plausible grasp axes of the target object. Potential grasp point pairs are generated, ranked based upon measurements taken from the contour, and a vision-guided grasp of the object using the highest ranked grasp point pair is executed. Several experimental results are presented.

HOLDeR (Hierarchical, Object-centered, Learning Dextrous Robotics) is a new approach to the problem of object-centered task execution. It balances stored object knowledge with flexibility to produce a system that is capable of manipulating a wide variety of known and unknown objects. HOLDeR utilizes a layered approach to sensory-motor coordination that enables a robot to use as much object-specific data as possible, given time and computational constraints. The architecture also incorporates learning to allow the system to adapt to new objects and new classes of objects that it encounters. HOLDeR is based upon a modification of an active deformable model technique called dynamic pressure snakes. The techniques can be adapted to a variety of robotic tasks.

Flexible operation of a robotic agent in an uncalibrated environment requires the ability to recover unknown or partially known parameters of the workspace through sensing. Of the sensors available to a robotic agent, visual sensors provide information that is richer and more complete than other sensors. In this paper we present robust techniques for the derivation of depth from feature points on a target's surface and for the accurate and high-speed tracking of moving targets. We use these techniques in a system that operates with little or no a priori knowledge of object- and camera-related parameters to robustly determine such object-related parameters as velocity and depth. Such determination of extrinsic environmental parameters is essential for performing higher level tasks such as inspection, exploration, tracking, grasping, and collision-free motion planning. For both applications, we use the Minnesota robotic visual tracker (MRVT) (a single visual sensor mounted on the end-effector of a robotic manipulator combined with a real-time vision system) to automatically select feature points on surfaces, to derive an estimate of the environmental parameter in question, and to supply a control vector based upon these estimates to guide the manipulator.

Many researchers have turned to sensing, and in particular computer vision, to create more flexible robotic systems. Computer vision is often required to provide data for the grasping of a target. Using a vision system for grasping presents several issues with respect to sensing, control, and system configuration. This paper presents some of these issues in concert with the options available to the researcher and the trade-offs to be expected when integrating a vision system with a robotic system for the purpose of grasping objects. The paper includes experimental results from a particular configuration that characterize the type and frequency of errors encountered while performing various vision-guided grasping tasks. These error classes and their frequency of occurrence lend insight into the problems encountered during visual grasping and into the possible solution of these problems.

Our previous work has introduced the framework of controlled active vision that provides a bridge between computer vision and control theory. This paper presents some of the computer vision techniques that were employed in order to detect targets, automatically select features, measure feature displacements, and evaluate measurements during the various applications of the controlled active vision framework. We experimented with a lot of different techniques, but the most robust proved to be the sum-of-squared differences (SSD) optical flow technique. Search optimizations and dynamic pyramiding are proposed in order to provide real-time performance. In addition, several techniques for the evaluation of the measurements are presented. One important characteristic of these techniques is that they can also be used for the selection of features for tracking in conjunction with several numerical criteria that guarantee the robustness of the servoing. These techniques are important aspects of our work since they can be implemented either on-line or off-line. The paper concludes with results from the application of these techniques to real images.

Recent work combining robotics with vision has emphasized an active vision paradigm where the system changes the pose of the camera to improve environmental knowledge or to establish and presence a desired relationship between the robot and objects in the environment. Much of this work has concentrated upon the active observation of objects by the robotic agent. In this paper we present extensions to the controlled active vision framework that focus upon the autonomous grasping of a moving or static object in the manipulator's workspace. Our work extends the capabilities of an eye-in-hand system beyond those as a "pointer" or a "camera orienter" to provide the flexibility required to robustly interact with the environment in the presence of uncertainty. The proposed work is experimentally verified using the Minnesota robotic visual tracker to automatically select object features, to derive estimates of unknown environmental parameters, and to supply a control vector based upon these estimates to guide the manipulator in the grasping of a moving or static object.

Flexible operation of a robotic agent in an uncalibrated environment requires the ability to recover unknown or partially known parameters of the workspace through sensing. Of the sensors available to a robotic agent, visual sensors provide information that is richer and more complete than other sensors. In this paper we present robust techniques for the derivation of depth from feature points on a target's surface and for the accurate and high-speed tracking of moving targets. We use these techniques in a system that operates with little or no a priori knowledge of the object- and camera-related parameters to robustly determine such object-related parameters as velocity and depth. Such determination of extrinsic environmental parameters is essential for performing higher level tasks such as inspection, exploration, tracking, grasping, and collision-free motion planning. For both applications, we use the Minnesota Robotic Visual Tracker (a single visual sensor mounted on the end-effector of a robotic manipulator combined with a real-time vision system) to automatically select feature points on surfaces, to derive an estimate of the environmental parameter in question, and to supply a control vector based upon these estimates to guide the manipulator.>

Accurate knowledge of depth continues to be of critical importance in robotic systems. Without accurate depth knowledge, tasks such as inspection, tracking, grasping, and collision-free motion planning prove to be difficult and often unattainable. Traditional visual depth recovery has relied upon techniques that require the solution of the correspondence problem or require known lighting conditions and Lambertian surfaces. In this paper, we present a technique for the derivation of depth from feature points on a target's surface using the controlled active vision framework. We use a single visual sensor mounted on the end-effector of a robotic manipulator to automatically select feature points and to derive depth estimates for those features using adaptive control techniques. Movements of the manipulator produce displacements that are measured using a sum-of-squared difference (SSD) optical flow. The measured displacements are fed into the controller to alter the path of the manipulator and to refine the depth estimate.>

Flexible operation of a robotic agent in an uncalibrated environment requires the ability to recover unknown or partially known parameters of the workspace through sensing. Of the sensors available to a robotic agent, visual sensors provide information that is richer and more complete than other sensors. In this paper we present robust techniques for the derivation of depth from feature points on a target's surface and for the accurate and high-speed tracking of moving targets. We use these techniques in a system that operates with little or no a priori knowledge of the object-related parameters present in the environment. The system is designed under the controlled active vision framework and robustly determines parameters such as velocity for tracking moving objects and depth maps of objects with unknown depths and surface structure. Such determination of intrinsic environmental parameters is essential for performing higher level tasks such as inspection, exploration, tracking grasping, and collision-free motion planning. For both applications, we use the Minnesota Robotic Visual Tracker (a single visual sensor mounted on the end-effector of a robotic manipulator combined with a real-time vision system) to automatically select feature points on surfaces, to derive an estimate of the environmental parameter in question, and to apply a control vector based upon these estimates to guide the manipulator. The paper concludes with applications of these techniques to transportation problems such as vehicle tracking.>