Interactive sensors are an important component of robotic systems but often require manual replacement due to wear and tear. Automating this process can enhance system autonomy and facilitate long-term deployment. We developed an autonomous sensor exchange and calibration system for an agriculture crop monitoring robot that inserts a nitrate sensor into cornstalks. A novel gripper and replacement mechanism, featuring a reliable funneling design, were developed to enable efficient and reliable sensor exchanges. To maintain consistent nitrate sensor measurement, an on-board sensor calibration station was integrated to provide in-field sensor cleaning and calibration. The system was deployed at the Ames Curtis Farm in June 2024, where it successfully inserted nitrate sensors with high accuracy into$\mathbf{3 0}$cornstalks with a$\mathbf{7 7} {\%}$success rate.

Autonomous navigation is crucial for various robotics applications in agriculture. However, many existing methods depend on RTK-GPS devices, which can be susceptible to loss of radio signal or intermittent reception of corrections from the internet. Consequently, research has increasingly focused on using RGB cameras for crop-row detection, though challenges persist when dealing with grown plants. This paper introduces a LiDAR-based navigation system that can achieve crop-agnostic over-canopy autonomous navigation in row-crop fields, even when the canopy fully blocks the inter-row spacing. Our algorithm can detect crop rows across diverse scenarios, encompassing various crop types, growth stages, illumination conditions, the presence of weeds, curved rows, and discontinuities. Without utilizing a global localization method (i.e., based on GPS), our navigation system can perform autonomous navigation in these challenging scenarios, detect the end of the crop rows, and navigate to the next crop row autonomously, providing a crop-agnostic approach to navigate an entire field. The proposed navigation system has undergone tests in various simulated and real agricultural fields, achieving an average cross-track error of 3.55 cm without human intervention. The system has been deployed on a customized UGV robot, which can be reconfigured depending on the field conditions.

Sim2Real transfer, particularly for manipulation policies relying on RGB images, remains a critical challenge in robotics due to the significant domain shift between syn-thetic and real-world visual data. In this paper, we propose SplatSim, a novel framework that leverages Gaussian Splatting as the primary rendering primitive to reduce the Sim2Real gap for RGB-based manipulation policies. By replacing traditional mesh representations with Gaussian Splats in simulators, SplatSim produces highly photorealistic synthetic data while maintaining the scalability and cost-efficiency of simulation. We demonstrate the effectiveness of our framework by training manipulation policies within SplatSim and deploying them in the real world in a zero-shot manner, achieving an average success rate of 86.25%, compared to 97.5% for policies trained on real-world data. Videos can be found on our project page: https://splatsim.github.io

In this paper, we present a transformer-based method to spatio-temporally associate apple fruitlets in stereo-images collected on different days and from different camera poses. State-of-the-art association methods in agriculture are dedicated towards matching larger crops using either high-resolution point clouds or temporally stable features, which are both difficult to obtain for smaller fruit in the field. To address these challenges, we propose a transformer-based architecture that encodes the shape and position of each fruitlet, and propagates and refines these features through a series of transformer encoder layers with alternating self and cross-attention. We demonstrate that our method is able to achieve an F1-score of 92.4% on data collected in a commercial apple orchard and outperforms all baselines and ablations. The code and data can be found at https://kantor-lab.github.io/fruitassociator/

In this paper, we present a next-best-view planning approach to autonomously size apple fruitlets. State-of-the-art viewpoint planners in agriculture are designed to size large and more sparsely populated fruit. They rely on lower resolution maps and sizing methods that do not generalize to smaller fruit sizes. To overcome these limitations, our method combines viewpoint sampling around semantically labeled regions of interest, along with an attention-guided information gain mechanism to more strategically select viewpoints that target the small fruits’ volume. Additionally, we integrate a dual-map representation of the environment that is able to both speed up expensive ray casting operations and maintain the high occupancy resolution required to informatively plan around the fruit. When sizing, a robust estimation and graph clustering approach is introduced to associate fruit detections across images. Through simulated experiments, we demonstrate that our viewpoint planner improves sizing accuracy compared to state of the art and ablations. We also provide quantitative results on data collected by a real robotic system in the field.

There is growing interest in automating agricultural tasks that require intricate and precise interaction with specialty crops, such as trees and vines. However, developing robotic solutions for crop manipulation remains a difficult challenge due to complexities involved in modeling their deformable behavior. In this study, we present a framework for learning the deformation behavior of tree-like crops under contact interaction. Our proposed method involves encoding the state of a spring-damper modeled tree crop as a graph. This representation allows us to employ graph networks to learn both a forward model for predicting resulting deformations, and a contact policy for inferring actions to manipulate tree crops. We conduct a comprehensive set of experiments in a simulated environment and demonstrate generalizability of our method on previously unseen trees. Videos can be found on the project website: https://kantor-lab.github.io/tree_gnn

In this paper, we present a method for creating high-quality 3D models of sorghum panicles for phenotyping in breeding experiments. This is achieved with a novel reconstruction approach that uses seeds as semantic landmarks in both 2D and 3D. To evaluate the performance, we develop a new metric for assessing the quality of reconstructed point clouds without ground-truth. Finally, a counting method is presented where the density of seed centers in the 3D model allows 2D counts from multiple views to be effectively combined into a whole-panicle count. We demonstrate that using this method to estimate seed count and weight for sorghum outperforms count extrapolation from 2D images, an approach used in most state of the art methods for seeds and grains of comparable size.

In this work, we present a method to extract the skeleton of a self-occluded tree canopy by estimating the unobserved structures of the tree. A tree skeleton compactly describes the topological structure and contains useful information such as branch geometry, positions and hierarchy. This can be critical to planning contact interactions for agricultural manipulation, yet is difficult to gain due to occlusion by leaves, fruits and other branches. Our method uses an instance segmentation network to detect visible trunk, branches, and twigs. Then, based on the observed tree structures, we build a custom 3D likelihood map in the form of an occupancy grid to hypothesize on the presence of occluded skeletons through a series of minimum cost path searches. We show that our method outperforms baseline methods in highly occluded scenes, demonstrated through a set of experiments on a synthetic tree dataset. Qualitative results are also presented on a real tree dataset collected from the field.

Robotic pruning of dormant grapevines is an area of active research in order to promote vine balance and grape quality, but so far robotic efforts have largely focused on planar, simplified vines not representative of commercial vineyards. This paper aims to advance the robotic perception capabilities necessary for pruning in denser and more complex vine structures by extending plant skeletonization techniques. The proposed pipeline generates skeletal grapevine models that have lower reprojection error and higher connectivity than baseline algorithms. We also show how 3D and skeletal information enables prediction accuracy of pruning weight for dense vines surpassing prior work, where pruning weight is an important vine metric influencing pruning site selection.

We outline a neural network-based pipeline for perception, control and planning of a 7 DoF robot for tasks that involve reaching into a dormant grapevine canopy. The proposed system consists of a 6 DoF industrial robot arm and a linear slider that can actuate on an entire grape vine. Our approach uses Convolutional Neural Networks to detect buds in dormant grape vines and a Reinforcement Learning based control strategy to reach desired cut-point locations for pruning tasks. Within this framework, three methodologies are developed and compared to reach the desired locations: the learned policy-based approach (RL), a hybrid method that uses the learned policy and an inverse kinematics solver (RL+IK), and lastly a classical approach commonly used in robotics. We first tested and validated the suitability of the proposed learning methodology in a simulated environment that resembled laboratory conditions. A reaching accuracy of up to 61.90% and 85.71% for the RL and RL+IK approaches respectively was obtained for a vine that the agent observed while learning. When testing in a new vine, the accuracy was up to 66.66% and 76.19% for RL and RL+IK, respectively. The same methods were then deployed on a real system in an end to end procedure: autonomously scan the vine using a vision system, create its model and finally use the learned policy to reach cutting points. The reaching accuracy obtained in these tests was 73.08%.

Object detection and semantic segmentation are two of the most widely adopted deep learning algorithms in agricultural applications. One of the major sources of variability in image quality acquired outdoors for such tasks is changing lighting conditions that can alter the appearance of the objects or the contents of the entire image. While transfer learning and data augmentation reduce the need for large amount of data to train deep neural networks to some extent, the large variety of cultivars and the lack of shared datasets in agriculture makes wide-scale field deployments difficult. In this paper, we present an active lighting-based camera system that generates robust and uniform images in any lighting conditions. We provide extensive validation experiments to evaluate the consistency in the quality of the images. Metrics for assessing image uniformity like Structural Similarity (SSIM) index and the Peak Signal to Noise Ratio (PSNR) ranged from 83.78 to 93.79 and from 25.30 to 31.78 respectively, showing stability over a day with changing sunlight. The validation stage also showed that the generated images effectively reduce the amount of training samples for object detection using deep neural networks. The camera system was then deployed in real field experiments for counting buds in dormant vines and shoots in early season grape vines as well as for counting apples in orchards. The mean absolute errors obtained when compared to ground truth were 5%, 2.55% and 8.57%, respectively.

Simultaneous Localization and Mapping (SLAM) is a fundamental task to mobile and aerial robotics. LiDAR based systems have proven to be superior compared to vision based systems due to its accuracy and robustness. In spite of its superiority, pure LiDAR based systems fail in certain degenerate cases like traveling through a tunnel. We propose Stereo Visual Inertial LiDAR (VIL) SLAM that performs better on these degenerate cases and has comparable performance on all other cases. VIL-SLAM accomplishes this by incorporating tightly-coupled stereo visual inertial odometry (VIO) with LiDAR mapping and LiDAR enhanced visual loop closure. The system generates loop-closure corrected 6-DOF LiDAR poses in real-time and lcm voxel dense maps near real-time. VIL-SLAM demonstrates improved accuracy and robustness compared to state-of-the-art LiDAR methods.

Reinforcement learning is known to be sample inefficient, preventing its application to many real-world problems, especially with high dimensional observations like images. Transferring knowledge from other auxiliary tasks is a powerful tool for improving the learning efficiency. However, the usage of auxiliary tasks has been limited so far due to the difficulty in selecting and combining different auxiliary tasks. In this work, we propose a principled online learning algorithm that dynamically combines different auxiliary tasks to speed up training for reinforcement learning. Our method is based on the idea that auxiliary tasks should provide gradient directions that, in the long term, help to decrease the loss of the main task. We show in various environments that our algorithm can effectively combine a variety of different auxiliary tasks and achieves significant speedup compared to previous heuristic approches of adapting auxiliary task weights.

The need for fast and precise measurements of plant attributes makes robotic solutions an ideal replacement for labor-intensive phenotyping processes. In this work we present a deep learning-based high throughput, online pipeline for in-situ sorghum stalk detection and grasping. We use a variation of Generative Adversarial Network (GAN) for stalk segmentation trained on a relatively small number of images followed by a grasp point generation pipeline. The presented pipeline is robust to field challenges such as occlusions, high stalk density and lighting variation, and was deployed on a custom-built ground robot. We tested our end-to-end system in a field of Sorghum bicolor in South Carolina, USA, achieving an average grasping accuracy of 74.13% and a stalk detection F1 score of 0.90. Grasp point detection for plant manipulation takes an average of 0.98 seconds, and pixel-wise stalk detection takes 0.2 seconds per image.

Perception system performance can vary dramatically with contextual factors such as environmental geometry, appearance, and other phenomena. In this work we present a theoretical framework for understanding the role of context in perception and discuss three approaches for predicting probabilistic performance from observations by efficiently learning local performance models. We compare these approaches with experiments on the monocular and stereo visual odometry systems for a ground robot, and show that they can effectively predict system failures in a wide variety of environments.

The established processes for measuring physiological and morphological traits (phenotypes) of crops in outdoor test plots are labor intensive and error-prone. Low-cost, reliable, field-based robotic phenotyping will enable geneticists to more easily map genotypes to phenotypes, which in turn will improve crop yields. In this paper, we present a novel robotic ground-based platform capable of autonomously navigating below the canopy of row crops such as sorghum or corn. The robot is also capable of deploying a manipulator to measure plant stalk strength and gathering phenotypic data with a modular array of non-contact sensors. We present data obtained from deployments to Sorghum bicolor test plots at various sites in South Carolina, USA.

This paper describes an algorithm for visually detecting crop stalks in-situ. Field-based crop stalk detection is a challenging computer vision problem due to occlusion by leaves, similarity of color and texture between stalks and surrounding foliage, and high stalk density within rows. Detecting stalks for grasping adds further challenges such as computation time and computing hardware. The proposed algorithm utilizes multiple stereo images taken from an unmanned ground vehicle and exploits geometric features specific to stalks such as vertical continuity, height, and the direction of surface normals. The described hardware and software pipeline is capable of detecting stalks for a manipulator grasp in approximately 12 seconds, and preliminary results show that hardware improvements can reduce detection time to 4 seconds. The algorithm is tested on 378 point clouds generated from 916 images of Sorghum bicolor, a grain crop. Performance is evaluated according to two methods, demonstrating precision greater than 93%.

The popularity of cameras for perception is enabled in part by powerful intrinsic calibration routines, commonly requiring a user to manually collect images of a known calibration target. The manual nature of this process produces training data that is unevenly spread in the camera relative pose space. If we desire camera parameters that perform well on average over the entire relative pose space, training on such a dataset results in poor performance. To address this, we show that reasoning about the training data distribution to select a more uniformly-spread subset of images produces more accurate and stable calibrations with fewer images. Our approach can be used easily with most camera calibration algorithms. We demonstrate in large-scale physical experiments the effect of non-uniform training data and show that our approach outperforms baselines in reprojection error and parameter variance.

Assembly of large structures requires large fixtures, often referred to as monuments. Their cost and massive size limit flexibility and scalability of the manufacturing process. Numerous small mobile robots can replace these large structures and, therefore, replicate the efficiency of the assembly line with far more flexibility. An assembly line made up of mobile manipulators can easily and rapidly be reconfigured to support scalability and a varied product mix, while allowing for near optimal resource assignment. The challenge to using small robots in place of monuments is making their joint behavior precise enough to accomplish the task and efficient enough to execute subtasks in a reasonable period of time. In this paper, we describe a set of techniques that we combine to achieve the necessary precision and overall efficiency to build a large structure. We describe and demonstrate these techniques in the context of a testbed we implemented for assembling a wing ladder.

The ballbot is a human sized, dynamically stable mobile robot that balances on a single, spherical wheel. The current framework for navigation and control makes the assumption that the robot is operating on a level surface without any center-of-mass offset; however, in practice, such a dynamically stable robot has to be able to successfully navigate sloped surfaces and with such offsets. This work develops the equations of motion for the ballbot system on a sloped surface with a center-of-mass offset. The equilibria of this system are analyzed, and a compensation strategy is formulated that allows the ballbot to operate in the presence of slopes and center-of-mass offsets. This work also develops estimation algorithms for slope and center-of-mass offset angles during station-keeping and trajectory following. Results for compensation and estimation are demonstrated experimentally.

The ubiquitous additive Gaussian noise model is favored in statistical modeling applications for its flexibility and ease of use. Often noise is assumed to be well-represented by a constant covariance, while in reality error characteristics may change predictably. We present an efficient parametric covariance predictor based on the modified Cholesky decomposition that maps from features of the input to covariance matrices. In addition, we discuss fitting the predictor parameters using noise samples with simple regularization techniques. We demonstrate our approach by estimating observation covariances for range-bearing localization with simulated and experimental datasets and show that this results in increased filtering performance compared to traditional covariance adaptation and constant covariance baselines.

In this paper, we present a novel approach for mapping periodically moving visual features with a monocular camera. Our target application is the estimation of moving surfaces during minimally invasive surgery for the purpose of aiding in the guidance of surgical tools. Our approach uses a bank of Kalman filters to estimate FFT parameters that encode the periodic motion of visually detected features. To ensure convergent estimation for this highly nonlinear problem, we have developed an iterative update procedure that treats the Kalman filter measurement update step as an optimization problem. Unlike existing solutions that rely on stereo vision, our approach estimates periodic motion with a single moving camera. With an experiment involving a beating heart phantom, we have shown that our approach is able to successfully estimate the periodic motion of visual features.

In this paper, we present a novel algorithm for deterministically mapping an undirected graph-like world with multiple synchronized agents. The application of this algorithm is the collective mapping of an indoor environment with multiple mobile robots while leveraging an embedded topological decomposition of the environment. Our algorithm relies on a group of agents that all depart from the same initial vertex in the graph and spread out to explore the graph. A centralized tree of graph hypotheses is maintained to consider loop-closure, which is deterministically verified when agents observe each other at a common vertex. To achieve efficient mapping, we introduce an active exploration method in which agents dynamically request rendezvous tasks from other available agents to validate graph hypotheses.

This paper presents controllers called motion policies that achieve fast, graceful motions in small, collision-free domains of the position space for balancing mobile robots like the ballbot. The motion policies are designed such that their valid compositions will produce overall graceful motions. An automatic instantiation procedure deploys motion policies on a 2D map of the environment to form a library and the validity of their composition is given by a gracefully prepares graph. Dijsktra's algorithm is used to plan in the space of these motion policies to achieve the desired navigation task. A hybrid controller is used to switch between the motion policies. The results of successful experimental testing of two navigation tasks, namely, point-point and surveillance motions on the ballbot platform are presented.

This paper presents a novel filtering technique that uses contact detection data and environmental stiffness estimates to register and localize a robot with respect to an a priori 3D surface model. The algorithm leverages geometric constraints within a Kalman filter framework and relies on two distinct update procedures: 1) an equality constrained step for when the robot is forcefully contacting the environment, and 2) an inequality constrained step for when the robot lies in the free-space of the environment. This filtering procedure registers the robot by incrementally eliminating probabilistically infeasible state space regions until a high likelihood solution emerges. In addition to registration and localization, the algorithm can estimate the deformation of the surface model and can detect false positives with respect to contact estimation. This method is experimentally evaluated with an experiment involving a continuum robot interacting with a bench-top flexible structure. The presented algorithm produces an experimental error in registration (with respect to the end-effector position) of 1.1 mm, which is less than 0.8 percent of the robot length.

We present a monocular visual navigation methodology for autonomous orchard vehicles. Modern orchards are usually planted with straight and parallel tree rows that form a corridor-like environment. Our task consists of driving a vehicle autonomously along the tree rows. The original contributions of this paper are: 1) a method to recover vehicle rotation independently of translation by modeling the vehicle as a car-like robot driving on a 3D ground surface-the rotation is estimated from monocular images while the translation is measured by a wheel encoder; and 2) a method to fit the 3D points corresponding to the trees into straight lines via an optimization algorithm that minimizes the error variance on the robot lookahead point. Additionally, we use a simple vanishing point detection approach to find the ends of the tree rows. The vanishing point detection is integrated into the system via an extended Kalman filter. The methodology's robustness to environmental changes is validated in more than fifty experiments in research and commercial orchards, six of which are presented and discussed in detail.

This paper presents a perception-based GPS free approach for localizing a mobile robot in an orchard environment. An extended Kalman filter (EKF) algorithm is presented that uses a wheel odometry prediction step and laser rangefinder update steps. There are two update steps, one that uses measurements to reflective point features and one that uses measurements to linear features formed by tree rows. The features are associated to landmarks in previously surveyed maps. The practical issues of dealing with uncertainty both from the environment and the on-board sensors are discussed and accounted for. The resulting algorithm is demonstrated in over 20km of online operation in a variety of real orchard environments.

The generalized Voronoi graph (GVG) is a topological representation of an environment that can be incrementally constructed with a mobile robot using sensor-based control. However, because of sensor range limitations, the GVG control law will fail when the robot moves into a large open area. This paper discusses an extended GVG approach to topological navigation and mapping: the saturated generalized Voronoi graph (S-GVG), for which the robot employs an additional wall-following behavior to navigate along obstacles at the range limit of the sensor. In this paper, we build upon previous work related to the S-GVG and provide two important contributions: 1) a rigorous discussion of the control laws and algorithm modifications that are necessary for incremental construction of the S-GVG with a mobile robot, and 2) a method for incorporating the S-GVG into a novel multi-hypothesis SLAM algorithm for loop-closing and localization. Experiments with a wheeled mobile robot in an office-like environment validate the effectiveness of the proposed approach.

This paper uses KLD-based (Kullback-Leibler Divergence) Monte Carlo Localization (MCL) to localize a mobile robot in an indoor environment represented by 3D texture maps. A 3D texture map is a simplified model that includes vertical planes with colored texture information associated with each vertical plane. At each time step, a distance measurement and an observed texture from an omnidirectional camera are compared to the expected distance measurement and the expected texture according to each hypothesis of the robot's pose in an MCL framework. Compared to previous implementations of MCL, our proposed approach converges faster than distance-only MCL and localizes the robot more precisely than SIFT-based MCL. We demonstrate this new MCL algorithm for robot localization with experiments in several hallways.

We present a novel method for enforcing nonlinear inequality constraints in the estimation of a high degree of freedom robotic system within a Kalman filter. Our constrained Kalman filtering technique is based on a new concept, which we call uncertainty projection, that projects the portion of the uncertainty ellipsoid that does not satisfy the constraint onto the constraint surface. A new PDF is then generated with an efficient update procedure that is guaranteed to reduce the uncertainty of the system. The application we have targeted for this work is the localization and automatic registration of a robotic surgical probe relative to preoperative images during image-guided surgery. We demonstrate the feasibility of our constrained filtering approach with data collected from an experiment involving a surgical robot navigating on the epicardial surface of a porcine heart.

In this paper, we present a filtering method for estimating the shape and end effector pose of a highly articulated surgical snake robot. Our algorithm introduces new kinematic models that are used in the prediction step of an extended Kalman filter whose update step incorporates measurements from a 5-DOF electromagnetic tracking sensor situated at the distal end of the robot. A single tracking sensor is sufficient for estimating the shape of the system because the robot is inherently a follow-the-leader mechanism with well defined motion characteristics. We therefore show that, with appropriate steering motion, the state of the filter is fully observable. The goal of our shape estimation algorithm is to create a more accurate and representative 3D rendered visualization for image-guided surgery.We demonstrate the feasibility of our method with results from an animal experiment in which our shape and pose estimate was used as feedback in a control scheme that semi-autonomously drove the robot along the epicardial surface of a porcine heart.

This paper presents a novel recursive maximum a posteriori update for the Kalman formulation of undelayed bearing-only SLAM. The estimation update step is cast as an optimization problem for which we can prove the global minimum is reachable via a bidirectional search using Gauss-Newton's method along a one-dimensional manifold. While the filter is designed for mapping just one landmark, it is easily extended to full-scale multiple-landmark SLAM. We provide this extension via a formulation of bearing-only FastSLAM. With experiments, we demonstrate accurate and convergent estimation in situations where an EKF solution would diverge.

Dynamically balancing, two-wheeled robots with high centers of gravity have been researched over the past few years in order to tackle the restrictions of robots in human environments. These robots are designed to have the same dimensions as humans and be able to actively maintain stability. However these systems are only able to compensate in the fore-aft directions and are limited to slow motions when turning in order to prevent tipping due to centrifugal forces. In this paper we present a two-wheeled balancing robot whose body is an actuated four-bar linkage that can be used to lean. This allows it to control the position of its center of mass in the sideways direction. An overview of the robot concept is presented, and controllers are designed to balance, turn, and lean. Results are presented in dynamic simulation.

Developed by Prof. Ralph Hollis in the Microdynamic Systems Laboratory at Carnegie Mellon University, Ballbot is a dynamically stable mobile robot moving on a single spherical wheel providing omni-directional motion. Unlike statically stable mobile robots, dynamically stable mobile robots can be tall and skinny with high center of gravity and small base. The ball drive mechanism is a four motor inverse mouse-ball setup. An Inertial Measuring Unit (IMU) and encoders on the motors provide all information needed for full-state feedback. Ballbot has three legs that provide static stability when powered down and is capable of auto-transitioning from the statically stable state to the dynamically stable state and vice versa. It is also capable of yaw rotation about its vertical axis. An absolute encoder provides the relative angle between the IMU and the ball drive unit.

Human-Robot Physical Interaction is an important attribute for robots operating in human environments. The authors illustrate some basic physically interactive behaviors with dynamically stable mobile robots using the ballbot as an example. The ballbot is a dynamically stable mobile robot moving on a single spherical wheel. The dynamic stability and robust controllers enable the ballbot to be physically moved with ease. The authors also demonstrate other behaviors like human intent detection and learn-repeat behavior on the real robot.

The ballbot is a dynamically stable mobile robot that moves on a single spherical wheel and is capable of omnidirectional movement. The ballbot is an underactuated system with nonholonomic dynamic constraints. The authors propose an offline trajectory planning algorithm that provides a class of parametric trajectories to the unactuated joint in order to reach desired static configurations of the system with regard to the dynamic constraint. The parameters of the trajectories are obtained using optimization techniques. A feedback controller is proposed that ensures accurate trajectory tracking. The trajectory planning algorithm and tracking controller are validated experimentally. The authors also extend the offline trajectory planning algorithm to a generalized case of motion between non-static configurations.

Unlike statically stable wheeled mobile robots, dynamically stable mobile robots can have higher centers of gravity, smaller bases of support and can be tall and thin resembling the shape of an adult human. This paper concerns the ballbot mobile robot, which balances dynamically on a single spherical wheel. The ballbot is omni-directional and can also rotate about its vertical axis (yaw motion). It uses a triad of legs to remain statically stable when powered off. This paper presents the evolved design with a four-motor inverse mouse-ball drive, yaw drive, leg drive, control system, and results including dynamic balancing, station keeping, yaw motion while balancing, and automatic transition between statically stable and dynamically stable states.

We present a method for topological SLAM that specifically targets loop closing for edge-ordered graphs. Instead of using a heuristic approach to accept or reject loop closing, we propose a probabilistically grounded multi-hypothesis technique that relies on the incremental construction of a map/state hypothesis tree. Loop closing is introduced automatically within the tree expansion, and likely hypotheses are chosen based on their posterior probability after a sequence of sensor measurements. Careful pruning of the hypothesis tree keeps the growing number of hypotheses under control and a recursive formulation reduces storage and computational costs. Experiments are used to validate the approach.

Perceptual aliasing makes topological navigation a difficult task. In this paper we present a general approach for topological SLAM (simultaneous localisation and mapping) which does not require motion or odometry information but only a sequence of noisy measurements from visited places. We propose a particle filtering technique for topological SLAM which relies on a method for disambiguating places which appear indistinguishable using neighbourhood information extracted from the sequence of observations. The algorithm aims to induce a small topological map which is consistent with the observations and simultaneously estimate the location of the robot. The proposed approach is evaluated using a data set of sonar measurements from an indoor environment which contains several similar places. It is demonstrated that our approach is capable of dealing with severe ambiguities and, and that it infers a small map in terms of vertices which is consistent with the sequence of observations.

This paper discusses the importance of iteration when performing the measurement update step for the problem of bearing-only SLAM. We focus on an undelayed approach that initializes a landmark after only one bearing measurement. Traditionally, the extended Kalman filter (EKF) has been used for SLAM, but the EKF measurement update rule can often lead to a divergent state estimate due to its inconsistency in linearization. We discuss the flaws of the EKF in this paper, and show that even the well established inverse-depth parametrization for bearing-only SLAM can be affected. We then show that representing the bearing-only update as a numerical optimization problem (solved with an iterative approach such as Gauss-Newton minimization) prevents divergence of the Kalman filter state and produces accurate SLAM results for a bearing-only sensor. More specifically, we propose the use of an iterated Kalman filter to resolve the issues normally associated with the EKF measurement update. Two outdoor mobile robot experiments are discussed to compare algorithm performance.

This paper presents a hybrid localization scheme for a mobile robot using the hierarchical atlas. The hierarchical atlas is a map that consists of a higher level topological graph with lower level feature-based metric submaps associated with the graph edges. Our method employs both a discrete Bayes filter and a Kalman filter to localize the robot in the map. This framework accommodates localization in a map with no prior information (global localization) and localization in a map with an incorrect pose estimate (kidnapped robot). Our approach efficiently scales to large environments without sacrificing accuracy or robustness. We have verified our method with large-scale experiments in a multi-floor office environment.

A mobile robot we have developed is equipped with sensors to measure range to landmarks and can simultaneously localize itself as well as locate the landmarks. This modality is useful in those cases where environmental conditions preclude measurement of bearing (typically done optically) to landmarks. Here we extend the paradigm to consider the case where the landmarks (nodes of a sensor network) are able to measure range to each other. We show how the two capabilities are complimentary in being able to achieve a map of the landmarks and to provide localization for the moving robot. We present recent results with experiments on a robot operating in a randomly arranged network of nodes that can communicate via radio and range to each other using sonar. We find that incorporation of inter-node measurements helps reduce drift in positioning as well as leads to faster convergence of the map of the nodes. We find that addition of a mobile node makes the SLAM feasible in a sparsely connected network of nodes

This paper describes the application of a RaoBlackwellized Particle Filter to the problem of simultaneous localization and mapping onboard a hovering autonomous underwater vehicle. This vehicle, called DEPTHX, equipped with a large array of pencil-beam sonars for mapping, and autonomously explore a system of flooded tunnels associated with the Zacaton sinkhole in Tamaulipas, Mexico. Due to the three-dimensional nature of the tunnels, we describe an extension of traditional two dimensional evidence grids to three dimensions. In May 2005, we collected a sonar data set in Zacaton. We present successful SLAM results using both the real-world data and simulated data

Multi-wheel statically-stable mobile robots tall enough to interact meaningfully with people must have low centers of gravity, wide bases of support, and low accelerations to avoid tipping over. These conditions present a number of performance limitations. Accordingly, we are developing an inverse of this type of mobile robot that is the height, width, and weight of a person, having a high center of gravity, that balances dynamically on a single spherical wheel. Unlike balancing 2-wheel platforms which must turn before driving in some direction, the single-wheel robot can move directly in any direction. We present the overall design, actuator mechanism based on an inverse mouse-ball drive, control system, and initial results including dynamic balancing, station keeping, and point-to-point motion

This paper presents a new map specifically designed for robots operating in large environments and possibly in higher dimensions. We call this map the hierarchical atlas because it is a multilevel and multiresolution representation. For this paper, the hierarchical atlas has two levels: at the highest level there is a topological map that organizes the free space into submaps at the lower level. The lower-level submaps are simply a collection of features. The hierarchical atlas allows us to perform calculations and run estimation techniques, such as Kalman filtering, in local areas without having to correlate and associate data for the entire map. This provides a means to explore and map large environments in the presence of uncertainty with a process named hierarchical simultaneous localization and mapping. As well as organizing information of the free space, the map also induces well-defined sensor-based control laws and a provably complete policy to explore unknown regions. The resulting map is also useful for other tasks such as navigation, obstacle avoidance, and global localization. Experimental results are presented showing successful map building and subsequent use of the map in large-scale spaces.

It is essential in many applications that mobile robots localize themselves with respect to an unknown environment. This means that the robot must build a map of its environment and then localize using the map. This process is called simultaneous localization and mapping (SLAM). This paper presents an iterative solution to the landmark initialization problem inherent in a bearing-only implementation of SLAM. No prior knowledge of the environment is required, and furthermore, there are no requirements about having the data association problem solved. Once landmarks are initialized, they are inserted into an extended Kalman Filter (EKF) to solve the SLAM problem. Both indoor and outdoor experiments are presented to validate the method.

No abstract available.

We present an early experimental result toward solving the localization problem with range-only sensors. We perform an experiment in which a mobile robot localizes using dead reckoning and range measurements to stationary radio-frequency beacons in its environment, incorporating the range measurements into the position estimate using a Kalman filter. This data set involves over 20,000 range readings to surveyed beacons while a robot moved continuously over a path for nearly 1 hour. Careful groundtruth accurate to a few centimeters was recorded during this motion. We show the improvement of the robot's position estimate over dead reckoning even when the range readings are very noisy. We extend this approach to the problem of simultaneous localization and mapping (SLAM), localizing both the robot and tag positions from noisy initial estimates.

This paper presents a novel method of combining topological and feature-based mapping strategies to create a hierarchical approach to simultaneous localization and mapping (SLAM). More than simply running both processes in parallel, we use the topological mapping procedure to organize local feature-based methods. The result is an autonomous exploration and mapping strategy that scales well to large environments and higher dimensions while confronting the issue of obstacle avoidance. We have obtained successful results of our approach in an area spanning 5000 square meters.

This paper presents preliminary development of autonomous sensor-guided behaviors for a six-legged dynamical robot (RHex). The behaviors represent the exteroceptive closed-loop locomotion strategies for this system. Simple motion models for RHex are described and used to deploy controllers for inertial straight line locomotion and visual line following. Results are experimentally validated on the robot.

This paper presents methods of localization using cooperating landmarks (beacons) that provide the ability to measure range only. Recent advances in radio frequency technology make it possible to measure range between inexpensive beacons and a transponder Such a method has tremendous benefit since line of sight is not required between the beacons and the transponder and because the data association problem can be completely avoided. If the positions of the beacons are known, measurements from multiple beacons can be combined using probability grids to provide an accurate estimate of robot location. This estimate can be improved by using Monte Carlo techniques and Kalman filters to incorporate odometry data. Similar methods can be used to solve the simultaneous localization and mapping problem (SLAM) when beacon locations are uncertain. Experimental results are presented for robot localization. Tracking and SLAM algorithms are demonstrated in simulation.