We describe a method of generating and utilizing visual landmarks that is well suited for SLAM applications. The landmarks created are highly distinctive and reliably detected, virtually eliminating the data association problem present in other landmark schemes. Upon subsequent detections of a landmark, a 3-D pose can be estimated. The scheme requires a single camera.

This paper presents the Visual Simultaneous Localization and Mapping (vSLAMTM) algorithm, a novel algorithm for simultaneous localization and mapping (SLAM). The algorithm is vision-and odometry-based, and enables low-cost navigation in cluttered and populated environments. No initial map is required, and it satisfactorily handles dynamic changes in the environment, for example, lighting changes, moving objects and/or people. Typically, vSLAM recovers quickly from dramatic disturbances, such as “kidnapping”.

In this paper we describe the need for, and the characteristics of a software architecture for commercial robotic products. We describe the Evolution Robotics Software Platform (ERSP/spl trade/), which provides a commercial-grade software architecture for mobile robots. The architecture has been designed to be modular, scalable, lightweight, portable, and reusable. It follows a hybrid model of data flow, combining behavior-based processing modules for real-time reactions with event-based task planning routines. We provide a detailed description of the main architectural components and some basic usage studies. We also highlight areas where compromises have been made and for which continuing work is needed.

Service robotics products are becoming a reality. This paper describes three core technologies that enable the next generation of service robots. They are low-cost, make use of low-cost hardware, and prepare for a short time-to-market for product development. The first technology is an object recognition system, which can be used by the robot to interact with the environment The second technology is a vision-based navigation system (vSLAM/spl trade/), which simultaneously can build a map and localize the robot in the map. Finally, the third technology is a flexible and rich software platform (ERSP/spl trade/) that assists developers in rapid design and prototyping of robotics applications.

In this paper, a vision-based stabilization and output tracking control method for a four-rotor helicopter has been proposed. A novel 2 camera method has been described for estimating the full 6 DOF pose of the helicopter. This two camera system is consisting of a pan-tilt ground camera and an onboard camera. The pose estimation algorithm is compared in simulation to other methods (such as four point method, and a stereo method) and is shown to be less sensitive to feature detection errors on the image plane. The proposed pose estimation algorithm and non-linear control techniques have been implemented on a remote controlled quadrotor helicopter.

We describe some recent work on randomized motion planning algorithms and consider the problem of motion planning for systems with both kinematic and dynamic constraints. Such a problem is often referred to as kinodynamic motion planning. A rapidly-exploring random tree (RRT) is used for the motion planning of a blimp system, and some techniques for improving the performance of the planner are proposed. Based on a dynamic model of a blimp, dynamic constraints are introduced into the algorithm design and simulations are conducted for trajectory generation.

We build on previous work, on the development of a computer controlled wheelchair equipped with a suite of sensors and a novel interface for human-robot interaction. In this paper, we present experimental results and usability studies for the wheelchair. The architecture for human-robot interaction is hierarchical, with the lowest level corresponding to trajectory control, the intermediate level being behavioral and the highest level involving the composition of behaviors and navigation. Our experimental results illustrate the benefits of a shared-control paradigm where the human operator selects the appropriate behavior(s) or goals while the software is responsible for executing behaviors and generating safe trajectories. Experiments with human users highlight advantages of augmentation in wheelchairs.

We investigate issues of control and motion planning for a biomimetic robotic system. Previous work has shown that a successful approach to solving the motion planning problem is to decouple it into the two subproblems of trajectory generation (feedforward controls) and feedback regulation. In this paper, we investigate basic issues of momentum generation for a class of dynamic mobile robots, focusing on eel-like swimming robots. We develop theoretical justification for a forward gait that has been observed in nature, and for a turning gait, used in our control laws, that has not been extensively studied in the biological literature. We also explore theoretical predictions for novel gaits for turning and sideways swimming. Finally, we present results from experiments in motion planning for a biomimetic robotic system. We show good agreement with theory for both open and closed-loop control of our modular, five-link, underwater swimming robot using image-based position sensing in an aquatic environment.

We present control methods for an autonomous four-rotor helicopter, called a quadrotor, using visual feedback as the primary sensor. The vision system uses aground camera to estimate the pose (position and orientation) of the helicopter. Two methods of control are studied - one using a series of mode-based, feedback linearizing controllers, and the other using a backstepping-like control law. Various simulations of the model demonstrate the implementation of feedback linearization and the backstepping controllers. Finally,. we present initial flight experiments. where the helicopter is restricted to vertical and yaw motions.

Addresses the issue of developing a motion planning algorithm for a general class of modular mobile robots. A modular mobile robot is essentially a reconfigurable robot system in which locomotive modules such as legs, wheels, propellers, etc. can be attached at various locations on the body. The equations of motion for the robot are developed in terms of a set of generalized inputs related to the constraints and configurations of each individual module. The motion planning method developed in Lafferriere and Sussmann (1991) and extended in Goodwine and Burdick (1997) can then be modified to generate required trajectories for the modular robot. These methods generally lead to sequential motion plans-we also develop conditions under which two consecutive plans can be executed simultaneously.

We investigate basic issues of motion planning for a class of dynamic mobile robots, focusing on eel-like swimming robots. A primary characteristic of this class of robots is that drift plays a significant role in the generation of motion. We provide a brief summary of our previous work in eel-like robotics, in which we explored generic gait patterns that could be used to drive an eel-like robot. We make an analogy with kinematic car-like robots to develop feedback algorithms to perform trajectory trucking about a nominal path. We use these control laws in closed-loop experiments with an underwater eel-like robot, using a vision based position sensing system to provide feedback.

This paper presents a sensor based algorithm for guiding a nonholonomic platform, such as a wheelchair, through a doorway. The controller uses information from a camera system and a laser range finder to perform image-based navigation. Simulations of the resultant switching controller are presented along with experimental results. A simple obstacle avoidance algorithm is also implemented on the experimental platform. Finally, we have considered the input of limited field-of-view constraints on this controller. All of these components together lead to a modal, image-based approach that will safely and robustly navigate a nonholonomic robot with sensor constraints through a doorway.

Addresses the problem of human robot interaction with application to the design of assistive devices. We describe the design and development of a prototype of a smart wheelchair that can be commanded by a rider. Specifically, we focus on (a) the vision-based human interaction interface; (b) the suite of sensors on the chair, and (c) the software architecture and the control algorithms used to control the chair.

We describe a framework for cooperative control of a group of nonholonomic mobile robots that allows us to build complex systems from simple controllers and estimators. The resultant modular approach is attractive because of the potential for reusability. Our approach to composition also guarantees stability and convergence in a wide range of tasks. There are two key features in our approach: 1) a paradigm for switching between simple decentralized controllers that allows for changes in formation; 2) the use of information from a single type of sensor, an omnidirectional camera, for all our controllers. We describe estimators that abstract the sensory information at different levels, enabling both decentralized and centralized cooperative control. Our results include numerical simulations and experiments using a testbed consisting of three nonholonomic robots.

This paper presents a novel framework for image-based motion planning, which we term Visual Motion Planning. The method skips the step of transferring image features back to robot pose, and hence makes motion plans directly in the image plane. Analogous to visual servo control, the visual motion planning concept takes advantage of the image features to achieve a direct and fast motion planning solution. It provides a "virtual" trajectory in the image plane for the robot to track with standard visual servoing techniques. In this paper, we show the result of applying the idea to simulated 2-D and 3-D mobile robot systems. Within this motion planning paradigm, we also discuss the mechanisms for taking advantage of surplus features and incorporating image-based constraints, such as requiring that the images remain in the field of view. Experimental results using a Pioneer AT ground mobile robot are presented, showing excellent agreement with theory.

We describe a framework for controlling a group of nonholonomic mobile robots equipped with range sensors. The vehicles are required to follow a prescribed trajectory while maintaining a desired formation. By using the leader-following approach, we formulate the formation control problem as a hybrid (mode switching) control system. We then develop a decision module that allows the robots to automatically switch between continuous-state control laws to achieve a desired formation shape. The stability properties of the closed-loop hybrid system are studied using the Lyapunov theory. We do not use explicit communication between robots; instead we integrate optimal estimation techniques with nonlinear controllers. Simulation and experimental results verify the validity of our approach.

We present a novel modeling framework and control algorithms for multiple mobile manipulators cooperatively grasping and transporting an object. The planning and control tasks are decentralized, and the framework explicitly incorporates the protocols used to coordinate the robots in the team. The framework is flexible in the sense that it scales with the number of robots and controllers. Preliminary experimental results for teams of two-three robots are shown, while simulation results are used to illustrate the extensions of our approach to larger teams.

We present a control algorithm to control underactuated mechanical systems that evolve in the configuration space with the fiber bundle structure. We show that by using the geometric tools such as symmetric product and small amplitude periodic forcing, we can develop unique techniques to control the underactuated dynamic system, such as an unmanned blimp.

This paper presents several new insights and experimental results on the problem of generating omnidirectional walking gaits for quadrupedal robots. For statically stable gaits, by placing some minor restrictions on how the leg motions are generated, we develop an easily computable classification of the best gait patterns (leg phasings) to be used for any given motion. This simple geometric construction allows easy generation of stable, omnidirectional walking gaits for any quadruped satisfying only some very weak assumptions about the location of the center of mass relative to the workspace of the legs. It is also well-suited to a modular approach in which the control of individual legs is decentralized. We also implement an omnidirectional dynamic trot gait using a similar approach. Transitions involving a change in the direction of motion of the robot are initiated at one of two fixed points of the foot placement curves for each leg in all gaits. Experimental data is presented showing the effectiveness of these techniques.

We describe a framework for coordinating multiple robots in cooperative manipulation tasks in which vision is used for establishing relative position and orientation and maintaining formation. The two key contributions are a cooperative scheme for localizing the robots based on visual imagery that is more robust than decentralized localization, and a set of control algorithms that allow the robots to maintain a prescribed formation (shape and size). The ability to maintain a prescribed formation allows the robots to "trap" objects in their midst, and to "flow" the formation to a desired position. We derive the cooperative localization and control algorithms and present experimental results that illustrate the implementation and the performance of these algorithms.

This paper addresses the control of a team of nonholonomic mobile robots navigating in a terrain with obstacles while maintaining a desired formation and changing formations when required, using graph theory. We model the team as a triple, (g, r, H), consisting of a group element g that describes the gross position of the lead robot, a set of shape variables r that describe the relative positions of robots, and a control graph H that describes the behaviors of the robots in the formation. Our framework enables the representation and enumeration of possible control graphs and the coordination of transitions between any two formations.

We investigate basic issues of motion planning for a class of dynamic mobile robots, focusing on eel-like swimming robots. A primary characteristic of this class of robots is that drift plays a significant role in the generation of motion. We build on previous work (1999) in which we explored generic gait patterns that could be used to drive an eel-like robot. We make an analogy with kinematic car-like robots to develop a nominal path from an initial state to a goal state, and then develop feedback algorithms to perform trajectory tracking around this nominal path. We also address the central issues that arise when using cyclic gaits as the basis for control strategies.

The problem of controlling the spatial position and orientation of a robotic platform based on the image data obtained from a video camera mounted on that platform is considered. More specifically, we propose control laws that will cause the robot to achieve and maintain a fixed position and orientation with respect to a set of feature points in the scene. We demonstrate analytically that the proposed control scheme is globally convergent even in the presence of large calibration errors in both the intrinsic parameters of the camera and in the extrinsic parameters which relate the frame of reference of the camera to the body frame of the robot platform which is being controlled. Furthermore, no a priori knowledge about the structure of the scene is assumed.

This paper presents a novel approach to image-based visual servoing by making motion plans directly in the image plane. By skipping the step of transferring image features back to robot pose, we are able to utilize the image features for a direct and fast motion planning solution. Standard visual servoing techniques can then be applied to tract the given trajectory. In this paper, we have applied the idea to 2D and 3D (kinematic) motion planning and discussed the mechanisms for taking advantage of surplus features and incorporating other image-based constraints, such as that the targets always remain in the field of view.

We propose a hierarchical structure for control of autonomous robots. We draw on ideas from the hybrid systems theory, which describe systems governed by a discrete set of continuous modes of operation and transitions between these modes. We aim to enable rapid development of test applications and to allow easy performance upgrades. Our structure is based on the creation of agents to process inputs and outputs at the highest level of abstraction in the system (the application domain). We describe our implementation of this structure in two systems: a nonholonomic car-like robot and a quadrupedal entertainment robot. Both examples demonstrate how abstraction of planning away from the implementation details of the robot enables rapid development.

We investigate issues of control and motion planning for a biomimetic robotic system. Previous work has shown that the motion problem can be decoupled into trajectory generation and steering. We investigate basic issues of momentum generation for a class of dynamic mobile robots, focusing on eel-like swimming robots. We use control laws based on a series of gaits motivated by the biological literature on coupled oscillators and central pattern generators. A primary characteristic of this class of robots is that drift plays a significant role in the generation of motion. We develop theoretical justification for a forward gait that has been observed in nature and for a turning gait, used in our control laws, that has not yet been studied in the biological literature. We also explore theoretical predictions of novel gaits for turning and sideways swimming.

This paper addresses the use of nonholonomic gears in robotic applications to reduce the number of actuators necessary to control a given system. Previous research by (Sordalen et al., 1994) has shown that an n-degree of freedom robot can be controlled using only two actuators. This paper focuses on the advantages of using three or more actuators to control a system of nonholonomic gears, and derives the principles for doing so. In particular, it shows that using one or two additional actuators aids in reducing the complexity of the control laws, while at the same time increasing the number of degrees of freedom that can feasibly be controlled. The paper illustrates the ideas using three novel proposed systems: a three-degree of freedom reorientable table, a six-degree of freedom mobile manipulator, and a (human) head-driven feeding device for quadraplegics.

We present a software architecture used to demonstrate the ability of a tank-like robot to climb multiple flights of stairs. The algorithm we use incorporates intelligent sensor fusion and a hierarchical modular structure. This structure simplifies the overall design, enables reuse, and promotes extensibility. The modular nature of the design also allows for a variety of user interface modes with different levels of user autonomy. The robustness of our algorithm allows us to ascend and descend stairs autonomously in many environments with different textures and configurations, all with minimal adjustment to the algorithm. Our approach develops an extensible sensor and task model that reduces the effect of sensor inaccuracies and geometric constraints, uses minimal information about the environment, and utilizes characteristic features in the environment for navigation.

In this paper the problem of controlling the spatial position and orientation of a robotic platform based on the image data obtained from a video camera mounted on that platform is considered. More specifically, we propose control laws that generate translational and angular velocities that will cause the robot to achieve and maintain a fixed position and orientation with respect to a set of feature points in the scene. The proposed control schemes make use of well established techniques for computing estimates for the relative orientation of two camera positions from a set of feature correspondences. An important advantage of these control schemes is that it is possible to demonstrate analytically that they are globally convergent even in the presence of large calibration errors in both the intrinsic parameters of the camera and in the extrinsic parameters which relate the frame of reference of the camera to the body frame of the robot platform which is being controlled. Furthermore no a priori knowledge about the structure of the scene is assumed.

Addresses the control of a team of robots navigating in a terrain with obstacles while maintaining a desired formation and changing formations when required. We model the team as a triple consisting of a group element that describes the gross motion of the team, a set of shape variables that describe the relative positions of robots, and a control graph that describes the behaviors of the robots in the formation. We assume that a lead robot is equipped with the appropriate sensors and plans the gross motion (path) for the team. This path is derived from optimal control theory. All other robots are coordinated by continuous controllers that are prescribed by the control graph. Our framework allows us to enumerate the number of control graphs and the possible transitions between them. Further, we describe an algorithm that allows the team of robots to move between any two formations, while avoiding obstacles. We illustrate the methodology with examples involving teams of 5 and 6 robots in the presence of obstacles.

In this paper, we approach the modeling, simulation, and control of snake and eel-like robots from a geometric perspective. We propose basic viscous and fluid drag models to capture the effect of external forces acting on the bodies of land-based snakes and aquatic eels, respectively. Using these models, we show that the dynamics can be decomposed into motions in the position and orientation of the overall system and changes in its internal shape. This allows us to write reduced dynamics for the serpentine and swimming motions that isolate the effect of shape changes on the momentum of the system. We explore in simulation several types of locomotive gaits, and report qualitative experimental results that provide validation for these simulations.

Image-based control of free-flying mechanical systems is addressed, with application to an indoor vision-guided blimp. A methodology is developed for incorporating the physical parameters of a mechanical system into the image plane for performing visual servoing. It is noted that under suitable conditions, namely the existence of a diffeomorphism between image features and robot pose, many tools from mechanical system control theory can be used directly when applied in the image plane. It is shown that when a suitable Jacobian map exists between pose and features, dynamics for vision-based control of aerial and underwater vehicles can easily be formulated. Experimental results show initial success at spatial tracking of simple objects.

Develops easily computable methods for deriving the reduced equations for mechanical systems with Lie group symmetries. These types of systems occur frequently in robotics, and are found generically in robotic locomotion, wheeled mobile robots, and satellites or underwater vehicles with robotic arms. Results are presented for two important cases: (1) the unconstrained case, for both body and spatial representations; and (2) the constrained (mixed kinematic and dynamic) case. In each case, the dynamic equations for these nonholonomic mechanical systems are given, and illustrated by the appropriate calculations for an example system. A primary result of the paper is to show that the spectrum of possible constraints-ranging from no constraints to fully constrained systems-can be expressed within a single unifying principle for calculating the reduced equations. In this process, the structure of the reduced Lagrangian directly reveals two useful components in the reduction process, namely the local forms of the locked inertia tensor and the mechanical connection. Finally, it is shown that the reduced dynamics decouple from any explicit dependence on the group configuration variables.

We investigate feedback laws used to control multiple robots moving together in a formation. We propose a method for controlling formations that uses only local sensor-based information, in a leader-follower motion. We use methods of feedback linearization to exponentially stabilize the relative distance and orientation of the follower, and show that the zero dynamics of the system are also (asymptotically) stable. We demonstrate in simulation the use of these algorithms to control six robots moving around an obstacle. These types of control laws can be used to control arbitrarily large numbers of robots moving in very general types of formations.

We present a general framework for motion planning of robots in the presence of obstacles and other robots. We use variational calculus and optimization to find optimal open loop and closed loop plans in the presence of uncertainty. The plans are based on world models with set-valued uncertainty associated with the positions and shape of the obstacles. The open loop plans are generated by an efficient method that allows successive refinements of a nominal motion plan and accommodates finer levels of granularity as additional information becomes available. The closed loop plans are control policies that are based on given sensor models. The optimal plan is a control policy that performs the best in the worst-case situation. We discuss how the open loop and closed loop plans can be viewed as optimal strategies in the framework of two-person, zero-sum, non-cooperative games.

This paper addresses the optimal control and selection of gaits in a class of nonholonomic locomotion systems that exhibit group symmetries. We study optimal gaits for the snakeboard, a representative example of this class of systems. We employ Lagrangian reduction techniques to simplify the optimal control problem, and describe a general framework and an algorithm to obtain numerical solutions to this problem. This paper represents an initial study in using optimal control techniques to study optimality of gaits and issues involving gait transitions. The general framework provided in this paper can easily be applied to other examples of biological and robotic locomotion.

This paper considers the problem of serpentine, or snake-like, locomotion from the perspective of geometric mechanics. A particular model based on Hirose's active cord mechanism is analyzed. Using the kinematic constraints, we develop a connection, which describes the net motion of the machine as a function of variations in the mechanism's shape variables. We present simulation results demonstrating three types of locomotive gaits, one of which bears an obvious resemblance to the serpentine motion of a snake. We also discuss how these algorithms can be used to optimize certain inputs given the particular choice of physical parameters for a snake robot.

This paper studies the mechanics of undulatory locomotion. This type of locomotion is generated by a coupling of internal shape changes to external non-holonomic constraints. Employing methods from geometric mechanics, the authors use the dynamic symmetries and kinematic constraints to develop a specialized form of the dynamic equations which govern undulatory systems. These equations are written in terms of physically meaningful and intuitively appealing variables that show the role of internal shape changes in driving locomotion.

Analysis and simulations are performed for a simplified model of a commercially available variant of the skateboard, known as the Snakeboard. Although the model exhibits basic gait patterns seen in a large number of locomotion problems, the analysis tools currently available do not apply to this problem. The difficulty lies primarily in the way in which the nonholonomic constraints enter into the system. As a first step towards understanding systems represented by their model the authors present the equations of motion and perform some controllability analysis for the snakeboard. The authors also perform numerical simulations of possible gait patterns which are characteristic of snakeboard locomotion.>