This work developed and evaluated a human-machine interface for the control of a roadable air vehicle (RAV), capable of surface driving, vertical takeoff, sustained flight, and landing. Military applications seek to combine the benefits of ground and air vehicles to maximize flexibility of movement but require that the operator have minimal pilot training. This makes the operator vulnerable to automation complexity issues; however, the operator will expect to be able to interact extensively and control the vehicle during flight. A mixed-initiative control approach mitigates these vulnerabilities by integrating the operator into many complex control domains in the way that they often expect---flexibly in charge, aware, but not required to issue every command. Intrinsic safety aspects were evaluated by comparing performance, decision making, precision, and workload for three RAV control paradigms: human-only, fully automated, and mixed-initiative control. The results suggest that the mixed-initiative paradigm leverages the benefits of human and automated control while also avoiding the drawbacks associated with each.

Mobile robots and vehicles with active articulated elements are well suited to drive on irregular and rough terrain because they are able to adjust their center of mass and decrease the risk of tip over or other accidents. The articulated mechanism, however, is usually constrained by the actuator's bandwidth, making it difficult for the vehicle to compensate for abrupt changes in terrain profile. In this paper we propose a terrain model-based control method to improve stability when traversing terrains with varying slopes, depressions, and rises. A model predictive control approach takes into account actuator bandwidth to anticipate interactions with the terrain and control the articulated elements to prevent tipping over. The method is applied to an autonomous orchard platform where workers stand while conducting production tasks on the trees, adjusting the platform height and increasing its stability. The feasibility is illustrated via numerical simulations performed with the MD Adams/Car software and Matlab, with both artificial and natural terrain data. Challenging yet realistic terrain profiles are considered to demonstrate the control effectiveness in preventing the platform from tipping over and therefore improving the workers' and the vehicle's safety.

Rows of trees such as in orchards, planted in straight parallel lines can provide navigation cues for autonomous machines that operate in between them. When the tree canopies are well managed, tree rows appear similar to corridor walls and a simple 2D sensing scheme suffices. However, when the tree canopies are three dimensional, or ground vegetation occludes tree trunks, it is necessary to use a three dimensional sensing mode. An additional complication in prolific canopies is that GPS is not reliable and hence is not suitable to register data from sensors onboard a traversing vehicle. Here, we present a method to register 3D data from a lidar sensor onboard a vehicle that must accurately determine its pose relative to the rows. We first register point cloud into a common reference frame and then determine the position of tree rows and trunks in the vicinity to determine the vehicle pose. Our method is tested online and with data from commercial orchards. Experimental results show that the accuracy is sufficient to enable accurate traversal between tree rows even when tree canopies do not approximate planar walls.

Specialty crops constitute a $45 billion/year industry. As opposed to crops such as wheat, cotton, corn and soybean, they are characterized by the need for intensive cultivation. Specialty crops growers currently face serious labor cost and availability problems, and few technological solutions exist to increase their efficiency given the past history of abundant supply of low-cost labor. This leads to an opportunity to use recent technological advances to not only increase efficiency and reduce labor costs in specialty crops production but also to support a domestic engineering solutions industry for specialty crops. We envision a family of reconfigurable vehicles that can be rapidly tasked to automate or augment pruning, thinning, harvesting, mowing, spraying, etc. They would share a common sensing and computing infrastructure, allowing applications created for one to be easily transferable to others - much like software applications today are transferable from one computer to another. In this paper we describe our work over the last three years designing and deploying a family of such vehicles, the Autonomous Prime Movers (APMs). The five vehicles completed so far have traveled autonomously over 300 km in research and commercial tree fruit orchards; preliminary results in time trials conducted by extension educators indicate efficiency gains of up to 58%.

Safe robot navigation in tree fruit orchards requires that the vehicle be capable of robustly navigating between rows of trees and turning from one aisle to another; that the vehicle be dynamically stable, especially when carrying workers; and that the vehicle be able to detect obstacles on its way and adjust its speed accordingly. In this paper we address the latter, in particular the problem of detecting people and apple bins in the aisles between rows. One of our requirements is that the obstacle avoidance subsystem shouldn't add to the robot's hardware cost, so as to keep the acquisition cost to growers as low as possible. Therefore, we confine ourselves to solutions that use only the sensor suite already installed on the robot for navigation-in our case, a laser scanner, low-cost inertial measurement unit, and steering and wheel encoders. Our methodology is based on the classification and clustering of registered 3D points as obstacles. In the current implementation, obstacle avoidance takes in 3D point clouds collected in apple orchards and generates an off-line assessment of obstacle position. Tests conducted at our experimental orchard-like environment in Pittsburgh and an actual apple orchard in Washington state indicate that the method is able to detect people and bins located along the vehicle path. Stretch tests indicate that it is also capable of dealing with objects as small as 15 cm tall as long as they aren't covered by grass, and to detect people crossing the aisles at walking speed.

We present a monocular visual navigation methodology for autonomous orchard vehicles. Modern orchards are usually planted with straight and parallel tree rows that form a corridor-like environment. Our task consists of driving a vehicle autonomously along the tree rows. The original contributions of this paper are: 1) a method to recover vehicle rotation independently of translation by modeling the vehicle as a car-like robot driving on a 3D ground surface-the rotation is estimated from monocular images while the translation is measured by a wheel encoder; and 2) a method to fit the 3D points corresponding to the trees into straight lines via an optimization algorithm that minimizes the error variance on the robot lookahead point. Additionally, we use a simple vanishing point detection approach to find the ends of the tree rows. The vanishing point detection is integrated into the system via an extended Kalman filter. The methodology's robustness to environmental changes is validated in more than fifty experiments in research and commercial orchards, six of which are presented and discussed in detail.

We present a cascaded control architecture for a Yamaha RMAX robotic helicopter. The controller is composed of an inner-loop that stabilizes the unstable poles of the helicopter's linear dynamic model; and an outer-loop that decouples the dynamics of the lateral, longitudinal, vertical, and heading axes and enables trajectory tracking. Actual flight results are presented to demonstrate the validity of the method. A discussion on the method's limitations and our plans on how to overcome them are also presented.

In January 2004, NASA began a bold enterprise to return to the Moon, and with the technologies and expertise gained, press on to Mars. The underlying Vision for Space Exploration calls for a sustained and affordable human and robotic program to explore the solar system and beyond; to conduct human expeditions to Mars after successfully demonstrating sustained human exploration missions on the Moon. The approach is to "send human and robotic explorers as partners, leveraging the capabilities of each where most useful." Human-robot interfacing technologies for this approach are required at readiness levels above any available today. In this paper, we describe the HRI aspects of a robot supervision architecture we are developing under NASA's auspices, based on the authors' extensive experience with field deployment of ground, underwater, lighter-than-air, and inspection autonomous and semi-autonomous robotic vehicles and systems.

This paper presents the Remotely Accessible Laboratory (REAL), a virtual laboratory accessible through the Internet. REAL allows a remote user to manipulate a mobile robot in a mode of interaction suitable to his or her level of expertise. A basic mode of interaction, dedicated to users with limited knowledge of robotics, supports interaction via teleoperation. In a more advanced level of interaction, expert users can plan and execute complex robotics experiments that exploit the full capabilities of the robot. In this mode of interaction, experiments in the field of autonomous navigation, environmental mapping, sensor fusion, mission planning, and robot control can be performed. Finally, a third mode of interaction allows a set of trainees to follow the interactions conducted by an instructor. The architecture of REAL departs from the commonplace World Wide Web applications, since it employs a sophisticated software architecture based on software components. This architecture presents a high degree of reusability that future developments in the field of Internet robots and virtual laboratories can take advantage of.

Internet robotic systems have a different role in the use and development of aerial robots compared to that for ground robots. While for ground robotic vehicles, Internet is useful for remote operation or to make remote development, in aerial vehicles, in addition, the safety aspect must be considered very carefully. This paper addresses the requirements to the specific case of Internet aerial robots. It describes the conceptual and implementation aspects of an Internet-based software architecture for the AURORA unmanned autonomous airship project, showing its use in the different project phases.

The problem of fault tolerance in cooperative manipulators rigidly connected to a solid object is addressed in this paper. Four faults are considered: free-swinging joint faults, locked joint faults, incorrect measured joint position, and incorrect measured joint velocity. The faults are first detected by a, fault detection and isolation system. Free-swinging and locked joint faults are isolated using artificial neural networks. The other faults are isolated based on the kinematic constraints imposed on the cooperative system. After the isolation of the faults, the control system is reconfigured. Control laws for the system with passive or locked joints are developed. Results of the fault tolerance system applied in simulations and in a real cooperative system are presented.

The increased utilization of manipulators in hazardous or hard-to-reach environments have led to a corresponding increase on the study of fault tolerant control methods for these mechanisms, ranging from robot design and trajectory planning to fault detection and isolation and post-failure control algorithms. In this article we focus on post-failure control of a mechanical manipulator from the point of view of optimal control, and present a novel method for controlling the positions of the failed, passive joints, in an optimal way. Although the optimization is performed locally, the results indicate the validity and feasibility of the proposed theory.

Project AURORA aims at the development of an unmanned airship capable of autonomous flight over user-defined locations for aerial inspection and imagery acquisition. In this article the authors report a successful autonomous flight achieved through a set of pre-defined points, one of the first of its kind in the literature. The guidance control strategy is based on a path tracking error generation methodology that takes into account both the distance and the angular errors of the airship with respect to the desired trajectory. The control strategy uses a PI controller for the tail surfaces' deflection.

Evolutionary computation is a generic name given to the resolution of computational problems that are planned and implemented based on models of the evolutionary process. Most of the evolutionary algorithms that have been proposed follow biological paradigms and the concepts of natural selection, mutation and reproduction. There are, however, other paradigms which may be adopted in the creation of evolutionary algorithms. Several problems involving unstructured environments may be addressed from the point of view of cultural paradigms, which offer plenty of categories of models where one does not know all possible solutions to a problem - a very common situation in real life. This work applies the computational properties of cultural technology to the solution of a specific problem, adapted from the robotics literature. A test environment denoted the "Cultural Algorithms Simulator" was developed to allow anyone to learn more about the rather unconventional characteristics of a cultural technology.

This paper discusses a dynamic approach to target recognition that is based on concepts from the theory of optimal design of experiments. The approach uses a cycle of hypothesis formulation, experiment planning for hypothesis validation, experiment execution, and hypothesis evaluation to confirm or reject the classification of targets into given object classes. Target classes of relevance to specific perceptual tasks of a robot mission are described through parametrization of sensor observations. We use spatial stochastic lattice models to encode sensor-based information and to provide potential target hypotheses. Information-theoretic uncertainty minimization metrics are employed to control the sensing processes and the robot vehicle. The approach presented was applied to an unmanned aerial robot vehicle developed for environmental research and monitoring applications, and initial results are presented showing aerial identification and tracking of large-scale man-made structures and biological targets.

Development of reliable control systems for semi-autonomous unmanned aerial vehicles require the use of extensive simulation data provided by an accurate six-degrees-of-freedom dynamic model. In this article we describe a Simulink-based control system development environment for Project AURORA's unmanned robotic airship, as well as the control algorithms and supervisory level of AURORA's control system. A complete take-off to landing mission illustrates the utilization of this environment.

We present one of the first Internet-accessible airship simulators, based on a comprehensive airship dynamic model. The simulator is meant to be used as a tool for the development of control and navigation methods for autonomous and semi-autonomous robotic airships and as testbed for airship pilot training. Realistic views of both the airship in flight and that of a virtual pilot are provided, as are commands to apply thrust to the engines, swivel them up and down, and deflect the control surfaces. This work is significant for providing robotics researchers with means to safely experiment with airship control.

This paper studies the problem of modeling and control of multiple cooperative underactuated manipulators handling a rigid object. We reveal holonomic property of such a system by presenting a smooth feedback controller subject to two conditions: 1) there are not fewer active joints than the degrees of freedom of the object; and 2) the Jacobian matrix with respect to passive joints is not singular. This controller is an extension of the PD plus gravity compensation scheme and its asymptotic stability is guaranteed by the LaSalle theorem. Furthermore, we develop a trajectory tracking controller that yields asymptotic convergence of position errors and bounded interaction forces simultaneously. The performance of the proposed controllers has been investigated by simulations on two 6-DOF underactuated manipulators and by experiments on the cooperative underactuated manipulator system developed at CMU.

This paper discusses Project AURORA (autonomous unmanned remote monitoring robotic airship) which focuses on the development of the control, navigation, sensing, and inference technologies required for substantially autonomous robotic airships. Our target application areas include the use of robotic airships for environmental, biodiversity, and climate research and monitoring. Based on typical mission requirements, we present arguments that favour airships over airplanes and helicopters as the ideal platforms for such missions. We outline the overall system architecture of the AURORA robotic airship, discuss its main subsystems, and mention the research and development issues involved.

We propose in this work the first model-based robust control method for a team of underactuated manipulators jointly manipulating a load. The method is based on feedback linearization of the nonlinear dynamic coupling between the torques applied at the actuated joints and the Cartesian acceleration of the load, combined with a variable structure controller. Singularities in the control method are addressed, and a sufficient condition for a singularity-free controller implementation is obtained. Simulation and experimental results are presented to validate the theory presented.

We propose a method to drive an underactuated manipulator among obstacles in its workspace. The method allows for collision-free trajectories to be generated from the dynamic equations of the manipulator. When the passive joints are locked, these trajectories lie on surfaces parallel to the axes of the active joints. When the passive joints are free, the trajectories lie on surfaces determined by the nonholonomic constraints imposed by the lack of actuation at the passive joints. By switching the joint brakes on and off, we obtain a sequence of trajectories that connect the start and the goal configurations. A robust controller is utilized to ensure that the manipulator follows the pre-planned trajectories closely despite modeling errors and external disturbances. Simulation and experimental studies demonstrate the validity of the proposed theory.

In this paper, we discuss the problem of how a free-floating space manipulator (SM) can be mapped to a conventional, fixed-base manipulator which preserves its dynamic and kinematic properties, and thus is called dynamically equivalent manipulator (DEM). The DEM concept allows one to use a conventional manipulator system to simulate a free-floating space manipulator connected to a space station, spacecraft, or satellite, without complicated experimental set-ups. This paper presents the theoretical development of the DEM concept, and demonstrates its dynamic and kinematic equivalence to the SM.

We propose the concept of the dynamically equivalent manipulator (DEM) of a free-floating space manipulator (SM) system. The dynamically equivalent manipulator can be physically built and used as an experimental testbed for the study of the dynamic performance and task execution of space robots. As it is a fixed-base manipulator, there is no need to resort to complex mechanisms to simulate the space environment. In this paper, we discuss two important issues associated with the DEM concept. First, we demonstrate the property of conservation of angular momentum and verify the validity of the DEM under free-flying conditions (i.e., when the SM base attitude is controlled via reaction wheels). Next, we investigate the effect of model uncertainty in the space manipulator and how it maps as errors in the parameters of the DEM. We derive explicit expressions for the error mapping and present a case study.

Considers the problem of controlling an underactuated manipulator with less actuators than passive joints. The control methodology consists of dividing the passive joints in several groups, and of controlling one group at a time via its dynamic coupling with the actuators. Among the many possible control sequences for a given robot, we choose the optimal one based on the dynamic programming method. The optimization is based on a control cost defined as the reciprocal of the coupling index, a measure of the dynamic coupling available between the active and the passive joints of the manipulator. The detailed theory, computational procedures, simulation results, and experimental results are presented.

Underactuated manipulators are a class of robotic mechanisms where passive joints are present. By controlling only the motion of the active joints, it is possible to control the entire system. Our goal is to develop control schemes using both classical nonlinear and modem learning techniques for underactuated manipulators. To examine the validity of the approaches, we developed an experimental setup known as U-ARM, or underactuated robot manipulator. In this paper we present the hardware development, dynamic parameters, control software and experimental results of real-time control of the U-ARM.