The Great Outdoors (GO) dataset is a multi-modal annotated data resource aimed at advancing ground robotics research in unstructured environments. Existing off-road datasets often lack sensor diversity and exclude vital modalities like thermal and radar that are critical for operation in degraded conditions (e.g., low visibility or adverse weather). To address these gaps, we introduce a large-scale multimodal off-road dataset with six complementary sensor modalities, along with semantic annotations and GPS traces, to support tasks such as semantic segmentation, object detection, and SLAM. The diverse environmental conditions represented in the dataset present significant real-world challenges, which provide opportunities to develop more robust solutions to support the continued advancement of field robotics, autonomous exploration, and perception systems in natural environments. The dataset can be downloaded at: https://www.unmannedlab.org/the-great-outdoors-dataset/

Object detection and subsequent perception of the environment surrounding a vehicle play a very important role in autonomous driving applications. Existing perception algorithms do not generalize well since most algorithms are trained in well-structured driving environment datasets. To deploy self-driving cars on the road, they should have a reliable and robust perception system to handle all corner cases. This paper introduces a multimodal dataset, TIAND*(TiHAN-IITH Autonomous Navigation Dataset), collected from structured and unstructured environments seen in and around the city of Hyderabad, India, as an aid to further research in the generalization of object detection algorithms. The sensor suite contains four cameras, six radars, one Lidar, and GPS and IMU. TIAND comprises 150 scenes, each spanning a duration ranging from 2 minutes to 4 minutes. Subsequently, we present the object detection model’s performance using camera, radar, and Lidar data. Additionally, we offer insights into projecting data from Lidar to camera and from radar to camera.

To ensure safe autonomous driving in urban environments with complex vehicle-pedestrian interactions, it is critical for Autonomous Vehicles (AVs) to have the ability to predict pedestrians’ short-term and immediate actions in real-time. In recent years, various methods have been developed to study estimating pedestrian behaviors for autonomous driving scenarios, but there is a lack of clear definitions for pedestrian behaviors. In this work, the literature gaps are investigated and a taxonomy is presented for pedestrian behavior characterization. Further, a novel multi-task sequence to sequence Transformer encoders-decoders (TF-ed) architecture is proposed for pedestrian action and trajectory prediction using only ego vehicle camera observations as inputs. The proposed approach is compared against an existing LSTM encoders decoders (LSTM-ed) architecture for action and trajectory prediction. The performance of both models is evaluated on the publicly available Joint Attention Autonomous Driving (JAAD) dataset, CARLA simulation data as well as real-time self-driving shuttle data collected on university campus. Evaluation results illustrate that the proposed method reaches an accuracy of 81% on action prediction task on JAAD testing data and outperforms the LSTM-ed by 7.4%, while LSTM counterpart performs much better on trajectory prediction task for a prediction sequence length of 25 frames.

LIDAR and RADAR are two commonly used sensors in autonomous driving systems. The extrinsic calibration between the two is crucial for effective sensor fusion. The challenge arises due to the low accuracy and sparse information in RADAR measurements. This paper presents a novel solution for 3D RADAR-LIDAR calibration in autonomous systems. The method employs simple targets to generate data, including correspondence registration and a one-step optimization algorithm. The optimization aims to minimize the reprojection error while utilizing a small multi-layer perception (MLP) to perform regression on the return energy of the sensor around the targets. The proposed approach uses a deep learning framework such as PyTorch and can be optimized through gradient descent. The experiment uses a 360-degree Ouster-128 LIDAR and a 360-degree Navtech RADAR, providing raw measurements. The results validate the effectiveness of the proposed method in achieving improved estimates of extrinsic calibration parameters.

The development and implementation of visual-inertial odometry (VIO) has focused on structured environments, but interest in localization in off-road environments is growing. In this paper, we present the RELLIS Off-road Odometry Analysis Dataset (ROOAD) which provides high-quality, time-synchronized off-road monocular visual-inertial data sequences to further the development of related research. We evaluated the dataset on two state-of-the-art VIO algorithms, (1) Open-VINS and (2) VINS-Fusion. Our findings indicate that both algorithms perform 2 to 30 times worse on the ROOAD dataset compared to their performance in structured environments. Furthermore, OpenVINS has better tracking stability and real-time performance than VINS-Fusion in the off-road environment, while VINS-Fusion outperformed OpenVINS in tracking accuracy in several data sequences. Since the camera-IMU calibration tool from Kalibr toolkit is used extensively in this work, we have included several calibration data sequences. Our hand measurements show Kalibr’s tool achieved ±1° for orientation error and ± 1 mm at best (x- and y-axis) and ± 10 mm (z-axis) at worse for position error in the camera frame between the camera and IMU. This novel dataset provides a new set of scenarios for researchers to design and test their localization algorithms on, as well as critical insights in the current performance of VIO in off-road environments. ROOAD Dataset: github.com/unmannedlab/ROOAD

We present a local 3D voxel mapping framework for off-road path planning and navigation. Our method provides both hard and soft positive obstacle detection, negative obstacle detection, slope estimation, and roughness estimation. By using a 3D array lookup table data structure and by leveraging the GPU it can provide online performance. We then demonstrate the system working on three vehicles, a Clearpath Robotics Warthog, Moose, and a Polaris Ranger, and compare against a set of pre-recorded waypoints. This was done at 4.5 m/s in autonomous operation and 12 m/s in manual operation with a map update rate of 10 Hz. Finally, an open-source ROS implementation is provided.https://github.com/unmannedlab/G-VOM

Semantic scene understanding is crucial for robust and safe autonomous navigation, particularly so in off-road environments. Recent deep learning advances for 3D semantic segmentation rely heavily on large sets of training data, however existing autonomy datasets either represent urban environments or lack multimodal off-road data. We fill this gap with RELLIS-3D, a multimodal dataset collected in an off-road environment, which contains annotations for 13,556 LiDAR scans and 6,235 images. The data was collected on the Rellis Campus of Texas A&M University, and presents challenges to existing algorithms related to class imbalance and environmental topography. Additionally, we evaluate the current state of the art deep learning semantic segmentation models on this dataset. Experimental results show that RELLIS-3D presents challenges for algorithms designed for segmentation in urban environments. This novel dataset provides the resources needed by researchers to continue to develop more advanced algorithms and investigate new research directions to enhance autonomous navigation in off-road environments. RELLIS-3D is available at https://github.com/unmannedlab/RELLIS-3D

We present a boundary-aware domain adaptation model for LiDAR scan full-scene semantic segmentation (LiDARNet). Our model can extract both the domain private features and the domain shared features with a two branch structure. We embedded Gated-SCNN into the segmentor component of LiDARNet to learn boundary information while learning to predict full-scene semantic segmentation labels. Moreover, we further reduce the domain gap by inducing the model to learn a mapping between two domains using the domain shared and private features. Additionally, we introduce a new dataset (SemanticUSL1) for domain adaptation for LiDAR point cloud semantic segmentation. The dataset has the same data format and ontology as SemanticKITTI. We conducted experiments on real-world datasets SemanticKITTI, SemanticPOSS, and SemanticUSL, which have differences in channel distributions, reflectivity distributions, diversity of scenes, and sensors setup. Using our approach, we can get a single projection-based Li-DAR full-scene semantic segmentation model working on both domains. Our model can keep almost the same performance on the source domain after adaptation and get an 8%-22% mIoU performance increase in the target domain.

We present a method for path optimization for ground vehicles in off-road environments at high speeds. This path optimization considers the kinematic constraints of the vehicle. By thinking in the actuator space we can represent such constraints as limits in the space rather than derived properties of the path. In this paper we present an actuator space approach to path optimization for off-road ground vehicles. This is done by representing the path as a list of steering angles over the path length. This transforms the set of kinematic constraints into constraints on the steering angle. We then put this path into a gradient descent solver. This produces paths that are kinematically feasible and optimized in accordance with our cost function. Finally, we tested the system both in simulation and on an off-road vehicle at speeds of 5 m/s.

In this paper, we present a fast, on-line mapping and planning solution for operation in unknown, off-road, environments. We combine obstacle detection along with a terrain gradient map to make simple and adaptable cost map. This map can be created and updated at 10 Hz. An A* planner finds optimal paths over the map. Finally, we take multiple samples over the control input space and do a kinematic forward simulation to generated feasible trajectories. Then the most optimal trajectory, as determined by the cost map and proximity to A* path, is chosen and sent to the controller. Our method allows real time operation at rates of 30 Hz. We demonstrate the efficiency of our method in various off-road terrain at high speed.

Drivers and other road users often encounter situations (e.g., arriving at an intersection simultaneously) where priority is ambiguous or unclear but must be resolved via communication to reach agreement. This poses a challenge for autonomous vehicles, for which no direct means for expressing intent and acknowledgment has yet been established. This paper contributes a minimal model to manage ambiguity and produce actions that are expressive and encode aspects of intent. Specifically, intent is treated as a latent variable, communicated implicitly through a partially observable Markov decision process (POMDP). We validate the model in a simple setting: a simulation of a prototypical crossing with a vehicle and one pedestrian at an unsignalized intersection. We further report use of our self-driving Ford Lincoln MKZ platform, through which we conducted experimental trials of the method involving real-time interaction. The experiment shows the method achieves safe and efficient navigation.

In this paper we perform an extensive experimental evaluation of three planar target based 3D-LIDAR camera calibration algorithms, on a sensor suite consisting multiple 3D-LIDARs and cameras, assessing their robustness to random initialization and by using metrics like Mean Line Re-projection Error (MLRE) and Factory Stereo Calibration Error. We briefly describe each method and provide insights into practical aspects like ease of data collection. We also show the effect of noisy sensor on the calibration result and conclude with a note on which calibration algorithm should be used under what circumstances.

This paper discusses a platform both in simulation and experimentation for testing autonomous heavy trucking. In simulation, we present a novel use of the video game American Truck Simulator (ATS) as the simulation platform, only costing a fraction of commercial simulator software. In experimentation, we present a modified ProStar 122+ using the PACMod system from AutonomouStuff, a popular by-wire kit. Discussion and review of the by-wire kit and sensors is provided. A proof-of-concept of the platform is shown by performing lane keeping at 65 mph using the Stanley lateral controller and MobilEye detection system. Further, we introduce a rapidly developed longitudinal control algorithm using a pedal actuation map, and 3D lookup tables created from braking and acceleration data. Introductory results are presented to aid the research community for single vehicle autonomous trucking.

This paper summarizes the progress in developing a rugged, low-cost, automated ground cone robot network capable of traffic delineation at lane-level precision. A holonomic omnidirectional base with a traffic delineator was developed to allow flexibility in initialization. RTK GPS was utilized to reduce minimum position error to 2 centimeters. Due to recent developments, the cost of the platform is now less than $1,600. To minimize the effects of GPS-denied environments, wheel encoders and an Extended Kalman Filter were implemented to maintain lane-level accuracy during operation and a maximum error of 1.97 meters through 50 meters with little to no GPS signal. Future work includes increasing the operational speed of the platforms, incorporating lanelet information for path planning, and cross-platform estimation.

This work presents an extrinsic parameter estimation algorithm between a 3D LIDAR and a Projective Camera using a marker-less planar target, by exploiting Planar Surface Point to Plane and Planar Edge Point to back-projected Plane geometric constraints. The proposed method uses the data collected by placing the planar board at different poses in the common Field of View (FoV) of the LIDAR and the Camera. The steps include, detection of the target and the edges of the target in LIDAR and Camera frames, matching the detected planes and lines across both the sensing modalities and finally solving a cost function formed by the aforementioned geometric constraints that link the features detected in both the LIDAR and the Camera using non-linear least squares. We have extensively validated our algorithm using two Basler Cameras, Velodyne VLP-32 and Ouster OS1 LIDARs.

In this work we evaluate Iterative Linear Quadratic Regulator(ILQR) for trajectory tracking of two different kinds of wheeled mobile robots namely Warthog (Fig. 1), an off-road holonomic robot with skid-steering and Polaris GEM e6 [1], a non-holonomic six seater vehicle (Fig. 2). We use multilayer neural network to learn the discrete dynamic model of these robots which is used in ILQR controller to compute the control law. We use model predictive control (MPC) to deal with model imperfections and perform extensive experiments to evaluate the performance of the controller on human driven reference trajectories with vehicle speeds of 3m/s-4m/s for warthog and 7m/s-10m/s for the Polaris GEM.

In this paper, we present a collaborative path-planning framework for a group of micro aerial vehicles that are capable of localizing through vision. Each of the micro aerial vehicles is assumed to be equipped with a forward facing monocular camera. The vehicles initially use their captured images to build 3D maps through common features; and subsequently track these features to localize through 3D-2D correspondences. The planning algorithm, while connecting start locations to provided goal locations, also aims to reduce the localization uncertainty of the vehicles in the group. To achieve this, we develop a two-step planning framework: the first step attempts to build an improved map of the environment by solving the next-best-view problem for multiple cameras. We express this as a black-box optimization problem and solve it using the Covariance Matrix Adaption evolution strategy (CMA-ES). Once an improved map is available, the second stage of the planning framework performs belief space planning for the vehicles individually using the rapidly exploring random belief tree (RRBT) algorithm. Through the RRBT approach, the planner generates paths that ensure feature visibility while attempting to optimize path cost and reduce localization uncertainty. We validate our approach using experiments conducted in a high visual-fidelity aerial vehicle simulator, Microsoft AirSim.

Obstacle avoidance is a key feature for safe Unmanned Aerial Vehicle (UAV) navigation. While solutions have been proposed for static obstacle avoidance, systems enabling avoidance of dynamic objects, such as drones, are hard to implement due to the detection range and field-of-view (FOV) requirements, as well as the constraints for integrating such systems on-board small UAVs. In this work, a dataset of 6k synthetic depth maps of drones has been generated and used to train a state-of-the-art deep learning-based drone detection model. While many sensing technologies can only provide relative altitude and azimuth of an obstacle, our depth map-based approach enables full 3D localization of the obstacle. This is extremely useful for collision avoidance, as 3D localization of detected drones is key to perform efficient collision-free path planning. The proposed detection technique has been validated in several real depth map sequences, with multiple types of drones flying at up to 2 m/s, achieving an average precision of 98.7 %, an average recall of 74.7 % and a record detection range of 9.5 meters.

In this paper, we describe the method and implementation of a real-time tumor tracking system for a pencil beam scanning (PBS) proton therapy system. PBS is an advanced cancer treatment system that can benefit from precise localization of the tumors through motion. We utilize techniques such as cross-correlation matching, correlation filters and small object saliency, creating an array of methods that can detect and track fiducial markers implanted in the cancer tumors. The final aim is to control the proton beam using real-time image guidance. Our technique works robustly on various types of markers such as ceramic/metallic fiducials, visicoil markers and surgical clips. Left and right views of an X-ray fluoroscopy system were utilized to also triangulate the marker positions in full 3D as they are tracked through normal breathing movement and organ motion. We have tested our detection system on data from several patients with different tumor locations both offline and in real-time and wish to implement it within a full treatment system soon. To the best of the authors knowledge, this is the first real time tracking system for PBS therapy that is applicable for various types of fiducials and tumor locations.

Though autonomous vehicles are currently operating in several places, many important questions within the field of autonomous vehicle research remain to be addressed satisfactorily. In this paper, we examine the role of communication between pedestrians and autonomous vehicles at unsignalized intersections. The nature of interaction between pedestrians and autonomous vehicles remains mostly in the realm of speculation currently. Of course, pedestrian's reactions towards autonomous vehicles will gradually change over time owing to habituation, but it is clear that this topic requires urgent and ongoing study, not least of all because engineers require some working model for pedestrian- autonomous-vehicle communication. Our paper proposes a decision-theoretic model that expresses the interaction between a pedestrian and a vehicle. The model considers the interaction between a pedestrian and a vehicle as expressed an MDP, based on prior work conducted by psychologists examining similar experimental conditions. We describe this model and our simulation study of behavior it exhibits. The preliminary results on evaluating the behavior of the autonomous vehicle are promising and we believe it can help reduce the data needed to develop fuller models.

The ability to avoid collisions with moving obstacles, such as commercial aircraft is critical to the safe operation of unmanned aerial vehicles (UAVs) and other air traffic. This paper presents the design and implementation of sampling-based path planning methods for a UAV to avoid collision with commercial aircraft and other moving obstacles. In detail, the authors develop and demonstrate a method based on the closed-loop rapidly-exploring random tree algorithm and three variations of it. The variations are: 1) simplification of trajectory generation strategy; 2) utilization of intermediate waypoints; 3) collision prediction using reachable set. The methods were validated in software-in-the-loop simulations, hardware-in-the-loop simulations, and real flight experiments. It is shown that the algorithms are able to generate collision free paths in real time for the different types of UAVs among moving obstacles of different numbers, approaching angles, and speeds.

We present a framework for vision based localization for two or more multirotor aerial vehicles relative to each other. This collaborative localization technique is built upon a relative pose estimation strategy between two or more cameras with the capability of estimating accurate metric poses between each other even through fast motion and continually changing environments. Through synchronized feature detection and tracking with a robust outlier rejection process, classical multiple view geometry concepts have been utilized for obtaining scale-ambiguous relative poses, which are then refined through reconstruction and pose optimization to provide a metric estimate. Furthermore, we present the implementation details of this technique followed by a set of results which involves evaluation of the accuracy of the pose estimates through test cases in both simulated and real experiments. Test cases include keeping one camera stationary as the other is mounted on a quadrotor which is then flown through various types of trajectories. We also perform a quantitative comparison with a GPS/IMU localization technique to demonstrate the accuracy of our method.

We present the design and implementation of an aircraft collision avoidance algorithm for Unmanned Aerial Vehicles (UAVs). Automatic Dependent Surveillance-Broadcast (ADS-B) is used to detect aircraft. The UAV needs to fly through pre-assigned waypoints while avoiding collisions with other aircraft. The aircraft are indifferent to the UAV. A collision with aircraft are detected by simulating the UAV's trajectory along the path of assigned waypoints using its closed-loop dynamics. A sampling based algorithm is used for collision avoidance path planning. A second collision check is performed on the generated path with the updated UAV and aircraft's states. The path will be re-planned if it leads to a collision. The algorithm was validated in Software-In-the-Loop Simulation (SITL). ADS-B data obtained from commercial aircraft flying over the Phoenix Skyharbor airport were used for simulating the collisions. The paper shows that the algorithm enables the UAV to avoid multiple aircraft with different approaching angles and speeds.

We present a method to compute an initial alignment for pairwise registration of point clouds. This method uses the properties of a rigid body transformation - the ratio of lengths is preserved, the euclidean distance between points is preserved - to find congruent pyramids in two point clouds. The corresponding vertices of the congruent pyramids are used to derive a closed form solution for initial alignment. The alignment is refined further using the Iterative Closest Point algorithm. We validate the method on challenging datasets - which include airborne LIDAR, outdoor, and indoor - having initial offsets and varying densities.

A critical problem in planning sampling paths for autonomous underwater vehicles is balancing obtaining an accurate scalar field estimation against efficiently utilizing the stored energy capacity of the sampling vehicle. Adaptive sampling approaches can only provide solutions when real-time and a priori environmental data is available. Through utilizing a cost-evaluation function to experimentally evaluate various sampling path strategies for a wide range of scalar fields and sampling densities, it is found that a systematic spiral sampling path strategy is optimal for high-variance scalar fields for all sampling densities and low-variance scalar fields when sampling is sparse. The random spiral sampling path strategy is found to be optimal for low-variance scalar fields when sampling is dense.

We experimentally evaluated the efficacy of various autonomous supervised classification techniques for detecting transient geophysical phenomena. We demonstrated methods of detecting volcanic plumes on the planetary satellites Io and Enceladus using spacecraft images from the Voyager, Galileo, New Horizons, and Cassini missions. We successfully detected 73–95% of known plumes in images from all four mission datasets.Additionally, we showed that the same techniques are applicable to differentiating geologic features, such as plumes and mountains, which exhibit similar appearances in images.

We present a fast, robust road detection algorithm for aerial images taken from an Unmanned Aerial Vehicle. A histogram-based adaptive threshold algorithm is used to detect possible road regions in an image. A probabilistic hough transform based line segment detection combined with a clustering method is implemented to further extract the road. The proposed algorithm has been extensively tested on desert and urban images obtained using an Unmanned Aerial Vehicle. Our results indicate that we are able to successfully and accurately detect roads in 97% of the images. We experimentally validated our algorithm on over ten thousand (10,000) aerial images obtained using our UAV. These images consist of intersecting roads, bifurcating roads and roundabouts in various conditions with significant changes in lighting and intensity. Our algorithm is able to successfully detect single roads effectively in almost all the images. It is also able to detect at least one road in over 95% of the images containing bifurcating or intersecting roads.

In this paper we introduce a novel framework of generating trajectories which explicitly satisfies the stability constraints such as no-slip and permanent ground contact on uneven terrain. The main contributions of this paper are: (1) It derives analytical functions depicting the evolution of the vehicle on uneven terrain. These functional descriptions enable us to have a fast evaluation of possible vehicle stability along various directions on the terrain and this information is used to control the shape of the trajectory. (2) It introduces a novel paradigm wherein non-linear time scaling brought about by parametrized exponential functions are used to modify the velocity and acceleration profile of the vehicle so that these satisfy the no-slip and contact constraints. We show that nonlinear time scaling manipulates velocity and acceleration profile in a versatile manner and consequently has exceptional utility not only in uneven terrain navigation but also in general in any problem where it is required to change the velocity of the robot while keeping the path unchanged like collision avoidance.

We present the design of an optimal trajectory controller for landing a helicopter on a moving target. The trajectory planner is based on the variational Hamiltonian and Euler-Lagrange equations. We use a kinematic model of the helicopter to derive an optimal controller that is able to track an arbitrarily moving target and then land on it. Simulations are shown to verify the performance of the optimal trajectory controller. Data from real flight trials is presented to validate the inputs obtained from the trajectory planner to track a desired trajectory. We present initial trials in simulation for landing the helicopter autonomously on a moving target.

Safe autonomous flight is essential for widespread acceptance of aircraft that must fly close to the ground. We have developed a method of collision avoidance that can be used in three dimensions in much the same way as autonomous ground vehicles that navigate over unexplored terrain. Safe navigation is accomplished by a combination of online environmental sensing, path planning and collision avoidance. Here we report results with an autonomous helicopter that operates at low elevations in uncharted environments some of which are densely populated with obstacles such as buildings, trees and wires. We have recently completed over 1000 successful runs in which the helicopter traveled between coarsely specified waypoints separated by hundreds of meters, at speeds up to 10 meters/sec at elevations of 5-10 meters above ground level. The helicopter safely avoids large objects like buildings and trees but also wires as thin as 6 mm. We believe this represents the first time an air vehicle has traveled this fast so close to obstacles. Here we focus on the collision avoidance method that learns to avoid obstacles by observing the performance of a human operator.

The use of unmanned aerial vehicles (UAVs) in civilian and domestic applications is highly demanding, requiring a high-level of capability from the vehicles. This work addresses the design and implementation of a vision-based feature tracker for an autonomous helicopter. Using vision in the control loop allows estimating the position and velocity of a set of features with respect to the helicopter. The helicopter is then autonomously guided to track these features (in this case windows in an urban environment) in real time. The results obtained from flight trials in a real world scenario demonstrate that the algorithm for tracking features in an urban environment, used for visual servoing of an autonomous helicopter is reliable and robust

We present the design and implementation of a real-time vision-based approach to detect and track features in a structured environment using an autonomous helicopter. Using vision as a sensor enables the helicopter to track features in an urban environment. We use vision for feature detection and a combination of vision and GPS for navigation and tracking. The vision algorithm sends high level velocity commands to the helicopter controller which is then able to command the helicopter to track them. We present results obtained from flight trials that demonstrate our algorithms for detection and tracking are applicable in real world scenarios by applying them to the task of tracking rectangular features in structured environments.

We describe a sensor network deployment method using autonomous flying robots. Such networks are suitable for tasks such as large-scale environmental monitoring or for command and control in emergency situations. We describe in detail the algorithms used for deployment and for measuring network connectivity and provide experimental data we collected from field trials. A particular focus is on determining gaps in connectivity of the deployed network and generating a plan for a second, repair, pass to complete the connectivity. This project is the result of a collaboration between three robotics labs (CSIRO, USC, and Dartmouth.).

This paper discusses similarities and differences in autonomous helicopters developed at USC and CSIRO. The most significant differences are in the accuracy and sample rate of the sensor systems used for control. The USC vehicle, like a number of others, makes use of a sensor suite that costs an order of magnitude more than the vehicle. The CSIRO system, by contrast, utilizes low-cost inertial, magnetic, vision and GPS to achieve the same ends. We describe the architecture of both autonomous helicopters, discuss the design issues and present comparative results.

We present the design and implementation of a real-time, vision-based landing algorithm for an autonomous helicopter. The landing algorithm is integrated with algorithms for visual acquisition of the target (a helipad) and navigation to the target, from an arbitrary initial position and orientation. We use vision for precise target detection and recognition, and a combination of vision and Global Positioning System for navigation. The helicopter updates its landing target parameters based on vision and uses an onboard behavior-based controller to follow a path to the landing site. We present significant results from flight trials in the field which demonstrate that our detection, recognition, and control algorithms are accurate, robust, and repeatable.

We present the design and implementation of a real-time, vision-based landing algorithm for an autonomous helicopter. The helicopter is required to navigate from an initial position to a final position in a partially known environment based on GPS and vision, locate a landing target (a helipad of a known shape) and land on it. We use vision for precise target detection and recognition. The helicopter updates its landing target parameters based on vision and uses an on-board behavior-based controller to follow a path to the landing site. We present results from flight trials in the field which demonstrate that our detection, recognition and control algorithms are accurate and repeatable.

We propose the use of a model helicopter to emulate the landing dynamics of a spacecraft. Our controller accepts thruster inputs (like those on a spacecraft) and converts them into appropriate helicopter stick controls such that the resulting trajectory of the helicopter is close to the trajectory that would have been achieved by simply providing the same thruster inputs to a spacecraft. The approach relies on simplified models of the spacecraft and helicopter dynamics. Initial results in simulation indicate that the approach is feasible, with tracking accuracies on the order of 5 m.