Events cameras are ideal sensors for enabling robots to detect and track objects in highly dynamic environments due to their low latency output, high temporal resolution, and high dynamic range. In this paper, we present the Asynchronous Event Multi-Object Tracking (AEMOT) algorithm for detecting and tracking multiple objects by processing individual raw events asynchronously. AEMOT detects salient event blob features by identifying regions of consistent optical flow using a novel Field of Active Flow Directions built from the Surface of Active Events. Detected features are tracked as candidate objects using the recently proposed Asynchronous Event Blob (AEB) tracker in order to construct small intensity patches of each candidate object. A novel learnt validation stage promotes or discards candidate objects based on classification of their intensity patches, with promoted objects having their position, velocity, size, and orientation estimated at their event rate. We evaluate AEMOT on a new Bee Swarm Dataset, where it tracks dozens of small bees with precision and recall performance exceeding that of alternative event-based detection and tracking algorithms by over 37%. Source code and the labelled event Bee Swarm Dataset will be open sourced.11https://github.com/angus-apps/AEMOT

Range-only Simultaneous Localisation and Mapping (RO-SLAM) is of interest due to its practical applications in ultra-wideband (UWB) and Bluetooth Low Energy (BLE) localisation in terrestrial and aerial applications and acoustic beacon localisation in submarine applications. In this work, we consider a mobile robot equipped with an inertial measurement unit (IMU) and a range sensor that measures distances to a collection of fixed landmarks. We derive an equivariant filter (EqF) for the RO-SLAM problem based on a symmetry Lie group that is compatible with the range measurements. The proposed filter does not require bootstrapping or initialisation of landmark positions, and demonstrates robustness to the noprior situation. The filter is demonstrated on a real-world dataset, and it is shown to significantly outperform a state-of-the-art EKF alternative in terms of both accuracy and robustness.

The majority of commercial and open-source autopilot software for uncrewed aerial vehicles rely on the tried and tested extended Kalman filter (EKF) to provide the state estimation solution for the inertial navigation system (INS). While modern implementations achieve remarkable robustness, it is often due to the careful implementation of exception code for a multitude of corner cases along with significant skilled tuning effort. In this paper, we use the data wealth of the ArduPilot community to identify and highlight the most common real-world challenges in INS state estimation, including sensor self-calibration, robustness in static conditions, global navigation satellite system (GNSS) outliers and shifts, and robustness to faulty inertial measurement units (IMUs). We propose a novel equivariant filter (EqF) formulation for the INS solution that exploits a Semi-Direct-Bias symmetry group for multi-sensor fusion with self-calibration capabilities and incorporates equivariant velocity-type measurements. We augment the filter with a simple innovation-covariance inflation strategy that seamlessly handles GNSS outliers and shifts without requiring coding of a whole set of exception cases. We use real-world data from the Ardupilot community to demonstrate the performance of the proposed filter on known cases where existing filters fail without careful exception handling or case-specific tuning and benchmark against the ArduPilot’s EKF3, the most sophisticated EKF implementation currently available.

Event cameras are ideally suited to capture High Dynamic Range (HDR) visual information without blur but provide poor imaging capability for static or slowly varying scenes. Conversely, conventional image sensors measure absolute intensity of slowly changing scenes effectively but do poorly on HDR or quickly changing scenes. In this paper, we present an asynchronous linear filter architecture, fusing event and frame camera data, for HDR video reconstruction and spatial convolution that exploits the advantages of both sensor modalities. The key idea is the introduction of a state that directly encodes the integrated or convolved image information and that is updated asynchronously as each event or each frame arrives from the camera. The state can be read-off as-often-as and whenever required to feed into subsequent vision modules for real-time robotic systems. Our experimental results are evaluated on both publicly available datasets with challenging lighting conditions and fast motions, along with a new dataset with HDR reference that we provide. The proposed AKF pipeline outperforms other state-of-the-art methods in both absolute intensity error (69.4% reduction) and image similarity indexes (average 35.5% improvement). We also demonstrate the integration of image convolution with linear spatial kernels Gaussian, Sobel, and Laplacian as an application of our architecture.

Event-based cameras are popular for tracking fast-moving objects due to their high temporal resolution, low latency, and high dynamic range. In this article, we propose a novel algorithm for tracking event blobs using raw eventsasynchronouslyin real time. We introduce the concept of anevent blobas a spatio-temporal likelihood of event occurrence where the conditional spatial likelihood is blob-like. Many real-world objects, such as car headlights or any quickly moving foreground objects, generate event blob data. The proposed algorithm uses a nearest neighbor classifier with a dynamic threshold criteria for data association coupled with an extended Kalman filter to track the event blob state. Our algorithm achieves highly accurate blob tracking, velocity estimation, and shape estimation even under challenging lighting conditions and high-speed motions ($>$11 000 pixels/s). The microsecond time resolution achieved means that the filter output can be used to derive secondary information, such as time-to-contact or range estimation, that will enable applications to real-world problems, such as collision avoidance, in autonomous driving.

Visual-inertial odometry (VIO) is the problem of estimating a robot's trajectory by combining information from an inertial measurement unit (IMU) and a camera and is of great interest to the robotics community. This article develops a novel Lie group symmetry for the VIO problem and applies the recently proposed equivariant filter. The proposed symmetry is compatible with the invariance of the VIO reference frame, leading to improved filter consistency. The bias-free IMU dynamics are group-affine, ensuring that filter linearization errors depend only on the bias estimation error and measurement noise. Furthermore, visual measurements are equivariant with respect to the symmetry, enabling the application of the higher order equivariant output approximation to reduce the approximation error in the filter update equation. As a result, the equivariant filter based on this Lie group is a consistent estimator for VIO with lower linearization error in the propagation of state dynamics and a higher order equivariant output approximation than standard formulations. Experimental results on the popularEuRoCandUZH FPVdatasets demonstrate that the proposed system outperforms other state-of-the-art VIO algorithms in terms of both speed and accuracy.

Event cameras are bio-inspired sensors that capture per-pixel asynchronous intensity change rather than the synchronous absolute intensity frames captured by a classical camera sensor. Such cameras are ideal for robotics applications since they have high temporal resolution, high dynamic range and low latency. However, due to their high temporal resolution, event cameras are particularly sensitive to flicker such as from fluorescent or LED lights. During every cycle from bright to dark, pixels that image a flickering light source generate many events that provide little or no useful information for a robot, swamping the useful data in the scene. In this paper, we propose a novel linear filter to preprocess event data to remove unwanted flicker events from an event stream. The proposed algorithm achieves over 4.6 times relative improvement in the signal-to-noise ratio when compared to the raw event stream due to the effective removal of flicker from fluorescent lighting. Thus, it is ideally suited to robotics applications that operate in indoor settings or scenes illuminated by flickering light sources. Code, Datasets and Video: https://github.com/ziweiWWANG/EFR

Inertial Navigation Systems (INS) are a key technology for autonomous vehicles applications. Recent advances in estimation and filter design for the INS problem have exploited geometry and symmetry to overcome limitations of the classical Extended Kalman Filter (EKF) approach that formed the mainstay of INS systems since the mid-twentieth century. The industry standard INS filter, the Multiplicative Extended Kalman Filter (MEKF), uses a geometric construction for attitude estimation coupled with classical Euclidean construction for position, velocity and bias estimation. The recent Invariant Extended Kalman Filter (IEKF) provides a geometric framework for the full navigation states, integrating attitude, position and velocity, but still uses the classical Euclidean construction to model the bias states. In this paper, we use the recently proposed Equivariant Filter (EqF) framework to derive a novel observer for biased inertial-based navigation in a fully geometric framework. The introduction of virtual velocity inputs with associated virtual bias leads to a full equivariant symmetry on the augmented system. The resulting filter performance is evaluated with both simulated and real-world data, and demonstrates increased robustness to a wide range of erroneous initial conditions, and improved accuracy when compared with the industry standard Multiplicative EKF (MEKF) approach.

Event cameras are bio-inspired dynamic vision sensors that respond to changes in image intensity with a high temporal resolution, high dynamic range and low latency. These sensor characteristics are ideally suited to enable visual target tracking in concert with a broadcast visual communication channel for smart visual beacons with applications in distributed robotics. Visual beacons can be constructed by high-frequency modulation of Light Emitting Diodes (LEDs) such as vehicle headlights, Internet of Things (IoT) LEDs, smart building lights, etc., that are already present in many real-world scenarios. The high temporal resolution characteristic of the event cameras allows them to capture visual signals at far higher data rates compared to classical frame-based cameras. In this paper, we propose a novel smart visual beacon architecture with both LED modulation and event camera demodulation algorithms. We quantitatively evaluate the relationship between LED transmission rate, communication distance and the message transmission accuracy for the smart visual beacon communication system that we prototyped. The proposed method achieves up to 4 kbps in an indoor environment and lossless transmission over a distance of 100 meters, at a transmission rate of 500 bps, in full sunlight, demonstrating the potential of the technology in an outdoor environment.

The current transformation towards smart manufacturing has led to a growing demand for human-robot collaboration (HRC) in the manufacturing process. Perceiving and understanding the human co-worker's behaviour introduces challenges for collaborative robots to efficiently and effectively perform tasks in unstructured and dynamic environments. Integrating recent data-driven machine vision capabilities into HRC systems is a logical next step in addressing these challenges. However, in these cases, off-the-shelf components struggle due to generalisation limitations. Real-world evaluation is required in order to fully appreciate the maturity and robustness of these approaches. Furthermore, understanding the pure-vision aspects is a crucial first step before combining multiple modalities in order to understand the limitations. In this paper, we propose GoferBot, a novel vision-based semantic HRC system for a real-world assembly task. It is composed of a visual servoing module that reaches and grasps assembly parts in an unstructured multi-instance and dynamic environment, an action recognition module that performs human action prediction for implicit communication, and a visual handover module that uses the perceptual understanding of human behaviour to produce an intuitive and efficient collaborative assembly experience. GoferBot is a novel assembly system that seamlessly integrates all sub-modules by utilising implicit semantic information purely from visual perception.

Event cameras are ideally suited to capture HDR visual information without blur but perform poorly on static or slowly changing scenes. Conversely, conventional image sensors measure absolute intensity of slowly changing scenes effectively but do poorly on high dynamic range or quickly changing scenes. In this paper, we present an event-based video reconstruction pipeline for High Dynamic Range (HDR) scenarios. The proposed algorithm includes a frame augmentation pre-processing step that deblurs and temporally interpolates frame data using events. The augmented frame and event data are then fused using a novel asynchronous Kalman filter under a unifying uncertainty model for both sensors. Our experimental results are evaluated on both publicly available datasets with challenging lighting conditions and fast motions and our new dataset with HDR reference. The proposed algorithm outperforms state-of-the-art methods in both absolute intensity error (48% reduction) and image similarity indexes (average 11% improvement).

Visual Inertial Odometry (VIO) is of great interest due the ubiquity of devices equipped with both a monocular camera and Inertial Measurement Unit (IMU). Methods based on the extended Kalman Filter remain popular in VIO due to their low memory requirements, CPU usage, and processing time when compared to optimisation-based methods. In this paper, we analyse the VIO problem from a geometric perspective and propose a novel formulation on a smooth quotient manifold where the equivalence relationship is the well-known invariance of VIO to choice of reference frame. We propose a novel Lie group that acts transitively on this manifold and is compatible with the visual measurements. This structure allows for the application of Equivariant Filter (EqF) design leading to a novel filter for the VIO problem. Combined with a very simple vision processing front-end, the proposed filter demonstrates state-of-the-art performance on the EuRoC dataset compared to other EKF-based VIO algorithms.

Multi-instance scenes are especially challenging for end-to-end visuomotor (image-to-control) learning algorithms. "Pipeline" visual servo control algorithms use separate detection, selection and servo stages, allowing algorithms to focus on a single object instance during servo control. End-to-end systems do not have separate detection and selection stages and need to address the visual ambiguities introduced by the presence of an arbitrary number of visually identical or similar objects during servo control. However, end-to-end schemes avoid embedding errors from detection and selection stages in the servo control behaviour, are more dynamically robust to changing scenes and are algorithmically simpler. In this paper, we present a reactive real-time end-to-end visuomotor learning algorithm for multi-instance reaching. The proposed algorithm uses a monocular RGB image and the manipulator’s joint angles as the input to a light-weight fully-convolutional network (FCN) to generate control candidates. A key innovation of the proposed method is identifying the optimal control candidate by regressing a control-Lyapunov function (cLf) value. The multi-instance capability emerges naturally from the stability analysis associated with the cLf formulation. We demonstrate the proposed algorithm effectively reaching and grasping objects from different categories on a table-top amid other instances and distractors from an over-the-shoulder monocular RGB camera. The network is able to run up to ∼160 fps during inference on one GTX 1080 Ti GPU.

Deep learning algorithms such as Convolutional Neural Networks (CNNs) are currently used to solve a range of robotics and computer vision problems. These networks typically estimate the desired representation in a single forward pass and must therefore learn to converge from a wide range of initial conditions to a precise result. This is challenging, and has led to increased interest in the development of separate refinement modules which learn to improve a given initial estimate, thus reducing the required search space. Such modules are usually developed ad-hoc for each given application, often requiring significant engineering investment. In this work we propose a generic innovation-based CNN. Our CNN is implemented along with a stochastic gradient descent (SGD) algorithm to iteratively refine a given initial estimate. The proposed approach provides a general framework for the development of refinement modules applicable to a wide range of robotics problems. We apply this framework to object pose estimation and depth estimation and demonstrate significant improvement over the initial estimates, in the range of 4.2 - 8.1%, for both applications.

Stereo camera systems play an important role in robotics applications to perceive the 3D world. However, conventional cameras have drawbacks such as low dynamic range, motion blur and latency due to the underlying frame- based mechanism. Event cameras address these limitations as they report the brightness changes of each pixel independently with a fine temporal resolution, but they are unable to acquire absolute intensity information directly. Although integrated hybrid event-frame sensors (e.g., DAVIS) are available, the quality of data is compromised by coupling at the pixel level in the circuit fabrication of such cameras. This paper proposes a stereo hybrid event-frame (SHEF) camera system that offers a sensor modality with separate high-quality pure event and pure frame cameras, overcoming the limitations of each separate sensor and allowing for stereo depth estimation. We provide a SHEF dataset targeted at evaluating disparity estimation algorithms and introduce a stereo disparity estimation algorithm that uses edge information extracted from the event stream correlated with the edge detected in the frame data. Our disparity estimation outperforms the state-of-the-art stereo matching algorithm on the SHEF dataset.

The static world assumption is standard in most simultaneous localisation and mapping (SLAM) algorithms. Increased deployment of autonomous systems to unstructured dynamic environments is driving a need to identify moving objects and estimate their velocity in real-time. Most existing SLAM based approaches rely on a database of 3D models of objects or impose significant motion constraints. In this paper, we propose a new feature-based, model-free, object-aware dynamic SLAM algorithm that exploits semantic segmentation to allow estimation of motion of rigid objects in a scene without the need to estimate the object poses or have any prior knowledge of their 3D models. The algorithm generates a map of dynamic and static structure and has the ability to extract velocities of rigid moving objects in the scene. Its performance is demonstrated on simulated, synthetic and real-world datasets.

We propose a closed-loop, multi-instance control algorithm for visually guided reaching based on novel learning principles. A control Lyapunov function methodology is used to design a reaching action for a complex multi-instance task in the case where full state information (poses of all potential reaching points) is available. The proposed algorithm uses monocular vision and manipulator joint angles as the input to a deep convolution neural network to predict the value of the control Lyapunov function (cLf) and corresponding velocity control. The resulting network output is used in real-time as visual control for the grasping task with the multi-instance capability emerging naturally from the design of the control Lyapunov function. We demonstrate the proposed algorithm grasping mugs (textureless and symmetric objects) on a table-top from an over-the-shoulder monocular RGB camera. The manipulator dynamically converges to the best-suited target among multiple identical instances from any random initial pose within the workspace. The system trained with only simulated data is able to achieve 90.3% grasp success rate in the real-world experiments with up to 85Hz closed-loop control on one GTX 1080Ti GPU and significantly outperforms a Pose-Based-Visual-Servo (PBVS) grasping system adapted from a state-of-the-art single shot RGB 6D pose estimation algorithm. A key contribution of the paper is the inclusion of a first-order differential constraint associated with the cLf as a regularisation term during learning, and we provide evidence that this leads to more robust and reliable reaching/grasping performance than vanilla regression on general control inputs.

The problem of tracking self-motion as well as motion of objects in the scene using information from a camera is known as multi-body visual odometry and is a challenging task. This paper proposes a robust solution to achieve accurate estimation and consistent track-ability for dynamic multi-body visual odometry. A compact and effective framework is proposed leveraging recent advances in semantic instance-level segmentation and accurate optical flow estimation. A novel formulation, jointly optimizing SE(3) motion and optical flow is introduced that improves the quality of the tracked points and the motion estimation accuracy. The proposed approach is evaluated on the virtual KITTI Dataset and tested on the real KITTI Dataset, demonstrating its applicability to autonomous driving applications. For the benefit of the community, we make the source code public†.

Event cameras are powerful new sensors able to capture high dynamic range with microsecond temporal resolution and no motion blur. Their strength is detecting brightness changes (called events) rather than capturing direct brightness images; however, algorithms can be used to convert events into usable image representations for applications such as classification. Previous works rely on hand-crafted spatial and temporal smoothing techniques to reconstruct images from events. State-of-the-art video reconstruction has recently been achieved using neural networks that are large (10M parameters) and computationally expensive, requiring 30ms for a forward-pass at 640 × 480 resolution on a modern GPU. We propose a novel neural network architecture for video reconstruction from events that is smaller (38k vs. 10M parameters) and faster (10ms vs. 30ms) than state-of-the-art with minimal impact to performance.

Solar energy is seen as a sustainable and non-depletable source of energy supply. Worldwide, large-scale solar power infrastructure is being installed every day. Such structures can suffer from many faults and defects that degrade their energy output during their operational life. Detecting such faults and defects requires regular inspection over physically large and distributed solar infrastructure. On-site manual human inspection tends to be impractical, risky and costly. As such, replacing humans with autonomous robotic aerial inspection systems has great potential. In this work, we propose an unmanned aerial vehicle (UAV) waypoint generation system that is specifically designed for aerial inspection of solar infrastructure. Our system takes into consideration the physical structure and the dynamic nature of sun-tracking solar modules and generates waypoints with the right camera viewing pose and drone orientation. Statistical methods are used to generate a randomly selected set of modules as a representation of the entire solar farm. The set is guaranteed to satisfy a user-defined confidence level and margin of error requirements. A path is generated to visit selected modules in an optimal way by deploying the traveling-salesman shortest path algorithm, allowing the vehicle to maximize battery use. Illustrative flights and preliminary inspection results are presented and discussed.

Visual reaching and grasping is a fundamental problem in robotics research. This paper proposes a novel approach based on deep learning a control Lyapunov function and its derivatives by encouraging a differential constraint in addition to vanilla regression that directly regresses independent joint control inputs. A key advantage of the proposed approach is that an estimate of the value of the control Lyapunov function is available in real-time that can be used to monitor the system performance and provide a level of assurance concerning progress towards the goal. The results we obtain demonstrate that the proposed approach is more robust and more reliable than vanilla regression.

This paper proposes a new method for estimating calibration parameters of plenoptic cameras by minimizing the nonlinear plenoptic reprojection error. Novel plenoptic feature types are proposed as data for the calibration method. These plenoptic disc features are in a natural one-to-one correspondence with physical points in front of the camera. We exploit the intrinsic geometry of plenoptic cameras in a novel projection model that relates the plenoptic disc features to physical points. The resulting calibration quality, as quantified by mean reprojection error and 3D reconstruction error, outperforms recently published results.

Deployment of aerial robotic vehicles for real world tasks such as home deliveries, close range aerial inspection, etc., require robotic vehicles to fly through complex and cluttered 3D environments such as forests, shrubbery or into balconies, garages, or sheds. Dense high-speed optical flow can provide real-time motion cues for obstacle avoidance that does not require 3D full reconstruction of the environment. However, classical reactive control does not `look ahead' and tends to bounce off obstacles rather than generating a smooth trajectory that anticipates and avoids upcoming obstacles. In this paper, we consider deriving a fully image based control criteria that forward predicts a cylinder of free space into the image flow representation of the environment and steers the vehicle by manoeuvering this cylinder through the upcoming environment. The length and radius of the cylinder provide a guarantee that the vehicle can indeed fly through the space identified and the fact that it is predicted forward into the environment leads to smooth anticipation of upcoming obstacles. Results are obtained for a quadrotor flying autonomously through a forest environment.

We consider the problem in which a supervisor or remote pilot provides a real-time linear velocity reference to a multirotor aerial robot, either through a traditional remote control handset, a modern haptic interface, or semi-autonomous guidance control system. In all such cases, the goal is to servo-control the vehicle's velocity to the set point as quickly and as efficiently as possible. The challenge is to achieve this robustly in the presence of unknown wind disturbances and in situations in which the vehicle moves into global position system (GPS) denied environments (indoors, urban canyons, forests) where estimation of the vehicle's velocity is challenging. These situations include unclutterred environments, poor visibility environments caused by poor lighting, and poorly textured visual environments where laser- and vision-based sensors become unreliable. The approach taken is to develop a coupled nonlinear complementary velocity aided attitude filter that provides estimates of both the inertial and body-fixed frame linear velocities, as well as the attitude of a multirotor aerial vehicle, that functions effectively even when only the inertial measurement unit and barometric sensor measurements are available. When full inertial velocity measurements are available (from GPS, Vicon, or a vision system), the filter additionally estimates the external wind speed. In this paper, we formally present the proposed filter along with experimental results and a comparison of the filter to recent results in the literature and in situations in which inertial reference frame velocities are available intermittently. The proposed filter is computationally simple to implement, easy to calibrate and tune, and provides an excellent base level functionality for modern multirotor aerial robotic systems that will be required to function robustly in a variety of environments.

This paper proposes a discrete-time geometric attitude observer for fusing monocular vision with GPS velocity measurements. The observer takes the relative transformations obtained from processing monocular images with any visual odometry algorithm and fuses them with GPS velocity measurements. The objectives of this sensor fusion are twofold; first to mitigate the inherent drift of the attitude estimates of the visual odometry, and second, to estimate the orientation directly with respect to the North-East-Down frame. A key contribution of the paper is to present a rigorous stability analysis showing that the attitude estimates of the observer converge exponentially to the true attitude and to provide a lower bound for the convergence rate of the observer. Through experimental studies, we demonstrate that the observer effectively compensates for the inherent drift of the pure monocular vision based attitude estimation and is able to recover the North-East-Down orientation even if it is initialized with a very large attitude error.

Current self-driving car systems operate well in sunny weather but struggle in adverse conditions. One of the most commonly encountered adverse conditions involves water on the road caused by rain, sleet, melting snow or flooding. While some advances have been made in using conventional RGB camera and LIDAR technology for detecting water hazards, other sources of information such as polarization offer a promising and potentially superior approach to this problem in terms of performance and cost. In this paper, we present a novel stereo-polarization system for detecting and tracking water hazards based on polarization and color variation of reflected light, with consideration of the effect of polarized light from sky as function of reflection and azimuth angles. To evaluate this system, we present a new large `water on road' datasets spanning approximately 2 km of driving in various on-road and off-road conditions and demonstrate for the first time reliable water detection and tracking over a wide range of realistic car driving water conditions using polarized vision as the primary sensing modality. Our system successfully detects water hazards up to more than 100m. Finally, we discuss several interesting challenges and propose future research directions for further improving robust autonomous car perception in hazardous wet conditions using polarization sensors.

Accurate online estimation of the environment structure simultaneously with the robot pose is a key capability for autonomous robotic vehicles. Classical simultaneous localization and mapping (SLAM) algorithms make no assumptions about the configuration of the points in the environment, however, real world scenes have significant structure (ground planes, buildings, walls, ceilings, etc.) that can be exploited. In this paper, we introduce meta-structural information associated with geometric primitives into the estimation problem and analyze their effect on the global structural consistency of the resulting map. Although we only consider the effect of adding planar and orthogonality information for the estimation of 3D points in a Manhattan-like world, this framework can be extended to any type of geometric, kinematic, dynamic or even semantic information. We evaluate our approach on a city-like simulated environment. We highlight the advantages of the proposed solution over SLAM formulation considering no prior knowledge about the configuration of 3D points in the environment.

This paper presents a novel control algorithm to regulate the aerodynamic thrust produced by fixed-pitch rotors commonly used on small-scale electrically powered multirotor aerial vehicles. The proposed controller significantly improves the disturbance rejection and gust tolerance of rotor thrust control compared to state-of-the-art RPM (revolutions per minute) rotor control schemes. The thrust modeling approach taken is based on a model of aerodynamic power generated by a fixed-pitch rotor and computed in real time on the embedded electronic speed controllers using measurements of electrical power and rotor angular velocity. Static and dynamic flight tests were carried out in downdrafts and updrafts of varying strengths to quantify the resulting improvement in maintaining a desired thrust setpoint. The performance of the proposed approach in flight conditions is demonstrated by a path tracking experiment, where a quadrotor was flown through an artificial wind gust and the trajectory tracking error was measured. The proposed approach for thrust control demonstrably reduced the tracking error compared to classical RPM rotor control.

Flight performance of aerial robotic vehicles is critically dependent on the quality of the state estimates provided by onboard sensor systems. The attitude estimation problem has been extensively studied over the last ten years and the development of low complexity, high performance, robust non-linear observers for attitude has been one of the enabling technologies fueling the growth of small scale aerial robotic systems. The velocity aided attitude estimation problem, that is simultaneous estimation of attitude and linear velocity of an aerial platform, has only been tackled using the non-linear observer approach in the last few years. Prior contributions have lead to non-linear observers for which either there is no stability analysis or for which the analysis is extremely complex. In this paper, we propose a simple relaxation of the state space, allowing scaled rotation matrices R ∈ ℝ3×3such that RXT= uI where X = uR̂ and u > 0 is a positive scalar, along with additional observer dynamics to force u → 1 asymptotically. With this simple augmentation of the observer state space, we propose a non-linear observer with a straightforward Lyapunov stability analysis that demonstrates almost global asymptotic convergence along with local exponential convergence. Simulations as well as experimental results are provided to demonstrate the performance of the proposed observer.

This paper introduces a novel constraint in the master joystick workspace, termed the dynamic kinesthetic boundary, that aids a pilot to navigate an aerial robotic vehicle through a cluttered environment. The proposed approach exploits spatial cues by projecting the remote environment into a hard boundary in the master device workspace that provides a pilot with a natural representation of approaching obstacles. The approach is distinguished from classical force feedback approaches by allowing normal operation of the vehicle in free flight and only imposing constraints when approaching and interacting with the environment. A key advantage is that contact with the environment constraint is immediately perceptible to a pilot, allowing them to make suitable adjustments to their inputs. Modulation of the velocity reference for the slave robot ensures obstacle avoidance while allowing a vehicle to approach as close as desired to an object, albeit at a slow speed. A comprehensive user study was performed to systematically test the proposed algorithm and comparisons to two existing state-of-the-art approaches are provided to demonstrate the relative performance of the proposed approach.

Force feedback is a powerful sensory cue that can provide a user, remotely teleoperating a vertical takeoff and landing (VTOL) aerial robotic vehicle, with an enhanced perception of the robot motion and interaction with the local environment. The accepted framework for mobile vehicle teleoperation, inspired by decades of research into manipulator teleoperation, maps joystick displacements to vehicle velocity set points and reflects interactive forces from the vehicle to the human operator. This paper proposes a novel admittance configured haptic interface where the force input from the user is used as a velocity set point and the joystick position is servo-controlled to reflect the vehicle velocity. This paper provides a comparative study, both analytic and experimental, of the two interfaces to investigate the performance and perceptual differences in the context of teleoperation of a VTOL aerial robot.

In this paper, a new motor control input and controller for small-scale electrically powered multirotor aerial vehicles is proposed. The proposed scheme is based on controlling aerodynamic power as opposed to the rotor speed of each motor-rotor system. Electrical properties of the brushless direct current motor are used to both estimate and control the mechanical power of the motor system which is coupled with aerodynamic power using momentum theory analysis. In comparison to current state-of-the-art motor control for multirotor aerial vehicles, the proposed approach is robust to unmodelled aerodynamic effects such as wind disturbances and ground effects. Theory and experimental results are presented to illustrate the performance of the proposed motor control.

This paper presents a novel intuitive multi-modal force feedback interface for teleoperation of mobile robotic vehicles. Two different force feedback interfaces are considered: a force feedback joystick and a novel force feedback trackball. The joystick considered is based on the admittance user interface developed by the authors in earlier work and is configured to servo velocity of the vehicle. The force feedback trackball is configured to map vehicle velocity directly to trackball velocity, exploiting the effectively infinite workspace of the trackball to overcome the classical challenge of servo controlling a slave with infinite workspace using a master device with finite workspace. A key contribution of the paper is to provide a modeling framework, based on the bond graph formalism, that allows the energy consistent modeling of input from an admittance joystick as reference to an internal velocity regulation loop for the vehicle. Once this is implemented it is straightforward to interconnect multiple input devices, and in particular the trackball device, using standard interconnection rules in bond graphs. Experiments were performed, and the outcomes verify the feasibility and effectiveness of the proposed interface.

This paper considers the question of providing effective feedback of vehicle dynamic forces to a pilot in haptic teleoperation of aerial robots. We claim that the usual state-of-the-art haptic interface, based on research motivated by robotic manipulator slaves and virtual haptic environments, does a poor job of reflecting dynamic forces of a mobile robotic vehicle to the user. This leads us to propose a novel force feedback user interface for mobile robotic vehicles with dynamics. An analysis of the closed-loop force-displacement transfer functions experienced by the master joystick for the classical and the new approach clearly indicate the advantages of the proposed formulation. Both the classical and the proposed approach have been implemented in the teleoperation of a quadrotor vehicle and we present quantitative and cognitive performance data from a user study that corroborates the expected performance advantages.

We introduce a new image-based visual navigation algorithm that allows the Cartesian velocity of a robot to be defined with respect to a set of visually observed features corresponding to previously unseen and unmapped world points. The technique is well suited to mobile robot tasks such as moving along a road or flying over the ground. We describe the algorithm in general form and present detailed simulation results for an aerial robot scenario using a spherical camera and a wide angle perspective camera, and present experimental results for a mobile ground robot.

The paper concerns visuo-inertial filtering and estimation based on homography and angular velocity measurements, i.e. data obtained from a mono-camera/IMU sensor. We extend recently developed nonlinear filters on the special linear group of homographies to the estimation of scene parameters and velocity of the sensor. A validation of the proposed solution and a comparative evaluation based on real data is presented.

This paper introduces a novel dynamic kinesthetic boundary to aid a pilot to navigate an aerial robotic vehicle through a cluttered environment. Classical haptic teleoperation interfaces for aerial vehicles utilize force feedback to provide the pilot with a haptic feel of the robot's interaction with an environment. The proposed approach constructs a dynamic kinesthetic boundary on the master device that provides the pilot with hard boundaries in the haptic workspace to indicate approaching obstacles. An advantage of the proposed approach is that when the vehicle is flying free of obstacles, then the haptic feedback of the joystick can be used to provide a more natural feel of the vehicle dynamics. Furthermore, rather than a gradual onset of virtual potential forces that are felt in the classical approaches, a pilot encountering the dynamic kinesthetic boundary is immediately aware of the presence of the obstacle and can act accordingly. The approach is implemented on an admittance haptic joystick to ensure that the haptic boundaries are faithfully rendered. We prove that in the case of perfect velocity tracking, the proposed algorithm will ensure the vehicle never colliding with the environment. Experiments were conducted on a robotic platform and the results provide verification of the novel approach.

This paper describes the theory and practice for a stable haptic teleoperation of a flying vehicle. It extends passivity-based control framework for haptic teleoperation of aerial vehicles in the longest intercontinental setting that presents great challenges. The practicality of the control architecture has been shown in maneuvering and obstacle-avoidance tasks over the internet with the presence of significant time-varying delays and packet losses. Experimental results are presented for teleoperation of a slave quadrotor in Australia from a master station in the Netherlands. The results show that the remote operator is able to safely maneuver the flying vehicle through a structure using haptic feedback of the state of the slave and the perceived obstacles.

This paper presents a nonlinear controller for a vertical take-off and landing (VTOL) unmanned aerial vehicle (UAV) that exploits a measurement optical flow to enable hover and landing control on a moving platform, such as, for example, the deck of a sea-going vessel. The VTOL vehicle is assumed to be equipped with a minimum sensor suite [i.e., a camera and an inertial measurement unit (IMU)], manoeuvring over a textured flat target plane. Two different tasks are considered in this paper. The first concerns the stabilization of the vehicle relative to the moving platform that maintains a constant offset from a moving reference. The second concerns regulation of automatic vertical landing onto a moving platform. Rigorous analysis of system stability is provided, and simulations are presented. Experimental results are provided for a quadrotor UAV to demonstrate the performance of the proposed control strategy.

This paper proposes an adaptation of classical image-based visual servoing to a generalised imaging model where cameras are modelled as sets of 3D viewing rays. This new model leads to a generalised visual servoing control formalism that can be applied to any type of imaging system whether it be multi-camera, catadioptric, non-central, etc. In this paper the generalised 3D viewing cones are parameterised geometrically via Plucker line coordinates. The new visual servoing model is tested on an a-typical stereo-camera system with non-overlapping cameras. In this case no 3D information is available from triangulation and the system is comparable to a 2D visual servoing system with non-central ray-based control law. I.

Micro aerial vehicles (MAVs) are a rapidly growing area of research and development in robotics. For autonomous robot operations, localization has typically been calculated using GPS, external camera arrays, or onboard range or vision sensing. In cluttered indoor or outdoor environments, onboard sensing is the only viable option. In this paper we present an appearance-based approach to visual SLAM on a flying MAV using only low quality vision. Our approach consists of a visual place recognition algorithm that operates on 1000 pixel images, a lightweight visual odometry algorithm, and a visual expectation algorithm that improves the recall of place sequences and the precision with which they are recalled as the robot flies along a similar path. Using data gathered from outdoor datasets, we show that the system is able to perform visual recognition with low quality, intermittent visual sensory data. By combining the visual algorithms with the RatSLAM system, we also demonstrate how the algorithms enable successful SLAM.

This paper presents a general approach for terrain following (including obstacle avoidance) of a vertical take-off and landing vehicle (VTOL) using multiple observation points. The VTOL vehicle is assumed to be a rigid body, equipped with a minimum sensor suite (camera, IMU and barometric altimeter), manoeuvering over a textured rough terrain made of a concatenation of planar surfaces. Assuming that the forward velocity is separately regulated to a desired non-zero value, the proposed control approach ensures terrain following and guarantees the vehicle does not collide with obstacles during the task. The proposed control acquires an optical flow from multiple spatially separate observation points, typically obtained via multiple cameras or non collinear directions in a unique camera. The proposed control algorithm has been tested extensively in simulation and then implemented on a quadrotor UAV to demonstrate the performance of the closed loop system.

We present a novel, simple and effective approach for tele-operation of aerial robotic vehicles with haptic feedback. Such feedback provides the remote pilot with an intuitive feel of the robot's state and perceived local environment that will ensure simple and safe operation in cluttered 3D environments common in inspection and surveillance tasks. Our approach is based on energetic considerations and uses the concepts of network theory and port-Hamiltonian systems. We provide a general framework for addressing problems such as mapping the limited stroke of a `master' joystick to the infinite stroke of a `slave' vehicle, while preserving passivity of the closed-loop system in the face of potential time delays in communications links and limited sensor data.

This paper presents a nonlinear controller for hovering flight and landing control on a moving platform for a Vertical Take-off and Landing (VTOL) Unmanned Aerial Vehicle (UAV) by exploiting the measurement of the average optical flow. The VTOL vehicle is assumed to be equipped with a minimum sensor suite (a camera and an IMU), manoeuvring over a textured flat target plane. Two different tasks are considered in this paper: the first one concerns the stability of hovering flight and the second one concerns regulation of automatic vertical landing on a moving platform using the divergent optical flow as feedback information. Simulation and experimental results performed on a quad-rotor UAV demonstrate the performance of the proposed control strategy.

This paper considers the problem of estimating pose from inertial and bearing-only vision measurements. We present a non-linear observer that evolves directly on the special Euclidean group SE(3) from inertial measurements and bearing measurements, such as provided by a visual system tracking known landmarks. Local asymptotic convergence of the observer is proved. The observer is computationally simple and its gains are easy to tune. Simulation results demonstrate robustness to measurement noise and initial conditions.

This paper presents a nonlinear controller for terrain following of a vertical take-off and landing vehicle (VTOL). The VTOL vehicle is assumed to be a rigid body, equipped with a minimum sensor suite (camera and IMU) along with a measure of the forward speed from another sensor such as global positioning system, maneuvering over a textured terrain made of planar surfaces. Assuming that the forward velocity is separately set to a desired value, the proposed control approach ensures terrain following and guaranties the vehicle does not collide with the ground during the task. The proposed control acquires an optical flow from three spatially separate observation points, typically obtained via three cameras or three non collinear directions in a unique camera. The proposed control algorithm has been tested extensively in simulation and then implemented on a quadrotor UAV to demonstrate the performance of the closed loop system.

This paper proposes the use of optical flow from a moving robot to provide force feedback to an operator's joystick to facilitate collision free teleoperation. Optic flow is measured by wide angle cameras on board the vehicle and used to generate a virtual environmental force that is reflected to the user through the joystick, as well as feeding back into the control of the vehicle. The coupling between optical flow (velocity) and force is modelled as an impedance - in this case an optical impedance. We show that the proposed control is dissipative and prevents the vehicle colliding with the environment as well as providing the operator with a natural feel for the remote environment. The paper focuses on applications to aerial robotics vehicles, however, the ideas apply directly to other force actuated vehicles such as submersibles or space vehicles, and the authors believe the approach has potential for control of terrestrial vehicles and even teleoperation of manipulators. Experimental results are provided for a simulated aerial robot in a virtual environment controlled by a haptic joystick.

In the last decade, many vision-based robot controllers have been designed using Cartesian information encoded in the homography transformation that links two images of a planar object. For any approach, the performance of the closed-loop system depends on the quality of the homography estimates obtained. In this paper, we exploit the special linear Lie-group structure of the set of all homographies to develop a dynamic observer to estimate homographies online. The resulting estimates are effective and can be used to improve closed-loop response of several visual servoing algorithms.

Virtually all quadrotors used in research weigh less than 2 kg, and carry payload measured in hundreds of grams. To be useful platforms for expanded operations, these vehicles must be capable of carrying greater weight. Several obstacles in aerodynamics, design and control must be overcome to enable the construction of larger craft with payloads in excess of 1 kg. We report the key design considerations essential for the construction of heavy quadrotor MAVs and demonstrate a 4 kg quadrotor with 1 kg payload.

In this paper, we investigate a range of image-based visual servo control algorithms for regulation of the position of a quadrotor aerial vehicle. The most promising control algorithms have been successfully implemented on an autonomous aerial vehicle and demonstrate excellent performance.

This paper considers the question of using a nonlinear complementary filter for attitude estimation of fixed-wing unmanned aerial vehicle (UAV) given only measurements from a low-cost inertial measurement unit. A nonlinear complementary filter is proposed that combines accelerometer output for low frequency attitude estimation with integrated gyrometer output for high frequency estimation. The raw accelerometer output includes a component corresponding to airframe acceleration, occurring primarily when the aircraft turns, as well as the gravitational acceleration that is required for the filter. The airframe acceleration is estimated using a simple centripetal force model (based on additional airspeed measurements), augmented by a first order dynamic model for angle-of-attack, and used to obtain estimates of the gravitational direction independent of the airplane manoeuvres. Experimental results are provided on a real-world data set and the performance of the filter is evaluated against the output from a full GPS/INS that was available for the data set.

This paper presents a nonlinear controller for hovering flight and touchdown control for a vertical take-off and landing (VTOL) unmanned aerial vehicle (UAV) using inertial optical flow. The VTOL vehicle is assumed to be a rigid body, equipped with a minimum sensor suite (camera and IMU), manoeuvring over a textured flat target plane. Two different tasks are considered in this paper: the first concerns the stability of hovering flight and the second one concerns regulation of automatic landing using the divergent optical flow as feedback information. Experimental results on a quad-rotor UAV demonstrate the performance of the proposed control strategy.

An image-based visual servo control is presented for an unmanned aerial vehicle (UAV) capable of stationary or quasi-stationary flight with the camera mounted onboard the vehicle. The target considered consists of a finite set of stationary and disjoint points lying in a plane. Control of the position and orientation dynamics is decoupled using a visual error based on spherical centroid data, along with estimations of the linear velocity and the gravitational inertial direction extracted from image features and an embedded inertial measurement unit. The visual error used compensates for poor conditioning of the image Jacobian matrix by introducing a nonhomogeneous gain term adapted to the visual sensitivity of the error measurements. A nonlinear controller, that ensures exponential convergence of the system considered, is derived for the full dynamics of the system using control Lyapunov function design techniques. Experimental results on a quadrotor UAV, developed by the French Atomic Energy Commission, demonstrate the robustness and performance of the proposed control strategy.

We present a robust strategy for docking a mobile robot in close proximity with an upright surface using optical flow field divergence and proportional feedback control. Unlike previous approaches, we achieve this without the need for explicit segmentation of features in the image, and using complete gradient-based optical flow estimation (i.e., no affine models) in the optical flow computation. A key contribution is the development of an algorithm to compute the flow field divergence, or time-to-contact, in a manner that is robust to small rotations of the robot during ego-motion. This is done by tracking thefocusofexpansionof the flow field and using this to compensate for ego rotation of the image. The control law used is a simple proportional feedback, using the unfiltered flow field divergence as an input, for a dynamic vehicle model. Closed-loop stability analysis of docking under the proposed feedback is provided. Performance of the flow field divergence algorithm is demonstrated using offboard natural image sequences, and the performance of the closed-loop system is experimentally demonstrated by control of a mobile robot approaching a wall.

This paper presents a coupled observer that uses accelerometer, gyrometer and vision sensors to provide estimates of pose and linear velocity for an aerial robotic vehicle. The observer is based on a non-linear complimentary filter framework and incorporates adaptive estimates of measurement bias in gyrometers and accelerometers commonly encountered in low-cost inertial measurement systems. Asymptotic stability of the observer estimates is proved as well as bounded energy of the observer error signals. Experimental data is provided for the proposed filter run on data obtained from an experiment involving a remotely controlled helicopter.

An image-based visual servo control is presented for an unmanned aerial vehicle (UAV) capable of stationary or quasi-stationary flight. The proposed control design addresses visual servo of 'eye-in-hand' type systems. The control of the position and orientation dynamics are decoupled using a visual error based on a spherical centroid data, along with estimation of the gravitational inertial direction. The error used compensates for the poor conditioning of the Jacobian matrix seen in earlier work in this area by introducing a non-homogeneous gain term adapted to the visual sensitivity of the error measurements. A nonlinear controller is derived for the full dynamics of the system. Experimental results on an experimental UAV known as an X4-flyer made by the French Atomic Energy Commission (CEA) demonstrate the robustness and performances of the proposed control strategy.

This paper considers the question of obtaining high quality attitude estimates from typical low cost inertial measurement units for applications in control of unmanned aerial vehicles. A nonlinear complimentary filter exploiting the structure of special orthogonal group S0(3) is proposed. The filter is expressed explicitly in terms of direct and untreated measurements. For a typical low cost inertial measurement where two inertial directions are measured (gravitational and magnetic fields) along with angular velocity, it is shown that the filter is well conditioned. If only a single direction is available (typically the gravitational field) along with angular velocity, it is shown that the full gyro bias vector is correctly estimated and that the estimated orientation converges to a set consistent with the measurements. Experimental results, for flight data from the HoverEyecopy UAV, demonstrate the efficiency of the proposed filter

This paper presents a bounded torque control design to solve the set-point regulation problem for robot manipulators subject to joint velocity constraints. The control objectives are achieved by exploiting the passivity properties of the system and utilizing barrier function ideas to reshape the control Lyapunov function. The structure of the modified control Lyapunov function is reminiscent of those used in the artificial potential field method. The resulting controllers are modified proportional-derivative controllers which are simple, intuitive, and can easily be implemented in practice. In addition, asymptotic stability of the closed-loop system is guaranteed, all joint velocity constraints are strictly satisfied for all time, and the demanded torque input is bounded in norm, irrespective of the initial condition. The effectiveness of the proposed control design is demonstrated through simulations on a 2-link planar manipulator

This paper describes a control strategy to stabilize the position of a vertical takeoff and landing (VTOL) unmanned aerial vehicle (UAV) in wind gusts. The proposed approach takes advantage of the cascade structure of the system to design a hierarchical controller. The idea is to separate the controller in a high level controller devoted to position control and a low level controller devoted to stabilization and attitude control. Both controllers are designed by means of backstepping techniques that allow the stabilization of the vehicle's position while on-line estimation of the unknown aerodynamic forces. The global stability of the connected system is proven, and simulations as well as experimental results are presented

Image moments provide an important class of image features used for image-based visual servo control. Spherical image moments have the additional desirable property that they are invariant under rotation of the camera frame. For these features one can study the local and global stability and performance of the position control independently of the rotation control. In this paper we study a range of control algorithms including the classical approximately linearising control, a recently proposed robust control based on Lyapunov function design methodology, and modifications of these designs to improve global and asymptotic performance and robustness of the schemes. The comparison of performance demonstrates that the choice of image feature and control design for image-based visual servo are each equally important and highly coupled. We finally propose a control law using a modified image feature and a Lyapunov control design that ensures global asymptotic stability and good performance equally in image space as in task space

An image-based "eye-in-hand" visual servo-control design is proposed for underactuated rigid-body dynamics. The dynamic model considered is motivated by recent work on vertical takeoff and landing aerial robotic vehicles. The task considered is that of tracking parallel linear visual features. The proposed design exploits the geometry of the task considered and passivity-like properties of rigid-body dynamics to derive a control Lyapunov function using backstepping techniques.

A pure image-based strategy for visual servo control of a class of dynamic systems is proposed. The proposed design concerns the dynamics of unmanned aerial vehicles capable of quasi-stationary flight (hover and near hover flight). The visual servo control task considered is to locate the camera relative to a stationary target. The paper extends earlier work, by weakening the assumption on the construction of the visual error used. In prior work some inertial information was used in the error construction to guarantee the passivity properties of the control design. In this paper the visual error is defined purely in terms of the 2D image features derived from, the camera signal.

This paper proposes a new control method for stabilising control design for docking unicycle-like vehicles based on bearing-only information. An omni-directional panoramic camera is used to detect visual targets around the docking station and provides bearing (or heading) data for each observed landmark. The convergence of the controlled system is fully analysed and simulations are provided to demonstrate the ideal behaviour of the system. A robust and computationally cheap blob detection algorithm is proposed and results are provided to demonstrate its performance in extracting targets from cluttered scenes. Experimental results are presented demonstrating the performance of the algorithm on the ANU Nomadic Technologies Nomad XR4000 robot.

We present control methods for an autonomous four-rotor helicopter, called a quadrotor, using visual feedback as the primary sensor. The vision system uses aground camera to estimate the pose (position and orientation) of the helicopter. Two methods of control are studied - one using a series of mode-based, feedback linearizing controllers, and the other using a backstepping-like control law. Various simulations of the model demonstrate the implementation of feedback linearization and the backstepping controllers. Finally,. we present initial flight experiments. where the helicopter is restricted to vertical and yaw motions.

An image-based visual servo control design is proposed for a four rotor vertical take-off and landing (VTOL) craft, known as an X4-flyer, accomplishing a trajectory tracking task. The approach taken is an image-based visual servo design that is applicable to under-actuated dynamic systems. The work is an extension of the authors earlier work (2000) to the trajectory tracking problem. Semiglobal stability of the closed loop system is proved for bounded trajectories.

An image-based visual servo control is presented for a robotic manipulator. The proposed control design addresses visual servo of 'eye-in-hand' type systems. Using a novel representation of the visual error based on a spherical representation of target centroid information along with a measure of the rotation between the camera and the target, the control of the position and orientation is decoupled. A non-linear gain introduced into the orientation feedback kinematics prevents the target image from leaving the visual field. Semi-global convergence of the closed loop system is proved.

This paper is concerned with choosing image features for image based visual servo control and how this choice influences the closed-loop dynamics of the system. In prior work, image features tend to be chosen on the basis of image processing simplicity and noise sensitivity. In this paper we show that the choice of feature directly influences the closed-loop dynamics in task-space. We focus on the depth axis control of a visual servo system and compare analytically various approaches that have been reported recently in the literature. The theoretical predictions axe verified by experiment.

A new image-based control strategy for visual servoing of a class of under-actuated rigid body systems is presented. The proposed control design applies to "eye-in-hand" systems where the camera is fixed to a rigid body with actuated dynamics. The control design is motivated by a theoretical analysis of the dynamic equations of motion of a rigid body and exploits passivity-like properties of these dynamics to derive a Lyapunov control algorithm using robust backstepping techniques. The proposed control is novel in considering the full dynamic system incorporating all degrees of freedom (albeit for a restricted class of dynamics) and in not requiring measurement of the relative depths of the observed image points. A motivating application is the stabilization of a scale model autonomous helicopter over a marked landing pad.

A control design to stabilise a reduced scale autonomous helicopter equipped with a camera is presented. The proposed algorithm is motivated by recent work in image-based visual servoing control for under-actuated dynamic systems. This work is extended by considering models that contain weakly nonminimum phase zero dynamics. This is an important class of systems since an offset between the camera and the centre of mass for a typical 'dynamic' autonomous vehicle will result in zero dynamics occurring in the image dynamics. In this paper we propose a simplified model of dynamics of the helicopter and show that by placing some constraints in the choice of the position of the camera we can minimise the effect of the zero dynamics.

An image-based 'eye-in-hand' type visual servoing control design is presented for under-actuated rigid body dynamics. The dynamic model considered is often used to model the dynamics of unmanned aerial vehicles (UAV) such as helicopters and aeroplanes. The task considered is that of tracking parallel linear visual features. The proposed design exploits the geometry of the task considered and passivity-like properties of rigid body dynamics to derive a Lyapunov control algorithm using backstepping design techniques.

A simplified model of the dynamics of an autonomous model helicopter valid for manoeuvres close to hover is considered. The control task is to track a given path in the position co-ordinates as accurately as possible. This is made difficult since the model contains weakly non-minimum phase zero dynamics which cannot be stably inverted. By placing constraints on the physical weight distribution associated with the necessary sensor and computer components that must be carried by the autonomous unit a new representation the dynamics of the system in block pure feedback form is derived. The exact tracking problem is solved using input/output linearization techniques.

The classical Viterbi algorithm (ML sequence estimation) can be computed using a forward-backward structure, similar to that of the classical hidden Markov model forward-backward algorithm (MAP state estimation). This similarity is exploited to develop a hybrid algorithm which provides a mathematical connection between ML sequence estimation and MAP state estimation.