We describe a technique for object detection that uses a combination of global shape descriptors and local point descriptors. Our system is able to represent pose using a global shape descriptor, rather than the commonly used part based representation. This approach considerably reduces computational complexity and achieves a significant performance improvement on an extensive dataset: CUB-200-2011 [31]. Our methodology is valuable for the detection of textured objects that are viewed against background clutter and possess a high degree of articulation and variation of pose, as for example in birds. We demonstrate how high and low frequency gradients can be separated to better deal with the presence of interfering textures or stripes within the body, which is a major problem in the detection of bird-like objects. Furthermore, detection accuracy is improved by integrating appropriately designed scale invariant color features into the algorithm.

This paper presents a novel visual navigation strategy applied to real-time guidance and control of autonomous rotorcraft in outdoor environments. The proposed approach acquires a sequence of snapshots during an initial exploratory journey and computes the rotorcraft's 3D position and velocity relative to the locations of these memorised snaphots during its long-range homing return. This mapless algorithm provides an efficient strategy that enables route learning and drift-free navigation in unknown and unstructured 3D environments, based on a simple and sparse visual route description. The performance of our approach is evaluated in closed-loop flight experiments onboard a small-size rotorcraft. Dynamic path optimisation is observed when navigating back and forth between two locations of interest.

Birds flying through a cluttered environment require the ability to choose routes that will take them through the environment safely and quickly. We have investigated some of the strategies by which they achieve this. We trained budgerigars to fly through a tunnel in which they encountered a barrier that offered two passages, positioned side by side, at the halfway point. When one of the passages was substantially wider than the other, the birds tended to fly through the wider passage to continue their transit to the end of the tunnel, regardless of whether this passage was on the right or the left. Evidently, the birds were selecting the safest and quickest route. However, when the two passages were of equal or nearly equal width, some individuals consistently preferred the left-hand passage, while others consistently preferred the passage on the right. Thus, the birds displayed idiosyncratic biases when choosing between alternative routes. Surprisingly--and unlike most of the instances in which behavioral lateralization has previously been discovered--the bias was found to vary from individual to individual, in its direction as well as its magnitude. This is very different from handedness in humans, where the majority of humans are right-handed, giving rise to a so-called 'population' bias. Our experimental results and mathematical model of this behavior suggest that individually varying lateralization, working in concert with a tendency to choose the wider aperture, can expedite the passage of a flock of birds through a cluttered environment.

This study describes a novel method for estimating the strength and direction of the local wind field from a mobile airborne platform. An iterative optimisation is derived that allows the properties of the wind field to be determined from successive measurements of the heading direction and ground track of the aircraft only. We have previously described methods for estimating these parameters using a single vision system. This approach therefore constitutes a purely visual method for estimating the properties of the local wind field. We present results from simulated and real-world flight tests that demonstrate the accuracy and robustness of the proposed method and its practicality in uncontrolled environmental conditions. These properties and the simplicity of the implementation should make this approach useful as an alternative means for estimating the properties of the local wind field from small-scale, fixed-wing UAVs.

This study describes a novel method for automatically obtaining the attitude of an aircraft from the visual horizon. A wide-angle view of the environment, including the visual horizon, is captured and the input images are classified into fuzzy sky and ground regions using the spectral and intensity properties of the pixels. The classifier is updated continuously using an online reinforcement strategy and is therefore able to adapt to the changing appearance of the sky and ground, without requiring prior training offline. A novel approach to obtaining the attitude of the aircraft from the classified images is described, which is reliable, accurate, and computationally efficient to implement. This method is therefore suited to real-time operation and we present results from flight tests that demonstrate the ability of this vision-based approach to outperform an inexpensive inertial system.

This study describes a novel, vision-based system for guidance of UAVs. The system uses two coaxially aligned cameras, each associated with a specially-shaped reflective surface, to obtain stereo information on the height above ground and the distances to potential obstacles. The camera-mirror system has the advantage that it remaps the world onto a cylindrical co-ordinate system that simplifies and speeds up range computations, and defines a collision-free cylinder through which the aircraft can pass without encountering obstacles. We describe an approach, using this vision system, in which the attitude and altitude of an aircraft can be controlled directly, making the system particularly suited to terrain following, obstacle avoidance, and landing. The autonomous guidance of an aircraft performing a terrain following task using the system is demonstrated in field tests.

A method for calculating visual odometry for ground vehicles with car-like kinematic motion constraints similar to Ackerman's steering model is presented. By taking advantage of this non-holonomic driving constraint we show a simple and practical solution to the odometry calculation by clever placement of a single camera. The method has been implemented successfully on a large industrial forklift and a Toyota Prado SUV. Results from our industrial test site is presented demonstrating the applicability of this method as a replacement for wheel encoder-based odometry for these vehicles.

This study describes a novel, vision-based system for guidance of UAVs. The system uses two cameras, each associated with a specially-shaped reflective surface, to obtain stereo information on the height above ground and the distances to potential obstacles. The camera-mirror system has the advantage that it remaps the world onto a cylindrical co-ordinate system that simplifies and speeds up range computations, and defines a collision-free cylinder through which the aircraft can pass without encountering obstacles. The result is a computationally efficient approach to vision-based aircraft guidance that is particularly suited to terrain and gorge following, obstacle avoidance, and landing. The feasibility of the system is demonstrated in laboratory and field tests.

This paper describes a technique for estimating the attitude of a UAV by monitoring the visual horizon. An algorithm is developed that makes the best use of color and intensity information in an image to determine the position and orientation of the horizon, and infer the aircraft's attitude. The technique is accurate, reliable, and fully capable of real-time operation. Furthermore, it can be incorporated into any existing vision system, irrespective of the way in which the environment is imaged (e.g. through lenses or mirrors).

Behavior analysis of social insects has garnered impetus in recent years and has led to some advances in fields like control systems, flight navigation etc. Manual labeling of insect motions required for analyzing the behaviors of insects requires significant investment of time and effort. In this paper, we propose certain general principles that help in simultaneous automatic tracking and behavior analysis with applications in tracking bees and recognizing specific behaviors exhibited by them. The state space for tracking is defined using position, orientation and the current behavior of the insect being tracked. The position and orientation are parametrized using a shape model while the behavior is explicitly modeled using a three-tier hierarchical motion model. The first tier (dynamics) models the local motions exhibited and the models built in this tier act as a vocabulary for behavior modeling. The second tier is a Markov motion model built on top of the local motion vocabulary which serves as the behavior model. The third tier of the hierarchy models the switching between behaviors and this is also modeled as a Markov model. We address issues in learning the three-tier behavioral model, in discriminating between models, detecting and in modeling abnormal behaviors. Another important aspect of this work is that it leads to joint tracking and behavior analysis instead of the traditional track and then recognize approach. We apply these principles for tracking bees in a hive while they are executing the waggle dance and the round dance.

We present a strategy for generating real-time relative depth maps of an environment from optical flow, under general motion. We achieve this using an insect-inspired hemispherical fish-eye sensor with 190 degree FOV, and a de-rotated optical flow field. The de-rotation algorithm applied is based on the theoretical work of Nelson and Aloimonos (1988). From this we obtain the translational component of motion, and construct full relative depth maps on the sphere. We examine the robustness of this strategy in both simulation and real-world experiments, for a variety of environmental scenarios. To our knowledge, this is the first demonstrated implementation of the Nelson and Aloimonos algorithm working in real-time, over real image sequences. In addition, we apply this algorithm to the real-time recovery of full relative depth maps. These preliminary results demonstrate the feasibility of this approach for closed-loop control of a robot.

There is considerable interest in designing guidance systems for UAVs that use passive sensing (such as vision), rather than active sensing which can be bulky, expensive and stealth-compromising. Here we describe an optical sensor, based partly on principles of insect vision and optic flow analysis, for measurement and control of height above the ground. A video camera is used in conjunction with a specially shaped reflective surface to simplify the computation of optic flow, and extend the range of aircraft speeds over which accurate data can be obtained. The imaging system also provides a useful geometrical remapping of the environment, which facilitates obstacle avoidance and computation of 3-D terrain maps.

Tracking multiple objects in a video is a demanding task that is frequently encountered in several systems such as surveillance and motion analysis. Ability to track objects in 3D requires the use of multiple cameras. While tracking multiple objects using multiples video cameras, establishing correspondence between objects in the various cameras is a non-trivial task. Specifically, when the targets are dim or are very far away from the camera, appearance cannot be used in order to establish this correspondence. Here, we propose a technique to establish correspondence across cameras using the motion features extracted from the targets, even when the relative position of the cameras is unknown. Experimental results are provided for the problem of tracking multiple bees in natural flight using two cameras. The reconstructed 3D flight paths of the bees show some interesting flight patterns.

A new approach called bioinspired engineering of exploration systems (BEES) and its value for solving pressing NASA and DoD needs are described. Insects (for example honeybees and dragonflies) cope remarkably well with their world, despite possessing a brain containing less than 0.01% as many neurons as the human brain. Although most insects have immobile eyes with fixed focus optics and lack stereo vision, they use a number of ingenious, computationally simple strategies for perceiving their world in three dimensions and navigating successfully within it. We are distilling selected insect-inspired strategies to obtain novel solutions for navigation, hazard avoidance, altitude hold, stable flight, terrain following, and gentle deployment of payload. Such functionality provides potential solutions for future autonomous robotic space and planetary explorers. A BEES approach to developing lightweight low-power autonomous flight systems should be useful for flight control of such biomorphic flyers for both NASA and DoD needs. Recent biological studies of mammalian retinas confirm that representations of multiple features of the visual world are systematically parsed and processed in parallel. Features are mapped to a stack of cellular strata within the retina. Each of these representations can be efficiently modeled in semiconductor cellular nonlinear network (CNN) chips. We describe recent breakthroughs in exploring the feasibility of the unique blending of insect strategies of navigation with mammalian visual search, pattern recognition, and image understanding into hybrid biomorphic flyers for future planetary and terrestrial applications. We describe a few future mission scenarios for Mars exploration, uniquely enabled by these newly developed biomorphic flyers.

Many animals, including insects, successfully engage in visual homing. We describe a system that allows a mobile robot to home. Specifically we propose a simple extension to our original homing scheme which significantly improves its performance by incorporating a richer view of the environment. The addition of landmark apparent-size cues assists homing by providing a more robust homing vector as well as providing a simple and effective method of reinforcing landmark avoidance. The homing algorithm allows a mobile robot to incrementally home by moving in such a way as to gradually reduce the discrepancy between the current view and the view obtained from the home position. Both simulation and mobile robot experiments are used to demonstrate the feasibility of this approach. By matching the bearings of features extracted from panoramic views and using a vector summation technique to compute a homing vector we are able to provide a simple, parsimonious and robust robotic homing algorithm.

Insects such as ants and bees are capable of surprisingly good navigation, despite the small size and relative simplicity of their brains. Recent experimental research in our laboratory, summarized in this chapter, indicates that honeybees estimate the distance travelled to a food site in terms of the image motion that they experience en route. This finding has inspired us to design and build a robot that navigates using a visually-driven odometer.

Animals navigate through various uncontrolled environments with seemingly little effort. Flying insects, especially, are quite adept at manoeuvring in complex, unpredictable and possibly hostile environments. Through both simulation and real-world experiments, we demonstrate the feasibility of equipping a mobile robot with the ability to navigate a corridor environment, in real time, using principles based on insect-based visual guidance. In particular we have used the bees' navigational strategy of measuring object range in terms of image velocity. We have also shown the viability and usefulness of various other insect behaviours: 1) keeping walls equidistant, 2) slowing down when approaching an object, 3) regulating speed according to tunnel width, and 4) using visual motion as a measure of the distance travelled.

A hypothesis is proposed that predicts regions where fixations are most likely to occur on a visual pattern viewed by a human test subject. A scheme is developed for evaluating a fixation probability density function for any given pattern, based on an assumed fixation probability density function for a pattern consisting of a single dot. Predictions of the hypothesis agree with experimental data available in the literature. When the hypothesis is used in conjunction with some assumptions, it also provides interpretations for some well-known geometrical illusions, such as the Muller-Lyer illusion. It is hoped that a study of this kind will aid in understanding some aspects of the mechanisms subserving human visual pattern recognition.