Algorithms for classifying road signs have a high computational cost per pixel processed. A detection stage that has a lower computational cost can facilitate real-time processing. Various authors have used shape and color-based detectors. Shape-based detectors have an advantage under variable lighting conditions and sign deterioration that, although the apparent color may change, the shape is preserved. In this paper, we present the radial symmetry detector for detecting speed signs. We evaluate the detector itself in a system that is mounted within a road vehicle. We also evaluate its performance that is integrated with classification over a series of sequences from roads around Canberra and demonstrate it while running online in our road vehicle. We show that it can detect signs with high reliability in real time. We examine the internal parameters of the algorithm to adapt it to road sign detection. We demonstrate the stability of the system under the variation of these parameters and show computational speed gains through their tuning. The detector is demonstrated to work under a wide variety of visual conditions.

Cars are becoming, in effect, a robotic system with an embedded human. It is not possible to know what the driver is thinking. We can, however, monitor their gaze and compare it with information in their view-field to make an inference. In this paper we present a complete system that reads speed signs in real-time, compares the driver gaze, and provides immediate feedback if the sign has been missed by the driver. This paper focuses on correlating measures of driver gaze direction with the position of signs in the road scene and improving recognition of signs through image enhancement.

This paper presents a framework for interactive driver assistance systems including techniques for fast speed sign detection and classification, car detection and tracking, and lane departure warning. In addition, the driver's actions are monitored. The integrated system uses information extracted from the road scene (speed signs, position within the lane, relative position to other cars, etc.) together with information about the driver's state such as eye gaze and head pose, to issue adequate warnings. A touch screen monitor presents relevant information and allows the driver to interact with the system. The research is focused around robust on-line algorithms. Initial results of online speed sign detection and car tracking are presented in the context of a driver assistance system.

We have incorporated interactive skills into an active gaze tracking system. Our active gaze tracking system can identify an object in a cluttered scene that a person is looking at. By following the user's 3-D gaze direction together with a zero-disparity filter, we can determine the object's position. Our active vision system also directs attention to a user by tracking anything with both motion and skin color. A Particle Filter fuses skin color and motion from optical flow techniques together to locate a hand or a face in an image. The active vision then uses stereo camera geometry, Kalman Filtering and position and velocity controllers to track the feature in real-time. These skills are integrated together such that they cooperate with each other in order to track the user's face and gaze at all times. Results and video demos provide interesting insights on how active gaze tracking can be utilized and improved to make human-friendly user interfaces.

Smooth control using an active vision head's verge-axis joint is performed through continuous state and action reinforcement learning. The system learns to perform visual servoing based on rewards given relative to tracking performance. The learned controller compensates for the velocity of the target and performs lag-free pursuit of a swinging target. By comparing controllers exposed to different environments we show that the controller is predicting the motion of the target by forming an implicit model of the target's motion. Experimental results are presented that demonstrate the advantages and disadvantages of implicit modelling.

About 1.17 million people die in road crashes around the world each year. It is estimated that up to 30% of these fatalities are caused by fatigue and inattention. There are systems able to detect what is happening outside of the car, e.g., lane tracking, obstacle detection, pedestrian detection etc. Further on, there are also means for monitoring the actions of the driver. A natural step is to fuse the available data from within and outside of the car, and suggest a suitable response. This paper discusses driver assistance systems, lists a set of necessary core competencies of such a system and in particular presents a system for force-feedback in the steering wheel when crossing lanes. The presented system utilises a robust lane tracker which is experimentally evaluated for the purpose of driver assistance. In addition, preliminary results from simultaneous driver monitoring and lane tracking are presented that indicates a good correlation between the two, i.e. the driver's gaze direction and the structure of the road. These data can in turn be used for more advanced driver assistance systems in the future.

A new transform is presented that utilizes local radial symmetry to highlight points of interest within a scene. Its low-computational complexity and fast runtimes makes this method well-suited for real-time vision applications. The performance of the transform is demonstrated on a wide variety of images and compared with leading techniques from the literature. Both as a facial feature detector and as a generic region of interest detector the new transform is seen to offer equal or superior performance to contemporary techniques at a relatively low-computational cost. A real-time implementation of the transform is presented running at over 60 frames per second on a standard Pentium III PC.

In our effort to make human-robot interfaces more user-friendly, we built an active gaze tracking system that can measure a person's gaze direction in real-time. Gaze normally tells which object in his/her surrounding a person is interested in. Therefore, it can be used as a medium for human-robot interaction like instructing a robot arm to pick a certain object a user is looking at. We discuss how we developed and put together algorithms for zoom camera calibration, low-level control of active head, face and gaze tracking to create an active gaze tracking system.

Presents a method for representing places using a set of visual landmarks from a panoramic sensor that allows for accurate local positioning while still providing efficient global localisation. For each place landmarks are selected for their local uniqueness in the panoramic visual field and their dynamic reliability over a turn back and look movement. During this movement, the depth of landmarks is also estimated using a bearing only SLAM approach. Accurate local position tracking within places equal to that of laser range finder systems is obtained by the application of the condensation algorithm over individual places. A topological map of such places is built and both global localisation and position tracking experiments are carried out.

The use of landmarks is a natural and instinctive method to determine the whereabouts of a location or a means to proceed to a particular location. Results provided in this paper indicate that landmark-based navigation possesses a corrective or feedback trait that produces a convergence bound on the movements to the goal position, in contrast to the odometry-based movement, which leads to the drift between successive navigation movements. Experiments show that the vector field approach can be used to explain the convergence property of landmark-based guidance tasks. Experiments have been carried out operating with a Nomad mobile robot equipped with real-time visual landmark tracking system.

Virtual environments provide a whole new way of viewing and manipulating 3D data. Current technology moves the images out of desktop monitors and into the space immediately surrounding the user. Users can literally put their hands on the virtual objects.

Removing suboptimal actions that can exist in a demonstration is a key problem to be solved in robot programming by demonstration. In this paper we present the first step of an approach for solving this problem. We present how the configuration space (C-space) of a task can be derived from demonstration. A demonstration traces out paths on a number of C-surfaces in C-space. The idea is to use statistical regression analysis on data from these paths to determine the unknown equation parameters of a C-surface. Experimental results show the validity of the approach. Accurate parameter estimates were obtained so long as a sufficiently rich set of demonstrated paths existed on the C-surface. The approach has the advantage that it tends to provide accurate parameter estimates for C-surfaces where they were most needed; that is, for C-surfaces (i) critical to task completion, and (ii) whose paths contained suboptimal actions.

Programming by demonstration is a promising approach to automatic robot programming, however, methods are required to remove suboptimal actions that can be demonstrated by end users. In this paper we use the partial knowledge of configuration space (C-space) derived in the previous work by Chen et al. (2000) to remove suboptimal actions from a demonstration. Our idea is to use demonstrated paths to predict what regions in C-space are obstacle free. Suboptimal actions in a demonstration are then avoided by planning alternative actions that pass through the obstacle free regions. Experimental results show the validity of the approach. A demonstrated path containing significant sub-optimality was converted by the approach into a short, efficient path suitable for execution by the robot.

Explores the issues involved in deployment of mobile robots in real-world situations and presents solutions and approaches under development at the Australian National University. For deployment of mobile robots outside of the laboratory, long-term operation is required. Hence, we have developed an automatic recharging system. In addition, a Web-based teleoperation system is used to provide missions to test the long-term reliability of the robot. The final aspect of real-world operation that is explored here is operations in dynamic environments. To date, researchers have assumed static environments for mapping and localisation. We propose methods to avoid this restriction.

We are developing a visually-guided autonomous underwater vehicle. We have achieved a position-based visual servo control of fixed and slow moving targets using visual position feedback and sensor-based orientation feedback. The visual position feedback has been implemented on a stereo camera system. We use a compass and an inclinometer for orientation feedback. We have also implemented a computed torque controller using Euler parameters to represent the orientation state, for vehicle motion control. Using Euler parameters eliminates singularities in the model and the controller. Preliminary experimental results of visual servo control are reported. 1

To build smart human interfaces, it is necessary for a system to know a user's intention and point of attention. Since the motion of a person's head pose and gaze direction are deeply related with his/her intention and attention, detection of such information can be utilized to build natural and intuitive interfaces. We describe our real-time stereo face tracking and gaze detection system to measure head pose and gaze direction simultaneously. The key aspect of our system is the use of real-time stereo vision together with a simple algorithm which is suitable for real-time processing. Since the 3D coordinates of the features on a face can be directly measured in our system, we can significantly simplify the algorithm for 3D model fitting to obtain the full 3D pose of the head compared with conventional systems that use monocular camera. Consequently we achieved a non-contact, passive, real-time, robust, accurate and compact measurement system for head pose and gaze direction.

Computer systems which analyse human face/head motion have attracted significant attention recently as there are a number of interesting and useful applications. Not least among these is the goal of tracking the head in real time. A useful extension of this problem is to estimate the subject's gaze point in addition to his/her head pose. This paper describes a real-time stereo vision system which determines the head pose and gaze direction of a human subject. Its accuracy makes it useful for a number of applications including human/computer interaction, consumer research and ergonomic assessment.

This paper introduces a theory to formally and practically analyze the robustness issues of visual guidance methods for robot navigation. The first aspect is related to the convergence of the navigation system to the goal. It is shown how the dynamic system which drives the strategies can be analyzed by using classical concepts such as the Lyapunov functions. The second aspect concerns the conservativeness of the resulting navigation vector fields. It is shown how this deals with the repeatability of the trials. Furthermore, the selection of the best landmarks to perform the navigation processes strongly affects the conservativeness thus providing a formal way to do landmark learning. The theory has been tested with two different visual methods that have been derived from the biological world: the snapshot model and the landmark model.

Biology often offers valuable example of systems both for learning and for controlling motion. Work in robotics has often been inspired by these findings in diverse ways. Though the fundamental aspects that involve visual landmark learning and motion control mechanisms have almost exclusively been approached heuristically rather than examining the underlying principles. In this paper we introduce theoretical tools that might explain how the visual learning works and why the motion is attracted by the pre-learnt goal position. Basically, the theoretical tools emerge from the navigation vector field produced by the visual behaviors. Both the learning process and the navigation scheme influence the motion field. We apply classical mathematical and dynamic control to analyze the efficiency of our method.

Reinforcement learning systems improve behaviour based on scalar rewards from a critic. In this work vision based behaviours, servoing and wandering, are learned through a Q-learning method which handles continuous states and actions. There is no requirement for camera calibration, an actuator model, or a knowledgeable teacher. Learning through observing the actions of other behaviours improves learning speed. Experiments were performed on a mobile robot using a real-time vision system.

To build smart human interfaces, it is necessary for a system to know a user's intention and point of attention. Since the motion of a person's head pose and gaze direction are closely related to his/her intention and attention, detection of such information can be utilized to build natural and intuitive interfaces. In this paper, we describe a behavior recognition system based on the real-time stereo face tracking and gaze detection system to measure head pose and gaze direction simultaneously. The key aspect of our system is the use of real-time stereo vision together with a simple algorithm which is suitable for real-time processing. Our system we can significantly simplify the algorithm for 3D model fitting to obtain the full 3D pose of the head compared with conventional systems that use monocular camera. The recognition of attentions and gestures of a person is demonstrated in the experiments.

Presents a biologically-inspired method for navigating using visual landmarks which have been self-selected within natural environments. A landmark is a region of the grabbed image which is chosen according to its reliability measured through a phase (turn back and look) that mimics the behavior of some social insects. From the self-chosen landmarks suitable navigation information can be extracted following a well known model introduced in biology to explain the bee's navigation behavior. The landmark selection phase affects the conservativeness of the navigation vector field thus allowing us to explain the navigation model in terms of a visual potential function which drives the navigation to the goal. The experiments have been performed using a Nomad200 mobile robot equipped with monocular color vision.

We have been developing an autonomous robotic agent that helps people in a real world environment, such as in an office. When a robotic agent works by cooperating with a person in a real world environment, it must manage a lot of information and deal with the knowledge and languages that people usually use. Therefore it is important for the agent to recognize what people request as soon as possible. To realize common communication with people, the agent should provide robust communicative functions to obtain information from people. We discuss communicative functions of our robotic agent called Jijo-2. Especially we focus on the problem of detecting human faces, and discuss how a method of detecting a human face can be robustly archived.

Introduces an approach to the control of robot manipulators in a way that is safe for humans in the robot's workspace. Conceptually the robot is viewed as a tool with limited autonomy. The limited perception capabilities of automatic systems prohibits the construction of failsafe robots with the capabilities of people. Instead, the goal of our control scheme is to make the interaction with a robot manipulator safe by making the robots actions predictable and understandable to the human operator. At the same time the forces the robot applies with any part of its body to its environment have to be controllable and limited. Experimental results are presented of a human-friendly robot controller that is under development for a Barrett Whole Arm Manipulator robot.

According to the behaviour-based philosophy, the structure of an agent's internal representations of the environment should not be explicitly imposed by the designer; they should be grounded in its sensor-action space. The paper presents a scheme in which the agent's action selection mechanism gives rise to an integrated spatial and topological navigation and mapping capability. The navigation behaviour emerges from the notion of location feature detectors and homogeneous action selection. The scheme is demonstrated using two autonomous mobile robots in a multi-robot cooperation scenario.

We describe a mobile robot system that performs goal-oriented visual navigation using a behaviour-based architecture. The system was constructed as a result of our previous research (Zelinsky et al. (1995)) into behaviour-based mobile robot systems. Since the system is implemented using real-time vision it is able to navigate in dynamic and unknown environments. The most important feature of this navigation system is that the mobile robot can learn to resolve conflicts between its own internal behaviours, such as moving to a goal while avoiding obstacles. Our robot is able to efficiently avoid obstacles while trying to reach a goal. Experimental results of an implementation on a Yamabico mobile robot are presented.

This paper describes the implementation of behaviour for real-time visual servoing on a mobile robot. The behaviour is a component of a multi-robot cleaning system developed in the context of our investigation into architectures for cooperative systems. An important feature for support of cooperation is the awareness of one robot by another, which this behaviour realises. Robust feature tracking aided by a hardware vision system is described. This forms the basis for range and pose estimation using a 3D projective model.

We describe an unsupervised robot learning method which is based on the robot learning a mapping between sensors and trajectory velocities. This enables the robot to acquire object avoidance, wall following and goal seeking behaviors simultaneously without incurring the credit assignment problem. To improve the robot's perception and behaviors we provide the robot with 7 fuzzy associative matrices (FAMs) so that sensors can be mapped to each trajectory independently. We provide results demonstrating how a mobile robot equipped with 16 sonar sensors is able to achieve improved perception and behaviors by using 7 FAMs to map sensors to trajectories.

The increase in hardware performance versus component cost has brought vision firmly into the realm of practical robotic sensors. We present an overview of an integrated research program to create a general-purpose robotic vision system from an expansive rather than linear perspective, incorporating a multimodal approach to real-time visual interaction with the environment. Design and construction of a high-performance active camera platform using primarily off-the-shelf components has proceeded in parallel with algorithm design and implementation towards a suite of low-level behaviours suitable for feeding quantised data to a multitude of visually guided tasks, from mobile robot navigation and visual servoing to human-robot interaction and instruction. We describe these initiatives, including performance specifications and experimental results obtained under real-world conditions.

In this paper we propose a new paradigm for teleoperating a mobile robot. Our teleoperation paradigm is made up of five major components: "self-preservation"; "instructive feedback"; "qualitative instructions"; "qualitative explanations"; and a "user interface". Our aim is to provide a mobile robot with autonomy while being teleoperated. Our approach can be used to combat the continuous closed-loop problem. A qualitative approach has been taken in the design of each of the components. The usefulness of our approach is demonstrated with an implementation of a user interface to teleoperate an autonomous mobile robot equipped with a vision based navigation system. In this paper we describe the implementation of our teleoperation system.

People naturally express themselves through facial gestures and expressions. Our goal is to build a facial gesture human-computer interface for use in robot applications. We have implemented an interface that tracks a person's facial features in real time (30 Hz). Our system does not require special illumination nor facial makeup. By using multiple Kalman filters we accurately predict and robustly track facial features. This is despite disturbances and rapid movements of the head (including both translational and rotational motion). Since we reliably track the face in real-time we are also able to recognise motion gestures of the face. Our system can recognise a large set of gestures (13) ranging from "yes", "no" and "may be" to detecting winks, blinks and sleeping.

We present an approach for using vision as the primary source of sensing to guide a mobile robot in an unknown environment. We define a set of primitive visual behaviours for navigating a mobile robot in real-time. By combining such behaviours with a purposive map, our mobile robot exhibits a goal seeking behaviour. We present a fast segmentation technique for vision processing. This processing technique is used by different behaviours to produce an overall competent behaviour in our Yamabico robot. Experimental results show that our robot can navigate competently in dynamic indoor environments.

In this paper we describe the design and implementation of a unique proportional whisker sensor and our general behaviour based software architecture. The architecture for behaviour based agents is used to realise purposive mobile robot navigation in a cluttered indoor environment. The whisker was developed specifically to meet the requirements of high speed and close wall following. We also discuss the specific architecture we have constructed.

Presents a scheme for specifying and executing purposive navigation tasks for a behaviour-based mobile robot. A user specifies the robot's navigation task in general and qualitative terms using a graphical resource called the purposive map (PM). The robot navigates using the incomplete and approximate information stored in the PM as an aid to achieve the specified mission. The robot is able to augment the knowledge provided in the PM with environment information learnt by the robot. Using the augmented PM, the robot learns how to perform efficient obstacle avoidance. The authors present experimental results using a real robot to show their scheme is robust. The authors' robot can escape from dead-ends, can deduce that goals are unreachable and can withstand disturbances to the environment between missions.

This paper presents a new scheme of purposive navigation for mobile agents. The new scheme is robust, qualitative and provides a mechanism for combining mapping, planning and mission execution for a mobile agent into a single data structure called the purposive map (PM). The agent can navigate using incomplete and approximate information stored the PM. We present a novel approach to perform obstacle avoidance for a behaviour based robot. Our approach is based on using a physically grounded search while monitoring and co-ordinating behaviours. The physically grounded search exploits stagnation points (local minima) to guide the search for the shortest path to a target. This scheme enables our robot to escape from dead-end situations and allows it to deduce that a target location is unreachable. Simulation results are presented.>

An algorithm for path planning to a goal with a mobile robot in an unknown environment is presented. The robot maps the environment only to the extent necessary to achieve the goal. Mapping is achieved using tactile sensing while the robot is executing a path to the specified goal. Paths are generated by treating unknown regions in the environment as free space. As obstacles are encountered en route to a goal, the model of the environment is updated and a new path to the goal is planned and executed. Initially the paths to the goal generated by this algorithm will be negotiable paths. However, as the robot acquires more knowledge about the environment, the length of the planned paths will be optimized. The optimization criteria can be modified to favor or avoid unexplored regions in the environment. The algorithm makes use of the quadtree data structure to model the environment and uses the distance transform methodology to generate paths for the robot to execute.>