We present a real-time pedestrian detection system based on structure and appearance classification. We discuss several novel ideas that contribute to having low-false alarms and high detection rates, while at the same time achieving computational efficiency: (i) At the front end of our system we employ stereo to detect pedestrians in 3D range maps using template matching with a representative 3D shape model, and to classify other background objects in the scene such as buildings, trees and poles. The structure classification efficiently labels substantial amount of non-relevant image regions and guides the further computationally expensive process to focus on relatively small image parts; (ii)We improve the appearance-based classifiers based on HoG descriptors by performing template matching with 2D human shape contour fragments that results in improved localization and accuracy; (iii) We build a suite of classifiers tuned to specific distance ranges for optimized system performance. Our method is evaluated on publicly available datasets and is shown to match or exceed the performance of leading pedestrian detectors in terms of accuracy as well as achieving real-time computation (10 Hz), which makes it adequate for in-vehicle navigation platform.

This paper presents a real-time egomotion estimation scheme that is specifically designed for measuring vehicle motion from a monocular infra-red image sequence at night time. Conventional methods of estimating camera motion such as optical-flow based method or direct-method that depends on brightness constancy do not work well in relatively low resolution infra-red imagery due to lack of textureness. In this paper, we propose a method that is based on aggregate feature tracking that is constrained by focus of expansion from instantaneous vehicle motion. Qualitative analysis of error is presented by comparing with groundtruth accelerometer data. An application of ground plane estimation combined with vanishing point estimation is also presented for robust pose estimation

This paper describes a framework for constructing a three-dimensional immersive environment that can be used for training physical activities. In the proposed system, an immersive environment is constructed through realistic reconstruction of a scene by applying a stereo algorithm on a set of images that is captured from multiple viewpoints. Specifically, twelve camera clusters that consist of four camera quadruples are used to capture full body human motions, where each camera cluster is processed by a pc independently and synchronously so that they can compute 3d information of the scene in real-time fashion. This paper discusses in detail system architectures that enable synchronous operations across multiple computers while achieving parallel computations within each multi-processor system. A set of experiments are performed to learn taichi lessons in this environment where students are instructed to follow pre-recorded teacher's movements while observing both their own motions and the teacher in real-time from arbitrary viewpoints.

Tele-immersive 3D multi-camera room environments are starting to emerge and with them new challenging research questions. One important question is how to organize the large amount of visual data, being captured, processed, transmitted and displayed, and their corresponding resources, over current COTS computing and networking infrastructures so that "everybody" would be able to install and use tele-immersive environments for conferencing and other activities. In this paper, we propose a novel cross-layer control and streaming framework over general purpose delivery infrastructure, called TEEVE (tele-immersive environments for everybody). TEEVE aims for effective and adaptive coordination, synchronization, and soft QoS-enabled delivery of tele-immersive visual streams to remote room(s). The TEEVE experiments between two tele-immersive rooms residing in different institutions more than 2000 miles apart show that we can sustain communication of up to 12 3D video streams with 4/spl sim/5 3D frames per second for each stream, yielding 4/spl sim/5 tele-immersive video rate.

We build on previous work, on the development of a computer controlled wheelchair equipped with a suite of sensors and a novel interface for human-robot interaction. In this paper, we present experimental results and usability studies for the wheelchair. The architecture for human-robot interaction is hierarchical, with the lowest level corresponding to trajectory control, the intermediate level being behavioral and the highest level involving the composition of behaviors and navigation. Our experimental results illustrate the benefits of a shared-control paradigm where the human operator selects the appropriate behavior(s) or goals while the software is responsible for executing behaviors and generating safe trajectories. Experiments with human users highlight advantages of augmentation in wheelchairs.

This paper presents a sensor based algorithm for guiding a nonholonomic platform, such as a wheelchair, through a doorway. The controller uses information from a camera system and a laser range finder to perform image-based navigation. Simulations of the resultant switching controller are presented along with experimental results. A simple obstacle avoidance algorithm is also implemented on the experimental platform. Finally, we have considered the input of limited field-of-view constraints on this controller. All of these components together lead to a modal, image-based approach that will safely and robustly navigate a nonholonomic robot with sensor constraints through a doorway.

Addresses the problem of human robot interaction with application to the design of assistive devices. We describe the design and development of a prototype of a smart wheelchair that can be commanded by a rider. Specifically, we focus on (a) the vision-based human interaction interface; (b) the suite of sensors on the chair, and (c) the software architecture and the control algorithms used to control the chair.

This paper describes an approach to estimating the trajectory of a moving camera based on the measurements acquired with an inertial sensor and estimates obtained by applying a structure from motion algorithm to a small set of keyframes in the video sequence. The problem is formulated as an offline trajectory fitting task rather than an online integration problem. This approach avoids many of the issues usually associated with inertial estimation schemes. One of the main advantages of the proposed technique is that it can be applied in situations where approaches based on feature tracking would have significant difficulties. Results obtained by applying the procedure to extended sequences acquired with both conventional and omnidirectional cameras are presented.

We present a software architecture used to demonstrate the ability of a tank-like robot to climb multiple flights of stairs. The algorithm we use incorporates intelligent sensor fusion and a hierarchical modular structure. This structure simplifies the overall design, enables reuse, and promotes extensibility. The modular nature of the design also allows for a variety of user interface modes with different levels of user autonomy. The robustness of our algorithm allows us to ascend and descend stairs autonomously in many environments with different textures and configurations, all with minimal adjustment to the algorithm. Our approach develops an extensible sensor and task model that reduces the effect of sensor inaccuracies and geometric constraints, uses minimal information about the environment, and utilizes characteristic features in the environment for navigation.

In this paper the problem of controlling the spatial position and orientation of a robotic platform based on the image data obtained from a video camera mounted on that platform is considered. More specifically, we propose control laws that generate translational and angular velocities that will cause the robot to achieve and maintain a fixed position and orientation with respect to a set of feature points in the scene. The proposed control schemes make use of well established techniques for computing estimates for the relative orientation of two camera positions from a set of feature correspondences. An important advantage of these control schemes is that it is possible to demonstrate analytically that they are globally convergent even in the presence of large calibration errors in both the intrinsic parameters of the camera and in the extrinsic parameters which relate the frame of reference of the camera to the body frame of the robot platform which is being controlled. Furthermore no a priori knowledge about the structure of the scene is assumed.