Although RANSAC is the most widely used robust estimator in computer vision, it has certain limitations making it ineffective in some situations, such as the motion estimation problem, in which uncertainty on the image features changes according to the capturing conditions. The greatest problem is that the threshold used by RANSAC to detect inliers cannot be changed adaptively; instead it is fixed by the user. An adaptive scale algorithm must therefore be applied in such cases. In this paper, we propose a new adaptive scale robust estimator that adaptively finds the best solution with the best scale to fit the inliers, without the need for predefined information. Our new adaptive scale estimator matches the residual probability density from an estimate and the standard Gaussian probability density function to find the best inlier scale. Our algorithm is evaluated in several motion estimation experiments under varying conditions and the results are compared with several of the latest adaptive-scale robust estimators.

In this paper, we propose a novel learning-based video super resolution algorithm with less memory requirements and computational cost. To this end, we adopt discrete cosine transform (DCT) coefficients for feature vector components. Moreover, we design an example selection procedure to construct a compact database. We conducted evaluative experiments using MPEG test sequences to synthesize a high resolution video. Experimental results show that our method can improve effectiveness of super-resolution algorithm, while preserving the quality of synthesized image.

We propose a new egomotion estimation algorithm for a compound omnidirectional camera. Image features are detected by a conventional feature detector and then quickly classified into near and far features by checking infinity on the omnidirectional image of the compound omnidirectional sensor. Egomotion estimation is performed in two steps: first, rotation is recovered using far features; then translation is estimated from near features using the estimated rotation. RANSAC is used for estimations of both rotation and translation. Experiments in various environments show that our approach is robust and provides good accuracy in real-time for large motions.

Image based modeling methods has been well studied for generating a 3D model from an image sequence. Most of them require redundant and huge spatio-temporal images for estimating a scene depth. It is not good characteristic for taking a higher resolution of texture. A route panorama is a continuous panoramic image along a path. It is suitable for modeling large environments such as a city or town. The panorama captured by a line scan sensor also has advantage for capturing higher resolution easily. In this paper, we propose a method for depth estimation from the panorama. The route panorama has color drifts that correspond to the distances of captured objects. We use these color drifts to estimate the depth of an image. The proposed method detects the color drift by window matching using belief propagation. It also uses a Gaussian pyramid to stabilize the estimation and decrease its computation cost. We confirmed that the proposed method estimated depth maps from a single high-resolution panorama in experiments.

Many omnidirectional vision sensors based on convex mirrors have been proposed for observing a 360 degrees field of view. Some have a single viewpoint using a hyperboloidal or parabolic mirror, others have a constant resolution property. Until now, there have not been any that have had both these properties. In this paper, we propose an omnidirectional vision sensor that has both properties; that is, a single viewpoint and a constant resolution. The proposed omnidirectional sensor uses two mirrors, which improves the degree of freedom of the design for satisfying each property. We discuss optimization of the design in terms of geometry and optics.

Camera ego-motion consists of translation and rotation, in which rotation can be described simply by distant features. We present a robust rotation estimation using distant features given by our compound omnidirectional sensor. Features are detected by a conventional feature detector, and then distant features are identified by checking the infinity on the omnidirectional image of the compound sensor. The rotation matrix is estimated between consecutive video frames using RANSAC with only distant features. Experiments with various environments show that our approach is robust and also gives reasonable accuracy in real-time.

In recent years, many different image features have been used for face recognition. The Gabor wavelet feature is the most widely used image feature in face recognition systems because its recognition rate is superior to that of Eigenface systems. However, it remains unclear as to whether Gabor wavelet features are indeed the best wavelet features for face recognition. In this paper, we extract image features of facial images from various wavelet transforms (e.g., Haar, French hat, Mexican hat, Daubechies, Coiflet, Symlet, and O-spline) and evaluate their face recognition performance. We also compare the recognition performance of fixed- and adaptive-scale wavelet features. The results demonstrate that the performance of the wavelets assessed here is similar to that of the Gabor wavelet, and that the performance of adaptive-scale wavelet features is superior to that of fixed-scale features

Many omnidirectional vision sensors using a convex mirror have been proposed for observing a 360 degrees field of view. Some of them have a single viewpoint using a hyperboloidal or parabolic mirror, some of the others have a constant resolution property. Until now, there have not been any that have had both these properties. In this paper, we propose an omnidirectional vision sensor that has both a single viewpoint and a constant resolution. The proposed omnidirectional sensor uses two mirrors, which improves the degree of freedom of the design for satisfying each property. We introduce three practical settings of constant resolution; angular constant, horizontal constant and vertical constant resolutions

In accordance with advances in camera technology, the requirement of high-quality video has greatly increased. Among the factors required for high-quality video are a high-resolution and a high frame rate. However, the limitation of the pixel transfer rate restricts compatibility of a high-resolution and a high frame rate in commercial camera. We propose a dual-sensor camera that consists of two cameras: one with a high-resolution and a low frame rate and the other with a low-resolution and a high frame rate. The system is capable of capturing two different image sequences: high-resolution images and high-frame-rate images. A sensor calibration for the dual-sensor camera is also proposed.

A real-time omnidirectional imaging system that can acquire an omnidirectional field of view at video rate using a convex mirror was applied to a variety of conditions. The imaging system consists of an isotropic convex mirror and a camera pointing vertically toward the mirror with its optical axis aligned with the mirror's optical axis. Because of these optics, angular resolution is independent of the azimuth angle. However, it is important for a mobile robot to find and avoid obstacles in its path. We consider that angular resolution in the direction of the robot's moving needs higher resolution than that of its lateral view. In this paper, we propose a non-isotropic omnidirectional imaging system for navigating a mobile robot.

A route navigation method for a mobile robot with an omnidirectional image sensor is described. The route is memorized from a series of consecutive omnidirectional images of the horizon when the robot moves to its goal. While the robot is navigating to the goal point, input is matched against the memorized spatio-temporal route pattern by using dual active contour models and the exact robot position and orientation is estimated from the converged shape of the active contour models.

As the Internet becomes the basic resource of information, not only texts but image retrieval systems have appeared. However, many supply only a list of images, so we must seek desired images one by one. Image labeling is one solution to such a problem, in which various words are labeled to an image when extracted from a single Web page. Therefore, this paper proposes an image clustering system that labels images by words related to a search keyword. Relationships are measured by Web pages in WWW. In experiments, users found the intended images faster than ordinary image search system.

A route navigation method for a mobile robot with an omni-directional image sensor is described. The route is memorized from a series of consecutive omni-directional images at the horizon when the robot moves to the goal. While the robot is navigating to the goal point, the input is matched against the memorized spatio-temporal route pattern by using dual active contour models and the exact robot position and orientation is estimated from the converged shape of active contour models. In this paper, we evaluated the precision of the localization of the robot under several different moving conditions and environments.

Many applications have used a head-mounted display (HMD), such as in virtual and mixed realities and telepresence. However, the field of view (FOV) of commercial HMD systems is too narrow for feeling immersion. In this paper, we propose a super-wide field of view head-mounted display consisting of an ellipsoidal mirror and a hyperboloidal curved mirror. The horizontal FOV of the proposed HMD is 180 degrees and includes the peripheral vision of humans. It increases the reality and immersion for users.

A route navigation method for a mobile robot with an omni-directional image sensor is described. The route is memorized from a series of consecutive omnidirectional images at the horizon when the robot moves to the goal. While the robot is navigating to the goal point, the input is matched against the memorized spatio-temporal route pattern by using dual active contour models and the exact robot position and orientation is estimated from the converged shape of active contour models.

Many applications have used a Head-Mounted Display (HMD), such as in virtual and mixed realities, and tele-presence. The advantage of HMD systems is the ease of feeling a 3D world in the display of animation or movies. However, the field of view (FOV) of commercial HMD systems is too narrow for feeling immersion. The horizontal FOV of many commercial HMDs is around 60 degrees, significantly narrower than that of humans. In this paper, we propose a super wide field of view catadioptrical head-mounted display consisting of an ellipsoidal and a hyperboloidal curved mirror. The horizontal FOV of the proposed HMD is 180 degrees and includes the peripheral view of humans. It increases reality and immersion of users. As well, the central region (60 degrees) of the FOV can measure 3D distances using stereoscopics.

Many applications have used a Head-Mounted Display (HMD), such as in virtual and mixed realities, and tele-presence. The advantage of HMD systems is the ease of feeling a 3D world in the display of animation or movies. However, the field of view (FOV) of commercial HMD systems is too narrow for feeling immersion. The horizontal FOV of many commercial HMDs is around 60 degrees, significantly narrower than that of humans. In this paper, we propose a super wide field of view head-mounted display consisting of an ellipsoidal and a hyperboloidal curved mirror. The horizontal FOV of the proposed HMD is 180 degrees and includes the peripheral view of humans. It increases reality and immersion of users. As well, the central region (60 degrees) of the FOV can measure 3D distances using stereoscopics.

Recently, many virtual reality and robotics applications have been called on to create virtual environments from real scenes. A catadioptric omnidirectional image sensor composed of a convex mirror can simultaneously observe a 360-degree field of view making it useful for modeling man-made environments such as rooms, corridors, and buildings, because any landmarks around the sensor can be taken in and tracked in its large field of view. However, the angular resolution of the omnidirectional image is low because of the large field of view captured. Hence, the resolution of surface texture patterns on the three-dimensional (3-D) scene model generated is not sufficient for monitoring details. To overcome this, we propose a high resolution scene texture generation method that combines an omnidirectional image sequence using image mosaic and superresolution techniques.

Recently, many applications in virtual reality and robotics require to create virtual environment from a real scene. A catadioptric omnidirectional image sensor composed of a convex mirror can observe a 360-degree field of view at once. It is useful for modeling a man-made environment such as a room, a corridor and a building, because the landmarks around the sensor can be taken and tracked by its large field of view. However, the angular resolution of the omnidirectional image is low owing to capturing the large field of view. Therefore, the resolution of texture patterns of each surface on the generated 3D scene model is not enough for monitoring details. To solve this problem, we propose a high-resolution scene texture generation method that combines an omnidirectional image sequence using image mosaic and super-resolution techniques.

Face is one of the most attractive information for personal identification. In this paper, we propose the face identification method from an omnidirectional image sequence. Since the omnidirectional image sensor, HyperOmni Vision, observes a 360-degree view around the robot, it can observe a global azimuth information of the person (face). It tracks the human face while the person walks around the camera. Under an assumption of smooth human motion, we identify the corresponding person from facial database.

The omnidirectional image sensor named HyperOmniVision, is composed of a hyperboloidal mirror and conventional video camera. It can observe a 360 degree field of view and can transform an input image to a perspective image. However, it has an intrinsic problem where the image resolution of HyperOmniVision is lower than that of an ordinary video camera, because it has a structure whereby only one CCD captures whole periphery scene. We propose a resolution improvement method using sub-pixel displaced and multi-focused images.

The paper deals with the detection of human faces in an image and recognition of their facial expressions using the potential net. Previously, we proposed a method to recognize facial expression using the potential net and KL expansion. In this method, it was necessary to assign a face area manually. This manual process will affect the application of recognition of facial expression to man-machine interaction. In this paper, we propose a method to automatically detect human faces by pattern matching using the potential net and to recognize facial expressions of the detected face area.

We propose a method to solve the general problem of road tracking and 3D-shape reconstruction for a smart vehicle. The method assumes that the road boundaries are parallel and that the width of the road is constant. We then detect and track the road region in the image using active contour models subject to a parallelism constraint. The system then generates a 3D-road model from a single image. We evaluate the effectiveness of the method by applying to real road scenes comprising more than 2000 images.

We (1995) have proposed an omnidirectional image sensor called HyperOmni Vision. This sensor can acquire omnidirectional and perspective images in real time. In this paper we present a new Hough transform method for an omnidirectional image by using a cubic Hough space. Furthermore, we describe a method for reconstructing 3D line segments from an omnidirectional input image sequence obtained by HyperOmni Vision with known motion.

Generation of a stationary environmental map is one of the important tasks for vision based robot navigation. Observational errors in the generated environmental map accumulate in long movements of the robot. To generate a large environmental map, it is desirable not to assume known robot motion. In this paper, under the assumption of unknown translational motions of the robot, we propose a method to generate a stationary environmental map, and estimate the egomotion of a robot in a dynamic environment by using an omnidirectional image sensor. Since both robot and objects move in the environment, the stationary map generation and the robot egomotion estimation by using a single camera are difficult because of correspondence ambiguity caused by occlusion. The proposed method can detect a moving object and find occlusion and mismatching by evaluating the estimation error of each object location.

Under the assumption of known motion of a robot, environmental maps of a real scene can be successfully generated by monitoring azimuth changes in an image. Several researchers have used this property for robot navigation. However, it is difficult to observe the exact motion parameters of the robot because of encoder measurement error. Therefore, observational errors in the generated environmental map accumulate in long movements of the robot. To generate a large environmental map, it is desirable not to assume known robot motion. In this paper, under the assumption of unknown motions of the robot, we propose a method to generate an environmental map and estimate the egomotion of a robot, by using an omnidirectional image sensor.

Most Internet users use search engines to find information according to their intentions. However, this approach excludes serendipity; that is, opportunities exist on other routes from the most direct. We propose a system to assist Internet surfing which employs a search engine. This system reminds users of potential interest by supplying keywords related to search keywords. As many of words do not follow from the user's thought processes, the user can discover words which express their potential interest and find information using these keywords. Associations of the keywords are formed from relationships between newly supplied keywords and newly entered search keywords. Therefore, users can take pleasure in their surfing and follow many unanticipated opportunities.

Sensing of human motion is very important for human-computer interactive applications such as virtual reality, gesture recognition, and communication. A vision system is suitable for human-computer interaction since this involves passive sensing and the system can estimate the motion of the user without any discomfort for the user. In this paper, we propose an algorithm for fast posture estimation of the user from an image sequence by using the position information of the hands and head. Our algorithm is founded on a model based method. The parameters of a geometric model are calculated from human kinematics. It is inadequate to obtain a unique solution from the image information such as position of the head and hands. The unknown parameters are predicted by the motion models and the previous posture parameters, then the unknown parameters are adjusted by a minimization method.

Described here is a visual navigation method for navigating a mobile robot along a man-made route such as a corridor or a street. We have proposed an image sensor, named HyperOmniVision, with a hyperboloidal mirror for vision based navigation of the mobile robot. This sensing system can acquire an omnidirectional view around the robot in real time. In the case of the man-made route, road boundaries between the ground plane and wall appears as a close-looped curve in the image. By making use of this optical characteristic the robot can avoid obstacles and move along the corridor by tracking the close-looped curve with an active contour model. Experiments that have been done in a real environment are described.

Most Internet users use search engines to get information on the WWW. However, users cannot be content with the output of search engines because it's hard to express the user's own interest as a search query. Therefore, we propose a system which provides users with keywords related to user interest. Users can recall keywords expressing user's hidden (not described in a search query) interest as search keywords. The system includes an interface to restructure a part of the user's search query. Users can compose a new search query expressing concrete user interest with the interface. Previous refinements are simple modifications of the search queries, for example, adding narrow keywords or relational keywords. However, the restructuring described in the paper includes replacement of the search keywords.

This paper describes a new method to detect faces in color images based on the fuzzy theory. We make two fuzzy models to describe the skin color and hair color, respectively. In these models, we use a perceptually uniform color space to describe the color information to increase the accuracy and stableness. We use the two models to extract the skin color regions and the hair color regions, and then comparing them with the prebuilt head-shape models by using a fuzzy theory based pattern-matching method to detect face candidates.

An automatic face and facial feature points (FFPs) detection system is proposed. A face is represented as a graph where the nodes are placed at facial feature points (FFPs) labeled by their Gabor features and the edges describe their spatial relations. An innovative flexible feature matching is proposed to perform features correspondence between models and the input image. This matching model works like a random diffusion process in the image space by employing the locally competitive and globally corporative mechanism. The system works nicely on the face images under complicated background, pose variations and distorted by facial accessories. We demonstrate the benefits of our approach by its implementation on a face identification system.

We propose a multiple-view-based tracking algorithm for multiple-human motions. In vision-based human tracking, self-occlusions and human-human occlusions are a part of the more significant problems. Employing multiple viewpoints and a viewpoint selection mechanism, however can reduce these problems. In our system, human positions are tracked with a sequence of multiple-viewpoint images. This tracking is based on the Kalman filtering approach. The estimation results are utilized to select proper viewpoints in other sub-tasks (rotation angle detection and body-side detection). Each sub-task has a different criterion for selecting viewpoints. We also describe the criterions for accomplishing individual sub-tasks and relationships between sub-tasks. We have already built an experimental system based on a small number of reliable image features. We confirm the stability of our algorithm through simulations. We also performed fundamental examinations on the experimental system.

Describes a method for recognition of continuous facial expression change from time-sequential images. The method is composed of two parts, (1) facial feature tracking by using a labeled graph matching with weighted links and (2) facial expression recognition by making a comparison between the trajectory of features and the expression change model. The expression change model is constructed by using B-spline curves, which represent the relationship between the motion of facial features and the change of expression. Not only the type, but also the degree of expression change can be recognized. Furthermore, the obtained expressional information is then fed back to guide the tracking in the next frame.

Presents a real time gesture recognition method for interactive systems which can perform an interaction like human communication. This method recognizes a user's gesture and the degree information of gesture using the eigenspace constructed from multi input image sequences. This eigenspace represents an approximate 3 dimensional information of gesture, so that complicated gestures which may have a self-occlusion and confusion in a single input image can be recognized correctly. Using our method, we realize a real time interactive system, the virtual conductor system, which indicates the usefulness of our method.

In this paper, we propose a facial contour extraction model. The influence of environmental changes on the model is small, and it is based on an active contour model (ACM). Our method has three characteristics: global shape constraint of axis-symmetry, dual-scale filtering and iterative initialization. We apply the method to real human faces of more than 60 people with our face recognition system for evaluating effectiveness.

In this paper, we propose a new fusion approach which uses the ultrasonic sensor aided by an omnidirectional vision sensor to give a grid based free space around the robot. By use of the ultrasonic sensor, the robot can obtain a conservative range information based on our nearby range filtering method. This filtering can give a more reliable result considering the sensor's problem of specular reflection. Also, by use of the special omni-directional vision sensor we developed, the color and edge information can be obtained in a single picture and mapped to the ground plane by the inverse perspective transformation. Thus the range, color and edge information can all be expressed on a metric grid-based representation which forms the basis of our radial fusion processing. Results in an indoor cluttered environment are given which show the usefulness of our proposed sensor fusion approach.

Described here is a route navigation method for a mobile robot with an omnidirectional image sensor. The route is memorized by a series of two dimensional Fourier power spectra of consecutive omnidirectional images at the horizon while the robot moves to the goal position. While the robot is navigating to the goal point, it is controlled by comparing the principal axis of inertia of its current position with that of the memorized Fourier power spectrum.

The purpose of this study is not only to recognize a facial expression which is associated with human emotion but also to estimate its degree. Our method is based on the idea that facial expression recognition can be achieved by extracting a variation from an expressionless face considering the face area as a whole pattern. For the purpose of extracting subtle changes in the face such as the degree of expressions, it is necessary to eliminate the individuality appearing in the facial image. Using an elastic net model, a variation of facial expression is represented as motion vectors of the deformed net from a facial edge image. Then, applying K-L expansion, the change of facial expression represented as the motion vectors of nodes is mapped into a low dimensional eigenspace, and estimation is achieved by projecting input images on to the emotion space. We have constructed three kinds of expression models; happiness, anger, surprise, and experimental results are evaluated.

This paper proposes a method to recognize human gestures from an image sequence based on hidden Markov model and subspace method. As our method uses the motion vector field of the scene for recognition, it is robust for variety in the background of the scene and it doesn't require the users to wear a sensor or a marker. The motion vector field of the scene is projected to an eigensubspace and is used as the input symbols for the HMM.

This paper presents a method of real time gesture recognition from image sequences. Real time gesture recognition is required for man machine interfaces and many multimedia applications. Our method recognizes a user's gesture in a low dimensional 'gesture space' which is constituted by Karhunen-Loeve (KL) expansion of model image sequences. We first segment a human part image in real time using the maskable template model which is an improved template matching method for gesture recognition. The segmented image sequence is represented as a gesture curve in the gesture space, and gesture is recognized by analyzing the curve. Using our method, we realize a real time interactive system, the Virtual Conductor System, which can control music played by a computer using gesture recognition results and indicates the usefulness of our method.

We design a new multiple image sensing sensor (MISS), which consists of a single camera and two different types of optics: an omnidirectional image sensor COPIS (conic projection image sensor) and binocular vision, for navigating the robot and understanding interesting objects in an environment by integrating both sensory data. Since COPIS observes a 360 degree view around the robot, it can observe global information of features (azimuths of vertical edges). On the other hand, binocular vision can obtain a sequence of stereo images although a field of view is limited by the visual angle of lens. In this paper, we integrate merits of omnidirectional vision and binocular vision, and propose an efficient sensing system for generating an environmental map by using MISS. Without an assumption of the exact known motion of the robot, MISS can generate an environmental map and estimate egomotion of the robot from both sensory information sources. In particular, the method is suitable for map generation during long robot movement because deadlocking error of the robot does not accumulate. The system has been evaluated on a prototype of MISS in a real environment.

To realize a speed up in learning without homogenizing the agents' behaviors in a multi-agent system, it is important to selectively share learning results. This paper describes a method designed to permit multiple agents to learn cooperatively. The advantage of our method is to dynamically switch the learning mode between mimetism and reinforcement learning according to the situation. Mimetism seeks stability in its behavior, while individual reinforcement leaning seeks the better solution. Accordingly, selective mimetism that allows the agents to partially share learning results,works to prevent homogenization among the agents. Experimental results are given for a ball-pushing task between the two virtual agents for evaluating the effectiveness of our method. This method will be useful for cooperative reinforcement learning with adaptive mimetism based on propagating the learned behaviors of a virtual agent to a physical robot in order to accelerate leaning in a physical environment.

This paper presents an automatic processing of human face from color images. The system works hierarchically from detecting the position of human face and its features (such as eyes, nose, mouth, etc.) to contours and feature points extraction. The position of human face and its parts are detected from the image by applying the integral projection method, which synthesize the color information (skin and hair color) and the edge information (intensity and sign). In order to extract the contour-line of face features we used a multiple active contour model with color information based energy terms. Facial feature points are decided based on the optimized contours. The proposed system is confirmed to be very effective and robust to deal with the image of faces in the complex background.

This paper proposes a method to recognize hand gestures from a monocular image. We use colored gloves to detect specific region of hand easily. By using this method, we can treat easily the occlusion problem due to color information. Hand gesture are recognized by the decision tree made from the image features automatically by our proposed method. There are another pattern recognition methods using image features like nearest-neighbor method. But it is better for computation time to use the decision tree because the decision tree uses only few selected important features. In this paper, we show a learning result by the CAD model and recognition results in real images by our proposed method.

This paper presents a new method for real-time gesture recognition and its application. The key idea of the method is that we use multiple template models for specific feature parts of body such as arms and estimate the pose of this part by considering similarities of an input image to the models. To calculate the similarity for the part, we developed maskable template model (MTM) with which both masking and template matching are done simultaneously. As an application of this method, we realize a real-time interactive system, gesture game system, which can activate the characters of video game by gestures in real-time.

This paper describes a fuzzy theory based face detection system. We use a perceptually uniform chromatic system to represent color information for increasing the robustness of our system. We build two models that describe the skin color and hair color, and use them to estimate the skin color likeness and the hair color likeness. We treat the appearance of faces in images as a combination of the skin part and the hair part, and model it with several two-dimensional patterns. Then we detect the "face like" regions by finding out patterns that are similar to one of the face models from input images using the fuzzy pattern matching method. Extensive experiments show the effectiveness of our method in the face detection from images with many people and complex background.

The outputs of many face detection systems are face candidates that may contain some false faces. This paper describes an algorithm to verify the face candidates. Once a face candidate is detected from an image, the positions and the sizes of the facial features on the face are predicted based on the knowledge about the arrangement of the facial features on a human face. Then the facial features are detected from the predicted positions with a coarse to fine approach. The face candidates are verified by considering whether facial features can be extracted and how well they match with a relational face model that describes the geometric relationship among the facial features of a generic human face.

This paper presents an approach to obtain optimal images for face recognition with an active camera. Once the face is detected from an input image, the 3-dimensional position and the pose of the face relative to the camera are estimated by making use of the active camera and both the skin and hair parts of the face extracted from input images. It is then used to guide the active camera system to change its view point and direction to obtain a face image where the face is in the desired size and pose.

Described here is a method for estimating a rolling motion of a mobile robot from optical flows. We have proposed an image sensor with a hyperboloidal mirror for vision based navigation of the mobile robot. Its name is HyperOmniVision. This sensing system can acquire an omnidirectional view around the robot, in real-time, with use of the hyperboloidal mirror. The radial component of optical flow in HyperOmniVision has a periodic characteristic. The proposed method makes use of this characteristic to estimate robustly the rolling motion of the robot.

The factorization method, first developed by Tomasi and Kanade (1992), recovers both shape and motion from a sequence of images, by tracking a large number of feature points and using singular value decomposition (SVD). However, in this approach, the nonlinear perspective projection is linearized by approximating it using orthographic projection, weak perspective projection, and para-perspective projection, which limits the range of motions that the approach can be accommodated. In this paper, we present a new approach based on a higher-order approximation of perspective projection to recover 3D shape and motion from image sequences. The accuracy of this approximation is higher than orthographic projection, weak perspective projection and para-perspective projection, so it can be used in wider circumstances in the real world. Experimental results with synthesized and real data show that this approach is promising.

We propose a real-time algorithm for path planning in a dynamically changing 3-D environment where the motion of the obstacles cannot be predicted. The method represents everything in the environment (an arbitrarily shaped robot and static/dynamic obstacles) with an octree that makes no distinction of the movability of objects. A collision free path for a robot with arbitrary motion (translation and rotation) in a dynamic 3-D environment is efficiently sought by using the real-time A* algorithm and the potential field generated from each black cell of the octree. Parallel computation is used to speed up the generation of the potential field. Finally, the experimental results demonstrate the efficiency of the proposed path planning method for a dynamically-changing 3-D environment with arbitrary motion.

We describe a method for generating the environmental map by cooperative observation among multiple mobile robots with omnidirectional visual sensor. Under the assumption of the known motion of the robot, an environmental map of a scene is generated by monitoring azimuth change in the image. First, each robot communicates and exchanges information of sensory date, then, can estimate a cooperative robot by evaluating relative directions of motion and estimated locations of vertical edges. Finally, a global environmental map is generated by integrating both environmental maps.

Described here is a method for estimating rolling and swaying motions of a mobile robot from optical flows. We have proposed an image sensor with a hyperboloidal mirror for vision based navigation of the mobile robot. Its name is HyperOmni Vision. This sensing system can acquire an omnidirectional view around the robot, in real-time, with use of the hyperboloidal mirror. The radial component of optical flow in HyperOmniVision has a periodic characteristic. The circumferential component of optical flow has a symmetric characteristic. The proposed method makes use of these characteristics to estimate robustly the rolling and swaying motions of the robot.

For a real robot to acquire behaviors, it is important for it to learn in a real environment. Most reinforcement learning research has been made by simulation because real-environment learning requires large computation costs as well as a lot of time. Realizing reinforcement learning of a physical robot in a real environment requires both an adaptation for the diversity of possible situations and a high-speed learning method that can learn from fewer trials. This paper describes the realtime reinforcement learning for a real robot in the real environment based on the exploitation oriented reinforcement learning method where the learning cost is very small and has strict incrementality to realize realtime reinforcement learning with an automated sub-rewards generation method achieved by the abstraction of the task state to accelerate the learning process. Successive learning experiments in the real environment for the ball pushing task for the real robot are performed.

The paper describes an approach to detect faces whose size and position are unknown in an image with a complex background. The candidates of faces are detected by finding out "face like" regions in the input image using the fuzzy pattern matching method. The perceptually uniform color space is used in our research in order to obtain reliable results. The skin color that is used to detect face like regions, is represented by a model developed by us called skin color distribution function. The skin color regions are then extracted by estimating a measure that describes how well the color of a pixel looks like the skin color for each pixel in the input image. The faces which appear in images are modeled as several 2 dimensional patterns. The face like regions are extracted by a fuzzy pattern matching approach using these face models. The face candidates are then verified by estimating how well the extracted facial features fit a face model which describes the geometrical relations among facial features.>

We designed a new omnidirectional image sensor, called COPIS (Conic Projection Image Sensor), to guide the navigation of a mobile robot. The feature of COPIS is passive sensing of the omnidirectional image of the environment, in real-time, using a conic mirror. COPIS is a suitable sensor for visual navigation in a real environment. Under the assumption of the known motion of the robot, an environmental map of an indoor scene is generated by monitoring the azimuth change in the image. We did several experiments in the simple indoor environment. The precision of the obtained environmental maps was sufficient for robot navigation in such environment. In these experiments, to examine the potential of COPIS against the effects of observational errors in real-time navigation, we simplified the image processing method and the experimental environment. In this paper, we improve the image processing method for extracting and tracking vertical edges, taking care of the reliability, and also evaluate the effectiveness of COPIS in a real indoor and outdoor environment.

Described here is an image sensor with a hyperboloidal mirror for vision based navigation of a mobile robot. Its name is HyperOmni Vision. This sensing system can acquire an omnidirectional view around the robot, in real-time, with use of a hyperboloidal mirror. The authors show a prototype of a mobile robot system with HyperOmni Vision and a method for estimating the motion of the robot and finding unknown obstacles.

Proposes an efficient and simple method for finding a collision-free path and orientation for a rigid robot in a dynamic observable 3-D environment for unmanned aerial vehicles (UAVs). The method uses an octree for representing every object (robot and static/dynamic obstacles) in the environment without any distinction of the movability of objects; therefore, all objects are dealt with in the same way. The path of a robot from its starting position to the given goal with arbitrary motion (i.e., translation and rotation) in a 3-D environment is efficiently searched for in successive adjoining regions (octree white cells) by using the potential field generated from each black cell of the octree. The algorithm is simple, so it can easily be accelerated by using parallelization techniques. Experimental results obtained under several conditions, such as a point shaped robot and arbitrarily shaped robots in static/dynamic environments, are reported.

We designed a new omnidirectional image sensor COPIS (Conic Projection Image Sensor) to guide the navigation of a mobile robot. The feature of COPIS is passive sensing of the omnidirectional image of the environment, in real-time (at the frame rate of a TV camera), using a conic mirror. COPIS is a suitable sensor for visual navigation in a real world environment. We report here a method for navigating a robot by detecting the azimuth of each object in the omnidirectional image. The azimuth is matched with the given environmental map. The robot can precisely estimate its own location and motion (the velocity of the robot) because COPIS observes a 360/spl deg/ view around the robot, even when all edges are not extracted correctly from the omnidirectional image. The robot can avoid colliding against unknown obstacles and estimate locations by detecting azimuth changes, while moving about in the environment. Under the assumption of the known motion of the robot, an environmental map of an indoor scene is generated by monitoring azimuth change in the image.>

In this paper, we propose a new multiple visual sensing sensor (MISS), which combines with an omnidirectional image sensor COPIS (COnic Projection Image Sensor) and binocular vision, for navigating the robot and understanding interesting objects in an environment by integrating both sensory data. Since COPIS observes a 360 degree view around the robot, the robot can always estimate its own location and motion precisely. The location of unknown objects can also be estimated. COPIS can observe a global and precise information of features (vertical edges) in real-time, however, it is difficult to understand their details of shapes. On the other hand, the mobile robot has binocular vision and obtains a sequence of stereo images from the environment. We extend the principle of trinocular vision to establish correspondences between a sequence of binocular images. Although a view field of the binocular vision is limited by a visual angle of lens, the binocular vision is useful for understand spatial configuration of the environment. Therefore we integrate both merits and propose an efficient sensing system. The system has been evaluated on the prototype sensor in actual environment.>

Described here is a method for navigating multiple mobile robots which estimate locations and motions of unknown moving objects. Each robot with conic projection image sensor COPIS can observe an omnidirectional view around the robot in real-time with use of a conic mirror. First, each robot communicates and exchanges information of sensory data. Then, unknown moving objects and a reciprocating robot are discriminated, and motions and locations of unknown moving objects are estimated by cooperative observation of azimuth changes between the two reciprocating robots. >

Described here is an image sensor with a hyperboloidal mirror for vision based navigation of a mobile robot. Its name is HyperOmni Vision. This sensing system can acquire an omnidirectional view around the robot, in real-time, with use of a hyperboloidal mirror.

The authors propose a novel omnidirectional image sensor, COPIS (Conic Projection Image Sensor), for guiding the navigation of a mobile robot. It performs passive sensing of the omnidirectional image of the environment in real time (at the frame rate of a TV camera) using a conic mirror. COPIS is a suitable sensor for visual navigation in the real-world environment with moving objects. The authors describe a method for estimating the location and the motion of the robot by detecting the azimuth of each object in the omnidirectional image. In this method, the azimuth is matched with the given environmental map. They also present a method to avoid collision against unknown obstacles and estimate their locations by detecting their azimuth changes while the robot is moving in the environment. Using the COPIS system, several experiments in the real world were performed.>

An omnidirectional image sensor COPIS (conic projection image sensor) is proposed for guiding navigation of a mobile robot. It features passive sensing of the omnidirectional environment in real-time using a conic mirror. Because the conic mirror is used, its image is under conic projection; where the azimuth of each point in the scene appears in the image as its direction from the image center. The authors describe COPIS and its application to guide the navigation of a mobile robot. The COPIS system acquires an omnidirectional view around the robot in real-time by using a conic mirror. Under the assumption of constant motion of the robot, locations of objects around the robot can be estimated by detecting their azimuth changes in the omnidirectional image. Using this method, the robot generates an environmental map of an indoor scene while it is moving in the environment. A method to avoid collision against objects by detecting their azimuth changes is presented.>

When a robot moves around, it obtains a large number of input images of its environment taken at different viewpoints. The mobile robot considered has binocular vision and it obtains a sequence of stereo images from the environment. This paper proposes a method to build the 3D model of the robot's environment and to understand its spatial configuration from these input images, which is one of the central problems for the study of an intelligent mobile robot.>

Proposes a new omnidirectional image sensor, COPIS (conic projection image sensor), for guiding navigation of a mobile robot. It features passive sensing of omnidirectional images of the environment in real-time (at the frame rate of a TV camera) using a conic mirror. COPIS is a suitable sensor for visual navigation in real world environment with moving objects. The paper describes a method for estimating the location and the motion of the robot by detecting the azimuth of each object in the omnidirectional image. It also presents a method to avoid collision against unknown obstacles and estimates their locations by detecting their azimuth changes while the robot is moving in the environment.>

Cine film and videotape are used to record a variety of natural processes in biology, medicine, meteorology, etc. This paper describes a system which detects and tracks moving objects from these records to obtain meaningful measures of their movements, such as linear and angular velocities. Features of the system are as follows. 1) In order to detect moving objects that are usually blurred, temporal differences of gray values (differences between consecutive frames) are used to separate moving objects from stationary objects, in addition to spatial differences of gray values. 2) The results of previous frames are used to guide feature extraction process of the next frame so that efficient processing of moving pictures which consists of a large number of frames is possible. 3) Uncertain parts in the current frame, such as occluded objects, are deduced using information of previous frames. 4) Misinterpreted or unknown parts in previous frames are reanalyzed using the results of later frames where those parts could be found.

As a step to automate assembly of various industrial parts, this paper describes a versatile machine vision system that can recognize a variety of complex industrial parts and measure the necessary parameters for assembly, such as the locations of screw holes. Emphasis is given to a method for extracting useful features from the scene data for complex industrial parts so that accurate recognition of them is possible. The proposed method has the following features: 1) simple features are detected first in the scene and more complex features are examined later, using the locations of the previously found features; 2) the system is provided with a high-level supervisor that analyzes the current information obtained from the scene and structural models of various objects, and proposes the feartures to be examined next for recognizing the objects in the scene; 3) the supervisor has problem-solving capabilities to select the most promising feature among many others; 4) the structural models are used to suggest the locations of the features to be examined; and 5) several sophisticated feature extractors are used to detect the complex features. An effort is also made to make the system versatile so that it can be readily applied to a variety of different industrial parts. The proposed system has been tested on several sets of parts for small industrial gasoline engines and the results were satisfactory.