We are developing the Robotic Travel Aid, the primary function of which is to guide the visually impaired in an outdoor environment. According to our investigation, the visually impaired feel fear and stress when they walk along an unknown route. In this paper, a pre-journey system helping the visually impaired acquire route information is proposed. The system generates route guidance message for the visually impaired based on the route information described in XML. By using a speech synthesis device and push button switches, the visually impaired can easily utilize this system to understand route information before setting out. From the results of our experiments, it is clear that the proposed system is useful and efficient for the visually impaired.

Since 1982, we have conducted research and development on an autonomous mobile robot, the "Harunobu" series, and studied a robot that travels autonomously on the university campus. In 1995, we started the development of the robotic travel aid application to guide the visually impaired. The robotic travel aid (RoTA) is a motorized wheel chair equipped with vision, sonar and tactile sensors, as well as a map database system. The visually impaired can get mobility, orientation and navigation information from the RoTA. The paper describes a strategy for a crossing of our robotic travel aid "Harunobu". We implemented this strategy on the mobile robot "Harunobu-6". Experimental results in a real environment showed the effectiveness of our proposed strategy.

This paper describes a navigation system for an autonomous mobile robot in outdoor environments. The robot uses vision to detect landmarks and differential GPS (DGPS) information to determine robot's initial position and orientation. The vision system detects landmarks in the environment by referring to an environment model. As the robot moves, it estimates its position by conventional dead-reckoning, and matches the landmarks with the environment model in order to reduce the error in the robot position estimate. The robot initial position and orientation are calculated from the coordinate values of the first and second locations which are acquired by DGPS. Subsequent orientations and positions are derived from map matching. We implemented the system on a mobile robot "Harunobu-6". Experimental results in real environments showed the effectiveness of our proposed navigation methods.

The color vision in the outdoor environment has two problems. The first is the hue shift of the object under the changes of the season, weather, the hour and the view position. The second is the hue shift of the object between in the sunny place and in the shaded place. We propose the color impression factor to memory the hue of the object. The color impression factor is defined as R-B. It is the function of the sunlight condition (sunny/shaded) and the hour. We show this factor is useful to discriminate objects from the shadows in the asphalt paved road-yellow tactile blocks from the asphalt paved road.

We have developed a real time vision system for the Robotic Travel Aid system which detects vehicles at the intersection of roads, their positions, velocities and moving directions. This system can also evaluate danger coefficient to have a mobile robot stay still or go and cross the road. In the previous work we used a single standard lens for this purpose and detected vehicles coming in the front of the camera. However this system could not watch the right side of the crossing. The present system has two cameras. One of them is wide angle lens to sec both sides of the crossing. The other one is telephoto lens to look at forward. In this paper we describe the system configuration and evaluation of danger coefficient. The obtained success rate for finding dangerous vehicles is over 85% and the processing time is 180 msec.

This paper deals with dynamic vision, vehicle avoidance, navigation, and applications to guide dog robot and vehicle intelligent control system. Danger estimation of vehicles at the intersection is proposed in this paper. Traffic rules of vehicles and the locomotion strategy of robot is formulated, and finally a danger matrix is defined for each vehicle represented by its location and its moving direction. This method is implemented on an image processing system. The location, velocity, and the moving direction of a vehicle are measured by computer vision and its danger coefficient is evaluated. About 90% of vehicles at a T shape intersection have been successfully voted in danger judgment.

Improvement of the environmental model to avoid the lapse time of observation and action by analyzing the data of the last run is called locomotion performance learning. The algorithm of the learning is expressed by rewriting rules on the trajectory data. The algorithm is implemented on mobile robot HARUNOBU-4, and tested in the university campus. Experimental results show that after the learning 83% of the observation times and 18% of locomotion times are cleared.

Ultrasonic rangefinders have already been used successfully for indoor mobile robots. However, in an outdoor environment, the sensor should be robust against noise from vehicle engines and other sound sources. The robust sensor has already been proposed, but it requires a high-performance processor. In this paper the authors propose a new ultrasonic ranging sensor which is simple and low-cost. This sonar is robust in noisy environments. A RTZ (return to zero) signal having a certain time duration modulated with a 40 KHz carrier is transmitted from a transducer, then cross-correlation is calculated between the transmitted wave form and the demodulated received wave form. If objects are present the calculated cross-correlation has relatively large value. The authors can measure the range from the sensor to the objects by the time interval between the starting time of the transmitted signal and the time which has the peak value. The experimental results show the robustness of this sensor against the impulse noises. Furthermore, the authors propose that several sonar sensors can work at the same time without mutual interference, when the different time duration signals are used as the transmitting signals.

The general paradigm for vehicle detection and avoidance by proper selection of visual and sonar sensors are proposed. The most appropriate sensor for robot detection and navigation is properly selected depending on its functions. Four types of actions based on stereo-typed motion are used for the navigation of the robot: moving along wall, moving alone;, moving forward and tracking motion. The first approach is based on sonar, and the remaining are vision based actions. The visual detection of vehicle is based on the locomotion strategy: the sign pattern based stereo-typed motion. The sign pattern of vehicle is the shadow underneath the vehicle which is darker than any other parts of the asphalt paved road. The robot detects the sign pattern of vehicle from the distance of 30m and extracts its front shape in real time processing.>

The robotic travel aid(RTA) described is a motor wheel chair equipped with vision, sonar, and tactile sensors in addition to a map database system. The visually impaired can get orientation, mobility and obstacle information from the RTA, and can inquire about the present location, land marks and an advanced part of the route. This concept is implemented on an RTA called "HITOMI". It can guide the impaired in a lane marked road and on yellow Braille marked sidewalks, avoiding vehicles.>

In this paper, a method for automatic pedestrian detection from monocular image sequence is described. It discriminates the pedestrians from the other moving objects based on the rhythmical motion of human walking. When both feet of a pedestrian are on the ground in walking, their motions are found to have a relatively small changes of intensity in subtracted image. When one of the feet is moving forward, its motion is found to have a relatively large changes of intensity. Therefore, the periodical changing of image intensity caused by walking can be observed by applying the maximum entropy method. The state of a moving object, such as position and velocity, can be estimated with a kinematic model, a measurement model and a tracking filter. Corresponding to the pace and the 1/stride of walking, the rhythm is decomposed into two features; the temporal-frequency and the spatial-frequency. The model matching is performed based on these features.>

This paper proposes a method for detecting and tracking a moving vehicle on the asphalt road based on vision. The distance between the robot and a moving vehicle, velocity, and its width are estimated during the tracking process. The authors show a real time moving vehicle detection method which is enough for detecting a moving vehicle for any weather conditions including fine, partly cloudy and cloudy. Furthermore, the method is applicable for the road covered partly or completely by shadows. Stationary vehicle is also distinguished from moving one by evaluating its velocity. The proposed method detects a vehicle from a distance of 25 m with a domestic video camera and extracts its front shape in real time.

The authors propose a method for detecting and following a half-faded lane mark on an asphalt road using image processing. It ensures reliable detection and high speed following in the actual outdoor environment. The method is based on a template matching between a lane mark model and a detected lane mark. The authors built a testbed robot, which aims to repaint a half-faded lane mark semi-automatically. The details of this testbed robot and experimental results of a half-faded lane mark detection and following are described. Experimental results (trajectories of this robot) for a actual half-faded lane mark show the availability of this method.

Presents the fundamental behavior of a compact mobile robot which moves in an outdoor environment. The robot's behavior is actived by the visual pattern of a straight line, such as a road boundary. This behavior makes the robot move along the visual pattern. The behavior is achieved using control of the TV camera direction and estimation of the position where the visual pattern will appear on the image plane. The control of the TV camera direction keeps the visual pattern within the image plane, and the position is estimated by measuring the TV camera motion with internal sensors. This is used for determining the search area on the image plane. Using this behavior, the robot can move along the visual pattern efficiently and robustly. The robot moves freely by selecting the visual patterns. The authors demonstrate its validity by experimental results in a real outdoor environment.

This paper presents a new concept, in robot visioii that we call sign pattern. Tlic sign patstern is visual pattern cliaracterizing tlie environnient. and resulting a stereotyped robot action. In other word, the sign pat,terii is a key wliicli activates t,lie robot reactive action. The visioiiguided mobile robot can iiiove on the road by detecting and following the sign patterns. Tlie sign patterns are used in tliree fiiiictioiis of the niobile robot: (1) Following. road boundaries, (2) Controlling t.he caniera posit.ion and (3) Approaching targcts. In this papcr. we describe t.he new concept of the sign pattern and image processing for detecting and tracking the sigii patterns.

The concepts of guide dog robot Harunobu-5 and a stereotyped motion following a person are described. The guide dog robot consists of a mission planner, digital map, interactive navigator, vision system and undercarriage system. Following a person is activated when the robot fails to move along sign pattern. The robot stops and finds a moving pedestrian by color vision. After that the robot follows it by tracking its jacket or trousers image. The following person concept was implemented and tested on Harunobu-5 and was successful in following only one person.>