Dynamic environments present a significant challenge for mobile robot localization, as changes in object configuration and human movement can lead to incorrect position estimates. Conventional methods struggle with these variations, often resulting in mismatches and localization failures. To address this issue, we propose a Visual SLAM-based localization method that integrates semantic information using an RGB-D camera. Our approach utilizes instance segmentation to enhance localization robustness in 3D environments. During map creation, feature points extracted from images in a static environment are assigned label IDs corresponding to object regions, embedding semantic information into the 3D map. During localization, feature matching is constrained by label consistency, ensuring that only points with identical labels are matched. Additionally, feature points associated with high-mobility objects, such as chairs or people, are excluded to prevent errors caused by environmental changes. Furthermore, during pose optimization, the information matrix is weighted according to semantic labels, giving higher importance to static objects and reducing the influence of dynamic ones. We validate the proposed method through real-world experiments in cluttered indoor environments with varying furniture arrangements and human presence. The results demonstrate that our approach significantly reduces mismatches and improves localization accuracy compared to conventional ORB-SLAM2. Unlike existing methods that update maps online, our approach maintains a fixed pre-constructed map, reducing inconsistencies over time. This study enhances localization robustness in changing environments by incorporating semantic constraints into Visual SLAM.

This paper introduces YamaS, a simulator integrating Unity3D Engine with Robotic Operating System for robot navigation research and aims to facilitate the development of both Deep Reinforcement Learning (Deep-RL) and Natural Language Processing (NLP). It supports single and multi-agent configurations with features like procedural environment generation, RGB vision, and dynamic obstacle navigation. Unique to YamaS is its ability to construct single and multi-agent environments, as well as generating agent’s behaviour through textual descriptions. The simulator’s fidelity is underscored by comparisons with the real-world Yamabiko Beego robot, demonstrating high accuracy in sensor simulations and spatial reasoning. Moreover, YamaS integrates Virtual Reality (VR) to augment Human-Robot Interaction (HRI) studies, providing an immersive platform for developers and researchers. This fusion establishes YamaS as a versatile and valuable tool for the development and testing of autonomous systems, contributing to the fields of robot simulation and AI-driven training methodologies.

In the Robot Operating System (ROS), a major middleware for robots, the Transform Library (TF) is a mandatory package that manages transformation information between coordinate systems by using a single-rooted directed tree and providing methods for registering and computing the information. However, the tree has two fundamental problems. The first is its poor scalability: since it accepts only a single thread at a time due to using a single giant lock for mutual exclusion, the access to the tree is sequential. Second, there is a lack of data freshness: it retrieves non-latest synthetic data when computing coordinate transformations because it prioritizes temporal consistency over data freshness. In this paper, we propose methods to solve these problems. First, we decentralize the giant lock to provide performance scalability and show that this results in a throughput 243 times higher than conventional TF on a read-only workload. Second, we design transactional methods based on serializable protocols that prevent anomalies, thus retrieving the freshest data. These transactional methods show a freshness up to 1276 times higher than the conventional one on a read-write combined workload.

We have developed a new type of 3D LIDAR, which is small, light weight and low cost. The size and weight are important factors of the aimed sensor to use with wide range of robot vehicles, helicopters and airplane, and small vehicles. To achieve compact size and light weight construction the sensor has single pair of laser transmitter and receiver, and scanning motor on which a vibrating mirror is placed. The mirror is driven by the resonant frequency current supplied through contactless power supply unit. Experimental results of the prototype shows that the sensor of comparable size and weight with recent 2D sensors can obtain 3D range data of 8000 points in 20 Hz with 270 degree horizontal and 40 degree vertical views both in indoor and outdoor.

We propose a method to recognize cups on desks and to detect liquids in cups with a depth camera using triangulation. When we measure an opaque liquid in a cup with the depth camera, the liquid surface is measured. On the other hand, when we measure a transparent liquid, the raised bottom is measured due to the principle of light refraction and triangulation. We formulated it theoretically, and confirmed that the theory is consistent with the experimental results. Therefore, it is possible to detect the existence of liquids in cups using the measured height based on this theory. In our experiments, the proposed method have successfully detected various transparent or opaque liquids.

For the construction of the mobile robot system, it is necessary to determine a lot of parameters like the feedback gain and the clip parameters of artificial saturation. And it is difficult to decide suitable parameters by analytical technique because of its non-linearity and complexity. In this paper, we propose the method of the mobile robot control system parameter design by a numerical simulation and heuristic optimization using the enough computer power. We describe the model of the system as iterative numerical calculation instead of analytical time function. This model can suit the real control system, and it is straightforward to be built. And we show the experimental results of the automatically decided feedback gains by using proposed method for the linear feedback trajectory tracking controller of the small-sized autonomous mobile robot system, which runs in relatively high speed up to 5 meters per second as an example.

In this paper, we present the current progress of our approach to identify and follow a target person for a service robot application. The robot is equipped with LRF and Omni directional camera. Our approach is based on multi-sensor fusion in which a person is identified using the panoramic image and tracked using the LRF. The selection of the target person is implemented to improve the identification when multiple candidates are detected. Our approach is successfully implemented on a mobile robot. A simplified target person following behavior is implemented to focus on the proposed method's efficiency. Several experiments are conducted and showed the effectiveness of our approach to identify and follow human in indoor environments.

Effective detection of people is a basic requirement for robot coexistence in human environments. In our previous work [1] we proposed a method for people detection and position estimation using multiple layers of Laser Range Finders (LRF) in a mobile robot. We extend our work by introducing laser reflection intensity as a novel feature for people detection, achieving significant improvement of detection rates. In concrete, we propose a method for calibration of laser intensity data, a method for segment separation using laser intensity, and introduce two new intensity-based features for people detection: the variance of laser intensity and the variance of intensity differences. We present experimental results that confirm the effectiveness of our multi-layered detection method including laser intensity.

Successful detection of people is a basic requirement for a robot to achieve symbiosis in people's daily life. Specifically, a mobile robot designed to follow people needs to keep track of people's position through time, for it defines the robot's position and trajectory.

We are developing robots for inspection of the under floor of houses. The developed robot system is reported in this paper. The mobile robot platform is based on a crawer type robot developed for rescue purposes. As sensors, a pan-tilt-zoom camera, top-view camera and SOKUIKI sensor are used. We constructed a test field and verified the performance of the robot.

In this paper, we propose a novel human detecting system using several stationary and moving laser range finders (LRF). The occluded area where the stationary sensors cannot measure will be covered by the actively moving sensor. The occlusion is perceived by overlaying visible area of the stationary sensors. The moving sensor will move to the target position where the sensor can measure the occluded area. Finally, the measured information of all sensors will be merged for data fusion. The experimental results show the effectiveness of the system.

In this paper, we propose a solution for human identification and localization with a mobile robot problem that implements multi-sensor data fusion techniques. This solution is designed for an autonomous luggage cart. The system utilizes a new approach based on identifying the target human visually from an omni directional camera then localizing and tracking him using LRF. This approach is composed of "registration stage" and "identification and localization stage". The registration stage extracts all necessary information needed including patches from the clothes. The identification is made using a modified pattern-matching algorithm to fit to a real time application. The tracking is implemented using a positions history structure to keep record of all positions of surrounding objects and the identified human.We implemented the proposed approach in fixed configuration to test its effectiveness.

This paper explains the design and implementation of a discreet landmark capable of modifying its properties in order to be able to differentiate itself from its surrounding environment. In this case, differently from common devices, our landmark does not make use of neither light, wave reflection, or radio frequencies, but it makes use of temperature properties of a peltier device which is able to increase or decrease its temperature. By having different characteristics from its surroundings, in this case a temperature landmark can be robustly found and tracked in complex indoor environments using thermo infrared cameras. The proposed system composition and configuration, landmark finding, extraction and tracking methods are also explained and effectiveness of this system is described and evaluated.

The purpose of this research is to develop a robot system which helps humans to remotely accomplish a daily given task such as shopping. This teleoperated system is based on simple mutual human-robot cooperation where the person remotely communicates with the mobile robot. We define the specific task of buying FRESH foods in a supermarket from a remote location. In order to do shopping operations, the robot has to be able of grasping a large amount of objects with different textures, shapes and weights which include fruits and vegetables with soft and irregular forms. After the user chooses the product to buy, the process is the next, firstly the robot has to find the object, then grasp it and then place it into a shopping basket. In this work, we built a system according to the previously explained requirements which involves autonomous navigation, environment perception, object manipulation and teleoperation. In this paper, we present the developed suction hand for picking fresh foods and evaluate its operating performance through experimental results

We have developed a 2D laser range sensor suitable for mobile robot platforms. The sensor features compactness, lightweight, high precision and low power consumption, and provides the wide scan angle with high resolution, which is very important for environment recognition by mobile robots. The basic technology for measuring the distance between the sensor and objects are; using amplitude modulation of light waves and detecting the phase difference between transmitted and the received one. This paper explains the specification of the proto-type sensor, the method of distance measurement and examples of experimental results.

This paper describes a system which uses a mobile manipulator as a teleoperated tool for accessing and manipulating remote objects, in our challenge to extend mobile robot potentials and usage in human daily life. The specific task we set up in this research is to help humans browse books located in a library from a remote location via the Internet. This task was decomposed into three major parts which are, the extraction and return of a book from a bookshelf, its browsing, and the teleoperational interface needed for humans to access to this system via a media such as the Internet. We built and realized a robot system which enables remote humans to access and browse books in a library from the Internet.

Our purpose is the realization of a system able to access objects located at remote places, by integrating Internet technologies and robotic engineering developments such as mobile robots. As an example, in this research, we are developing a system for browsing books from remote places through the Internet with a robot. In order to do browsing operations, the robot has to recognize the alignment state of a bookshelf itself and to pick up a book correctly. This paper also describes the developed sensor for recognizing the posture of a book and the plan for extractional operation in the environment of a general library. In addition, we conducted experiments by the developed system, and summarize the consideration about this system.

Proposes an indoor navigation system for an autonomous mobile robot including the teaching of its environment. The self-localization of the vehicle is done by detecting the pose of corridor fluorescent tubes with a camera pointing to the ceiling. A map of the lights based on odometry and used later for navigation tasks is first built by remotely controlling the robot. The map distortion due to positioning errors is corrected to facilitate route definition by detecting and closing open cycles extremities. Paths defined on the modified map whose geometry differs from the robot one are then converted automatically into the robot map so that the robot can refer to it and cancel odometry errors during navigation.

This paper describes a system which uses a mobile robot as a teleoperated tool for accessing and manipulating remote objects. The purpose of this study is to develop a robot system which helps humans to accomplish remotely a given task in their daily life, based on simple communication and mutual cooperation between them and a teleoperated mobile robot. The specific task we set up in this research is to help humans browse books of a library from a remote location. In this work, we build such a system accordingly to its basic requirements involving autonomous navigation, environment perception, book manipulation and teleoperation. In this paper, we present the developed book browsing robot system, its teleoperation human interface and evaluate its performance and operating efficiency through experimental results.

This paper propose a new method of sensor fusion of an omni-directional ultrasonic sensor and an omnidirectional vision sensor. A new omni-directional sonar, which we developed, can measure accurate distance and direction of reflecting points, and an omnidirectional vision can give direction to edges of segment. We propose a sensor fusion method using both the reflecting points measured by the sonar and the edges of segment measured by the vision, based on the angles. Those data are different in character, so they compensate each other in the proposed method, and it becomes possible to get better information which are useful for environment recognition of mobile robots. We describe the proposed method and an experimental result to show its potential.

Proposes a method of adding the amplitude of ultrasound echo from reflecting points. An omni-directional sonar which we developed, can measure accurate distance and direction of the reflecting points, and it is possible to measure amplitude of the echo which corresponds to the reflecting points. We propose a method to add information from the amplitude of the echo from the reflecting points for environment recognition. This method can give useful information such as character of reflecting objects and it makes possible to correspond each reflecting point with others using this information. We describe the proposed method and an experimental result to show its usefulness.

A mobile robot can use the walls or the pillars to localize itself when it navigates in indoor environment. However, in open space or outdoor environment there are few easy detectable and stable objects that the robot can use. In this paper, we propose a method for the detection of braille blocks, which are originally used for visually handicapped people, for autonomous mobile robot navigation. To recognize the braille block, a CCD camera and a laser fan beam projector are used as sensor to detect bumps on road surface. This paper also presents the experimental results of braille block following using the sensor system to detect the braille block position and orientation. This experiment shows effectiveness of the sensor system for the braille block recognition and implies the possibility of braille block based mobile robot navigation.

This research aims at development of sensors for small size indoor mobile robots. We have proposed a new sonar-ring which can measure accurate direction and distance to reflecting points rapidly for this purpose. This new sonar-ring needs to receive echo signal by multiple receivers and to detect multiple Time-of-Flights (TOFs) in received signal at each receiver simultaneously for achieving fast and accurate measurement. However, it is impossible to satisfy them by conventional circuit. This paper proposes 30 channel 1 bit wave memory system to achieve these two points simultaneously and shows its usefulness.

A sonar-ring is one of the most popular sensors for indoor mobile robots, because it is simple and gives omni-directional distance information directly. However, it is difficult to measure the accurate direction of reflecting points by a conventional sonar-ring sensor. Further more, conventional sonar-ring sensors are slow to get full 360 degrees information due to sequential driving of transducers for avoiding interference. We propose a new sonar-ring sensor system for a mobile robot which can measure the accurate bearing angles of reflecting objects in a single measurement. The proposed system employs measurement by using differences of time-of-flight and by simultaneous transmit/receive of all directions, and consequently achieve fast measurement. We implemented a prototype of the proposed sonar-ring on a mobile robot. Experimental data show the effectiveness of the proposed system.

We developed APCS (autonomous position correction system) that can autonomously cancel the error of the estimated robot position from odometry by detecting flat walls using ultrasonic sensing. When it detects flat walls in the environment, this system corrects the estimated position using maximum likelihood estimation (MLE). The feature of this system is that it can correct the position not being concerned with the behavior of the robot because the system autonomously decides the trigger of the position correction. We show the algorithm and implementation of APCS, and some experimental results to confirm feasibility of the system in a disordered environment.

This paper proposes a new method of position correction for a mobile robot on rough terrain. The strong point of our method is that no external sensor is needed for position correction. The information about position correction is derived from comparing between the estimated position and elevation map of the terrain by using the knowledge that the robot must touch the ground. Some experimental results are also shown.

This paper describes a vision-based navigation method in an indoor environment for an autonomous mobile robot which can avoid obstacles. In this method, the self-localization of the robot is achieved by a model-based vision system, and nonstop navigation is realized by a retroactive position correction system. Stationary obstacles are avoided with single-camera vision and moving obstacles are detected with ultrasonic sensors. We report on experiments in a hallway using the YAMABICO robot.

In this paper, we report the outdoor navigation of an autonomous and self-contained mobile robot using the perceived route map (PRM) generated through human assisted route teaching. The PRM includes the path from the start to the goal taught by the operator and landmark information are automatically generated by the robot itself. The target environment is a paved or tiled pedestrian walkway. The pedestrian walkway can be assumed to be a two dimensional plane along with a wall, a hedge or a tree, etc., which can be utilized as landmarks for estimating the robot position. The robot is assumed to have sensors to detect landmarks and obstacles. A walking person or bicycle is assumed to be a moving obstacle.

This paper describes a vision-based navigation method in an indoor environment for an autonomous mobile robot which can avoid obstacles. In this method, the self-localization of the robot is done with a mode-based vision system, and a non-stop navigation is realized by a retroactive position correction system. Stationary obstacles are avoided with single-camera vision and moving obstacles are detected with ultrasonic sensors. We report on experiments in a hallway using the YAMABICO robot.

We present research on long distance outdoor navigation based on the estimated position for a mobile robot. We discuss the generation of the route map given to the robot in advance of autonomous navigation. We propose how the robot generates the route map including landmarks while a human operator takes it to the goal once. By our proposed method, it will be easy to make a large size route map for long distance outdoor navigation.

The purpose of this study is to realize autonomous navigation of a mobile robot with reference pre-recorded images, without any environment maps. The strategy of navigation in this research is that the differences of position and orientation of robot between the reference one and the present one are continuously estimated from true images, and are used in the locomotion control commands. In this paper, we use only vertical lines in the image as extracted features, and propose the method to calculate the estimated differences of position and orientation from two images. We also show some results of simulations and experiments using an actual mobile robot, and examine its reliability.

The usual method for position estimation of a wheeled mobile robot is odometry. However, it has the problem of gradual error accumulation when the robot moves. To solve this problem, we place several reflectors sparsely in the robot's work-space as landmarks, so that the robot can correct its estimated position when detecting the landmark. The new estimated position of the robot is calculated by the maximum likelihood estimation (MLE) by both the position information estimated by odometry and the reflector detecting sensor system (ReDS) using the laser beam. ReDS, which we developed, can detect the reflector at most 3 meters away from the robot in the indoor environment. Practical experiments show that the estimated position obtained by this system is precise enough to be useful.

We propose a position estimation technique for nonstop outdoor navigation of an autonomous mobile robot. The proposed position estimation technique is based on maximum likelihood estimation. To cope with the parallel processing of internal and external sensor information and time delay in the sensor data process, we introduce the retroactive positioning data fusion technique. The proposed technique is implemented on our small size autonomous mobile robot. An experimental result is shown, in which our robot could navigate itself without stopping even when it takes several seconds of processing time to detect landmark from external sensor data.

Ultrasonic sensors are very useful for mobile robots because they have the advantage that the systems are simple and are easy to handle. When a mobile robot is navigated using an ultrasonic sensor, the map of the environment should include information about the reflection characteristics of objects. For this purpose, the map using a vector expression is better than a grid map. In this paper, an environment reconstruction method using a vector map is proposed. In this method, the environment is reconstructed from the data obtained by using our novel ultrasonic sensor which can measure the normal direction of walls, and the next sensing point is determined automatically. The experimental results of a map construction using this algorithm are shown. As a result of the experiments, it is found that the measurement of walls is possible using this method and the effectiveness of the planning algorithm for the sensing point is verified.>

A 3-D range sensor system for an autonomous mobile robot is reported. It is important for a mobile robot to recognize spaces of passage. In this sensor system, the passage space is detected with an active vision sensor using a fiber grating, laser diode, and CCD camera. The fiber grating vision sensor can easily obtain 3-D information about the object. In the authors' system, the object is the floor and the system measures the height of the floor. In this case the height is nearly equal to zero, so the robot can perceive that the measured area is passable. An image processing module using a transputer (T805) is developed for this purpose. In this paper, the principle of the sensor and the method for passage space recognition are presented. The hardware and software of the system for the authors' autonomous mobile robot "Yamabico" is presented. Experimental results are also presented.>