Reinforcement Learning (RL) can be considered as a sequence modeling task, where an agent employs a sequence of past state-action-reward experiences to predict a sequence of future actions. In this work, we propose State-Action-Reward Transformer (StARformer), a Transformer architecture for robot learning with image inputs, which explicitly models short-term state-action-reward representations (StAR-representations), essentially introducing a Markovian-like inductive bias to improve long-term modeling. StARformer first extracts StAR-representations using self-attending patches of image states, action, and reward tokens within a short temporal window. These StAR-representations are combined with pure image state representations, extracted as convolutional features, to perform self-attention over the whole sequence. Our experimental results show that StARformer outperforms the state-of-the-art Transformer-based method on image-based Atari and DeepMind Control Suite benchmarks, under both offline-RL and imitation learning settings. We find that models can benefit from our combination of patch-wise and convolutional image embeddings. StARformer is also more compliant with longer sequences of inputs than the baseline method. Finally, we demonstrate how StARformer can be successfully applied to a real-world robot imitation learning setting via a human-following task.

One of the most challenging issues in radio received signal strength (RSS)-based localization systems is the generation and distribution of a radio map with a coordinate system linked with spatial information in a large indoor space. This study proposes a novel spatial-tagged radio-mapping system (SRS) that effectively combines the heterogeneous properties of LiDAR and mobile phones to simultaneously perform both spatial and radio mappings. The SRS consists of synchronization, localization, and map building processes, and enables real-time spatial and radio mapping. In the synchronization process, the distance range, motion data, and radio signals obtained through the LiDAR and mobile phone are collected in nodal units according to the sensing time. In the localization process, a feature variance filter is used to control the number of features generated from LiDAR and estimate the positions at which the nodes are generated in real time according to the motion data and radio signals. In map building, the estimated positions of the nodes are used to extract spatial and radio maps by using a unified location coordinate system. To ensure mobility, the SRS is manufactured in the form of a backpack supporting LiDAR and a mobile phone; the usefulness of the system is experimentally verified. The experiments are performed in a large indoor shopping mall with a complex structure. The experimental results demonstrated that a common coordinate system could be used to build spatial and radio maps with high accuracy and efficiency in real time. In addition, the field applicability of the SRS to location-based services is experimentally verified by applying the constructed radio map to well-known fingerprinting algorithms using the heterogeneous mobile phones.

This paper presents about the localization technology to estimate the positions of the mobile robot that is able to navigate through the multi-floor spaces, for expanding the application ranges of the robots. For this purpose, the mobile robots can use some maps having some information about the spaces in advance. In this paper, the maps are divided into two types according to the application; as to one, it is the grid-based map that represents the locations and the structures of objects in the robot surroundings with each floor. The other is the inter-floor map containing the stairs data to help the robot move into other floors. And the localization method is consisted of the floor and the inter-floor localizations. The floor localization uses the particle filter method to estimate the robot position in each floor. And the inter-floor localization can help the robot safely move to other floors by recognizing the stairs and change the coordinate system. Finally we performed the experiments in the multi-floor office spaces to verify the effectiveness of the proposed localization method by using the robot with the caterpillar wheels and the maps.

This paper presents an efficient method for building a topological map for robots in urban environments based on satellite image maps acquired from a geographic information system (GIS). In urban space, mobile robots need a special map, such as a topological map, to generate a path toward their goal. A topological map for mobile robot navigation should include semantic data, e.g., the width and type of road. We divide the satellite image based topological map building process into two steps. The first step is defining the topological map model to fit the mobile robot navigation in an urban environment. The second step is generating a topological map from existing satellite image maps downloaded from the GIS to reduce the cost and improve the accuracy in building the map. If a GIS-based topological map is constructed once, it is possible to use the map for the navigation of various robots as common spatial data. To validate the proposed mapping method, we show details of its implementation in map building as well as in a navigation experiment in real outdoor environments, such as a campus or city district.

This paper presents a localization method in urban environments by using dead reckoning sensors, Global Positioning System (GPS), and taking into account the benefits of map matching. Extended Kalman Filter (EKF) is used as the main framework to fuse the information from sensors. However, the result of the EKF greatly depends on how the robot utilizes and judges the position measurement which comes from GPS since the GPS easily gives wrong position measurement due to the phenomenon called multipath effect. Under the assumption that the robot must operate only on the main road, a map matching is used to filter out the wrong GPS measurements which fall outside the main road. An experiment has been conducted in urban environment to validate the proposed method. Experimental results show that our proposed method has superior performance compared to the EKF without map matching.

This paper proposes the artificial landmark map building method using a Grid-based Simultaneous Localization And Mapping (Grid SLAM). The Grid SLAM method is employed to simultaneously localize the position of mapping cart and construct the map of working area. Based on the estimated position of the mapping cart and the grid-based map, the artificial landmarks are localized and their positions are saved to the artificial landmark map. The proposed method reduces the complexity and the cost of mapping process which is usually done by hand. The real implementation has been carried out to build the artificial landmark map of large scale indoor environment named T-City. The implementation results show that the proposed method gives a convenient way to construct the artificial landmark map while still maintaining the accuracy of map. The correctness of the artificial landmark map has been confirmed through the real operation of the mobile robots which rely on the artificial landmark map for their navigation at T-City.

This paper presents a method to build the large-scale indoor maps by means of extended Kalman filter (EKF) localization which explores the statistical distribution of noise parameters. As typical method in many robot localization applications, EKF localization has shown considerable success history to locate the position of the robot. However, EKF has also lack which can degrade its performance, especially in the real environment due to incompleteness, incorrectness and imprecision of noise parameters. Moreover, although many kinds of sensors are used for EKF localization, it is still difficult to generate an accurate map because of noise parameters. The fundamental solution of this problem should be addressed to the utilization of adequate noise parameters setting. We have developed a new technique for searching the optimal noise parameters setting of EKF localization using a statistical distribution. The experiments carried out on mobile robot have been performed to build accurate maps by using EKF localization relied on statistical distribution of noise parameters. The mapping results show that the method based on statistical distribution can be useful for practical application.

We propose a network robot application utilizing a human-in-the-loop structure. The proposed structure hybridizes robot perception with perception capability of a human, rendering a robot immune to the uncontrolled environment. Network infrastructure enables the user to intervene in the operation of a robot whenever necessary, enabling seamless transition of task control between the robot and the remote user. We have developed a prototype security robot system by implementing the proposed human-in-the-loop structure. Introduction of network infrastructure, however, also invokes system instability due to network fluctuation, time delay, or jittering. We could resolve the instability by imparting task intelligence to the robot and employing wireless distribution system (WDS) configuration for seamless data communication beyond the coverage of a single access point. Although our approach may solve only part of problems inherent in networked robots, we found that the developed robot system bears sufficient applicability to real world robotic systems incorporating large-area robot navigation.

This study introduces the sonar quick mapping method using tethered-robot guiding system, Navi-guider, that is one of the human and robot interactive devices. Basic function of the Navi-guider is to actively control the robot through the tension length and an orientation of the cable. This is able to easily control the mobile robot, is applied to robot controller for quick mapping. Using Navi-guider, user can easily control the mobile robot, but the sonar sensors of the robot head occasionally make the wrong decision to return the incorrectly measured distance between the sensor and an object. Therefore, this paper presents a Altering method, called a cell conjunction Alter (CCF), for building a sonar grid-based map for a mobile robot using Navi-guider. The CCF can recover the wrongly updated map by the incorrect sonar data. The principle of the CCF was based on revaluation of the occupancy possibility in a certain cell through the probabilistic relation among the cells. In this paper, the CCF was applied to a basic grid map construction model, Bayesian model, to verify its effectiveness. Experimental results were also obtained using a mobile robot in a real world environment.

The Navi-Guider, which is a new tool to make it easy to handle mobile robots, is presented in this paper. Since a robot used in home should be operated easily, the proposed robot control scheme used for the Navi-Guider has to be designed to support the user-interface. The Navi-Guider that is easily mounted in a mobile robot could detect a length and a direction of the tether pulled out by a user. Those detected factors are utilized in the Navi-Guider for the robot guidance and control.

This paper addresses a new feature map building method that can minimizes the appearance of phantom features by using only sparse sonar data. The approach is composed of extraction of features and building a probability grid map using only the footprint of sparse sonar data, estimation of position uncertainty of the feature, and evaluation of the reliable features. A virtual circle association frame model has been developed, which associates two sonar footprints into a virtual circle frame. Using this model, the geometric primitives such as lines, points, and arc features are separately extracted. While extracting the features, a grid map is also built using the orientation probability approach. The position uncertainty of each extracted feature is, then, estimated by considering both the position uncertainty of the robot and the measurement uncertainty of the sonar sensor. Finally, the reliable features among all extracted ones are evaluated from grid association method. The proposed methods have been tested in a real home environment with a mobile robot