Cooperative recognition, a method to achieve human-automation cooperation in the recognition phase of the autonomous driving system, has been proposed to address the challenges in the conventional control phase cooperation, e.g., taking over vehicle control. In cooperative recognition, the operator intervenes in recognition tasks that are difficult for the automated system alone to improve driving efficiency and safety. The challenge is the integration of both human and automated systems while both participants have different characteristics, processing capabilities, and uncertainty in the decisions (recognition results). The objectives of this study are task allocation (i.e., when and for which targets the operator should intervene) taking into account the intervention efficiency and human state. And also combine the human intervention and recognition result of the automated systems to solve the uncertainties in both participants. We formulated this problem with a Partially Observable Markov Decision Process (POMDP). The simulator experiment indicated that the recognition result of the automated system and the operator’s intervention were stochastically combined. The intervention requests to the operator adapted to the operator state and could be reduced while maintaining driving efficiency and minimizing risk omissions.

Autonomous driving of vehicles and robots requires highly accurate position information, and RTK-GNSS is expected to be utilized for this purpose. In this paper, we propose a robust and real-time operation method by introducing graph optimization into the integrated RTK-GNSS/IMU method. The proposed method is an extension of a method using vehicle trajectories that can estimate positions with lane-level accuracy even in urban areas. The position is estimated by removing GNSS multipaths from the shape of a vehicle trajectory of several hundred meters and averaging the remaining GNSS results. This method does not take into account the errors in the vehicle trajectory and cannot fully benefit from the high accuracy positioning solution of RTK-GNSS. To solve this problem, we introduce graph optimization to the base method, which treats the error state as a probabilistic model. However, general graph optimization methods have problems with processing time and outlier elimination. The proposed method solves these problems by restricting the time series data to be optimized and using a two-step optimization structure. Evaluations show that the proposed method is effective because it satisfies the requirements for real-time operation and improves accuracy compared to conventional methods.

In this research, we proposed an active management method to tele-monitor and tele-operate more autonomous vehicles (AVs) with few observers by adjusting the movement of the AVs actively. A management system is created to get the status from the AVs and separate the monitoring requirement to the observers optimally. When the requirements might be intensive, the management system can also adjust the movements of the AVs actively to distribute the monitoring time, which can make the observers monitor more vehicles. We implement and verify the management system in an autonomous driving simulator - CARLA for the limited number of AVs and observers. Based on the data acquired from the driving simulator, we also create a numerical simulator and tested our method with the pairs of a large number of AVs and observers. The result of both simulators shows that our method can reduce the utilization degree of observers and make them monitor more AVs.

As research and development related to the practical use of autonomous driving systems (ADS) continue to advance, one of the remaining challenges is achieving both safe and natural autonomous driving. In part, this is due to the difficulty of achieving flawless, automated perception and understanding of the driving environment. In our previous study, we proposed a recognition assistance interface to solve this problem by sharing ADS recognition information with the passenger, allowing them to assist in the recognition stage of the autonomous driving process. In this study, we incorporate our recognition assistance interface into Autoware and test it in a simulated driving environment, using scenarios in which the ADS must recognize the intent of pedestrians and respond to the presence of trash in the road. Natural driving was dened as driving that avoids significant, unnecessary deceleration when performing these challenging recognition tasks. The results of our experiment with 11 participants showed that sharing recognition information with passengers is effective for avoiding unnecessary deceleration and achieving only minor variations in speed when encountering obstacles flagged in error by the recognition system.

Continuous point cloud data is being used more and more widely in practical applications such as mapping, localization and object detection in autonomous driving systems, but due to the huge volume of data involved, sharing and storing this data is currently expensive and difficult. One possible solution is the development of more efficient methods of compressing the data. Other researchers have proposed converting 3D point cloud data into 2D images, or using tree structures to store the data. In a previous study targeting streaming point cloud data, we proposed an MPEG-like compression method which utilizes simultaneous localization and mapping (SLAM) results to simulate LiDAR’s operating process. In this paper, instead of imitating MPEG, we propose new strategy for more efficient reference frame distribution and more natural frame prediction, and use a different algorithm to encode the residual, greatly improving the algorithm’s performance and its stability in different scenarios. We also discuss how various parameters affect compression performance. Using our proposed method, streaming point cloud data collected by LiDAR sensors can be compressed to 1/50th of its original size, with only 2 cm of Root Mean Square Error for each detected point. We evaluate our proposed method by comparing its performance with several other existing point cloud compression methods in three different driving scenarios, demonstrating that our proposed method outperforms them.

In this work, we present LIBRE: LiDAR Benchmarking and Reference, a first-of-its-kind dataset featuring 10 different LiDAR sensors, covering a range of manufacturers, models, and laser configurations. Data captured independently from each sensor includes three different environments and configurations: static targets, where objects were placed at known distances and measured from a fixed position within a controlled environment; adverse weather, where static obstacles were measured from a moving vehicle, captured in a weather chamber where LiDARs were exposed to different conditions (fog, rain, strong light); and finally, dynamic traffic, where dynamic objects were captured from a vehicle driven on public urban roads, multiple times at different times of the day, and including supporting sensors such as cameras, infrared imaging, and odometry devices. LIBRE will contribute to the research community to (1) provide a means for a fair comparison of currently available LiDARs, and (2) facilitate the improvement of existing self-driving vehicles and robotics-related software, in terms of development and tuning of LiDAR-based perception algorithms.

Teaching autonomous vehicles to imitate human driving in complex, urban traffic scenarios is a difficult task. “End-to-end” autonomous driving systems, based on “imitation learning”, are an expecting approach. A model learns the relationships between sensing input and vehicle control signal outputs. These methods can successfully achieve driving in simple scenarios such as lane keeping. In contrast, the “mid-to-mid” autonomous driving methods now being proposed. In such framework, the model learns the relationships between pre-processed feature maps from the model-based system as input and the future position of the ego vehicle as the output. Mid-to-mid driving methods can direct vehicles more robustly than end-to-end driving methods in some complex driving environments. However, mid-to-mid driving methods use the results of the object detection module to create the feature map. If object detection fails, or detection performance is poor due to changes in the driving environment, prediction performance may also be degraded. Our proposed method uses a prediction module that outputs point grid maps directly, without the use of an object detection module, which are then incorporated into the feature map. Point grid maps represent the locations of surrounding vehicles and obstacles directly, based on LiDAR point cloud data. Since the results of object detection are not used by the prediction module, detection performance does not affect prediction performance. In this study we conduct two experiments, an off-line evaluation using a Lyft dataset, and an on-line evaluation using the CARLA simulator. The results show that our model can achieve the same level of ego-vehicle position prediction performance as a model using annotated object location information.

The use of 3D LiDAR, which has proven its capabilities in autonomous driving systems, is now expanding into many other fields. The sharing and transmission of point cloud data from 3D LiDAR sensors has broad application prospects in robotics. However, due to the sparseness and disorderly nature of this data, it is difficult to compress it directly into a very low volume. A potential solution is utilizing raw LiDAR data. We can rearrange the raw data from each frame losslessly in a 2D matrix, making the data compact and orderly. Due to the special structure of 3D LiDAR data, the texture of the 2D matrix is irregular, in contrast to 2D matrices of camera images. In order to compress this raw, 2D formatted LiDAR data efficiently, in this paper we propose a method which uses a recurrent neural network and residual blocks to progressively compress one frame's information from 3D LiDAR. Compared to our previous image compression based method and generic octree point cloud compression method, the proposed approach needs much less volume while giving the same decompression accuracy. Potential application scenarios for point cloud compression are also considered in this paper. We describe how decompressed point cloud data can be used with SLAM (simultaneous localization and mapping) as well as for localization using a given map, illustrating potential uses of the proposed method in real robotics applications.

In this paper we present a method to learn how to drive in different types of blind intersections using expert driving data. We cluster different intersections based on the velocity of how drivers approach them, and train a linear SVM classifier for each class of intersection. Through clustering we found that there were three different classes of intersections in typical residential areas in Japan. We used inverse reinforcement learning (IRL) to build a driving model for each type of intersection. The models were trained from 308 trajectories traversed by 5 different drivers. The models and policies were implemented and evaluated in a ROS simulator where the agent is provided a global path, and upon it reaching an intersection, it selects the appropriate trained policy. By doing this, the simulated autonomous vehicle can perform proactive safe driving behaviors when approaching blind intersections.

This paper presents a model for safe driving at blind intersections and its integration to a local planner based on a Frenet frame. The model predicts potential moving obstacles from blind intersections to proactively slow down to avoid potential collisions. The derivation of the model is described and its parameters are detailed. The local planner computes smooth trajectories with smooth velocity profiles so that the vehicle can follow the paths without jerk and sudden accelerations resulting in safe and comfortable navigation. Experimental results in simulation and in the real field with an autonomous car, show that the proposed predictive driving framework can reproduce human expert driver's trajectories and velocities when facing blind intersections.

In this paper, we present a localization approach that is based on a point-cloud matching method (normal distribution transform “NDT”) and road-marker matching based on the light detection and ranging intensity. Point-cloud map-based localization methods enable autonomous vehicles to accurately estimate their own positions. However, accurate localization and “matching error” estimations cannot be performed when the appearance of the environment changes, and this is common in rural environments. To cope with these inaccuracies, in this work, we propose to estimate the error of NDT scan matching beforehand (off-line). Then, as the vehicle navigates in the environment, the appropriate uncertainty is assigned to the scan matching. 3D NDT scan matching utilizes the uncertainty information that is estimated off-line, and is combined with a road-marker matching approach using a particle-filtering algorithm. As a result, accurate localization can be performed in areas in which 3D NDT failed. In addition, the uncertainty of the localization is reduced. Experimental results show the performance of the proposed method.

This paper presents a model for velocity control in blind corners and intersections based on expert driver data. Accurate expert driver data was collected with a car equipped with a 3D LiDAR and high definition maps. A model based on human expert driver data is used to control the velocity of the ego-vehicle when facing blind intersections. The model regulates ego-vehicle velocity based on the visibility of the road at the blind intersection. As the vehicle approximates the intersection and crossing roads are not visible, the vehicle slows down, then as the roads become visible the vehicle accelerates. Experimental results show the performance of the velocity model compared towards 270 trajectories taken from 7 expert drivers towards 6 different intersections without mandatory stops.

Interest in continuous point cloud data has increased rapidly as it becomes more widely used in practical applications, such as the development of autonomous driving systems. Sharing and storing continuous point cloud data is currently expensive and difficult due to the huge volume of data involved, however. As a result, developing efficient methods of compressing cloud point data has become an urgent task. Current methods compress the data directly using tree structures or height map-based methods. Taking into consideration the specialized requirements of autonomous driving, in [1] we proposed a method of compressing raw point cloud data using image compression methods and obtained superior results. Image compression-based methods are not able to utilize the 3D characteristics of point clouds, however. Therefore, in this paper we propose a new compression method using location and orientation information from Simultaneous Localization and Mapping (SLAM). Proposed method can take advantage of the 3D characteristic of point cloud by a predicting process which simulating the working procedure of 3D LiDAR. Strategies from MPEG and DPCM compression are utilized and several issues which can affect performance are discussed. In addition, based on proposed method, this paper discusses the ways to segment steam point cloud for compression task. We call this segmentation process adaptive sequence decision. We then compare the proposed SLAM-based method with image compression-based methods in various situations. Our experimental results show that, while requiring a small margin of error, the SLAM-based method outperforms image compression-based methods.

A precise and efficient tracking technique is essential for an intelligent vehicle to interact with the surrounding vehicles on the road. In this paper, we propose a model-based vehicle tracking method using the Rao-Blackwellized particle filter (RBPF) and scaling series particle filter (SSPF).

Obstacle detection is an essential technique for intelligent vehicles. Environmental sensing especially plays a vital role to achieve accurate obstacle detection. Unlike classical 2D scan, emerging 3D Light Detection and Ranging (LiDAR) sensors can scan dense point cloud at one time, which represents detailed information of urban environments. The downside of obstacle detection using 3D LiDAR, on the other hand, is its computational cost posed by a large amount of 3D data. The virtual scan (VScan), first introduced by Petrovskaya et al. [1] for efficient vehicle detection and tracking, is a 2D compression of 3D point cloud to represent free space, obstacles and unknown areas. To overcome the computational problem of obstacle detection using 3D LiDAR, therefore, VScan is suitable. In addition, it can bridge across new-born 3D LiDAR sensors and many matured applications based on 2D scan, including occupancy grid map, SLAM, planning, detection, and tracking, due to its 2D representation of 3D point cloud.

We propose a method by which a micro air vehicle (MAV) can hover without propulsion power. In this paper, we describe the design of a magnetic adhesion mechanism and a winch mechanism for an MAV that is used to perform search operations inside buildings. An MAV equipped with these mechanisms can adhere to an iron ceiling and search for a long time from an appropriate position. We designed a magnetic adhesion mechanism with an adhesion force of over 20 N. The magnetic adhesion mechanism comprises a magnet, dual coil springs, and a switching mechanism. Using models of MAV and the adhesion mechanism, we determined the parameters of the mechanism. The magnetic adhesion mechanism can be detached easily by a force less than the magnetic adhesion force. The winch mechanism comprises a structure to retrieve the adhesion mechanism, a strong, lightweight tether and a servo-motor that produces enough torque to lift the MAV by the tether. We confirmed that the MAV can hover without propulsion power by using these mechanisms.

Remote robotic explorations for collapsed buildings in a severe disaster are demanded. However, rescue robots cannot approach the rubble due to safety risks. This study proposes a remote vertical exploration system for collapsed buildings with a robotic inspection system hoisted by a crane. An Active Scope Camera (ASC) has many advantages for the vertical exploration such as a light and flexible continuum body to produce distributed driving forces. The purpose of this paper is to confirm the feasibility of the vertical exploration system with the ASC. The vertical explorations have proper problems related to contact and hanging conditions of the scope cable. We developed a new ASC that has a two-step bending mechanism to produce larger head movement in multi-DOF. We also evaluated the performances of the prototype when the contact areas were small. Finally, we conducted a remote vertical exploration experiments at the simulated collapsed building in 6 m height. The robot could explore in six different pathways by changing head directions and running the rubbles within seven trials. The experimental results showed that the proposed system has high potential to get inserted in the deep area in the rubble.

The Fukushima nuclear power plant accident that occurred in 2011 eastern Japan has created local spots with a high dose level in residential areas in eastern Japan. It is necessary to build a map of the distribution of radiation activities that shows the locations and intensities of radiation sources to assist in decontamination work. The proposed method estimates the intensities of point-like sources from the radiation dose measured by moving a dosimeter in three dimensions and 3D map of the environment, assuming that the radioactive sources are located on the ground surface. The radiation dose is measured as a few counts including a wide variability because in the proposed method, low levels of radiation are measured by moving a dosimeter. For estimating the intensities, the maximum a posteriori probability (MAP) estimation is computed on the basis of the radiation characteristics of attenuation and stochastic counts. Experiments of estimation in real environments were conducted. The experimental results showed that using the proposed method, it is possible to estimate the distributions of radiation sources using the radiation dose measured by a dosimeter and the measured 3D shape of the ground and building surface.

This paper describes logical winnowing methods from multiple identification candidates using corresponding appearance identification results with chronological pedestrian tracking results. It is difficult to identify individual using appearance identification, because appearance identification has some properties. This research proposes two methods that logically winnow out the identification candidates as methods that effectively fuse different directional identification results without the directional information. Experiments were made to verify the validity of the proposed methods. A mobile robot equipped with a laser scanner and a camera was used in the experiments. A pedestrian tracking method uses the laser scanner. The appearance identification uses the camera. The experimental results verified the validity of the logical winnowing method taking the logical product of identification candidates determined by each round of identification. In this paper, the appearance identification properties, the proposed methods and the experiments are described.

In this paper, an estimation method using the flexible multi-body dynamics model, inertial/magnetic sensor system, and probabilistic state estimation is proposed for estimating the shape of a flexible cable. Sensors mounted on flexible cables, such as endoscopes, are often picked up noise, and their size and number are restricted. The proposed method can estimate motion using the flexible dynamics model if there are few sensors. In addition, this method is robust against noise because it can integrate some types of sensor information by probabilistic state estimation. The proposed method uses the Absolute Nodal Coordinate Formulation (ANCF) as a flexible dynamics model and the Unscented Kalman Filter (UKF) for probabilistic state estimation. It can model a greatly deformed cable at any point using ANCF, and it can fit nonlinear motion and achieve high versatility about models because of UKF. In the second half of this paper, experiments using Active Scope Camera (ASC) are conducted to evaluate the proposed method. The ASC is a snake-like rescue robot, and its sensor system picks up strong noise. In the experiments, the proposed method estimates three types of motion, and it shows the effectiveness if the number of sensors decreases.

Gyro-based odometry is an easy-to-use localization method for tracked vehicles because it uses only internal sensors. However, on account of track-terrain slippage and transformation caused by changes in sub-track angles, gyro-based odometry for tracked vehicles with sub-tracks experiences difficulties in estimating the exact location of the vehicles. In order to solve this problem, we propose an estimation method with 6 degrees of freedom (DOF) for determining the position and pose of the tracked vehicles using terrain information. (In this study, position refers to the robot's position and pose.) In the proposed method, position are estimated using a particle filter. The subsequent position of each particle are predicted using a motion model that separately considers each contact point of the vehicle with the ground. In addition, each particle is analyzed using terrain and gravity information. Experimental results demonstrate the effectiveness of this method.

This paper describes a 2D localization method for a differential drive mobile vehicle on real forested paths. The mobile vehicle is equipped with two rotary encoders, Crossbow’s NAV420CA Inertial Measurement Unit (IMU) and a NAVCOM SF-2050M GPS receiver (used in StarFire-DGPS dual mode). Loosely-coupled multisensor fusion and sensor fault detection issues are discussed as well. An extended Kalman Filter (EKF) is used for sensor fusion estimation where a GPS noise pre-filter is used to avoid introducing biased GPS data (affected by multi-path). Normalized Innovation Squared (NIS) tests are performed when a GPS measurement is incorporated to reject GPS data outliers and keep the consistency of the filter. Finally, experimental results show the performance of the localization system compared to a previously measured ground truth.

This paper presents and describes the approach for achieving long distance autonomous navigation with a mobile robot on outdoor cluttered pedestrian paths. The task was to finish an event launched by the City of Tsukuba in Japan, called “RealWorld Robot Challenge”, of navigating 1km autonomously in a real environment with real pedestrians and bicycles. The hardware, software and strategy for navigating in cluttered environments is explained. Moreover, the complementary functionality of the overall system where map-based and sensor-based navigation seamlessly change, is presented. The robustness of the system is validated with experimental results.

This paper proposes fast map building algorithm for multiple modal exteroceptive(external) sensors based on trajectory optimization. To realize localization using multiple modal exteroceptive sensors needs consistent environment maps (landmark maps). Multiple sensor SLAM (simultaneous localization and mapping) is one of solutions to build maps consistently. In implementation of SLAM, approximations of sensor model contribute to get speeded up the algorithm. However in several case, these approximations deteriorate quality of the map. This paper proposes fast and accurate map building algorithm using multiple modal sensors based on sparse linear equations solver. This algorithm needs less approximations-but computation speed is fast. In addition, application of measured data from multiple sensors to this algorithm is described.

The present authors take such an approach to build a service mobile robot that all the robot functions are categorized into common functions and specific functions. The common functions are assigned to a mobile robot and the specific functions are assigned to a specific mechanism. A "portable effector" is designed for a mechanism to accomplish the specific task. The portable effector is placed on the floor near the task space in the environment. A navigation and docking functions with the effector provided as common functions for the mobile robot. The robot changes the portable effector automatically according to the assigned task. In this paper, the authors describe techniques of automatic docking with portable effector for the mobile robot. They include position alignment technique proper to docking by PWS type mobile robot using simple sensor and mark, and firm holding mechanism between the robot and the effector. The docking mechanism is designed so as to allow alignment error at the time of docking. The authors illustrate portable effector changing operation using two kinds of portable effectors in the real environment

A 3D scan matching is an important component for sensor based localization and mapping by a mobile robot in natural environment. In this paper, the present authors propose a way to extend 2D normal distributions transform (NDT) scan matching method to 3D scan matching, and its improvement for faster processing time. This scan matching method divides scan into voxels, and approximates scan points in each cell into normal distribution. That matching time is O(N) with N of the number of input scan points. The authors describe in this paper, NDT for 3D scan points, its acceleration using the dual resolutions of NDT, and experiments of map building in large scale environments