A method is presented for enhancing the steering maneuvers of human drivers using peripheral visual information generated based on the vehicle’s lateral positions on the road. Human drivers control vehicles mainly using visual feedback information received from the road. However, such feedback may sometimes become degraded due to various reasons. On the other hand, recent advances in sensory technologies make it possible to detect vehicle behavior on the road quite stably. In this paper, we investigate a way to compensate such impaired feedback using artificial visual stimuli. A series of LED devices were installed in the car’s doors to display the stimuli. We designed three types of illumination patterns (flow, position, and width conditions) that react to the current lateral position of the vehicle. Simulated driving studies were conducted to evaluate the effect of each pattern on the driver’s control of the vehicle. As a result, we found that the presenting the artificial visual stimuli without explanation could significantly reduce inappropriate steering maneuvers in the degraded feedback situations.

In room-scale virtual reality, providing continuous haptic feedback from touching grounded objects, such as walls and handrails, has been challenging due to the user’s walking range and the required force. In this study, we propose LoopBot, a novel technique to provide continuous haptic feedback from grounded objects using only a single user-following robot. Specifically, LoopBot is equipped with a loop-shaped haptic prop attached to an omnidirectional robot that scrolls to cancel out the robot’s displacement, giving the user the haptic sensation that the prop is actually fixed in place, or “grounded.” We first introduce the interaction design space of LoopBot and, as one of its promising interaction scenarios, implement a prototype for the experience of walking while grasping handrails. A performance evaluation shows that scrolling the prop cancels 77.5% of the robot’s running speed on average. A preliminary user test (N = 10) also shows that the subjective realism of the experience and the sense of the virtual handrails being grounded were significantly higher than when the prop was not scrolled. Based on these findings, we discuss possible further development of LoopBot.

This paper proposes a new method to guide a person along with a route when a mobile robot explains an exhibition. In previous work, there were problems that people overtake the robot and go out of the guide route, and that people move to positions that interferes with the movement of the robot. In response to these problems, a conventional solution is to frequently instruct people to move by using voice, displays, etc., and it was not a comfortable for people who are guided. In this paper, we focus on a mobile robot with a projection function and propose a method of controlling human positions without explicit instructions by combining movement and projection of a robot. We introduce three basic guiding behaviors combining projection and movement, each controlling a person and a robot into different positional relationship according to the situation of guiding. Experiments of basic guiding behaviors showed that both the existence of the robot and the position of the projected image are combined to affect the movement of the guided person. The result indicates that by selecting appropriate guide behavior according to the situation of guiding, the robot can guide a person effectively while controlling the position of the person.

Recently a lot of robots have been developed for supporting our daily life or patient care. In order to instruct such a support robot that can work in such cluttered environments, we have conventionally developed an intuitive robot teaching interface using a Time-of-Flight laser sensor on a pan-tilt actuator driven by a user. This interface enables us to control the direction of the laser spot to “click” a real object and instruct a robot to manipulate it by drag-and-drop operation throughout a PC world and a real world. In our conventional system, however, the success rates of grasping object was very low because only the position of the object can be taught, not the orientation of the object. To cope with the problem, in this paper we propose a system to easily grasp an object of arbitrary posture by measuring the locus of the laser spot using our real world click system. Some grasping experiments on various daily objects showed the effectiveness of the proposed method.

Automated driving reduces the burden on the driver, however also makes it difficult for the driver to understand the current situation and predict the future movement of the vehicle. When the acceleration due to automated driving occurs without future prediction, the driver's anxiety and discomfort are increased compared to the case in manual driving. To facilitate the prediction of the future behavior of the vehicle by the driver, this paper aims to design and evaluate a haptic interface that actuates the vehicle seat. Our system displays to the driver the movement of the vehicle a few seconds in the future, which allows the driver to make predictions and preparations. Using a driving simulator, we compared the conditions where the movement of the car was displayed in advance for the length of different time. The subjective evaluation of the driver showed that the predictability of the behavior of the vehicle were significantly increased compared to the case without display. The experiment also showed that comfortable feeling significantly decreased if the preceding display is too early.

Robots that share daily environments with us are required to behave in a socially acceptable manner. There are two important approaches to this purpose: 1) robots model human behavior, understand it properly and behave appropriately 2) robots present their understanding and future behavior to surrounding people. In this paper, considering people present various cues to other people around them using gaze and face direction, we focus on the latter approach and propose a robot that presents cues to an opposing pedestrian by turning face. Another problem with the conventional research is that the evaluation of the pedestrian's ease of passing with the robot depends only on the subjective impression, so it was difficult to design the robot's behavior based on the temporal change of the ease of walking. In this paper, we evaluate the fluctuation of the pedestrian's moving velocity vector as an index of the ease of walking and analyze the temporal change. We have conducted preliminary experiments in which 12 subjects passed by the robot and compared the three types of presentation methods using the face. By presenting information using a face, we confirmed that the subjects tended to have better impressions of walking based on subjective evaluation and that the walking was relatively easy to walk for several seconds while approaching the robot based on analyzing the fluctuation of the moving speed vector.

This paper discusses an amusement content in which a robot dances in response to an electric guitar sound played by the guitarist. In our system, the motion of the robot changes according to the guitar performance, and simultaneously, the performer changes his ad-lib by receiving the stimulus from the robot motion and its acoustic percussion sound. In other words, our system realizes a so-to-speak jam session that can give an interactive music stimulus between the performer and robot. The main technical subject of this research is a method of determining an appropriate function from the guitar sound to the robot motion. Our system can extract various guitar sound features from the electric guitar sound. Based on the features, appropriate motions are selected from a motion database prepared in advance. The selection is based on the evaluation result of the audience who have seen the performance. Specifically a neural network that can output the correct answer is constructed by giving a set of highly evaluated sound and motion as training data. We introduce the results of experiments with a 6 degree-of-freedom animal robot holding a maracas by the hand.

In this work, we were interested in creating a robot service for offering help to people who appear to be in need for guidance. In order to achieve that, we first developed a technique to detect pedestrians who walk in an atypical way (e.g., people who do not know their way). In our approach, a motion model of typical pedestrians developed in our previous work was used, and a novel predictability feature was defined that quantifies how well can a person's future position be predicted using that model. The classification method based on this feature gave accurate results and outperformed alternative methods. Using this detection method, we created a robot service for offering guidance to people who were classified as atypical. Experiments done in a shopping mall have shown that the robot was successful in choosing the people to approach, and the reactions from users in the interviews were very positive.

This work proposes a Human-Comfortable Path Planner (HCoPP) system for autonomous passenger vehicles. The aim is to create a path planner that improves the feeling of comfort of the passenger, this topic is different from collision free planning and it has not received much attention. For this purpose, in addition to the shortest distance constraint conventionally used in path planning, constraints related to relevant environmental features are introduced. For straight segments, the constraint is based on the lane-circulation pattern preferred by humans. In curved segments and intersections, the constraint takes into account the visibility. A multi-layered cost map is proposed to integrate these additional constraints. To compute the human-comfortable path, a graph search algorithm was implemented. The evaluation of the proposed approach was conducted by having 30 participants riding an autonomous robotic wheelchair. The paths computed by the proposed path planner were compared towards a state of the art shortest-distance path planner implemented in the navigation stack of ROS. Experimental results show that the paths computed by the proposed approach are perceived as more comfortable.

This paper presents a study on intention communication in a navigational context using a robotic wheelchair. The robotic wheelchair uses light projection to communicate its motion intentions. The novelty of the work is threefold: the communication of robot intentions to the passenger, the consideration of passenger and robot as a group (“in-group”) [1] who share motion intentions and the communication of the in-group intentions to other pedestrians (the “out-group”). A comparison in an autonomous navigation task where the robotic wheelchair autonomously navigates the environment with and without intention communication was performed showing that passengers and walking people found intention communication intuitive and helpful for passing by actions. Evaluation results significantly show human participant preference for having navigational intention communication for the wheelchair passenger and the person passing by it. Quantitative results show the motion of the person passing by the wheelchair with intention communication was significantly smoother compared to without intention communication.

This work introduces a 3D visibility model for comfortable autonomous vehicles. The model computes a visibility index based on the pose of the wheelchair within the environment. We correlate this index with human navigational comfort (discomfort) and we discuss the importance of modeling visibility to improve human riding comfort. The proposed approach models the 3D visual field of view combined with a two-layered environmental representation. The field of view is modeled with information from the pose of the robot, a 3D laser sensor and a two-layered environmental representation composed of a 3D geometric map with traversale area information. Human navigational discomfort was extracted from participants riding the autonomous wheelchair. Results show that there is fair correlation between poor visibility locations (e.g., blind corners) and human discomfort. The approach can model places with identical traversable characteristics but different visibility and it differentiates visibility characteristics according to traveling direction.

The noninvasive brain-machine interface (BMI) is anticipated to be an effective tool of communication not only in laboratory settings but also in our daily livings. The direct communication channel created by BMI can assist aging societies and the handicapped and improve human welfare. In this paper we propose and experiment a BMI framework that combines BMI with a robotic house and autonomous robotic wheelchair. Autonomous navigation is achieved by placing waypoints within the house and, from the user side, the user performs BMI to give commands to the house and wheelchair. The waypoint framework can offer essential services to the user with an effectively improved information-transfer rate and is an excellent examples of the fusion of data measured by sensors in the house, which can offer insight into further studies.

A method for tracking the position, orientation, and height of persons in large public environments is presented. Such a piece of information is known to be useful both for understanding their actions, as well as for applications such as human-robot interaction. We use multiple 3-D range sensors, which are mounted above human height to have less occlusion between persons. A computationally simple-tracking method is proposed that works on single sensor data and combines multiple sensors so that large areas can be covered with a minimum number of sensors. Moreover, it can work with different sensor types and is robust to the imperfect sensor measurements; therefore, it is possible to combine currently available 3-D range sensor solutions to achieve tracking in wide public spaces. The method was implemented in a shopping center environment, and it was shown that good tracking performance can be achieved.

This study focuses on joint motion patterns of humans that move together with other humans or objects. Since this scope embraces `group motion', which relates only humans, and expands its extent of interactions accounting for various auxiliary instruments such as walking aids or pushcarts, we term this collective motion pattern as `coherent' motion. Coherence is proposed to be characterized by the distance between the moving parties, the scalar product of their velocities and the scalar product of the velocity vector and the displacement vector. The contribution of this study lies in the formulation of coherence in terms of the listed features through explicit mathematical models. The models are developed in accordance with a large database recorded in an uncontrolled environment involving a total of more than 500 mobile entities. The performance of the proposed models is evaluated qualitatively by comparing them to the empirical data and quantitatively by employing log-likelihoods. Comparison to an earlier work indicates that the proposed models improve the identification of coherence quality significantly well.

Recently, laser-based people-tracking systems have received increasing attention for their ability to localize human subjects precisely in crowded situations. However, in a real environment, there exist many kinds of moving objects other than people, and previous methods have focused only on humans. To design a more sophisticated system, it is necessary to distinguish humans from observed objects and recognize their individual condition-for example, the kind and amount of belongings they are carrying. However, in previous methods using 2D laser range finders (LRFs), it proved difficult to recognize the type of target since all sensors observe a common horizontal plane and only the 2D contours of their targets. In this study, to recognize the type of target, we observe the 3D shapes of objects moving in their environment by installing LRFs with an angle of inclination. So far, in the area of 3D modeling, LRFs have been used to construct 3D models of static objects by moving the sensor and registering multiple views. In contrast, our method observes moving objects by using a static LRF network in the environment. Experimental results are shown to confirm the effectiveness of the proposed method.

This paper proposes a simple yet novel method for recognition of certain sorts of moving entities incorporating their shape and motion patterns. Although shape features have been commonly employed in object recognition, motion characteristics are in general not integrated to geometric models. In the interest of utilizing the motion attributes, the trajectories are investigated to extract the `coherence quality' of the entities. Besides, at every step a geometric shape model is adopted and the parameters defining the shape model are utilized in obtaining the prior probabilities of the entities being a member of a particular class of interest. The coherence quality is used to get the posterior probabilities through a Bayesian approach. The main contribution of this paper is the incorporation of coherence quality in identification of moving entities. The proposed method is tested against clutter and occlusion in an uncontrolled environment with patterns collected from over 500 entities. It is shown to yield a satisfactory performance rate of 92% over the entire dataset with significant generalization capabilities without any restrictions on the application setting and with considerable occlusion and clutter.

To provide personal and location-dependent services in public spaces such as shopping malls, it is important to be able to estimate the positions and identities of people in the environment. Sensors in the environment reliably detect their current positions, but it is difficult to identify people using these sensors. On the other hand, signals from wearable sensors can be used to identify people correctly, but precise position estimation remains problematic. In this paper, we describe a novel method of integrating laser range finders (LRFs) in the environment and wearable inertial sensors. Time sequences of angular velocities estimated from both LRFs and wearable sensors are matched to identify people. Examples of tracking individuals in the environment that confirm the effectiveness of this method are shown.

To realize accurate tracking of people in the environment, many studies have been proposed using vision sensors, floor sensors, and wearable devices. The problem of using vision sensors is that they do not provide ID information of each people and there are ambiguities when people come across. To solve the problem, we propose to combine acceleration sensors that are attached to the human body. Since the signals from vision sensors and acceleration sensors synchronize when they observe same person who are acting or walking in the environment, these signals are not independent. The correlation between the signals is evaluated based on the canonical correlation analysis. Experimental results are shown to detect gesture and to track people to confirm the effectiveness of the proposed method.

The goal of this research is to develop systems that support human activities through environmental intelligence. These systems may include agents that interact with people in a sensor-rich setting. As a first step, we have developed a distributed vision system with 12 omnidirectional cameras to recognize what is happening at a subway station. Conventional recognition systems require intermediate models of human behavior, such as geometrical models. However, it is hard to calibrate many cameras with enough accuracy to use this framework. Therefore, we propose a memory-based method that does not require any calibration or a priori models. The developed system recognizes "people are walking toward a ticket gate," "people are going down the stairs," and so on. In addition, we have developed a method to select what information is necessary to make these discriminations. This method, which is analogous to an aspect of attention, reduces the computation involved in learning support vector machines to about 0.5% of its original value and increases classification accuracy by 2 to 23%.

The research approaches utilizing ubiquitous sensors to support human activities have become of major interest lately. Sensor fusion is one of the fundamental issues to develop such intelligent environments. The sensor fusion in previous works is performed in the task-level layer through individual representations of the sensors. Therefore, it does not provide new information by fusing sensors. This paper proposes another method that fuses sensory signals based on mutual information maximization in the signal-level layer. The fused signal provides us new information that cannot be obtained from individual sensors. As an example, this paper also shows experimental results in an audio-visual fusion task.