For a service robot, it is not adequate to let its navigational movement be based only on a single metric, such as minimum distance path. In the environment where the robot and humans are coexisting, the robot should always perform social navigation whenever it is moving. However, to perform social navigation, the robot needs to follow certain “social norms” of the environment. Recently, deep reinforcement learning (DRL) technique is popularly applied to the robotics field; yet, it is rarely used to solve the mentioned social navigation problem, generally deemed as a high dimension complex problem. In this paper, we propose the composite reinforcement learning (CRL) framework under which the robot learns appropriate social navigation with sensor input and reward update based on human feedback. For learning the aspect of human robot interaction (HRI), we provide a method to facilitate the training of DRL in real environment by incorporating prior knowledge to the system. It turns out that our CRL system not only can incrementally learn how to set its velocity and to perform HRI but also keep collecting human feedback to synchronize the reward functions to the current social norms. The experiments show that the proposed CRL system can safely learn how to navigate in the environment and show that our system is able to perform HRI for social navigation.

In an ageing society, we expect that a robotic caregiver is able to persuade the elderly to perform a healthier behavior. In this work, pragmatic argument is adopted to make the elderly realize that a choice beneficial for health is really worthwhile, such as eating suitable fruits. Based on this concept, an adaptive recommendation dialogue system through pragmatic argumentation is proposed. There are three objectives in this system. First, a knowledge base for pragmatic argument construction is built, which concerns not only the effect of a decision but also the reason for the effect. Secondly, the robot is endowed with the ability to do recommendation that adapts to different states of the elder, and the recommendation is determined based on the integration of both the robot's and the elder's preference for different perspectives so that the robot knows how to reach a compromise with the elder. Lastly, through learning about the elder's preference for perspectives in conversation, the robot will try to select such a perspective to construct arguments that the elder can be more easily convinced to accept its recommendation. We invited 21 volunteers to interact with the robot. The experimental result has proved that the recommendation system has potential to affect the decision making of the elderly and help him/her pursue a healthier life.

For a visual servo system, there usually exists the problem of time-delay likely caused by long image processing and data transmission. The visual servo system of our robot is subject to two main limitations stemming from the specific commercial mobile manipulator, of which one is the large time-delay due to image transmission, whereas the other is failure to directly command each joint velocity as the visual servo's control input. In this paper, we propose a novel visual servo system to compensate the large time-delay, and we use integral of velocity which incorporating the image estimation and prediction with the kinematic model so as to achieve indirect velocity control with position based control mode joint. Our proposed visual servo system controls the neck joints of robot and keeps the object within the field of view of the camera during the approaching phase. We evaluate the proposed approach by experiment on a real wheeled humanoid robot and find the resulting performance quite promising, showing that our method is feasible and practical.

In this paper, we propose a sensory cues guided robotic walker for improving gaits of Parkinson Disease (PD) patients. A completely non-intrusive, 3D real-time leg pose tracking and gait analysis are proposed by using a depth camera mounted on the rear of the robotic walker. It has been studied that the sensory cues can serve as effective stimuli to the PD patients for gait improvement. In our work, the sensory cues including visual and auditory cues are incorporated into the robotic walker. More specifically, both sensory cues are gait-adaptive, of which the visual cue in particular is projected onto the ground by a projector installed on the walker, in order to stimulate patients walking gait more easily. Since the adjustable cues can improve patients gaits and reduce their uncomfortableness simultaneously, the hereby developed robotic walker serves as a gait rehabilitation mechanism. To demonstrate the performance of the developed walker, several real experiments have been conducted. First, the accuracy of the proposed 3D leg pose tracking is verified by a standard motion capture system. Next, seven participants (4PD patients and 3 healthy elders) are recruited to test the system three days for verifying the effectiveness of participants gait improvement. The experimental results confirm the potential of the walker serving as a rehabilitation device for PD patients. Note to Practitioners-We proposed a completely non-intrusive, relatively inexpensive and real-time gait analyzer integrated with sensory cues guided rehabilitation system on an active robotic walker. Our goal is to provide a smart robotic walker with safety, reliability, and with rehabilitation function for elders whoever suffered from chronic disease or health problem. In this work, a reliable 3D leg pose tracking is proposed. Then, a gait analysis for acquiring spatio-temporal gait parameters such as stride length and gait velocity is proposed based on the tracking to analyze the walking gait of the patients. On the other hand, improving Parkinson disease patients gaits by using sensory cues which are known to have remarkable effects for Parkinson disease patients is in many research. Therefore, we concentrate on the sensory cues include visual cue and a rhythmic auditory cue cooperating with the robotic walker which aimed to stimulate and provide walkingassistance in their ambulation. For the sensory cues, an adaptive gait mechanism is proposed to improve patients gait based on their personal gait pattern as well as to reduce uncomfortableness of them by doing the rehabilitation. The experimental results confirm the potential of the walker serving as a rehabilitation device for Parkinson disease patients. We hope the new invention of this kind assistive robotic walker will become more popular and may provide more living aid facilities to elders.

For service robots to be able to enter a multi-human office environment, it is important to find a group of human users' social patterns and then to provide a proper service to them in time. Usually, human users' social patterns are represented in terms of nonverbal social signals. In this paper, a new integrated approach on recognizing multi-human social signals is proposed. Specifically, the nonverbal social signals are detected by a laser range finder and a RGB-D camera and are processed to find the multi-human (spatial) social patterns. Those recognized patterns are then applied to human-to-human, human-to-robot or multi-human-to-robot interactive formation. Experimental results shows that our robot successfully recognizes the aforementioned users' social patterns followed by appropriate services.

In this paper, we propose an online scene reconstruction algorithm with monocular camera since there are many advantages on modeling and visualization of an environment with physical scene reconstruction instead of resorting to sparse 3D points. The goal of this algorithm is to simultaneously track the camera position and map the 3D environment, which is close to the spirit of visual SLAM. There're plenty of visual SLAM algorithms in the current literature which can provide a high accuracy performance, but many of them rely on stereo cameras. It's true that we'll face many more challenges to accomplish this task with monocular camera. However, the advantages of cheaper and easier deployable hardware setting have made monocular approach more attractive. Specifically, we apply a maximum a posteriori Bayesian approach with optimization technique to simultaneously track the camera and build a dense point cloud. We also propose a feature expansion method to expand the density of points, and then online reconstruct the scene with a delayed approach. Furthermore, we utilize the reconstructed model to accomplish visual localization task without extracting the features. Finally, a number of experiments have been conducted to validate our proposed approach, and promising performance can be observed.

This paper introduces a context-aware assisted active robotic walker for Parkinson's disease (PD) patients. Most of PD patients suffer from not only loss of balance but also abnormal gaits. These symptoms tend to make PD patients fall down more easily and result in low quality of life. We use Hidden Markov Model (HMM) to analyze the gait of PD patients, and then use our walker to help patients adjust their gait to become normal while applying auditory cues when abnormal gaits are recognized. To prevent user from leaning forward before falling down, the walker locks the motors when sudden forward pushing by the user is detected. Moreover, the walker can record the statics of gait from the user, making the therapists monitor the rehabilitation process relatively easier. Finally, the road conditions in front of the walker will be automatically analyzed, making user able to adjust his/her walking pace dynamically. To our best knowledge, the hereby proposed active robotic walker should be the first system which can provide walking aid to PD patients. In our experiments, the feasibility and performance of this system are evaluated by PD patients at two actual senior care units.

RGB-D sensor has gained its popularity in the study of object recognition for its low cost as well as its capability to provide synchronized RGB and depth images. Thus, researchers have proposed new methods to extract features from RGB-D data. On the other hand, learning-based feature representation is a promising approach for 2D image classification. By exploiting sparsity in 2D image signals, we can learn image representation instead of using hand-crafted local descriptors like SIFT or HoG. This framework inspired us to learn features from RGB-D data. Our work focuses on two goals. First, we propose a novel Hierarchical Sparse Shape Descriptor (HSSD) to form learning-based representation for 3D shapes. To achieve this, we analyze several 3D feature extraction techniques and propose a unified view of them. Then, we learn hierarchical shape representation with sparse coding, max pooling and local grouping. Second, we investigate whether RGB and depth information should be fused at lower level or higher level. Experimental results show that, first, our HSSD algorithm can learn shape dictionary and provide shape cues in addition to the 2D cues. Using the proposed HSSD algorithm achieves 84% accuracy on a household RGB-D object dataset and outperforms a widely used VFH shape feature by 13%. Second, fusing RGB-D information at lower level does not improve recognition performance.

In this paper, we present a novel data structure representing the environment with occupancy grid cells while each grid map is associated with a set of line features extracted from laser scan points. Due to the fact that line segments are principal elements of artificial environments, they provide considerable geometric information about the environment which can be used for enhancing the accuracy of localization. Orthogonal characteristic of line features is the key issue to guarantee the consistency of the SLAM algorithm by allowing us to deal with lines that are parallel or perpendicular to each other. This behavior allows us to sample robot poses more correctly. As a result, the proposed algorithm can close bigger loops with the same number of particles. Experimental results are carried out using SICK LMS-100 laser scanner which has a maximum range of 20 m and Pioneer 3DX mobile robot mapping an indoor environment with the size of 40 m × 47 m.