Freezing of gait (FoG) is a common and debilitating symptom in patients with Parkinson ’s disease (PD), often leading to falls and reduced quality of life. This study proposes a robotic walker system that provides rhythmic self-induced stimuli by adapting its speed based on the user ’s gait phases. The walker utilizes pressure sensors embedded in insoles to detect stance and swing phases in real-time and adjusts its speed accordingly to promote more stable walking. Experimental results with both healthy older adults and PD patients suggest potential improvements in gait cadence and step length under the proposed control method. Challenges and future improvements are also discussed.

The challenge of human-robot collaboration, particularly in the context of enhancing the productivity of work processes, has been a pivotal area of research and development for many years. Despite significant advancements, there remains a substantial gap in the design of these systems to cater specifically to individuals with disabilities. This paper presents an innovative approach in assistive robotics, focusing on the development of a work cell designed to facilitate individuals with single-arm functionality. Through the integration of depth camera technology, machine learning algorithms, and Mediapipe human tracking, our system is capable of interpreting human intentions, thereby making interactions with robots more intuitive and effective. Central to our research is the design of a specialized workspace that assists in object handling and incorporates a fully functional One-Handed Soldering Tool, integrated within a robotic arm setup. This work cell is tailored for users with limited arm functionality, demonstrating the system’s versatility and providing invaluable insights into the practical implementation of applied robotics to bridge the theoretical and practical aspects of assistive technology.

The sense of touch plays an indispensable role in human perception by enabling the discernment of various objects based on their textures, roughness, and shapes. In the realm of robotics, touch sensors play a vital role in the manipulation and control of objects. By providing information on mechanical properties, these sensors facilitate the visualization of grip, shear, and pressure force at contact points. However, handling deformable objects such as clothing poses a significant challenge. While some research relies on visual sensors for the detection and classification of clothing types, this research focuses on leveraging tactile sensors for the discrimination and classification of deformable surfaces in real-time.The aim is centered on soft surfaces, with a specific focus on examining seven different types of materials. The system we present focuses on signal processing and machine learning techniques for cloth recognition, and classification in real time. The system integrates a soft touch sensor fingertip into a robotic arm to achieve these objectives.The current study has developed a classification model leveraging Long Short-Term Memory (LSTM) networks. The objective of this model is to accurately classify seven distinct types of clothing by analyzing their texture features.

In the field of human-robot interaction (HRI), social coordination, meaning the cooperation between humans and robots through emotional and cognitive states, can be an essential element in considering the social position of robots for social implementation. A handshake is an interaction involving physical and social coordination in that it is the first interaction between humans involving physical contact, but also the first opportunity for social interaction. In this paper, we developed a CPG (Central Pattern Generator) controlled handshake robot to clarify the relationship between this social and physical coordination. In the experiment, we investigated how the control method affects the robot’s social attributes as perceived by humans by using the Robot Social Attributes Scale (RoSAS) for passive control methods using simple impedance control and cooperative and active control methods using the CPG controller we developed. Results from a 14-subject experiment showed that the CPG-based control method improved the social attributes of being competent and comfortable with the robot. Based on the results of these studies, we propose guidelines for future research on handshake robots and contribute to the development of future social robots.

We propose a framework for a new control method for robotic dressing assistance systems that utilize the residual capabilities of the person being dressed. The proposed control method focuses on the swinging motion of the arm during dressing, and by passively controlling the arm in response to the human’s movement, the robot does not inhibit the human’s movement but only provides the minimum necessary support for dressing. The proposed control method is realized by assuming that the arm-swinging movements of the human during controlled dressing have some regularity and that modeling is possible. By acquiring and analyzing the skeletal information of the person being dressed during the dressing assistance by the robot, the possibility of the proposed method is discussed by proving the assumption, and the future prospects for the realization of a controller that makes use of residual abilities are also discussed.

In rapidly aging societies, robotic solutions for clothing assistance can significantly improve the quality of life of the elderly while coping with the shortage of caregivers. Previously, we proposed a framework for the same by employing imitation learning from a human demonstration to a compliant dual-arm robot. As the robot has a limited workspace, this framework involves a manual movement of the wheeled chair by pushing it while coordinating with the robot to stay within the workspace of the robot [1]. To avoid the manual push and coordination, we facilitate the automatic movement of the chair based on the trajectory of the robot's dual arms. In this paper, we present an approach for the collaboration of an electric wheelchair and a humanoid robot to achieve the clothing assistance task. Our approach incorporates Manifold Relevance Determination (MRD) to learn an offline latent model from the simultaneous observations of the clothing assistance task as well as the movement of the wheelchair. We trained and tested the latent model on different human subjects by dressing a sleeveless T-shirt. Experimental results verify the plausibility of our approach. To the best of our knowledge, this is the first work addressing collaboration between wheelchair and robot to perform clothing assistance.

The recent demographic trend across developed nations shows a dramatic increase in the aging population, fallen fertility rates and a shortage of caregivers. Robotic solutions to clothing assistance can significantly improve the Activity of Daily Living (ADL) for the elderly and disabled. We have developed a clothing assistance robot using dual arms and conducted many successful demonstrations with healthy people. It was, however, impossible to systematically evaluate its performance because human arms are not visible due to occlusion from a shirt and robot during dressing. To address this problem, we propose to use another robot, Whole-Body Robotic Simulator of the Elderly that can mimic the posture and movement of the elderly persons during the dressing task. The dressing task is accomplished by utilizing Dynamic Movement Primitives (DMP) wherein the control points of DMP are determined by applying forward kinematics on the robotic simulator. The experimental results show the plausibility of our approach.

Robotic solutions to clothing assistance can significantly improve quality of life for the elderly and disabled. Real-time estimation of the human-cloth relationship is crucial for efficient learning of motor skills for robotic clothing assistance. The major challenge involved is cloth-state estimation due to inherent nonrigidity and occlusion. In this study, we present a novel framework for real-time estimation of the cloth state using a low-cost depth sensor, making it suitable for a feasible social implementation. The framework relies on the hypothesis that clothing articles are constrained to a low-dimensional latent manifold during clothing tasks. We propose the use of manifold relevance determination (MRD) to learn an offline cloth model that can be used to perform informed cloth-state estimation in real time. The cloth model is trained using observations from a motion capture system and depth sensor. MRD provides a principled probabilistic framework for inferring the accurate motion-capture state when only the noisy depth sensor feature state is available in real time. The experimental results demonstrate that our framework is capable of learning consistent task-specific latent features using few data samples and has the ability to generalize to unseen environmental settings. We further present several factors that affect the predictive performance of the learned cloth-state model.

Real-time estimation of human-cloth relationship is crucial for efficient learning of motor skills in robotic clothing assistance. However, cloth state estimation using a depth sensor is a challenging problem with inherent ambiguity. To address this problem, we propose the offline learning of a cloth dynamics model by incorporating reliable motion capture data and applying this model for the online tracking of human-cloth relationship using a depth sensor. In this study, we evaluate the performance of using a shared Gaussian Process Latent Variable Model in learning the dynamics of clothing articles. The experimental results demonstrate the effectiveness of shared GP-LVM in capturing cloth dynamics using few data samples and the ability to generalize to unseen settings. We further demonstrate three key factors that affect the predictive performance of the trained dynamics model.

In this study, we propose a novel method for the real-time estimation of Human-Cloth relationship, which is crucial for efficient motor skill learning in Robotic Clothing Assistance. This system relies on the use of low cost depth sensor, which provides color and depth images without requiring an elaborate setup making it suitable for real-world applications. We present an efficient algorithm to estimate the parameters that represent the topological relationship between human and the clothing article. At the core of our approach are low dimensional representation of Human-Cloth relationship using topology coordinates for fast learning of motor skills and a unified ellipse fitting algorithm for the compact representation of the state of clothing articles. We conducted experiments that illustrate the robustness of these feature representations. Furthermore, we evaluated the performance of our proposed method by applying it to real-time clothing assistance tasks and compared the estimates provided by our method with the ground truth.

Prediction of driving behaviors is important problem in developing the next-generation driving support system. In order to take account of diverse driving situations, it is necessary to deal with multiple time series data considering commonalities and differences among them. In this paper we utilize the beta process autoregressive hidden Markov model (BP-AR-HMM) that can model multiple time series considering common and different features among them using the beta process as a prior distribution. We apply the BP-AR-HMM to actual driving behavior data to estimate VAR process parameters that represent the driving behaviors, and with the estimated parameters we predict the driving behaviors of unknown test data. The results suggest that it is possible to identify the dynamical behaviors of driving operations using BP-AR-HMM, and to predict driving behaviors in actual environment.

In this paper, we study probabilistic modeling of heterogeneously attributed multi-dimensional arrays. The model can manage the heterogeneity by employing an individual exponential-family distribution for each attribute of the tensor array. These entries are connected by latent variables and are shared information across the different attributes. Because a Bayesian inference for our model is intractable, we cast the EM algorithm approximated by using the Lap lace method and Gaussian process. This approximation enables us to derive a predictive distribution for missing values in a consistent manner. Simulation experiments show that our method outperforms other methods such as PARAFAC and Tucker decomposition in missing-values prediction for cross-national statistics and is also applicable to discover anomalies in heterogeneous office-logging data.

Although there have been much research on robot walking, the energy efficiency of central pattern generator (CPG)-based walking has not received much attention. This study proposes a novel method for acquiring energy-efficient CPG-based bipedal walking for a robot with knees and feet. In this method, we introduce a torque-free period for swing leg control into the swing leg control cycle. During this period, no torque is applied to the hip joint controller, and therefore no energy is consumed. When and for how long the torque-free period is inserted into the swing leg control cycle is adaptively acquired by reinforcement learning. Simulation experiments demonstrate the feasibility of our method. The energy consumed in acquiring walking is reduced by 40% compared with simple CPG-based walking without the torque-free period in the practical learning speed. Walking stability is maintained with respect to external disturbances on a level floor. Although the method is more unstable on slopes with the torque-free period, the torque-free-period can be adaptively eliminated to achieve stable walking on the slopes.

In this paper, we propose an improved mimesis method for interpolation and extrapolation of motion patterns in the proto-symbol space towards an ultimate goal that motion pattern synthesis and recognition of humanoid robots are achieved by means of natural language. The proto-symbol space is a topological space which abstracts motion patterns by utilizing continuous hidden Markov models. An interpolation algorithm for the proto-symbol space was proposed in a previous work, but an extrapolation algorithm was not. Therefore, in this study, we propose and extrapolation method which can further clarify the physical meaning of the dimension of the proto-symbol space that is one of the most essential issues for the realization of translation between motion patterns and language using the proto-symbol space. The extrapolation method also enables the robot to recognize and synthesis various kinds of motion patterns using a fewer number of proto-symbols. The feasibility of the proposed method is demonstrated through simulation experiments.

Recently, many researchers on humanoid robotics are interested in quasi-passive-dynamic walking (quasi-PDW) which is similar to human walking. It is desirable that control parameters in quasi-PDW are automatically adjusted because robots often suffer from changes in their physical parameters and the surrounding environment. Reinforcement learning (RL) can be a key technology to this adaptability, and it has been shown that RL realizes quasi-PDW in a simulation study. To apply the existing method to controlling real robots, however, requires further improvement to accelerate its learning, otherwise the robots will break down before acquiring appropriate controls. To accelerate the learning, this study employs off-policy natural actor-critic (off-NAC), and applies it to an acquisition problem of quasi-PDW. The most important feature of the off-NAC is that it reuses the samples that has already been obtained by previous controllers. This study also shows an adaptive method of the learning rate. Simulation as well as real experiments demonstrate that fast and stable learning of quasi-PDW of an unstable biped robot can be realized by our modified off-NAC

A class of biped locomotion called passive dynamic walking (PDW) has been recognized to be efficient in energy consumption and a key to understand human walking. Although PDW is sensitive to the initial condition and disturbances, some studies of quasi-PDW, which introduces supplementary actuators, are reported to overcome the sensitivity. In this article, for realization of the quasi-PDW, an on-line learning scheme of a feedback controller based on a policy gradient reinforcement learning method is proposed. Computer simulations show that the parameter in a quasi-PDW controller is automatically tuned by our method utilizing the passivity of the robot dynamics. The obtained controller is robust against variations in the slope gradient to some extent.

Following a moving target with a narrow-view foveal vision system is one of the essential oculomotor behaviors of humans and humanoids. This oculomotor behavior, called "smooth pursuit", requires accurate tracking control which cannot be achieved by a simple visual negative feedback controller due to the significant delays in visual information processing. In this paper, we present a biologically inspired smooth pursuit controller consisting of two cascaded subsystems: one is an inverse model controller for the oculomotor system; and the other is a learning controller for the dynamics of the visual target. The latter learns how to predict the target motion in head coordinates such that the tracking performance can be improved. We investigate our smooth pursuit system in simulations and experiments on a humanoid robot. By using a fast online statistical learning network, our humanoid oculomotor system is able to acquire a high performance smooth pursuit after about 5 seconds of learning despite significant processing delays in the system.

The goal of our research is to investigate the interplay between oculomotor control, visual processing, and limb control in humans and primates by exploring the computational issues of these processes with a biologically inspired artificial oculomotor system on an anthropomorphic robot. In this paper, we investigate the computational mechanisms for visual attention in such a system. Stimuli in the environment excite a dynamical neural network that implements a saliency map, i.e., a winner-take-all competition between stimuli while simultaneously smoothing out noise and suppressing irrelevant inputs. In real-time, this system computes new targets for the shift of gaze, executed by the head-eye system of the robot. The redundant degrees-of-freedom of the head-eye system are resolved through a learned inverse kinematics with optimization criterion. We also address important issues how to ensure that the coordinate system of the saliency map remains correct after movement of the robot. The presented attention system is built on principled modules and generally applicable for any sensory modality.

Accurate oculomotor control is one of the essential pre-requisites of successful visuomotor coordination. Given the variable nonlinearities of the geometry of binocular vision as well as the possible nonlinearities of the oculomotor plant, it is desirable to accomplish accurate oculomotor control through learning approaches. We investigate learning control for a biomimetic active vision system mounted on a humanoid robot. By combining a biologically inspired cerebellar learning scheme with a state-of-the-art statistical learning network, our robot system is able to acquire high performance visual stabilization reflexes after about 40 seconds of learning despite significant nonlinearities and processing delays in the system.

We propose an approach "online visual feature determination through discourse", which realizes the finding and the following task in a real environment. To segment the human's image from the complex background, it is possible to prepare many finds of basic visual features, and to combine them. The proposed approach is a solution for the problem of how to select and combine these visual features for each situation. Namely, a robot and a human make a conversation to search for the suitable visual features for the current situation. This idea enables the robot to find and follow a human successfully in a changeable background environment.

Oculomotor control is the foundation of most biological visual systems, as well as an important component in the entire perceptual-motor system. We review some of the most basic principles of biological oculomotor systems, and explore their usefulness from both the biological and computational point of view. As an example of biomimetic oculomotor control, we present the state of our implementations and experimental results using the vestibulo-ocular-reflex and opto-kinetic-reflex paradigm.

This paper discusses a stereo vision processing system which processes color and monochrome stereo video signals on a single vision processing board. Here, we propose a "field mixing" technique for multiplexing multiple video signals. A compact color stereo vision system based on this technique is developed for a mobile robot. This system can process multiple video signals simultaneously, and realizes flexible color stereo vision processing. In order to show the feasibility of this vision system, we installed it on our mobile robot, and implemented a correlation-based EZDF method for stereo tracking of an object. The experimental result of the tracking is shown.

Presents a practical mobile robot system, called Hyper Scooter, which a human can ride on, and share access to the environment through visual information. The advantage of these capabilities is significant. A user can give his/her instructions to the robot without skillful programming operations. This is supported by the authors' high speed tracking vision system. To show the advantage, a complex task is presented as an example. After being instructed the way to accomplish the task by a human, Hyper Scooter does it without explicit environmental models.