The arterial blood pressure waveform (ABPW) serves as a less invasive technique for evaluating hemodynamic parameters, offering a lower risk compared to the more invasive pulmonary artery catheter (PAC) thermodilution method. Various studies suggest that deep learning models can potentially predict the hemodynamic parameters of ABPW. However, the scarcity of ground truth data restricts the accuracy of these models, preventing them from gaining clinical acceptance. To mitigate this data and domain challenge, this work proposed a self-supervised generative learning model for hemodynamic parameter prediction, called SSHemo (Self-Supervised Hemodynamic model). Specifically, SSHemo suggests first to leverage large amounts of unlabeled ABPW data to learn the representative embedding and then to fine-tune for the downstream task with a small amount of hemodynamic parameters’ ground truth. To verify the effectiveness of SSHemo, we utilize the public available VitalDB data set to train the model, and evaluation was conducted on two public datasets: VitalDB and MIMIC. The experimental results reveal that SSHemo’s regression mean absolute error (MAE) improved significantly from 1.63 L/min to 1.25 L/min when predicting cardiac output (CO). The trending tracking ability for CO changes meets clinical acceptance (radial limit of agreement (LOA) is $$\pm 25.56$$ °, less than $$\pm 30$$ °). In addition, SSHemo demonstrates robust stability in various conditions and cohorts, as evidenced by subgroup analysis, varying systemic vascular resistance (SVR) range analysis, and rapid CO analysis, compared to the most widely used commercial devices, the EV1000. Computational analysis further underscores the value and potential of practical application of the model in various settings.

Object rearrangement is widely demanded in many of the manipulation tasks performed by industrial and service robots. Rearranging an object through planar pushing is deemed energy efficient and safer compared with the pick-and-place operation. However, due to the unknown physical properties of the object, re-arranging an object toward the target position is difficult to accomplish. Even though robots can benefit from multi-modal sensory data for estimating novel object dynamics, the exact estimation error bound is still unknown. In this work, firstly, we demonstrate a way to obtain an error bound on the center of mass (CoM) estimation for the novel object only using a position-controlled robot arm and a vision sensor. Specifically, we extend Mason's Voting Theorem (MVT) to object CoM estimation in the absence of accurate information on friction and object shape. The probable CoM locations are monotonously narrowed down to a convex region, and the Extended Voting Theorems (EVT's) guarantee that the convex region contains the CoM ground truth in the presence of contact normal estimation error and pushing execution error. For the object translation task, existing methods generally assume that the pusher-object system's physical properties and full-state feedback are available, or utilize iterative pushing executions, which limits the application of planar pushing to real-world settings. In this work, assuming a nominal friction coefficient between the pusher and object through contact normal error bound analysis, we leverage the estimated convex region and the Zero Moment Two Edge Pushing (ZMTEP) method [1] to select the contact configurations for object pure translation. It is ensured that the selected contact configurations are capable of tolerating the CoM estimation error. The experimental results show that the object can be accurately translated to the target position with only two controlled pushes at most.

Pushing is one of the fundamental nonprehensile manipulation skills to impart to an object changes in position and orientation. To exploit this skill to manipulate novel objects, explicit knowledge of their physical properties should be givena priori. In this work, we estimate the center of mass (CoM) of an object by narrowing down its probable location with a deep learning model and Mason’s voting theorem. In addition, we propose the Zero Moment Two Edge Pushing (ZMTEP) method to translate a novel object without rotation to a goal pose. The proposed method enables a pusher to select the most suitable two-edge-contact configuration for a given object using the estimated CoM and the geometrical shape of the object. Notably, neither the friction between the object and its support plane nor the friction between the object and the pusher are assumed to be known. We evaluate the proposed CoM estimation and ZMTEP methods through a series of experiments in both simulation and real robotic pusher settings. The result shows that the CoM estimation method has good mean squared error properties and small standard deviation, and the ZMTEP method significantly outperforms competitive baseline methods.Note to Practitioners—This article aims to endow robotic arms with the capability of moving or aligning objects by pushing, which is much more simple and secure than pick-and-place or in-hand manipulations. Most in-demand manipulation skills require sophisticated hand design and control, which might not be affordable for industrial applications staying cost-competitive. In contrast, robot pushing can be implemented with different types of simple pushers and straightforwardly applied to pre-grasp manipulation. This article makes the estimation of an object’s CoM location practical. Building upon the estimation method, a robust and noise-tolerant two-edge-contact pushing configuration selection method is presented to translate an arbitrarily shaped unknown object to its goal pose.

Cardiac Output (CO) is a key hemodynamic variable that can be estimated in a minimally invasive way via using Arterial Pressure Waveform Analysis (APWA). Many models use circulation mechanics to build the relationship between arterial pressure and CO. In this study, we attempt to apply machine learning and feature engineering to analyze the Arterial Pressure Waveform (APW) and create regression models to predict the CO. We utilize the traditional APWA model knowledge and the time-domain, frequency-domain, and other characteristics of time series data for feature engineering. We present the benchmarking results for several machine learning models using the MIMICII waveform database. We compare the predicted CO values from our proposed models with the “gold standard” TCO (CO measured by intermittent pulmonary artery thermodilution). Our results show that the Random forest model has the most accurate agreement (MSE: 1.421 $\displaystyle \text{L}/\min$, bias: $-0.01\displaystyle \text{L}/\min$, 95% limits of agreement: $-2.35\displaystyle \text{L}/\min$ to $+2.32\displaystyle \text{L}/\min$, percentage error: 39.44%). Notably, the XGBoost model demonstrates good tracking ability with TCO (radius bias: 11.79o, 95% radius limits of agreement: ±28.89°), achieving the clinically acceptable level.

Non-verbal behaviors have an indispensable role for social robots, which help them to interact with humans in a facile and transparent way. Especially, communicative gestures allow robots to have the capability of using bodily expressions for emphasizing the meaning of their speech, describing something, or showing clear intention. This paper presents an approach to learn the synthesis of human actions and natural language. The generative framework is inspired by Conditional Generative Adversarial Network (CGAN), and it makes use of the Convolutional Neural Network (CNN) with the Action Encoder/Decoder for action representation. The experimental and comparative results verified the efficiency of the proposed approach to produce human actions synthesized with text descriptions. Finally, through the Transformation model, the generated data were converted to a set of joint angles of the target robot, being the robot's communicative gestures. By employing the generated human-like actions for robots, it suggests that robots' social cues could be more understandable by humans.

The video describes the novel concept of “culturally competent robotics”, which is the main focus of the project CARESSES (Culturally-Aware Robots and Environmental Sensor Systems for Elderly Support). CARESSES a multidisciplinary project whose goal is to design the first socially assistive robots that can adapt to the culture of the older people they are taking care of. Socially assistive robots are required to help the users in many ways including reminding them to take their medication, encouraging them to keep active, helping them keep in touch with family and friends. The video describes a new generation of robots that will perform their actions with attention to the older person's customs, cultural practices and individual preferences.

In this report, a new framework is proposed for inferring the user's personality traits based on their habitual behaviors during face-to-face human-robot interactions, aiming to improve the quality of human-robot interactions. The proposed framework enables the robot to extract the person's visual features such as gaze, head and body motion, and vocal features such as pitch, energy, and Mel-Frequency Cepstral Coefficient (MFCC) during the conversation that is lead by Robot posing a series of questions to each participant. The participants are expected to answer each of the questions with their habitual behaviors. Each participant's personality traits can be assessed with a questionnaire. Then, all data will be used to train the regression or classification model for inferring the user's personality traits.

The way in which humans behave, speak and interact is deeply influenced by their culture. For example, greeting is done differently in France, in Sweden or in Japan; and the average interpersonal distance changes from one cultural group to the other. In order to successfully coexist with humans, robots should also adapt their behavior to the culture, customs and manners of the persons they interact with. In this paper, we deal with an important ingredient of cultural adaptation: how to generate robot plans that respect given cultural preferences, and how to execute them in a way that is sensitive to those preferences. We present initial results in this direction in the context of the CARESSES project, a joint EU-Japan effort to build culturally competent assistive robots.

Mobile robots are increasingly being used to assist with active pursuit and law enforcement. One major limitation for such missions is the resource (battery) allocated to the robot. Factors like nature and agility of evader, terrain over which pursuit is being carried out, plausible traversal velocity and the amount of necessary data to be collected all influence how long the robot can last in the field and how far it can travel. In this paper, we develop an analytical model that analyzes the energy utilization for a variety of components mounted on a robot to estimate the maximum operational range achievable by the robot operating on a single battery discharge. We categorize the major consumers of energy as: 1.) ancillary robotic functions such as computation, communication, sensing etc., and 2.) maneuvering which involves propulsion, steering etc. Both these consumers draw power from the common power source but the achievable range is largely affected by the proportion of power available for maneuvering. For this case study, we performed experiments with real robots on planar and graded surfaces and evaluated the estimation error for each case.

Generating emotional bodily expressions for culturally competent robots has been gaining increased attention to enhance the engagement and empathy between robots and humans in a multi-culture society. In this paper, we propose an incremental learning model for selecting the user's representative or habitual emotional behaviors which place emphasis on individual users' cultural traits identified through long term interaction. Furthermore, a transformation model is proposed to convert the obtained emotional behaviors into a specific robot's motion space. To validate the proposed approach, the models were evaluated by two example scenarios of interaction. The experimental results confirmed that the proposed approach endows a social robot with the capability to learn emotional behaviors from individual users, and to generate its emotional bodily expressions. It was also verified that the imitated robot motions are rated emotionally acceptable by the demonstrator and recognizable by the subjects from the same cultural background with the demonstrator.

In this paper, we focus on large-scale environment monitoring by utilizing a fully decentralized team of mobile robots. The robots utilize the resource constrained-decentralized active sensing scheme to select the most informative (uncertain) locations to observe while conserving allocated resources (battery, travel distance, etc.). We utilize a distributed Gaussian process (GP) framework to split the computational load over our fleet of robots. Since each robot is individually generating a model of the environment, there may be conflicting predictions for test locations. Thus, in this paper, we propose an algorithm for aggregating individual prediction models into a single globally consistent model that can be used to infer the overall spatial dynamics of the environment. To make a prediction at a previously unobserved location, we propose a novel gating network for a mixture-of-experts model wherein the weight of an expert is determined by the responsibility of the expert over the unvisited location. The benefit of posing our problem as a centralized fusion with a distributed GP computation approach is that the robots never communicate with each other, individually optimize their own GP models based on their respective observations, and off-load all their learnt models on the base station only at the end of their respective mission times. We demonstrate the effectiveness of our approach using publicly available datasets.

We propose a homing constrained bi-objective optimization variant of budget-limited informative path planning for monitoring a spatio-temporal environment. The objective function consists of weighted combination of two components: model performance which must be maximized and travel distance which must be bounded by the maximum operational range. Besides this, we have additional constraints that guarantee that the robots will return to home (base station) upon completion of their respective missions. Optimizing over this objective function is essentially NP-hard owing to the conflicting constituents. Moreover, the appropriate choice of weights and additional homing guarantees further adds to complications. We employ Gaussian Process (GP) model [1] which is highly data driven i.e., the larger the amount of training data, the better the model performance. However, owing to limited resources, a robot can only collect a limited amount of training samples. Thus, with the introduction of our bi-objective cost function, it becomes possible to plan budget-limited (e.g., battery, flight time, travel distance etc.) informative tours using autonomous mobile robots to effectively select only the most informative (uncertain) locations from the environment. In this work, we develop an algorithm to autonomously choose the appropriate weights for the components based on available resources while ensuring homing and maintaining model quality. We perform simulations to verify the effectiveness of our proposed objective function on the publicly available Ozone Concentration dataset gathered from USA.

Cultural competence is a well known requirement for an effective healthcare, widely investigated in the nursing literature. We claim that personal assistive robots should likewise be culturally competent, aware of general cultural characteristics and of the different forms they take in different individuals, and sensitive to cultural differences while perceiving, reasoning, and acting. Drawing inspiration from existing guidelines for culturally competent healthcare and the state-of-the-art in culturally competent robotics, we identify the key robot capabilities which enable culturally competent behaviours and discuss methodologies for their development and evaluation.

Cultural differences may influence interactions between humans with different social norms and cultural traits, incurring different emotional and behavioral responses. The same applies to human-robot interaction (HRI). We believe that controlling robot emotions based on the cultural context can help robots adapt to humans from culturally diverse backgrounds. Such culturally aligned robots are expected to be easily accepted by humans as part of daily life. In this paper, we aim at investigating the role of culture in representing robot emotions which are injected by humans during its early stage of development and subject to change through their own experience thereafter. Several public data sets of pictures labeled with affective ratings by Indian, American, and European subjects are presented to social humanoid Pepper robots. The result shows that robots can learn to behave socially in alignment with an individual's cultural background. Moreover, we have demonstrated that robots under the effect of different cultures can generate different behavioral responses to the same stimuli, which is considered one of the most important issues in socially assitive robotics.

In this paper, we present a novel cognitive framework allowing a robot to form memories of relevant traits of its perceptions and to recall them when necessary. The framework is based on two main principles: on the one hand, we propose an architecture inspired by current knowledge in human memory organisation; on the other hand, we integrate such an architecture with the notion of context, which is used to modulate the knowledge acquisition process when consolidating memories and forming new ones, as well as with the notion of familiarity, which is employed to retrieve proper memories given relevant cues. Although much research has been carried out, which exploits Machine Learning approaches to provide robots with internal models of their environment (including objects and occurring events therein), we argue that such approaches may not be the right direction to follow if a long-term, continuous knowledge acquisition is to be achieved. As a case study scenario, we focus on both robot–environment and human–robot interaction processes. In case of robot–environment interaction, a robot performs pick and place movements using the objects in the workspace, at the same time observing their displacement on a table in front of it, and progressively forms memories defined as relevant cues (e.g. colour, shape or relative position) in a context-aware fashion. As far as human–robot interaction is concerned, the robot can recall specific snapshots representing past events using both sensory information and contextual cues upon request by humans.

Toward a global picture of the radiation exposure of an area, particularly for fast emergency response, a UAV based exploration method is proposed. Without a priori knowledge of the radiation field, it is difficult to select the region of interest (ROI) which includes all radiation sources. For the case of a single radiation source, a greedy algorithm may localize the source by finding the maximum radiation value. However, when multiple sources generate a hotspot in a cumulative manner, the hotspot position does not coincide with one of the source positions. Therefore, we propose an efficient exploration method to quickly localize the radiation sources using the following procedures: (1) ROI selection using topographic maps with specific radiation level selection methods and (2) source localization estimating the number of sources and their positions with incremental variational Bayes inference of Gaussian mixtures. Under three different conditions according to the number of sources and their positions, we have shown that the proposed model can reduce the ROI and significantly improve the estimation accuracy than existing methods.

We present a robot cognitive framework based on (a) a memory-like architecture; and (b) the notion of “context”. We posit that relying solely on machine learning techniques may not be the right approach for a long-term, continuous knowledge acquisition. Since we are interested in long-term human-robot interaction, we focus on a scenario where a robot “remembers” relevant events happening in the environment. By visually sensing its surroundings, the robot is expected to infer and remember snapshots of events, and recall specific past events based on inputs and contextual information from humans. Using a COTS vision frameworks for the experiment, we show that the robot is able to form “memories” and recall related events based on cues and the context given during the human-robot interaction process.

An unsupervised attention-path planning algorithm is proposed and applied to large unknown area classification with small field-of-view cameras. Attention-path planning is formulated as the sequential feature selection problem that greedily finds a sequence of attentions to obtain more informative observations, yielding faster training and higher accuracies. In order to find the near-optimal attention-path, adaptive submodular optimization is employed, where the objective function for the internal belief is adaptive submodular and adaptive monotone. First, the amount of information of attention areas is modeled as the dissimilarity variance among the environment data set. With this model, the information gain function is defined as a function of variance reduction that has been shown to be submodular and monotone in many cases. Furthermore, adapting to increasing numbers of observations, each information gain for attention areas is iteratively updated by discarding the non-informative prior knowledge, enabling to maximize the expected information gain. The effectiveness of the proposed algorithm is verified through experiments that can significantly enhance the environment classification accuracy, with reduced number of limited field of view observations.

Our paper newly presents unsupervised feature representation method for very low-resolution (VLR) images called informative census transform (ICT) based on statistical analysis of CT binary features and submodular optimization. A new cost function is designed to measure the informativeness of each binary feature: (1) an individual informativeness of features to represent unlabeled image dataset and (2) relative informativeness between binary features to represent different binary features. Therefore, we considered informativeness of binary feature according to two relationship (1) between feature space and image space, and (2) between different features within same feature space. Moreover, two constraints are designed by considering sub-modular characteristics to guarantee theoretical performance and fast optimization via simple greedy algorithm. Experimental results show that the proposed ICT features with two constraints outperforms the traditional CT features in terms of recognition performance and computational cost at VLR problem.

This paper presents an interactive control for our assistive robotic walker, the JAIST Active Robotic Walker (JARoW), developed for elderly people in need of walking assistance. The focus of our paper is placed on how to estimate the user's walking parameters by sensing the locations of lower limbs and to predict his or her walking patterns. For this purpose, a particle-filter-based prediction technique and a motion controller are developed to help JARoW smoothly generate the direction and velocity of its movements in a way that reflects the prediction. The proposed scheme and its implementation are described in detail, and outdoor experiments are performed to demonstrate its effectiveness and feasibility in everyday environments.

This paper presents a novel interactive control for our assistive robotic walker, the JAIST Active Robotic Walker (JARoW), developed for elderly people in need of assistance. The aim of our research is to recognize characteristics of the user's gait and to generate the movement of JARoW accordingly. Specifically, the proposed control enables JARoW to accurately generate the direction and velocity of its movement in a way that corresponds to the user's variable walking behaviors. The algorithm and implementation of the control are explained in detail, and the effectiveness and usability of JARoW are verified through extensive experiments in everyday environments.

This paper proposes a new attention-path planning algorithm that allows robots with limited sensing coverage to identify an unknown entity efficiently. Our focus is placed on how to plan optimal sequences of views to access more useful information needed to understand the entity. The adaptive submodular optimization technique guaranteed to achieve near-optimal performance is used to maximize the expected information gain. We verified the validity of the proposed approach to the face recognition problem through preliminary experiments.

This paper presents a 3D online path planning algorithm for Unmanned Aerial Vehicles (UAVs) equipped with limited range sensors and computational resources in unknown cluttered environments. Even though quadrotor UAVs are considered to be a promising technology for surveillance purposes in indoor environments and for close observation in outdoor urban areas, it is very difficult to achieve autonomous aerial navigation toward a goal avoiding unpredicted collisions. Furthermore, greater attention and effort should be aimed at improving the computational efficiency and performance of path planning algorithms. The proposed heuristic algorithm offers on-the-fly path findings with a lesser computational complexity. We demonstrate the efficiency of our algorithm in a real world scenario implemented using the V-REP simulator.

The five papers in this special section were originally presented at the 2012 IEEE International Conference on Automation Science and Engineering (CASE), held in Seoul, Korea during the week of August 20.

In this paper, we describe the design and implementation of an integrated two dimensional and three dimensional location measurement system, where different types of range sensors can be mounted onto the spiral motion positioner. The proposed sensor/positioner system enables terrestrial and aerial robots to observe their surroundings in all directions without blind spots. Using a nut-and-bolt and link mechanism, the proposed positioner driven by a single stepper motor exhibits continuous three dimensional spiral trajectories over the upper hemisphere. This single axis motor driven system helps decrease the size, weight, and structural complexity of the system. Particular attention in this work is placed on how to effectively combine two dimensional and three dimensional measurement functions. We verify the validity and effectiveness of the proposed location measurement system through simulations and experiments. It is expected that the proposed system can be incorporated into a wide range of mobile robot platforms.

This paper presents a new control scheme of an assistive robotic cart that helps a user easily transport objects in various weights. The maneuverability of the cart would be highly affected by the loaded weight of the cart and the friction between the wheel and the floor. Our focus is placed on how to enable the cart to offer easy maneuverability by creating desired interactions with a user. For this, a switched observer based impedance control scheme is proposed to allow the cart to autonomously adapt to changes in the weight of the load and the friction. Specifically, a pre-determined impedance between the user and the cart is regulated to generate an assist force using the user's input force and the velocity of the cart. The switched observer is designed to estimate the loaded weight and the friction coefficient for the precise computation of the amount of assistance. Further, using the process integration and design optimization approach, the observer gains are automatically adjusted, resulting in enhancing the control performance. We describe the proposed scheme in detail, and perform extensive simulations to demonstrate its effectiveness.

This paper presents a new control scheme of JAIST Active Robotic Walker (JARoW) developed to provide elderly people with sufficient ambulatory capability. Toward its practical use, our focus is placed on how to allow easier and reliable maneuverability by creating a natural user interface. Specifically, our challenge lies in providing a well-functioning controller by detecting what the user wants to do or their intentions. A Kalman filter based tracking scheme is realized to estimate and predict the locations of the user's legs and body in real time. The feedback control can then adjust the motions of JARoW corresponding to the actual user's walking behaviors. Our experiments confirm that JARoW can autonomously adjust its motion direction and velocity without requiring any additional control inputs.

We present a new control scheme of JAIST Active Robotic Walker (JARoW) developed to provide potential users such as the elderly with sufficient ambulatory capability. Toward its practical use, we tackle JARoW's easy and reliable maneuverability by creating a natural user interface between a user and JARoW. Specifically, our focus is placed on how to realize the natural and smooth movement of JARoW despite different gait parameters of users. For this purpose, a particle filtered interface function (PFIF) is proposed to estimate and predict the locations of the user's legs and body. Then, the simple feedback motion control function adjusts the motions of JARoW corresponding to the estimation and prediction. Experimental results show that the proposed control scheme can be quite satisfactory for practical use without requiring any additional user effort.

This paper presents a novel low-cost position detection prototype from practical design to implementation of its control schemes. This prototype is designed to provide mobile robot swarms with advanced sensing capabilities in an efficient, cost-effective way. From the observation of bats' foraging behaviors, the prototype with a particular emphasis on variable rotation range and speed, as well as 360° observation capability has been developed. The prototype also aims at giving each robot reliable information about identification of neighboring robots from objects and their positions. For this purpose, an observation algorithm-based sensor is proposed. The implementation details are explained, and the effectiveness of the control schemes is verified through extensive experiments. The sensor provides real-time location of stationary targets positioned 100 cm away within an average error of 2.6 cm. Moreover, experimental results show that the prototype observation capability can be quite satisfactory for practical use of mobile robot swarms.

This paper addresses the deployment problem for a swarm of autonomous mobile robots initially randomly distributed in 3 dimensional space. A fully decentralized geometric self-configuration approach is proposed to deploy individual robots at a given spatial density. Specifically, each robot interacts with three neighboring robots in a selective and dynamic fashion without using any explicit communication so that four robots eventually form a regular tetrahedron. Using such local interactions, the proposed algorithms enable a swarm of robots to span a network of regular tetrahedrons in a designated space. The convergence of the algorithms is theoretically proved using Lyapunov theory. Through extensive simulations, we validate the effectiveness and scalability of the proposed algorithms.

Recently, biologically inspired control approaches for robotic systems that involve the use of central pattern generators (CPGs) have been attracting considerable attention owing to the fact that most humans or animals move and walk easily without explicitly controlling their movements. Furthermore, they exhibit natural adaptive motions against unexpected disturbances or environmental changes without considering their kinematic configurations. Inspired by such novel phenomena, this paper endeavors to achieve self-adapting robotic arm motion. For this, biologically inspired CPG based control is proposed. In particular, this approach deals with crucial problems such as motion generation and repeatability of the joints emerged remarkably in most of redundant DOF systems. These problems can be overcome by employing a control based on artificial neural oscillators, virtual force and virtual muscle damping instead of trajectories planning and inverse kinematics. Biologically inspired motions can be attained if the joints of a robotic arm are coupled to neural oscillators and virtual muscles. We experimentally demonstrate self-adaptation motions that that enables a 7-DOF robotic arm to make adaptive changes from the given motion to a compliant motion. In addition, it is verified with real a real robotic arm that human-like movements and motion repeatability are satisfied under kinematic redundancy of joints.

We address the problem of dispersing a large number of autonomous mobile robots toward building wireless ad hoc sensor networks performing environmental monitoring and control. For the purpose, we propose the adaptive triangular mesh generation algorithm that enables robots to generate triangular meshes of various sizes adapting to changing environmental conditions. A locally interacting, geometric technique allows robots to generate each triangular mesh with their two neighbor robots. Specifically, we have assumed that robots are not allowed to have the identifier, any pre-determined leaders or common coordinate systems, and any explicit communication. Under such minimal conditions, the positions of the robots were shown to converge to the desired distribution, which was mathematically proven and also verified through extensive simulations. Our preliminary results indicate that the proposed algorithm can be applied to the problem regarding the coverage of an area of interest by a swarm of mobile sensors.

This paper presents practical design and hardware implementation issues of self-configuring swarms of autonomous mobile robots. For the purpose, we develop a new low-cost position detection system that we call dual rotating infrared (DRIr) sensor. The DRIr sensor can provide robots with advanced sensing capabilities that give reliable information about the position and surface geometry of neighboring robots and obstacles. Special focus is placed on how to realize the observation and object identification of mobile robots through the use of DRIr sensors. We verify the functionality and performance of the DRIr sensors mounted on a commercial mobile robot. Experimental results show that a swarm of mobile robots equipped with the DRIr sensors can autonomously configure themselves into an area.

This paper proposes a neural oscillator based control to attain rhythmically dynamic movements of a robot arm. In human or animal, it is known that neural oscillators could produce rhythmic commands efficiently and robustly under the changing task environment. In particular, entrainments of the neural oscillator play a key role to adapt the nervous system to the natural frequency of the interacted environments. Hence, we discuss how a robot arm controls for exhibiting natural adaptive motions as a controller employing the entrainment property. To demonstrate the excellence of entrainment, we implement the proposed control scheme to a real robot arm. Then this work shows the performance of the robot arm coupled to neural oscillators in various tasks that the arm traces a trajectory. Exploiting the neural oscillator and its entrainment property, we experimentally verify an impressive capability of self-adaptation of the neural oscillator that enables the robot arm to make adaptive changes corresponding to an exterior environment.

It is known that biologically inspired neural systems could exhibit natural dynamics efficiently and robustly for motion control, especially for rhythmic motion tasks. In addition, humans or animals exhibit natural adaptive motions without considering their kinematic configurations against unexpected disturbances or environment changes. In this paper, we focus on rhythmic arm motions that can be achieved by using a controller based on neural oscillators and virtual force. In comparison with conventional researches, this work treats neither trajectories planning nor inverse kinematics. Instead of those, a few desired points in task-space and a control method with Jacobian transpose and joint velocity damping are merely adopted. In addition, if the joints of robotic arms are coupled to neural oscillators, they may be capable of achieving biologically inspired motions corresponding to environmental changes. To verify the proposed control scheme, we perform some simulations to trace a desired motion and show the potential features related with self-adaptation that enables a three-link planar arm to make adaptive changes from the given motion to a compliant motion. Specifically, we investigate that human-like movements and motion repeatability are satisfied under kinematic redundancy of joints.

This paper proposes an adaptive CPG based controller for a lower limb prosthesis consisting of online trajectory generation and interlimb coordination. The adaptive CPG can produce multidimensional rhythmic patterns and modulate their frequency by tuning relevant parameters in an autonomously way adapting to a changing periodicity of external signals. Also, to increase the stability of the prosthesis, a spring-damper component is attached between the hip and ankle joints, allowing the absorption of impulsive ground reaction forces at landing. We verify the validity of the proposed controller with a simulated humanoid robot through the investigation of the self-coordination between the healthy and robotic legs.

A self-contained direction sensing radio frequency identification (RFID) reader is developed employing a dual-directional antenna for automated target acquisition and docking of a mobile robot in indoor environments. The dual-directional antenna estimates the direction of arrival (DOA) of signals from a transponder by using the ratio of the received signal strengths between two adjacent antennas. This enables the robot to continuously monitor the changes in transponder directions and ensures reliable docking guidance to the target transponder. One of the technical challenges associated with this RFID direction finding is to sustain the accuracy of the estimated DOA that varies according to environmental conditions. It is often the case that the robot loses its way to the target in a cluttered environment. To cope with this problem, the direction correction algorithm is proposed to triangulate the location of the transponder with the most recent three DOA estimates. Theoretical simulation results verify the reliability of the proposed algorithm that quantifies the potential error in the DOA estimation. Using the algorithm, we validate mobile robot docking to an RFID transponder in an office environment occupied by obstacles.

We address the problem of deploying a swarm of autonomous mobile robots toward building an ad hoc network of robotic sensors with spatial uniform density. For the purpose, each of the robots configures themselves into an area with geographical constraints through local interactions with two adjacent neighboring robots. The basic idea underlying this work is that robots can be thought of as liquid particles that change their positions conforming to the shape of the container they occupy. The main challenge is how to cope with the accuracy limitations of sensors and possible holes in the configuration. Considering such realistic conditions, the convergence of the proposed method is proved using Lyapunov’s theorem. The proposed method is verified to be effective through the simulation for the secure deployments of robotic sensor network.

Humans or animals exhibit natural adaptive motions against unexpected disturbances or environment changes. In this paper, we focus on periodic, rhythmic arm motions that can be achieved by using a controller based on neural oscillators. The challenge of this work is to determine appropriate parameters of neural oscillators coupled to a robot arm, accomplishing a given task as well as self-sustaining natural rhythms. For this, an enhanced simulated annealing (SA) algorithm is developed. This work also demonstrates how to technically implement the proposed control scheme to a real robot. Exploiting the entrainment property of neural oscillators coupled to the joints of the arm, we verify that the arm traces a trajectory in such a way that the total energy consumption is minimized, responding to external disturbances.

Automated target acquisition and docking is key to enabling various applications of autonomous mobile robots in indoor environments. For the purpose, many researches have been devoted to the development of location sensing techniques employing the latest in RFID or GPS. However, it has not yet become possible to attain high accuracy in those techniques, particularly in cluttered or dynamically changing environments. In this paper, we propose a novel location sensing RFID reader equipped with a dual directional antenna that communicates with controllable RF transponders. The dual directional antenna estimates the direction of arrival (DOA) of signals from various transponders by using the ratio of the received strength between two antennas. This enables the robot to continuously monitor the changes in the ratio and find its way to the target transponder. To verify the validity of the proposed system in real environments populated with unknown obstacles, we perform detailed experiments using simulations and hardware implementations. Specifically, the target acquisition and docking guidance are demonstrated in a multiple transponder environment under various circumstances.

This paper presents a motion planning framework for a large number of autonomous robots that enables the robots to configure themselves adaptively into an area of an arbitrary geometry. A locally interacting geometric technique provides a unique solution that allows the robots to converge to the uniform distribution by forming an equilateral triangle with their two neighbors. The basic idea underlying the proposed solution is that robots can be thought of as liquid particles that change their relative positions conforming to the shape of the container they occupy. Specifically, it is assumed that robots are not allowed to have the identification number, a pre-determined leader, a common coordinate system, and communication capabilities. Under such minimal conditions, the convergence of the algorithm is mathematically proved and verified through extensive simulations. The results validate the feasibility of applying the algorithm to self-configuration of mobile sensors across the constrained environment.

Stable and robust dynamic locomotion has been gaining increasing attention in humanoid research. This paper presents a neural oscillator network for the generation of periodic locomotion patterns adapting to changes in the slope of the terrain. Specifically, locomotion trajectories of individual limbs are predetermined in the trajectory generator as a periodic function of the gait. The phase of the periodic function is coordinated with the output of the neural oscillator network incorporating sensory signals detecting the state of the foot in contact with the unknown changing terrain. For stability to be maintained, the neural oscillator plays an important role by controlling the trajectory of the COM in phase with the trajectory of the ZMP. In order to verify the validity of the proposed scheme, we carry out simulations and experiments. A preliminary investigation has yielded promising results, indicating that it may be applied to humanoid locomotion through uneven and uncertain terrain.

In this study, we focus on a self-deployment problem for a swarm of autonomous mobile robots that can be used to build a sensor networking infrastructure with equilateral triangle lattice configurations. In order to deploy the swarm, this paper proposes a self-stabilizing distributed self- deployment algorithm under a robot model with the following features: no identification numbers, no common coordinates, no predetermined leader, no memory for past actions and implicit communication. Regardless of the restricted model, our proposed algorithm based on local interactions provides a solution for the self-deployment problem. Moreover, the algorithm provides robust capability of swarm connectivity in spite of loss of several robots. We discuss in details the features of the algorithm, including self-organization, self-stabilization, and robustness. A simulation study demonstrates the validity of the algorithm.

A parameter tuning scheme for the neural oscillator is addressed to achieve biologically inspired robot control architectures based on a neural oscillator. It would be desirable to determine appropriately unknown parameters of the neural oscillator to accomplish a task of rhythmic movement under various changes of environment. Human or animal exhibits natural dynamics with efficient and performs robust motions against unexpected disturbances or environment changes. The neural oscillator needs to be tuned using its optimal parameters to generate such natural movement. As simple examples, this paper connects the neural oscillator to a pendulum system and a rotating crank system. To determine the optimal parameters of the neural oscillator for the examples, the optimization scheme based on the Simulated Annealing (SA) method is used. We verify the performance of the given tasks with the obtained optimal parameters of the neural oscillator, showing the adaptation motions of the example systems with entrainment property in numerical simulations.

This paper presents a novel flocking strategy for a large-scale swarm of robots that enables the robots to navigate autonomously in an environment populated with obstacles. Robot swarms are often required to move toward a goal while adapting to changes in environmental conditions in many applications. Based on the observation of the swimming behavior of a school of tunas, we apply their unique patterns of behavior to the autonomous adaptation of the shape of robot swarms. Specifically, each robot dynamically selects two neighboring robots within its sensing range and maintains a uniform distance with them. This enables three neighboring robots to form a regular triangle and remain stable in the presence of obstacles. Therefore, the swarm can be split into multiple groups or re-united into one according to environmental conditions. More specifically, assuming that robots are not allowed to have individual identification numbers, a pre-determined leader, memories of previous perceptions and actions, and direct communications to each other, we verify the validity of the proposed algorithm using the in-house simulator. The results show that a swarm of robots repeats the process of partition and maintenance passing through multiple narrow passageways

We propose a novel framework for imitation learning that helps a humanoid robot achieve its goal of learning. There are apparent discrepancies in shapes and sizes among humans and humanoid robots. It would be advantageous if robots could learn their behavior from different individuals. Toward this end, this paper discusses appropriate behavior generation method through imitation learning considering that demonstrator and imitator robots have different kinematics and dynamics. As part of a wider interest in behavior generation in general, this work mainly investigates how an imitator robot adapts a reference locomotion gait captured from a demonstrator robot. Specifically, a goal-directed adaptation process that we call self-adjusting adaptor is proposed to achieve stable locomotion of the imitator. The proposed adaptor has an important role that the perceived locomotion patterns are modified to keep the direction of lower leg contacting the ground identical between the demonstrator and the imitator, sustaining the dynamic stability by controlling the position of the center of mass. The validity of the proposed scheme is evaluated through simulations employing various imitator models on OpenHRP and then verified on a real robot

In this work, we propose a new framework for better deployment and utilization of robots in our uncertain, unstructured everyday environments. Programming robots can be a very time-consuming process and seems almost impossible for ordinary end users. To cope with many challenges in the user programming, this work is to provide an open environment for building robot programming automatically, where we have robots learn how to accomplish commanded tasks interacting with the object. An integrated sensing and computing tag is embedded into every single object in the environment. In the robot controller, only the basic software libraries for low-level robot motion control are provided by the robot manufacturer. The main contributions of this work is to develop the knowledge integrator platform that we call Omniscient Organizer that generates the application programs and send them to the robot controller through networks. In the Omniscient Organizer, the object-related information downloaded from the object Web server merges into robot control software based on the task commands from the human. We have built a test bed and demonstrated that a robot can perform common household tasks such as clearing the table within the proposed framework.

We propose a collaborative knowledge network that we call omniscient spaces in the attempt to generate sophisticated robotic behavior with minimal programming effort. New products are manufactured and brought into our daily life everyday. Robots should need a way to easily integrate new products into their existing recognizable environments. Radio frequency identification gains increasing attention to support context and ambient awareness in dynamically changing environments. To solve robot programming difficulties in our environments, the collaborative knowledge network connects heterogeneous knowledge resources to collectively build up the robot's knowledge required to accomplish a task. Specifically, a decentralized knowledge acquisition and task specific integration model is proposed, where the proposed knowledge integrator merges specific knowledge with existing knowledge into a task requiring knowledge. For this, manufacturers put their product data tailored to plan robot motions online and robots may access the data without authorization. In this work, the best possible scenario under current technological limitations is proposed for real world robot applications. A detailed analysis of the knowledge flow model is described. To verify the validity of the proposed model, a test bed is built and table clearing task is performed according to the distributed knowledge management framework.

The Intelligent Systems Laboratory (ISL) has been developing coordinated control technologies for multitelerobot collaboration, in a common environment remotely controlled from multiple operators physically at a distance from each other. We have built a test bed and conducted a series of experiments, where we learned more about how the transmission delay over the network deteriorates the performance of telerobots. Previously, to overcome the problems arising from the throughput of the network such as the operator's delayed visual perception, we have suggested several coordination approaches in the local operator site. Likewise, this paper discusses the use of virtual repulsive force field in the online predictive simulator to assist the operator to cope with the collision between telerobots in remote environments. In the test bed, the operators control their master robot to get remote telerobots to work cooperatively with the other telerobots in a task. Specifically, the operator detects a priori the possibility of collision in the predictive simulator that runs in near real-time and the use of virtual force field prevents the telerobots from coming into collision. We have demonstrated various tasks by two telerobots and two operators via an Ethernet local area network (LAN) subject to simulated communication delays and evaluated the validity of the virtual force field guided approach in multi-operator-multi-robot (MOMR) tele-collaboration.

The speed, accuracy, and adaptability of human movement depends on the brain performing an inverse kinematics transformation-that is, a transformation from visual to joint angle coordinates-based on learning from experience. In human motion control, it is important to learn a feedback controller for the hand position error in the human inverse kinematics solver. This paper proposes a novel model that uses disturbance noise and the feedback error signal to learn coordinate transformations of the human visual feedback controller. The proposed model redresses drawbacks in current models because it does not rely on complex signal switching, which does not seem neurophysiologically plausible. Numerical simulations show the effectiveness of the model.

Inverse kinematics computation using an artificial neural network that learns the inverse kinematics of a robot arm has been employed by many researchers. However, the inverse kinematics system of typical robot arms with joint limits is a multivalued and discontinuous function. Since it is difficult for a well-known multilayer neural network to approximate such a function, a correct inverse kinematics model cannot be obtained by using a single neural network. In order to overcome the discontinuity of the inverse kinematics function, we proposed a novel modular neural network system that consists of a number of expert neural networks. Each expert approximates the continuous part of the inverse kinematics function. The proposed system uses the forward kinematics model for selection of experts. When the number of the experts increases, the computation time for calculating the inverse kinematics solution also increases without using the parallel computing system. In order to reduce the computation time, we propose a novel expert selection by using the performance prediction networks which directly calculate the performances of the experts.

Various coordinated control schemes are explored in the multi-operator-multi-robot (MOMR) tele-collaborative system through a network with time delay. Multi-robot cooperation has rapidly emerged in many possible applications such as the plant maintenance, construction, and surgery, because it would have a significant advantage over a single robot in such cases. Thus, time-delayed control of a multi-robot system is expected to play an important role in remote operations, too. However, the effect of time-delay would pose a more difficult problem to the MOMR teleoperation systems and seriously affect their performance. In this work, first, we have built an experimental system to investigate the remote cooperation in MOMR teleoperation. Then, different coordinated control methods are proposed to cope with the collision, arising from the time delay over the network. To verify the validity of the proposed schemes, we have carried out various experiments on a planar block arrangement by two slave arms employing graphic simulators and a LAN subject to a significant communication delay.

The Mechanical Engineering Laboratory (MEL) is developing technologies for multi-robot collaboration in remote environments with the Toshiba Mechanical Systems Laboratory (TMSL). Human operators' delayed visual perception due to the communication delay over the network seriously affects the collaboration performance in the multi-operator-multi-robot tele-manipulation. We have built an experimental tele-manipulation test bed connecting the MEL and the TMSL to study the time-delayed remote tele-collaboration between two places. In the test bed, one operator controls her master robot nearby two slave robots in the work site and another from a distance. A local online graphics simulator without time delay is developed to cope with the communication delay and several multi-robot coordinated control strategies are devised to avoid the collision. To verify the validity of this simulator assisted approach, we demonstrated a maintenance work on a plant mock-up in the TMSL by two different slave robots and two distant operators in the MEL and TMSL respectively through an ISDN.

A compliance control strategy for robot manipulators is proposed by employing a self-adjusting stiffness function. To be specific, each entry of the diagonal stiffness matrix corresponding to task coordinate in Cartesian space is adaptively adjusted during contact along the corresponding axis based on the contact force with its environment. It can also be used for both unconstrained and constrained motions without any switching mechanism which often causes undesirable instability and/or vibrational motion of the end-effector. The experimental results show the effectiveness of the proposed method by employing a two-link direct drive manipulator interacting with an unknown environment.

In this paper, remote robot collaboration using a network with communication time delay is discussed in multi-operator-multi-robot (MOMR) teleoperation. Recently, collaboration tasks have rapidly emerged in many possible applications such as plant maintenance, construction, and surgery, because multi-robot collaboration would have a significant advantage over a single robot in such cases. Problems and several noticeable results have been reported in a single-operator-single-robot (SOSR) teleoperation system. However, the effect of time-delay would pose a more difficult problem to the MOMR teleoperation systems and seriously affect their performance. In this work, first, some of the constraints on performance in MOMR teleoperation applications are examined. Then, the time-delay effects on tele-collaboration are investigated through several experimental studies. Finally, a method to cope with the time-delay in MOMR teleoperation systems is proposed exploiting the virtual thickness modification scheme.

A new concept of a robot manipulator with passive mobile base is proposed which can be safely used in a human-robot cooperation environment. A joint-based control scheme is developed to control the end-effector position of the manipulator which is tolerant to unexpected contact force with the manipulator's environment. The mobility the of base of the manipulator is passively exploited to attenuate the dynamic disturbance of the contact force and the joint configurations are re-adjusted according to the evolution of the position of the base to maintain performance of the given task. Effectiveness of the proposed method is verified by computer simulation and experimental evaluation is currently underway.

A hierarchical planning strategy for dextrous manipulation of multifingered hands with soft finger contact model is proposed. Dextrous manipulation planning can be divided into a high-level stage which specifies the position/orientation trajectories of the fingertips on the object and a low-level stage which determines the contact forces and joint trajectories for the fingers. In the low-level stage, various nonlinear optimization problems are formulated according to the contact modes and integrated into a manipulation planning algorithm to find contact forces and joint velocities at each time step. Montana's contact equations are used for the high-level planning. Quasistatic simulation results are presented and illustrated by employing a three-fingered hand manipulating a sphere to demonstrate the validity of the proposed low-level planning strategy.>

A hierarchical planning strategy for dextrous manipulation of multifingered hands with soft finger contact model is proposed. Dextrous manipulation planning can be divided into a high-level stage which specifies the position/orientation trajectories of the finger-tips on the object and a low-level stage which determines the contact forces and joint trajectories for the fingers. In the low-level stage, various nonlinear optimization problems are formulated according to the contact modes and integrated into a manipulation planning algorithm to find contact forces and joint velocities at each time step. Montana's contact equations (1988) are used for the high-level planning. A real-time compensation tactics to eliminate the trajectory errors of the object resulted from various uncertainties are also developed. Simulation results are presented and illustrated by employing a three-fingered hand manipulating a sphere to demonstrate the validity of the proposed strategy.>

A generalized algorithm for the motion/force planning of the multifingered hand is proposed to generate finite displacements and changes in orientation of objects by considering sliding contacts as well as rolling contacts between the fingertip and the object at the contact point. Specifically, a nonlinear optimization problem is firstly formulated and solved to find joint velocities and contact forces to impart a desired motion to the object at each time step. Then, the relative velocity at the contact point is found by calculating the velocity of the fingertip and the object at the contact point. Finally, time derivatives of the surface variables and the contact angle of the fingertip and the object at the current time step is computed using the Montana's contact equation to find the contact parameters of the fingertip and the object at the next time step. To show the validity of the proposed algorithm, a numerical example is illustrated by using the robotic hand manipulating a circular cylinder with three fingers each of which has four joints.