Jee-Hwan Ryu

dblp:58/2381 · DBLP profile ↗
← Back
77ranked-venue papers
14as first author
12since 2021 · last 2026
0000-0002-6497-7115ORCID · corroborated

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 59 · 10 first-author · 8 since 2021Systems, architecture and hardware · 56 · 9 first-author · 7 since 2021Applied, interdisciplinary, general and emerging computing · 16 · 5 first-author · 5 since 2021Human-computer interaction and ubiquitous computing · 7 · 1 first-author · 1 since 2021
YearPublicationVenuePosition
2026 MonoEndoCal: Autonomous Calibration for Tendon-Driven Endoscopic Robots Using Monocular Vision
abstract
Flexible endoscopic surgical robots provide enhanced access and maneuverability in complex anatomical environments, addressing limitations encountered by conventional rigid robotic systems. However, achieving precise control of these robots remains challenging due to inherent nonlinear hysteresis originating from friction and tendon slack in tendon-driven mechanisms, significantly impacting surgical accuracy and increasing the cognitive workload on surgeons. To overcome these challenges, this study introduces MonoEndoCal, a deep learning-based autonomous calibration framework designed to estimate bending angles and compensate for nonlinear hysteresis using only monocular endoscopic images, without the need for additional sensors or markers. Leveraging advanced vision models-Depth Anything V2 for dense depth estimation and Segment Anything Model 2 for zero-shot segmentation-MonoEndoCal accurately estimates robot bending angles in real-time. These angle estimations, along with corresponding actuation commands, are employed to derive a precise hysteresis model using differential equations capturing the robot's nonlinear dynamics. By integrating this hysteresis model with robot kinematics, MonoEndoCal implements an effective feedforward control strategy that enables precise hysteresis compensation and enhanced control accuracy. Experimental validations further demonstrate its robustness across varying robot geometries, input characteristics (both periodic and aperiodic), and blood contamination, highlighting its strong potential for improving control precision. Note to Practitioners-Flexible endoscopic surgical robots are increasingly used to enhance precision and reduce patient trauma in minimally invasive procedures. However, accurate control remains challenging due to nonlinear hysteresis in tendon-driven mechanisms, which can significantly affect surgical performance. This work presents MonoEndoCal, a deep learning-based autonomous calibration framework that uses only monocular endoscopic images-without external sensors or markers to estimate bending angles and compensate for hysteresis in real time. From a practical perspective, the proposed method can be integrated into existing endoscopic robotic platforms with minimal hardware modification, making it well-suited for clinical environments with strict space constraints. Practitioners adopting this framework should ensure that real-time computational resources are available and validate the hysteresis model with system-specific calibration data to achieve optimal performance. This approach offers a cost-effective, sensorless calibration solution that can improve control fidelity and reduce surgeon workload in tendon-driven surgical robots.
Dong-Geol Lee, Dong-Soo Kwon, Jee-Hwan Ryu, Hansoul Kim
IEEE Trans Autom. Sci. Eng.4
2025 Helical Structured Soft Growing Robot for Hazardous Gas Suction in Inaccessible Environments
abstract
Immediate removal of hazardous gases is critical for ensuring safety. Traditional methods, such as portable ventilation equipment, are difficult to use when hazardous gases are released in inaccessible environments. In this paper, we propose a novel mechanism that integrates an inflatable helical structure into a soft growing robot. The proposed mechanism is capable of performing suction through its inner channel after navigating complex environments, while maintaining the inherent advantages of the soft growing robot as it grows. The mechanism operates in two phases: a growing phase, in which the robot extends by eversion, and a suction phase, in which suction is performed through the inner channel of the robot. Experiments and demonstrations were conducted to evaluate the performance of the proposed mechanism. The experimental results confirmed the ability to maintain the passageway shape of the inner channel during suction operations and provided a design guideline. The demonstration validated that the mechanism can effectively navigate inaccessible environments and perform suction to remove hazardous gases.
Nam Gyun Kim, Dongoh Seo, Shinwoo Park, Jee-Hwan Ryu
ICRA5
2025 Guest Editorial: Special Issue on Automation and Artificial Intelligence (AI) in Construction and Building
Jingang Yi, Dikai Liu, Wei Yan 0006, Vineet R. Kamat, Chao Wang 0046, Jee-Hwan Ryu
IEEE Trans Autom. Sci. Eng.6
2024 Design of a Knee-joint Exoskeleton to Reduce Misalignment in Both the Sagittal and Coronal Planes
abstract
Many individuals experience knee dysfunctions attributed to the natural aging process and degenerative conditions. To aid individuals in regaining knee functionality, supportive exoskeletons were designed to be affixed to both the shin and thigh. However, a common issue encountered in knee exoskeletons involves the misalignment of joints between the exoskeleton and the user, resulting in discomfort and potential injuries. To reduce misalignment with the knee joint, it is essential for the thigh and shin harnesses of the exoskeleton to replicate the natural trajectories of the knee. However, achieving this is a complex task due to the shifting center of rotation of the knee in both the Sagittal and Coronal planes. Previous knee exoskeletons primarily focus on aligning the joint in the Sagittal plane, neglecting alignment in the other dimension due to inherent design constraints. For the first time, this study introduces a knee-joint exoskeleton capable of conforming to the natural movement of the knee in both the Sagittal and Coronal planes, with the aim of minimizing joint misalignment without the use of inherently soft materials. A spherical scissor linkage mechanism (SSLM) was utilized in conjunction with a customized guide rail to adjust the center of rotation of the SSLM. This configuration facilitates knee flexion/extension while accommodating the knee joint’s center of rotation in both the Sagittal and Coronal planes. The experimental outcomes demonstrated a substantial reduction in misalignment with the knee when compared to a commercial knee-support brace with a one-degree-of-freedom revolute joint.
Shubhranil Sengupta, Jee-Hwan Ryu
ICRA2
2024 Is a Simulation better than Teleoperation for Acquiring Human Manipulation Skill Data?
abstract
This study explores the feasibility of using simulations as a better interface to collect human object manipulation skills for learning from demonstrations (LfD). Recently, numerous researchers have started introducing teleoperation systems to acquire human manipulation skills. However, capturing the subtle, force-involved interaction skills of humans in teleoperation is still challenging due to its inherent dynamic delays and feedback transparency. This research evaluates the effectiveness of demonstration data obtained through simulation versus teleoperation. To evaluate the efficacy of this approach, tasks such as plane cutting, tight peg-in-hole, and deformable pipe plugging were performed to assess the quality of demonstrations acquired. The experimental results highlight the effectiveness of demonstration through simulation in capturing the operator’s force-involved interaction skills. Simulation creates an environment similar to performing tasks with bare hands by minimising dynamic delays due to the exclusion of physical robots and effectively rendering high stiffness. As a result, the demonstration through simulation method has proven effective in extracting interaction data and capturing physical task performance skills.
Kwang-Hyun Lee, Dongheui Lee, Jee-Hwan Ryu
IROS5
2024 Latency-Free Driving Scene Prediction for On-Road Teledriving With Future-Image-Generation
abstract
Teledriving could serve as a practical solution for handling unforeseen situations in autonomous driving. However, the latency of transmission networks remains a prominent concern. Despite advancements like 5G networks, delays in remote driving scenes cannot be entirely eradicated, potentially leading to unwanted incidents. While a few attempts have been made to address this issue by predicting the future driving scenes, these efforts have been restricted in their ability to accurately foresee clear and relevant driving scenarios. This study presents a method to predict a latency-free future driving scene. Unlike prior approaches, our method incorporates the command signal of a remote driver into the prediction network, as well as the past driving video frames and vehicle status. As a result, we can accurately predict relevant and clear latency-free future driving scenes. A combination of convolutional long short-term memory (ConvLSTM) and generative adversarial networks (GAN) was utilized in a deep neural network to predict the future driving scenes based on latency. The dataset was gathered from on-road teledriving experiments, with a maximum vehicle speed of 53 km/h and a driving route length of approximately 1.3 km. The dataset used to train the deep neural network was gathered from on-road teledriving experiments. The proposed method can estimate the future driving scene for up to 0.5 s, surpassing the performance of both baseline video prediction methods and a method that does not utilize the input command of the driver.
Kang-Won Lee 0001, Dae-Kwan Ko, Yong-Jun Kim, Jee-Hwan Ryu, Soo-Chul Lim
IEEE Trans. Intell. Transp. Syst.4
2024 Autonomous Vehicle Localization Without Prior High-Definition Map
abstract
Accurate localization by which vehicles can arrive at their destination while accurately following a given route is one of the most important factors for autonomous driving. In recent years, numerous studies have been conducted to achieve accurate localization using high-definition (HD) maps. Based on the HD map information (e.g., spatial data, lane, and traffic sign), autonomous vehicles can localize themselves by matching the surrounding spatial information obtained from onboard sensors to the HD maps. However, generating HD maps is a time-consuming and costly task. This study introduces a time-saving, effective, and accurate localization method inspired by humans, using only onboard sensors and publicly available two-dimensional (2D) map information. Similar to the multi-level localization process performed by humans, the proposed method interprets and matches the surrounding spatial data to the publicly available 2D maps using deep-learning-based place recognition and simultaneous localization and mapping (SLAM), thereby enabling autonomous vehicles to localize even without prior HD maps. Through the proposed method, our framework enables autonomous vehicles to perform maximally decimeter-level accurate localization without using HD maps. Evaluation of the proposed method using various datasets and publicly available map sources demonstrates that accurate global localization can be achieved without prior HD maps.
Sangmin Lee 0011, Jee-Hwan Ryu
IEEE Trans. Robotics2
2023 Reconfigurable Inflated Soft Arms
abstract
Inflatable structures have attracted considerable research attention in many fields owing to their numerous advantages, such as being light and able to engage in interactions safely. However, in most cases, the inflatable structure can only have one stable configuration, which is undesirable for robotic arms. This study proposes a novel inflatable structure that can be easily reconfigured into multiple stable configurations, even with single-body inflation. In the proposed mechanism, the structure length can be freely adjusted, and its respective joints can be set in the desired directions to facilitate the reconfiguration of its pose. An additional advantage of the proposed mechanism is that it can withstand external forces as well as its own weight. This study analyzes and experimentally validates the shape locking and load-carrying properties of the proposed mechanism. Further, the fabrication process and design guidelines for the proposed mechanism are presented. Through a suitable demonstration, the proposed mechanism is shown to exhibit multiple stable configurations and lock its poses.
Nam Gyun Kim, Jee-Hwan Ryu
ICRA2
2023 Error-Domain Conservativity Control to Transparently Increase the Stability Range of Time-Discretized Controllers
abstract
Time-discretization introduces an explicit time dependency for control laws that were originally designed to depend exclusively on an error variable: At different times, the control actions at the same error value might differ. Integrating the control action over the error reveals that this time dependency translates into the energy. It can directly cause active behavior when energy values at given error values decrease over time, potentially destabilizing the system. In this work, we aim to prevent energy values at given error values from decreasing over time. To this end, energies are recorded when error values are encountered for the first time. Linear interpolation of the recorded energy values provides a lower limit for energy as a function of the error value. This limit is enforced using an adaptive damping. The main contributions of this work include increasing the stability range with minimal amplitude control modifications, while promoting a symmetric behavior of control actions and energy. The approach's characteristics are shown in simulation and validated in experiments.
Michael Rothammer, Jee-Hwan Ryu
ICRA2
2023 High-Curvature Consecutive Tip Steering of a Soft Growing Robot for Improved Target Reachability
abstract
Over the years, soft growing robots that allow the feeding of new materials at their tips have attracted considerable attention owing to their unique locomotion characteristics. However, accessing targets over highly curved passages by steering compliant continuum bodies remains challenging. To this end, this study proposes a new tip steering mechanism that imparts soft growing robots with consecutive high-curvature steering ability. We place a hyper-redundant rolling contact joint and twisted string actuator (TSA) at the tip of a soft growing robot to provide high-curvature continuous steering with a smaller scale factor, allowing consecutive turns. The proposed small form factor continuous curvature tip steering mechanism allows the steering mechanism to stay at the tip during steering, improving accessibility by enabling consecutive turns. The improved accessibility of the proposed mechanism is demonstrated based on better confined space reachability compared with that of conventional whole-body steering soft growing robots, along with a consecutive high-curvature pipe navigation demonstration.
Dong-Geol Lee, Nam Gyun Kim, Jee-Hwan Ryu
IROS3
2023 Enhancing Contact Stability in Admittance-Type Haptic Interaction Using Bidirectional Time-Domain Passivity Control
abstract
The present paper proposes a novel strategy to enhance admittance-type haptic interaction using bidirectional time-domain passivity control. While admittance-type haptic interaction is widely employed in human-robot collaboration, its ability to render low virtual inertia can be limited, leading to unstable interactions with rigid environments. The proposed strategy seeks to stabilize a lower range of virtual inertia while maintaining responsive behavior in free space. The Franka Emika Collaborative robot was utilized in various experiments to test the approach, and the results indicate that the bidirectional time-domain passivity controller can improve interaction performance relative to the conventional unidirectional time-domain passivity approach. This technique may aid in reducing operator fatigue during the manipulation of heavy, non-back drivable industrial robots while preserving a lightweight feel.
Huseyin Tugcan Dinc, Kwang-Hyun Lee, Jee-Hwan Ryu
RO-MAN4
2021 On Smooth Time-Optimal Trajectory Planning in Twisted String Actuators
Simeon Nedelchev, Daniil Kirsanov, Igor Gaponov, HyeonSeok Seong, Jee-Hwan Ryu
ICRA5
2020 Closing the Force Loop to Enhance Transparency in Time-delayed Teleoperation
abstract
In the present paper, we first adopt explicit force control from general robotics and embed it into teleoperation systems to enhance the transparency by reducing the effect of the perceived inertia to the human operator and simultaneously improve contact perception. To ensure stability of the proposed teleoperation system considering time-delays, we propose a sequential design procedure based on time domain passivity approach. Experimental results of master-slave teleoperation system, based on KUKA light-weight-robots, for different values of delays are presented. Comparative analysis is conducted considering two existing approaches, namely 2-channel and 4-channel architecture based bilateral controllers, and its results clearly indicate significant improvement in force transparency owing to the proposed method. The proposed system is finally validated considering a real industrial assembly scenario.
Ribin Balachandran, Jee-Hwan Ryu, Mikael Jorda, Christian Ott 0001, Alin Albu-Schäffer
ICRA2
2020 Mini-Batched Online Incremental Learning Through Supervisory Teleoperation with Kinesthetic Coupling
abstract
We propose an online incremental learning approach through teleoperation which allows an operator to partially modify a learned model, whenever it is necessary, during task execution. Compared to conventional incremental learning approaches, the proposed approach is applicable for teleoperation-based teaching and it needs only partial demonstration without any need to obstruct the task execution. Dynamic authority distribution and kinesthetic coupling between the operator and the agent helps the operator to correctly perceive the exact instance where modification needs to be asserted in the agent's behaviour online using partial trajectory. For this, we propose a variation of the Expectation-Maximization algorithm for updating original model through mini batches of the modified partial trajectory. The proposed approach reduces human workload and latency for a rhythmic peg-in-hole teleoperation task where online partial modification is required during the task operation.
Hiba Ovais Latifee, Affan Pervez, Jee-Hwan Ryu, Dongheui Lee
ICRA3
2020 Successive Stiffness Increment and Time Domain Passivity Approach for Stable and High Bandwidth Control of Series Elastic Actuator
abstract
For safe human-robot interaction, various type of flexible manipulators have been developed. Especially series elastic actuator (SEA) based manipulators have been getting huge attention since the elastic element of SEA prevents people from injury when undesirable collision happens. Moreover, it improves system durability by absorbing impact force, which could damage actuators. However, the elastic element inside SEA manipulator causes low system bandwidth which limits the speed performance of conventional impedance control approaches. To alleviate the low bandwidth issue of impedance controlled SEA while guaranteeing system stability, we implement Time Domain Passivity Approach (TDPA) and Successive Stiffness Increment (SSI) approach, which was invented in haptic and teleoperation domain. Impedance controlled SEA is reformulated as a two-port electrical circuit network for implementing TDPA. In addition, a pair of input and output power conjugate variable, dominating the system passivity is identified for implementing SSI approach. Experimental results showed that TDPA and SSI approach can render the stiffness of the impedance controller, which decides the bandwidth, upto 350 kN/m without any stability issue, while normal impedance controller only render upto 120 kN/m. Although both of the approaches significantly increased the bandwidth of the impedance controlled SEA, TDPA slightly outperformed in stability, and SSI outperformed in tracking.
Chan-Il Lee, Do-Hyeong Kim, Harsimran Singh, Jee-Hwan Ryu
ICRA4
2020 A Framework for Interactive Virtual Fixture Generation for Shared Teleoperation in Unstructured Environments
abstract
Virtual fixtures (VFs) improve human operator performance in teleoperation scenarios. However, the generation of VFs is challenging, especially in unstructured environments. In this work, we introduce a framework for the interactive generation of VF. The method is based on the observation that a human can easily understand just by looking at the remote environment which VF could help in task execution. We propose a user interface that detects features on camera images and permits interactive selection of the features. We demonstrate how the feature selection can be used for designing VF, providing 6-DOF haptic feedback. In order to make the proposed framework more generally applicable to a wider variety of applications, we formalize the process of virtual fixture generation (VFG) into the specification of features, geometric primitives, and constraints. The framework can be extended further by the introduction of additional components. Through the human subject study, we demonstrate the proposed framework is intuitive, easy to use while effective, especially for performing hard contact tasks.
Vitalii Pruks, Jee-Hwan Ryu
ICRA2
2020 Development of a Twisted String Actuator-based Exoskeleton for Hip Joint Assistance in Lifting Tasks
abstract
This paper presents a study on a compliant cable-driven exoskeleton for hip assistance in lifting tasks that is aimed at preventing low-back pain and injuries in the vocational setting. In the proposed concept, we used twisted string actuator (TSA) to design a light-weight and powerful exoskeleton that benefits from inherent TSA advantages. We have noted that nonlinear nature of twisted strings’ transmission ratio (decreasing with twisting) closely matched typical torque-speed requirements for hip assistance during lifting tasks and tried to use this fact in the exoskeleton design and motor selection. Hip-joint torque and speed required to lift a 10-kg load from stoop to stand were calculated, which gave us a baseline that we used to design and manufacture a practical exoskeleton prototype. Preliminary experimental trials demonstrated that the proposed device was capable of generating required torque and speed at the hip joint while weighing under 6 kg, including battery.
HyeonSeok Seong, Do-Hyeong Kim, Igor Gaponov, Jee-Hwan Ryu
ICRA4
2020 A Tip Mount for Transporting Sensors and Tools using Soft Growing Robots
abstract
Pneumatically operated soft growing robots that extend via tip eversion are well-suited for navigation in confined spaces. Adding the ability to interact with the environment using sensors and tools attached to the robot tip would greatly enhance the usefulness of these robots for exploration in the field. However, because the material at the tip of the robot body continually changes as the robot grows and retracts, it is challenging to keep sensors and tools attached to the robot tip during actuation and environment interaction. In this paper, we analyze previous designs for mounting to the tip of soft growing robots, and we present a novel device that successfully remains attached to the robot tip while providing a mounting point for sensors and tools. Our tip mount incorporates and builds on our previous work on a device to retract the robot without undesired buckling of its body. Using our tip mount, we demonstrate two new soft growing robot capabilities: (1) pulling on the environment while retracting, and (2) retrieving and delivering objects. Finally, we discuss the limitations of our design and opportunities for improvement in future soft growing robot tip mounts.
Sang-Goo Jeong, Margaret M. Coad, Laura H. Blumenschein, Ming Luo 0004, Usman Mehmood, Ji Hun Kim, Allison M. Okamura, Jee-Hwan Ryu
IROS8
2019 High-Bandwidth Control of Twisted String Actuators
abstract
Twisted string actuators are an emerging type of transmission systems that may benefit various applications of robotics and mechatronics. However, control of TSAs in applications that require high bandwidth has attracted comparatively little interest from research community, mainly due to complexity of twisted string behavior. This paper proposes a new adaptive control methodology that allows to sufficiently increase bandwidth of TSA-based systems. We reformulate mathematical model of the TSA into a suitable form for online parameter estimation, outline adaptive estimation methods and propose a method to design variable controller gain that rectifies nonlinearities in the system. we present experimental comparison of proposed adaptive control strategies with two conventional tsa control techniques. experimental results demonstrated that the proposed adaptive control architecture with feedforward speed term was nearly insensitive to increase in input signal frequency while reducing position tracking error by 80%. proposed algorithm can be applied in any tsa control system that has input and output signal measurements.
Simeon Nedelchev, Igor Gaponov, Jee-Hwan Ryu
ICRA3
2019 Enhancing the Force Transparency of Time Domain Passivity Approach: Observer-Based Gradient Controller
abstract
Passivity has been the most often used constraint for the stable controller design of bilateral teleoperation systems. Especially, Time Domain Passivity Approach (TDPA) has been used in many applications since it has been known as one of the least conservative passivity-based approaches. Although TDPA were able to stabilize the system with the least conservatism, it has its own drawbacks as the cost of achieving the least conservative passivity especially when there is communication time-delay. Due to the on/off bang-bang control-like modification for instantaneous passivity recovery, it has high frequency force vibrations on the slave and especially master side. By implementing a virtual mass-spring system between the passivity controller and master device, these high frequency vibration has been eliminated. However, the gains need proper tuning as they are dependent on the teleoperation setup and application. It also tends to make the system sluggish which further distorts the transparency. We propose a new observer-based gradient controller to eliminate the force jittering on the master side. It rectifies the delayed feedback force by removing the undesired increase in force which is generated by the delay in communication channel. It does not require any system parameters and there are no gains to tune, thus it can be added to any teleoperator irrespective of its dynamics and without having any prior system information. The proposed approach was implemented to a position-force bilateral teleoperation system, and compared with TDPA with virtual mass spring with round-trip delays of up to 500 ms for hard wall contacts.
Harsimran Singh, Aghil Jafari, Jee-Hwan Ryu
ICRA3
2019 Effect of Vibration on Twisted String Actuation Through Conduit at High Bending Angles
abstract
This paper studies an effect of vibration on twisted string actuation through conduit at high bending angles. In our previous work [1], we have demonstrated that twisted string actuators can be used to transmit power even at significant deflection angles of the conduit. However, several undesirable effects, namely pull-back, hysteresis, and chattering, were present during actuation due to friction between strings and the internal sheath of the conduit. This paper reports the results of experimental study on effects of vibration on twisted string actuation inside deflected conduits. We have demonstrated that applying vibration generated near natural frequency of the system during the later stages of twisting and untwisting cycles helped reduce pull-back and hysteresis and increase string contraction. In case when sheath was deflected by 180° under a constant load of 3 kg, we were able to achieve over 70% decrease in pull-back and 30% decrease in hysteresis, compared with no vibration case.
Donghyee Lee, Igor Gaponov, Jee-Hwan Ryu
IROS3
2019 Relaxing the Conservatism of Passivity Condition for Impedance Controlled Series Elastic Actuators
abstract
This paper proposes a practical and less conservative passivity analysis for series elastic actuators (SEAs) by introducing load port definition and shows that the achievable stiffness by the impedance control of SEA can be set higher than the inherent stiffness of SEA depending on the condition of the load dynamics. Since SEA can inherently measure or estimate a transmitted force thanks to its embedded spring element, impedance control is often exploited to render compliant behaviors related between the motion and the force. Although the stability of the SEA control system is of great importance, the conventional passivity analysis gives conservative criteria, and indeed limits the actual actuator performance. To tackle the conservatism of the conventional passivity in SEAs, we first explore the dynamic characteristics of SEA including load dynamics, which has been ignored for the sake of simplicity of the passivity analysis by excluding uncertain load dynamics. The inclusion of the load dynamics into the passivity analysis allows us to properly derive the less conservative limit of achievable stiffness by impedance control and the factors that determine the limit. The proposed analysis is verified by numerical simulations and applied to a passivity observer design for experimental validation on an actual SEA setup.
Hyunwook Lee, Jinoh Lee, Jee-Hwan Ryu, Sehoon Oh
IROS3
2019 Inverse discounted-based LQR algorithm for learning human movement behaviors
Haitham El-Hussieny, Jee-Hwan Ryu
Appl. Intell.2
2019 Robotic Artificial Muscles: Current Progress and Future Perspectives
abstract
Robotic artificial muscles are a subset of artificial muscles that are capable of producing biologically inspired motions useful for robot systems, i.e., large power-to-weight ratios, inherent compliance, and large range of motions. These actuators, ranging from shape memory alloys to dielectric elastomers, are increasingly popular for biomimetic robots as they may operate without using complex linkage designs or other cumbersome mechanisms. Recent achievements in fabrication, modeling, and control methods have significantly contributed to their potential utilization in a wide range of applications. However, no survey paper has gone into depth regarding considerations pertaining to their selection, design, and usage in generating biomimetic motions. In this paper, we discuss important characteristics and considerations in the selection, design, and implementation of various prominent and unique robotic artificial muscles for biomimetic robots, and provide perspectives on next-generation muscle-powered robots.
Jun Zhang 0025, Jun Sheng, Ciarán T. O'Neill, Conor J. Walsh, Robert J. Wood, Jee-Hwan Ryu, Jaydev P. Desai, Michael C. Yip
IEEE Trans. Robotics6
2018 Preliminary Study of Twisted String Actuation Through a Conduit Toward Soft and Wearable Actuation
abstract
Twisted string actuators (TSAs) are gaining popularity in modern engineering and robotic applications. However, in conventional actuators of this type, the twisted part of the string should not be in contact with any surfaces or objects because this may interfere with the propagation of twisting. This imposes significant constraint on potential applications of TSAs. In this paper, we investigated the feasibility of using TSAs inside conduit and demonstrated that the twists of the string can be fully propagated through the sheath and that consistent periodic behavior of the twisted string can be achieved. In addition, we investigated input-output position and force characteristics of TSAs for various deflection angles of the conduit, effect of lubrication on transmission efficiency, and compared it with conventional cable sliding transmission. We found that TSA has higher transmission efficiency than sliding due to decreased friction between the string and conduit, which is further improved by lubrication. We have managed to achieve 85 % of force transmission efficiency for the case of lubricated twisting, as opposed to the 71.74% for lubricated sliding.
Bhivraj Suthar, Hyunseok Seong, Igor Gaponov, Jee-Hwan Ryu
ICRA5
2018 Development and Evaluation of an Intuitive Flexible Interface for Teleoperating Soft Growing Robots
abstract
Mobility by growth is a new paradigm in robotic systems design and their applications in the real world. Soft, tip-extending, or “growing”, robots have potential applications including inspection and navigation in disaster scenarios. However, due to their growing capability, such robots create unique challenges for intuitive human control. In this paper, a new flexible interface is proposed to intuitively map human bending commands into movements of the growing robot while providing shape information of the robot in order to improve situational awareness. Several command mappings are proposed, and a subjective study was conducted to assess the intuitiveness of the developed interface and mappings compared with other commercially available interfaces. The interfaces were evaluated using four metrics in two virtual task scenarios. The proposed interface with shape mapping performed better than the other interfaces, especially when the vine robot rolls over unintentionally during complex tasks.
Haitham El-Hussieny, Usman Mehmood, Syed Zain Mehdi, Sang-Goo Jeong, Elliot Wright Hawkes, Allison M. Okarnura, Jee-Hwan Ryu
IROS8
2018 Design of Robotic Gripper with Constant Transmission Ratio Based on Twisted String Actuator: Concept and Evaluation
abstract
Robotic systems for object handling and manipulation are hugely important for modern engineering and industry, with their efficiency, agility and robustness often depending on gripper design and performance. In this work, we investigate a gripper design that, when driven by a twisted string actuator, exhibits nearly-constant transmission ratio throughout its motion range. This allows for design of a highly-compact, modular and efficient robotic gripper driven by a low-power motor. We investigate kinematics of the device, experimentally verify developed models with a practical gripper testbed, and analyze transmission ratio and efficiency of the designed device. The resulting system has a nearly-constant transmission ratio of 550, with the constancy coefficient of 0.985.
Simeon Nedelchev, Igor Gaponov, Jee-Hwan Ryu
IROS3
2018 Enhancing the Command-Following Bandwidth for Transparent Bilateral Teleoperation
abstract
Enhancing transparency of a teleoperation system by increasing the command-following bandwidth has not received lots of attention so far. This is considered a challenging task since in a teleoperation system the command-following bandwidth of the slave robot motion controller cannot be increased with a conventional motion controller as the desired trajectory is instantaneously commanded by the human user and thus, cannot be considered to be given in a pre-computed, smooth second order derivative form. We propose a method to increase the command-following bandwidth by extending the previously introduced Successive Stiffness Increment (SSI) approach to bilateral teleoperation. The approach allows realizing a very high motion controller gain, which cannot be realized with a conventional bilateral teleoperation controller as confirmed by experimental results.
Harsimran Singh, Aghil Jafari, Angelika Peer, Jee-Hwan Ryu
IROS4
2017 Passivity-based stability in explicit force control of robots
abstract
Direct force control of robots is challenging, particularly since the interaction with the environment can render the robot unstable. This paper presents the results of novel approaches for passivity-based stability for a particular direct force control method, namely explicit force control. A step-by-step procedure to passivate and stabilise the control loop is presented and it explains how Time Domain Passivity Approach, a passivity-based tool widely used in teleoperation and haptics has been extended and applied in explicit force control. The electrical circuit and network-port representations derived in the process allows the analytical evaluation of the system and can be applied in other control architectures as well. The stability methods are presented both qualitatively and quantitatively with simulations and hardware experiments. A discussion about the results obtained and the energy behavior is also provided. Results are promising and suggest that these methods can be used for stable and high-bandwidth force control of robotic manipulators.
Ribin Balachandran, Mikael Jorda, Jordi Artigas, Jee-Hwan Ryu, Oussama Khatib
ICRA4
2017 Passive returning mechanism for twisted string actuators
abstract
The twisted string actuator is an actuator that is gaining popularity in various engineering and robotics and applications. However, the fundamental limitation of actuators of this type is the uni-directional action, meaning that the actuator can contract but requires external power to return to its initial state. This paper proposes 2 novel passive extension mechanisms based on buckling effect to solve the uni-directional issue of the twisted string actuator. The proposed mechanisms are mechanically simple and compact and provide a nearly-constant extension force throughout the operation range. The constant force can fully extend the twisted string actuator with minimal loss of force during contraction. The designed mechanisms are evaluated in a series of practical tests, and their performances are compared and discussed.
Bhivraj Suthar, Hyunseok Seong, Elliot Wright Hawkes, Igor Gaponov, Jee-Hwan Ryu
ICRA6
2017 New passivity observers for improved robot force control
abstract
This paper extends the previously proposed Explicit Force Controller based on Time Domain Passivity Approach. When using the classical passivity observer, we encounter an energy accumulation problem: if the system stays a long time in stable contact, energy is dissipated and the passivity observer builds up a large value. This causes the passivity controller to be triggered late after the interaction becomes unstable, so we lose the advantage of the passivity controller. In order to deal with this energy accumulation issue, we propose two new passivity observers that allow us to quickly detect potential instability despite the accumulated energy. We prove the theoretical validity of these new observers. In addition, we propose a more generalized way of implementing the passivity based explicit force controller on a multi-DoF manipulator, using a model for the robot and environment that includes sensor flexibility, and a hybrid position/force controller in the operational space framework. The proposed method is experimentally tested with KUKA IIWA, and the improved performance is verified.
Mikael Jorda, Ribin Balachandran, Jee-Hwan Ryu, Oussama Khatib
IROS3
2017 Realizing low-impedance rendering in admittance-type haptic interfaces using the input-to-state stable approach
abstract
This paper proposes an approach to enlarge the impedance range of admittance-type haptic interfaces. Admittance-type haptic interfaces have advantages over impedance-type haptic interfaces in the interaction with high impedance virtual environments. However, the performance of admittance-type haptic interfaces is often limited by the lower boundary of the impedance that can be achieved without stability issue. Especially, it is well known that low value of inertia in an admittance model often causes unstable interaction. This paper extends recently proposed input-to-state stable approach [1] to further lower down the achievable impedance in admittance-type haptic interfaces with less conservative constraint compared with the passivity-based approaches. The primary challenge was identifying the nonlinear hysteresis components which are essential for the implementation of the input-to-state stable approach. Through experimental investigation and after separating and merging the admittance model and the position controller, the partial admittance model (from the measured human force to the desired velocity) and the velocity controller (from the velocity tracking error to the controller force) were found having counter-clockwise hysteresis nonlinear behavior. Therefore, it allows implementing the one-port input-to-state stable (ISS) approach for making both components dissipative and ISS. An additional advantage of the proposed ISS approach is the easiness of the implementation. No model information is required, and the network representation is not necessary, unlike the passivity-based approaches. Series of experiments verified the effectiveness of the proposed approach in term of significantly lowering the achievable impedance value compared with what the time-domain passivity approach can render.
Muhammad Nabeel, Aghil Jafari, Jee-Hwan Ryu
IROS3
2017 A study on life cycle of twisted string actuators: Preliminary results
abstract
Twisted string actuators are widely used in different areas of robotics, however, their lifetime remains one of the main concerns for robotics. There are no comprehensive studies that relate the lifetime of a twisted string to its material properties and operation conditions, which can be important to improve the life cycle. This paper provides preliminary results of an extensive experimental study on twisted string actuators, using strings made of different materials and operating in various working conditions. We experimentally investigated the life time of two different strings and provided empiric mathematical model which relates operating conditions of TSAs and string material and properties to their endurance. Based on experimental results, the observations are made that the life cycle of the strings decreases exponentially with the increase in load forces and stroke and increases linearly with the increase in the number of strings.
Hyunseok Seong, Bhivraj Suthar, Igor Gaponov, Jee-Hwan Ryu
IROS5
2017 The Input-to-State Stable (ISS) Approach for Stabilizing Haptic Interaction With Virtual Environments
abstract
Passivity has been a major criterion for designing a stable haptic interface due to its numerous advantages. However, passivity-based controllers have suffered from the design conservatism of the passivity criterion, particularly when users want to increase the maximum apparent impedance. Based on the input-to-state stable (ISS) criterion and an analogy between haptic interfaces and systems with hysteresis, this paper proposes a control framework that is less conservative than passivity-based controllers. The proposed ISS approach allows a non-predetermined finite amount of output energy to be extracted from the system. Therefore, the proposed method can increase the maximum apparent impedance compared with passivity-based approaches. The focus of this paper is on how the proposed approach is designed to satisfy the input-to-state stability criterion in real time without prior knowledge of the system. This paper also extends the primary single-port ISS approach to a two-port ISS approach for multiple-degree-of-freedom generalization. The experimental and numerical results demonstrate that the proposed ISS approach is able to stabilize a higher impedance range than the time-domain passivity approach. The experimental results also confirm that the proposed approach provides higher actual apparent impedance to the operator compared with the energy-bounding and force-bounding approaches.
Aghil Jafari, Muhammad Nabeel, Jee-Hwan Ryu
IEEE Trans. Robotics3
2016 KONTUR-2: Force-feedback teleoperation from the international space station
abstract
This paper presents a new robot controller for space telerobotics missions specially designed to meet the requirements of KONTUR-2, a German & Russian telerobotics mission that addressed scientific and technological questions for future planetary explorations. In KONTUR-2, Earth and ISS have been used as a test-bed to evaluate and demonstrate a new technology for real-time telemanipulation from space. During the August 2015' experiments campaign, a cosmonaut teleoperated a robot manipulator located in Germany, using a force-feedback joystick from the Russian segment of the International Space Station (ISS). The focus of the paper is on the design and performance of the bilateral controller between ISS joystick and Earth robot. The controller is based on a 4-Channels architecture in which stability is guaranteed through passivity and the Time Delay Power Network (TDPN) concept. We show how the proposed approach successfully fulfills mission requirements, specially those related to system operation through space links and internet channels, involving time delays and data losses of different nature.
Jordi Artigas, Ribin Balachandran, Cornelia Riecke, Martin Stelzer, Bernhard M. Weber, Jee-Hwan Ryu, Alin Albu-Schäffer
ICRA6
2016 Performance comparison of Wave Variable Transformation and Time Domain Passivity Approaches for time-delayed teleoperation: Preliminary results
abstract
This paper presents initial results of a set of experiments to compare time-delayed teleoperation performance of selected architectures based on two popular passivity based methods, namely, Wave Variables Transformation (WVT) and Time Domain Passivity Approach (TDPA). Five different architectures widely used and available in literature are selected: 2-channel and 3-channel for both WV and TDPA, and 4-channel based on TDPA. An empirical approach for measuring transparency is proposed, which presents three main qualities: 1) Controller independency 2) Frequency dependency of performance and 3) Removal of the human operator out of the loop. In order to achieve this, the human operator is replaced by a linear actuator equipped with a force sensor. Deterministic interactions between the simulated operator and the rest of the system can thus be easily implemented and registered. Therefore, this approach allows to isolate pure effects of any control method and perform a proper analysis in terms of selected attributes for a desired metrics. In this article, these are chosen to be effective impedance and position tracking error, both in the frequency domain. In addition to the quantitative results, a qualitative analysis is presented which underlines practical implications of each architecture and control method. The comparison sample presented contains five different delays, varying from 0 to 200 milliseconds, five different environmental impedances and 5 bilateral control architectures along with an analysis of the results obtained.
Ribin Balachandran, Jordi Artigas, Usman Mehmood, Jee-Hwan Ryu
IROS4
2016 Development of the human interactive autonomy for the shared teleoperation of mobile robots
abstract
Manipulation complexity of teleoperation increases the necessity of shared autonomy. However, designing an autonomy while keeping in mind the environmental uncertainty is challenging. This paper proposes a human interactive autonomy for a shared teleoperation. The central concept is a novel methodology of combining the innate cognitive ability of a human operator to arrive at an accurate, and precise solution of an autonomous system for mobile robot shared teleoperation. The resulting system is more efficient and less fatigued than direct teleoperation. This combined approach transfers the human operator's intention, shaped by his cognition about remote environment, to the autonomous system. The intention information input from the operator helps autonomous system perform the given teleoperation tasks more efficiently. For initial feasibility and proof of concept, the intention information is provided to the autonomous system in the form of a path which is acquired by our proposed sketch method. Our novel proposed sketch method allows the operator to sketch the path on the visual feedback image of the remote environments and the information is then transmitted to the autonomous system. Experimental results provide the feasibility and effectiveness of proposed method against prevalent control methods.
Kwang-Hyun Lee, Usman Mehmood, Jee-Hwan Ryu
IROS3
2015 Multi Degree-of-Freedom Input-to-State Stable approach for stable haptic interaction
abstract
Passivity has been the most often used constraint for the controller design of haptic interfaces. However, the designed controller based on passivity constraint has been suffering from its conservatism, especially when the user wants to increase the maximum achievable impedance. To overcome this problem, our group have proposed Input-to-State Stable (ISS) approach [1], which reduce the design conservatism of the passivity-based controller by allowing bigger output energy from the haptic interface compared with the passivity-based controller while guaranteeing the stability. However, the previous paper was limited to single Degree-of-Freedom (DoF) systems. This paper extends the ISS approach for multi-DoF haptic interaction. For multi-DoF haptic interaction, penetration depth-based rendering method using Virtual Proxy (VP) is adopted, and VP allows us to decouple the interaction into each axis. Although the interaction can be decoupled, previous ISS analysis “cannot” be directly implemented because the decoupled system, unlike to the previous case, has unconstrained end point, that is a moving Virtual Environment (VE). To include the moving VE into the ISS approach, we extend the previous one-port ISS approach to two-port ISS approach, and generalize this into multi-DoF ISS approach by augmenting each two-port analysis. Proposed approach is experimentally verified with Phantom Pre. 1.5, and showed the effectiveness of the proposed multi-DoF ISS approach.
Aghil Jafari, Muhammad Nabeel, Jee-Hwan Ryu
World Haptics3
2015 Input-to-state stable approach to release the conservatism of passivity-based stable haptic interaction
abstract
Passivity has been a major criterion on designing a stable haptic interface due to many advantages. However, passivity has been suffering from its intrinsic conservatism since it only represents a small set of the whole stable region. Therefore, there was always limitation to increase the performance due to the small design margin from the passivity criterion. In most of the cases, stability and performance has trade-off relationship. In this paper, we propose a less conservative control approach for stable haptic interaction based on Input to State Stable (ISS) criterion. The proposed approach is inspired from the analogy between virtual environments and systems with hysteresis nonlinearities. A system with hysteresis nonlinearity has sector bounded property, which allows us to guarantee that only a finite amount of energy can be extracted from the system, which leads the system to be dissipative [1] and also the states to be bounded by a function of the input [2]. Since the finite amount of energy is allowed to be extracted from the system, the proposed ISS approach has less conservative constraint compared with passivity-based approaches. Moreover, the proposed approach has a simple structure and does not use any system parameters which make it suitable for practical implementations. Experimental evaluation validates the effectiveness of the proposed approach.
Aghil Jafari, Jee-Hwan Ryu
ICRA2
2015 Passively adjustable gear based on twisted string actuator: Concept, model and evaluation
abstract
Actuators with adjustable transmission ratios are required in a variety of robotics and automation applications, spanning from human assistive devices and mobile robots to general manipulators. Most existing self-adjustable actuators are bulky and mechanically complex, which often makes their implementation challenging. In this work, we propose a novel passively-adjustable transmission mechanism based on a twisted string actuator. Twisted string actuator is a light, cheap, and mechanically simple actuator, in which twisted strings contract as a result of twisting and therefore act as a translational gear. If one introduces a physical offset between the twisted strings, this changes a ratio between the speed and output force provided by actuator. This work introduces a kinematical model of such actuator for the configuration when a physical offset between the strings is present. In order to experimentally verify the proposed mathematical model, we designed and manufactured a twisted string actuator with variable offsets between the strings. Additionally, in this paper we also propose an idea for passively adjustable gear, controlled by twisted string actuator according to the proposed kinematical model. The main advantages of the proposed passively adjustable gear are its light weight, compliancy and mechanical simplicity, which make it attractive for implementaion in various areas of haptics, teleoperation, wearable and lightweight robotics.
Harsimran Singh, Dmitry Popov 0001, Igor Gaponov, Jee-Hwan Ryu
ICRA4
2015 Stable bilateral teleoperation with input-to-state stable approach
abstract
Passivity has been the most often used constraint for the controller design of bilateral teleoperation systems. However passivity has been suffering from its own design conservatism since this is a sufficient condition for stability and representing only a small set of overall stability region. In our previous works [1], [2], a less conservatism control approach, we named it Input-to-Sate Stable (ISS) approach, has been introduced for reducing the intrinsic design conservatism of the passivity-based controllers by allowing bigger output energy compared with the passivity-based controller while guaranteeing the stability. However the previous works were limited to haptic interfaces, which is interacting with computer generated virtual environments. In this paper, we extended the ISS approach for bilateral teleoperation systems. For this extension, the previous one-port ISS approach was generalized to two-port ISS approach. The main difficulty of this extension was identifying the signal pair to show hysteresis nonlinear behavior. By introducing a linear function assumption for two-port bilateral controller, which is generally acceptable, we could formulate two-port ISS approach. The extended two-port approach was implemented to a position-force bilateral teleoperation system, and the performance was experimentally verified with a dual Phantom teleoperation system.
Aghil Jafari, Muhammad Nabeel, Jee-Hwan Ryu
IROS3
2015 Increasing the impedance range of admittance-type haptic interfaces by using Time Domain Passivity Approach
abstract
This paper proposes a method to increase the impedance range of admittance-type haptic interfaces. Admittance-type haptic interfaces are used in various applications that typically require interaction with high impedance virtual environments. However, the performance of admittance haptic interfaces is often judged by the lower boundary of the impedance that can be achieved without stability problem; in particular, minimum displayable inertia. It is well known that rendering the low value of inertia makes the admittance-type haptic interfaces unstable easily. This paper extends Time Domain Passivity Approach (TDPA) to lower down the minimum achievable inertia in an Admittance-type haptic interface. To use the well-developed TDPA framework, an admittance haptic interface should be represented in network domain with clear energy flows, which was not straightforward due to unclear causality. Therefore by introducing dependent effort and flow source concept, the admittance type haptic interface is represented in electrical network domain. This network representation allows us to have clear causality, and consequently allowing to implement TDPA. The proposed idea was experimentally verified, and found successful in bringing down the minimum inertia 10 times lower than without TDPA case.
Muhammad Nabeel, JaeJun Lee, Usman Mehmood, Aghil Jafari, Jung-Hoon Hwang, Jee-Hwan Ryu
IROS6
2015 Dynamic authority distribution for cooperative teleoperation
abstract
Dividing the control authority of the slave robot among operators, based on the available information to them and their controlling strategy, is absolutely essential for cooperative teleoperation systems. Although there have been many studies on this, most of the analysis were limited to fixed authority and its effect on stability. None of the research has discussed how to allocate the control authority in real-time. This paper presents a dynamic authority distribution method for dual master single slave Cooperative Teleoperation systems, which allocates the authority over slave to operators on the move in each DOF separately. This method observes the energy interaction in the kinesthetic coupling link between the operators, classifies the interaction into operators' leader/follower behavior and adapts the authorities accordingly. The proposed method is experimentally tested with unique set of camera views of task environment with 3DOF dual master single slave teleoperation system. Experimental results demonstrate accurate convergence of decomposition of authorities based on camera-view. The performance of proposed dynamic authority architecture is evaluated by comparing total energy generation with fixed authority setup, and is found to be more efficient in terms of less energy generation, and hence less effort is required by operators to execute the task.
Naveed Ahmed Usmani, Jee-Hwan Ryu
IROS3
2014 Towards variable stiffness control of antagonistic twisted string actuators
abstract
This paper shows the possibility of using two antagonistic twisted strings actuators as a new type of variable stiffness actuator. Variable stiffness model of the twisted string actuator is identified empirically, and the control strategy is proposed for simultaneous position and stiffness control of the actuator. A variable stiffness linear joint actuated by antagonistic twisted string actuators is proposed as a target system. The proposed model and control strategy make it possible to control the position and the stiffness of the joint without position and force sensor at the load side. The developed variable stiffness linear joint can be effectively used in applications where weight distribution is vital, such as exoskeleton systems and low weight manipulators.
Dmitry Popov 0001, Igor Gaponov, Jee-Hwan Ryu
IROS3
2013 A preliminary study on a twisted strings-based elbow exoskeleton
abstract
This paper presents a new concept of a 1-DOF elbow exoskeleton driven by a twisted strings-based actuator. A novel joint actuation mechanism is proposed and its kinematic model is presented along with its experimental evaluation, and guidelines on how to choose the strings suitable for such an exoskeleton are given. We also proposed and experimentally verified a human intention detection method which takes advantage of intrinsic compliance of the mechanism. The study showed that the developed twisted strings-driven elbow exoskeleton is light, compact and have a high payload-to-weight ratio, which suggests that the device can be effectively used in a variety of haptics, teleoperation, and rehabilitation applications.
Dmitry Popov 0001, Igor Gaponov, Jee-Hwan Ryu
World Haptics3
2013 Bidirectional elbow exoskeleton based on twisted-string actuators
abstract
In this paper, a bidirectional elbow exoskeleton device based on rotational twisted string actuators is proposed. A novel actuation mechanism incorporating antagonistic motors is proposed and its kinematic model is presented along with its experimental evaluation. Two antagonistic actuation mechanisms provide the motion of the forearm link in both directions thus allowing to position the forearm precisely even in the presence of such disturbances as friction, external forces, and at different positions of the upper arm link. In addition, we propose a method to control the antagonistic exoskeleton based on the actuators kinematics. The developed twisted strings-driven elbow exoskeleton can be effectively used in a variety of haptics, teleoperation, and rehabilitation applications.
Dmitry Popov 0001, Igor Gaponov, Jee-Hwan Ryu
IROS3
2013 Stable multilateral teleoperation with Time Domain Passivity Approach
abstract
In this paper, we propose a generalized method to represent multilateral teleoperation system as an electrical network with dependent effort/flow sources which allows us to implement Time Domain Passivity Approach (TDPA) to passivate the system. Using the conventional mechanical-electrical analogy, the multilateral teleoperation system with mechanical nature is modelled as an electrical circuit. Power correlated signal are then identified to extract the augmented network representation in which time-delay is taken into account. The passivity of multilateral teleoperation system are also analyzed and the method of using TDPA to passivate the network is proposed. This framework is independent of control architecture and communication delay. Experiment on a trilateral teleoperation system has been done and showed good performance with proposed method.
Ha Van Quang, Jee-Hwan Ryu
IROS2
2012 A study on twisted string actuation systems: Mathematical model and its experimental evaluation
abstract
This paper presents an improved mathematical model of a single twisted string transmission system. The proposed mathematical model has been validated experimentally and provided a better match with the practical data in comparison to the conventional model. Translational transmission systems based on twisted strings coupled with electrical motors can compose light-weight, compact, and mechanically simple actuators which can be used in various robotic applications. Hands-on instructions on design of twisted string transmission systems presented in this paper are intended to help engineers in development of such actuation systems.
Dmitry Popov 0001, Igor Gaponov, Jee-Hwan Ryu
IROS3
2012 Passivity of delayed bilateral teleoperation of mobile robots with ambiguous causalities: Time Domain Passivity Approach
abstract
Rate mode commanding together with obstacle based force feedback makes mobile robot teleoperation difficult to stabilize even without time-delay. This paper proposes a method for stable time-delayed teleoperation of a mobile robot. Rate mode teleoperation with three different types of force feedback is considered to develop generally applicable method. We reformulate mobile robot bilateral teleoperation architecture based on recently proposed Time Delayed Power Network framework. It allows clarifying ambiguous energy ports and makes it possible to implement Time Domain Passivity Approach in order to secure the system stability. Experimental results show the effectiveness of the proposed formulation for a mobile robot teleoperation with time-delay.
Ha Van Quang, Ildar Farkhatdinov, Jee-Hwan Ryu
IROS3
2011 Memory based passivation method for stable haptic interaction
abstract
This paper proposes a new concept of passivation method to increase the dynamic range of impedance that a haptic interface can passively interact. The proposed method is motivated by force vs. position graph, which shows a pressing and releasing path when the haptic interface is interacting with the Virtual Environment (VE). The main reason of the active behavior was on the fact that the releasing path was higher than the pressing path in the force vs. position graph. To solve this issue, when the haptic interface is pressed, the computed force output from the VE is saved into fast accessible memory (such as FPGA) together with position data, and reused when the haptic interface is released to regulate the releasing path below the saved pressing path. The proposed method is tested with one-DOF haptic display, and shows better performance than recently proposed FPGA based time domain passivity approach [9].
Trung Hieu Do, Jee-Hwan Ryu
World Haptics2
2011 Haptic interface for intuitive teleoperation of wheeled and tracked vehicles
abstract
In this paper, a novel haptic interface is proposed for general teleoperation of wheeled and tracked vehicle. The new mechanism of the proposed haptic interface shown in Fig. 1 not only allows human operator to easily tele-operate various types of target vehicles including car-like vehicles, mobile robots, tracked vehicles, but also improve the operators' perception of target vehicles' operating status and its environment by introducing “cornering feel” in the field of vehicle teleoperation. In addition, the proposed interface enables human operators to give commands to (or drive) target vehicle in a way which is carried out in traditional direct control or driving. Experiments have been done to carefully evaluate and test the performance of the proposed interface. The results have shown that the developed haptic master device is sufficient and suitable for general wheeled and tracked-vehicle teleoperation.
Ba-Hai Nguyen, Jee-Hwan Ryu
World Haptics2
2011 Network representation and passivity of delayed teleoperation systems
abstract
In this paper, a decentralized platform for Simultaneous Localization and Mapping (SLAM) with multiple robots is developed. A novel occupancy grid map fusion algorithm is proposed. Map fusion is achieved through a multi-step process that includes image pre-processing, map learning, relative transformation extraction and then verification of the results. The proposed map learning method is a process based on the Self Organizing Map (SOM). In the learning phase, the obstacles of the map are learned by clustering the occupied cells of the map. The clusters represent the spatial form of the map and make further analyses of the map easier and faster. Also, clusters can be interpreted as features extracted from the occupancy grid map so the map fusion problem becomes a task of matching features. Results of the experiments from tests performed on a real environment with multiple robots prove the effectiveness of the proposed solution.
Jordi Artigas, Jee-Hwan Ryu, Carsten Preusche, Gerd Hirzinger
IROS2
2010 Plugfest 2009: Global interoperability in Telerobotics and telemedicine
abstract
Despite the great diversity of teleoperator designs and applications, their underlying control systems have many similarities. These similarities can be exploited to enable inter-operability between heterogeneous systems. We have developed a network data specification, the Interoperable Telerobotics Protocol, that can be used for Internet based control of a wide range of teleoperators. In this work we test interoperable telerobotics on the global Internet, focusing on the telesurgery application domain. Fourteen globally dispersed telerobotic master and slave systems were connected in thirty trials in one twenty four hour period. Users performed common manipulation tasks to demonstrate effective master-slave operation. With twenty eight (93%) successful, unique connections the results show a high potential for standardizing telerobotic operation. Furthermore, new paradigms for telesurgical operation and training are presented, including a networked surgery trainer and upper-limb exoskeleton control of micro-manipulators.
Hawkeye H. I. King, Blake Hannaford, Ka-Wai Kwok, Guang-Zhong Yang, Paul G. Griffiths, Allison M. Okamura, Ildar Farkhatdinov, Jee-Hwan Ryu, Ganesh Sankaranarayanan, Venkata Sreekanth Arikatla, Kotaro Tadano, Kenji Kawashima, Angelika Peer, Thomas Schauss, Martin Buss, Levi Makaio Miller, Daniel Glozman, Jacob Rosen 0001, Thomas Low
ICRA8
2010 Position drift compensation in time domain passivity based teleoperation
abstract
The Time Domain Passivity Control Approach is gathering interest in the robotics field. Simplicity and flexibility and the fact that system design emerges from ideal cases make it a powerful stability tool for teleoperation systems. Communication time delay is an inherent attribute of nearly every realistic teleoperation system. Unless the communication channel guarantees transmission delays of less than the system sampling time, the delay must be considered in the design in order to guarantee stability and satisfy a desired degree of performance. In previous work it has been shown how passivity can be considered in the time domain and how control rules are derived from it in order to dissipate the energy produced by the delayed communication. However, a weakness of these approaches is the impossibility of observing the exact amount of energy stored in the communication channel due to its delayed nature. A passive estimation is therefore needed which outcomes in an over-dissipation and in turn impacts on transparency. In constrained communications over-dissipation may become apparent in the form of a non-neglectful position drift between master and slave. This paper tackles the over-dissipative behavior of the Passivity Controller by resembling the energetic behavior of an ideal communication, i.e. where no delay is present and the transmission is lossless. Thus, the communication channel is not just controlled to be passive, as has been the case up to now, but also lossless. Energy can be dissipated to prevent activity, but activity can be also produced to prevent dissipative behaviors. The approach is sustained with experimental results.
Jordi Artigas, Jee-Hwan Ryu, Carsten Preusche
IROS2
2010 Improving mobile robot bilateral teleoperation by introducing variable force feedback gain
abstract
This paper presents new feedback force rendering scheme for the bilateral teleoperation of mobile robot. Previous research indicated that the feedback force based on obstacle range information prevented accurate motion control of the mobile robot since human operator's commands were distorted by the feedback force. To solve this problem, a new force rendering approach with variable feedback gain is proposed. In proposed scheme, force feedback gain is adaptively tuned based on measured distances to the obstacle and time derivatives of the distances. Stability of the proposed bilateral teleoperation architecture was analyzed and the performance is proved by simulations. Results of simulation and experimental study proved that the quality of the mobile robot bilateral teleoperation with variable force feedback gain is significantly better than the conventional approach with constant feedback gain.
Ildar Farkhatdinov, Jee-Hwan Ryu
IROS2
2007 Intelligent Filtering in Telerobotic System
Igor Gaponov, Hyun Cho, Jong-Won Kim 0003, Khalis Totorkulov, Seong Choi, Jee-Hwan Ryu, Tai-Hoon Cho
ICIC (3)6
2007 Stable Bilateral Control of Teleoperators Under Time-varying Communication Delay: Time Domain Passivity Approach
abstract
In this paper, modified two-port time-domain passivity approach is proposed for stable bilateral control of teleoperators under time-varying communication delay. We separate input and output energy at each port of a bilateral controller, and propose a sufficient condition for satisfying the passivity of the bilateral controller including time-delay. Output energy at the master port should be less than the transmitted input energy from the slave port with time-delay, and output energy at the slave port should be less than the transmitted input energy from the master port with time-delay. For satisfying above two conditions, two passivity controllers are attached at each port of the bilateral controller. Teleoperation experiment with about 120 (msec) of time-delay each way is performed. Stable teleoperation is achieved in free motion and hard contact as well.
Jee-Hwan Ryu, Carsten Preusche
ICRA1
2007 Bilateral Control with Time Domain Passivity Approach Under Time-varying Communication Delay
abstract
Recently, two-port time-domain passivity approach was modified for time-varying communication delay. The newly proposed approach could achieve stable teleoperation even under the serious time-varying delay and packet loss communication condition. However, after some operation hour, the accumulated energy difference between the input energy from one port and the output energy at the other port caused unstable behavior until the passivity controller is activated. Resetting scheme is introduced for solving this problem, and stable bilateral teleoperation can be guaranteed without worrying about the accumulated energy difference.
Jee-Hwan Ryu
RO-MAN1
2007 Landing Force Control for Humanoid Robot by Time-Domain Passivity Approach
abstract
This paper proposes a control method to absorb the landing force or the ground reaction force for a stable dynamic walking of a humanoid robot. Humanoid robot may become unstable during walking due to the impulsive contact force of the sudden landing of its foot. Therefore, a control method to decrease the landing force is required. In this paper, time-domain passivity control approach is applied for this purpose. Ground and the foot of the robot are modeled as two one-port network systems that are connected, and exchange energy with each other. The time-domain passivity controller with admittance causality is implemented, which has the landing force as input and foot's position to trim off the force as output. The proposed landing force controller can enhance the stability of the walking robot from simple computation. The small-sized humanoid robot, HanSaRam-VII that has 27 DOFs, is developed to verify the proposed scheme through dynamic walking experiments.
Yong-Duk Kim, Bum-Joo Lee, Jee-Hwan Ryu, Jong-Hwan Kim 0001
IEEE Trans. Robotics3
2006 Compensation for the Landing Impact Force of a Humanoid Robot by Time Domain Passivity Approach
abstract
In this paper, a method to reduce the landing impact force is proposed for a stable dynamic walking of a humanoid robot. To measure the meaningful landing impact force, a novel foot mechanism, which uses FSRs (force sensing resistors), is introduced as well. Humanoid robot might become unstable during the walking due to the impulsive contact force from the sudden landing of its foot. Therefore a new control method to decrease the landing impact force has been required. In this paper, time domain passivity control approach is applied for this purpose. Ground and the foot of the robot are modeled as two one-port network systems which are connected and exchanging energy each other. And, the time domain passivity controller which has the landing impact force as input and foot's position to trim off the force as output, is implemented. Unlike previous works, the proposed controller can guarantee the stability of the robot system without any dynamic model information at all. The small sized humanoid robot, HanSaRam-VI which has 25 DOFs, with the proposed foot mechanism is developed to verify the proposed approach through dynamic walking experiments
Yong-Duk Kim, Bum-Joo Lee, Jeong-Ki Yoo, Jong-Hwan Kim 0001, Jee-Hwan Ryu
ICRA5
2006 Landing Force Controller for a Humanoid Robot: Time-Domain Passivity Approach
abstract
For the purpose of a humanoid robot's stable walking or running, it is important to absorb landing force or ground reaction force which is generated when the robot's foot lands on the ground surface. The force can make the robot unstable, and the problem becomes serious if the robot runs. This paper proposes a control system, which can absorb the landing force of a humanoid robot. Time-domain passivity control approach is applied for this purpose. Ground and the robot's foot are modeled as two one-port network systems, which are connected and exchange energy with each other. The time-domain passivity controller has the landing force as input and controls the foot's position to reduce the force. The proposed controller can guarantee the stability of the robot system without need of any dynamic model information or control parameters. Using small sized humanoid robot, dynamic walking experiments are performed to verify the proposed scheme, and its efficiency is shown from the comparison with the other scheme.
Yong-Duk Kim, Bum-Joo Lee, Jeong-Ki Yoo, Jong-Hwan Kim 0001, Jee-Hwan Ryu
SMC5
2005 A Simulation/Experimental Study of the Noisy Behavior of the Time Domain Passivity Controller for Haptic Interfaces
abstract
A noisy behavior of the time domain passivity controller during the period of low velocity is analyzed. Main reasons of the noisy behavior are investigated through a simulation with a one-DOF haptic interface model. It is shown that the PO/PC is ineffective in dissipating the produced energy when the sign of the velocity, which is numerically calculated from the measured position, is suddenly changed, and when this velocity is zero. These cases happen during the period of low velocity due to the limited resolution of the position sensor. New methods, ignoring the produced energy from the velocity sign change, and holding the control force while the velocity is zero, are proposed for removing the noisy behavior. The feasibility of the developed methods is proved with both a simulation and a real experiment.
Jee-Hwan Ryu, Dong-Soo Kwon, Blake Hannaford
ICRA1
2005 A Simulation/Experimental Study of the Noisy Behavior of the Time-Domain Passivity Controller
abstract
A noisy behavior of the time-domain passivity controller during the period of low velocity is analyzed. Main reasons of the noisy behavior are investigated through a simulation with a one-DOF haptic interface model. It is shown that the PO/PC is ineffective in dissipating the produced energy when the sign of the velocity, which is numerically calculated from the measured position, is suddenly changed, and when this velocity is zero. These cases happen during the period of low velocity due to the limited resolution of the position sensor. New methods, ignoring the produced energy from the velocity sign change, and holding the control force while the velocity is zero, are proposed for removing the noisy behavior. The feasibility of the developed methods is proved with both a simulation and a real experiment.
Jee-Hwan Ryu, Blake Hannaford, Dong-Soo Kwon
IEEE Trans. Robotics1
2004 Stable teleoperation with time-domain passivity control
abstract
A new bilateral control scheme is proposed to ensure stable teleoperation under a wide variety of environments and operating speeds. System stability is analyzed in terms of the time-domain definition of passivity. A previously proposed energy-based method is extended to a 2-port network, and the issues in implementing the "passivity observer" and "passivity controller" to teleoperation systems are studied. The method is tested with our two-degrees-of-freedom master/slave teleoperation system. Stable teleoperation is achieved under conditions such as hard wall contact (stiffness >150 kN/m) and hard surface following.
Jee-Hwan Ryu, Dong-Soo Kwon, Blake Hannaford
IEEE Trans. Robotics Autom.1
2004 Sampled- and continuous-time passivity and stability of virtual environments
abstract
We propose a new time-domain passivity observer (PO) and passivity controller (PC) which removes the constant-velocity assumption during one sample time, which was used in our previous PO/PC approach. A new sampled-time definition of passivity is introduced, and this new definition is compared with the previous sampled-time definition of passivity. Through this comparison, we propose the more accurate PO/PC approach. The proposed new PO/PC approach is applied to the "Excalibur" haptic interface system with very high stiffness (K = 120 kN/spl middot/m) virtual environment, and stable contact is demonstrated.
Jee-Hwan Ryu, Yoon Sang Kim, Blake Hannaford
IEEE Trans. Robotics1
2004 Control of a flexible manipulator with noncollocated feedback: time-domain passivity approach
abstract
A new method to control a flexible manipulator with noncollocated feedback is proposed. We introduce a method to implement the time-domain passivity-control approach to a flexible manipulator with noncollocated feedback, which could not be treated with the previous time-domain passivity-control framework due to a possible active transfer function from the input to the noncollocated output. The proposed method is simulated with a single-link flexible manipulator, and a good control performance is obtained.
Jee-Hwan Ryu, Dong-Soo Kwon, Blake Hannaford
IEEE Trans. Robotics1
2003 Sampled and continuous time passivity and stability of virtual environments
abstract
We propose new time domain passivity observer (PO) and passivity controller (PC) which removes the constant velocity assumption during one sample time, which was used in our previous PO/PC approach. A new sampled time definition of passivity is introduced, and this new definition is compared with the previous sampled time definition of passivity. Through this comparison, we propose the more accurate PO/PC approach. The proposed new PO/PC approach is applied to "Excalibur" haptic interface system with very high stiffness (K = 120 KN/m) virtual environment (VE), and stable contact is demonstrated.
Jee-Hwan Ryu, Yoon Sang Kim, Blake Hannaford
ICRA1
2003 Time domain passivity control for 6 degrees of freedom haptic displays
abstract
In this paper a modification of the time domain passivity controller is presented to improve its performance and transparency in case of multi degrees of freedom (dof) haptic interaction. In multi-dof application the concept needs to be extended by additional conditions to distribute the adaptive damping appropriately among the degrees of freedom. This can be solved by using the geometrical information coded in the output signals of the system. Experiments show the validity of this concept.
Carsten Preusche, Gerd Hirzinger, Jee-Hwan Ryu, Blake Hannaford
IROS3
2003 Time domain passivity control with reference energy behavior
abstract
A recently proposed method for stabilizing haptic interfaces and teleoperation systems was tested with a "PHANToM" commercial haptic device. The "passivity observer" (PO) and "passivity controller" (PC) stabilization method was formed to stabilize the system but also excite high frequency mode in the device. To solve this problem, we propose a method to use a time-varying desired energy threshold instead of fixed zero energy threshold for the PO, and make the actual energy input follow the time-varying energy threshold. With the time-varying energy threshold, we make the PC control action smooth without sudden impulsive behavior by distributing the dissipation. The proposed new PO/PC approach is applied to PHANToM with high stiffness (K=500 N/m), and stable and smooth contact is guarantee. Resetting and active environment display problems can also be solved with the reference energy following idea.
Jee-Hwan Ryu, Blake Hannaford, Carsten Preusche, Gerd Hirzinger
IROS1
2002 Stable Teleoperation with Time Domain Passivity Control
abstract
A new bilateral control scheme is proposed to ensure stable teleoperation under a wide variety of operating conditions. To guarantee the stability of a teleoperation system, a previously proposed energy-based method is extended to a two-port network. The issues in implementing the "passivity observer" and "passivity controller" to teleoperation systems are studied. The method is tested with our two-DOF master/slave teleoperation system. Totally stable teleoperation is achieved under conditions such as hard wall contact (stiffness>150 kN/m) and hard surface following.
Jee-Hwan Ryu, Dong-Soo Kwon, Blake Hannaford
ICRA1
2002 Stability guaranteed control: Time domain passivity approach
abstract
A new, energy-based method is proposed for guaranteeing the stability of large classes of control systems with minimum performance losses. Based on a network presentation, the large classes of control systems are analyzed in a unified framework. In this unified network model, the concept of passivity is used to study the stability of large classes of control systems. For guaranteeing the stability condition, the time-domain passivity controller is extended to a 2-port network to make the controller 2-port passive. The developed method is tested with numerical simulation in the regulation of a single link flexible manipulator. Totally stable control is achieved under a wide variety of operating conditions and uncertainties without any model information.
Jee-Hwan Ryu, Dong-Soo Kwon, Blake Hannaford
IROS1
2002 Time-domain passivity control of haptic interfaces
abstract
A patent-pending, energy-based method is presented for controlling a haptic interface system to ensure stable contact under a wide variety of operating conditions. System stability is analyzed in terms of the time-domain definition of passivity. We define a "passivity observer" (PO) which measures energy flow in and out of one or more subsystems in real-time software. Active behavior is indicated by a negative value of the PO at any time. We also define the "passivity controller" (PC), an adaptive dissipative element which, at each time sample, absorbs exactly the net energy output (if any) measured by the PO. The method is tested with simulation and implementation in the Excalibur haptic interface system. Totally stable operation was achieved under conditions such as stiffness >100 N/mm or time delays of 15 ms. The PO/PC method requires very little additional computation and does not require a dynamical model to be identified.
Blake Hannaford, Jee-Hwan Ryu
IEEE Trans. Robotics Autom.2
2001 Time Domain Passivity Control of Haptic Interface
abstract
An energy-based method is presented for controlling a haptic interface system to ensure stable contact under a wide variety of operating conditions. System stability is analyzed in terms of the time-domain definition of passivity. We define a "passivity observer" (PO) which measures energy flow in and out of one or more subsystems in real-time software. Active behavior is indicated by a negative value of the PO at any time. We also define the "passivity controller" (PC), an adaptive dissipative element which, at each time sample, absorbs exactly the net energy output (if any) measured by the PO. The method is tested with simulation and implementation in the "Excalibur" haptic interface system. Totally stable operation was achieved under conditions such as stiffness >100 N/mm or time delays of 15 ms. The PO/PC method requires very little additional computation and does not require a dynamical model to be identified.
Blake Hannaford, Jee-Hwan Ryu
ICRA2
2001 Control of Underwater Manipulators Mounted on an ROV using the base Force Information
abstract
This paper presents a control scheme for obtaining high manoeuvrability of underwater robot manipulators mounted on a remotely operated vehicle (ROV). The motions of an underwater manipulator can affect the attitude and position of the ROV which should remain stationary in seabed operation. To compensate for the dynamic effect of the underwater manipulator on the ROV, the force-torque (F/T) information between the manipulator and the vehicle is used to regulate the states of the ROV. When an F/T sensor is practically unavailable, a disturbance observer can fill the role of the F/T sensor. This paper proposes a disturbance observer for estimating the interaction forces between the ROV and the manipulator. A two-link manipulator mounted on an ROV is considered and numerical simulations are performed to demonstrate the improvement on the manoeuvrability of the proposed controller.
Jee-Hwan Ryu, Dong-Soo Kwon, Pan-Mook Lee
ICRA1
2000 Design of a Teleoperation Controller for an Underwater Manipulator
abstract
A robust teleoperation controller design method for an underwater manipulator is proposed considering the master and the underwater slave separately. To achieve transparency and stability for a teleoperation of an underwater manipulator in unknown environments with time-varying uncertainties such as added mass, buoyancy, hydraulic drag and friction effect, an adaptive sliding mode control scheme is proposed for robust position tracking control of slave manipulator. To guarantee a force transparency in the master side, disturbance observer is used as a local controller for compensating a friction and coupled nonlinear dynamic effects of the master manipulator. Numerical simulations are performed to demonstrate the transparency and robustness of the proposed controller.
Dong-Soo Kwon, Jee-Hwan Ryu, Pan-Mook Lee, Seok-Won Hong
ICRA2
2000 A novel adaptive bilateral control scheme using dual closed-loop dynamic characteristics of master/slave manipulators
abstract
The paper presents a novel adaptive bilateral control scheme for obtaining transparency for teleoperation systems that has uncertainties. It has been found that a condition that is equivalent to getting an ideal response in teleoperation is to making the closed-loop dynamics of master and slave manipulators dual. An adaptive approach is applied to achieve the duality for the uncertain master and slave manipulators. Due to the dual closed-loop dynamic characteristics of master/slave teleoperation systems, excellent position and force tracking performance can be obtained without concerning the impedance variation of human and environment. The validity of the theoretical results is verified by experiments with a 1-DOF master/slave system.
Jee-Hwan Ryu, Dong-Soo Kwon
IROS1
1999 A Robust Controller Design Method for a Flexible Manipulator with a Time Varying Payload and Parameter Uncertainties
abstract
A robust controller design method is proposed to obtain a less conservative feedback controller, and applied to a single-link flexible manipulator. The objective is to maximize the control performance guaranteeing the robust stability when regulating the tip position of the flexible manipulator in the presence of a large time varying payload and parameter uncertainties such as stiffness, joint friction. Descriptor form representation, which allows separate treatment of payload uncertainty from other parametric uncertainties, is used to reduce the conservatism of the conventional robust control approaches. Uncertainty of the payload included in the left-hand side inertia matrix and the uncertain parameters included in the right-hand side damping and stiffness matrices are treated with polytopic and descaling techniques, respectively. Using the aforementioned techniques, the robust LQ controller design problem for a flexible manipulator is formulated based on the guaranteed cost approach. Then, the formulated problem has been solved by LMIs.
Jee-Hwan Ryu, Dong-Soo Kwon, Youngjin Park
ICRA1