Unsupervised skill discovery in reinforcement learning (RL) aims to learn diverse behaviors without relying on external rewards. However, current methods often overlook the periodic nature of learned skills, focusing instead on increasing the mutual dependency between states and skills or maximizing the distance traveled in latent space. Considering that many robotic tasks—particularly those involving locomotion—require periodic behaviors across varying timescales, the ability to discover diverse periodic skills is essential. Motivated by this, we propose Periodic Skill Discovery (PSD), a framework that discovers periodic behaviors in an unsupervised manner. The key idea of PSD is to train an encoder that maps states to a circular latent space, thereby naturally encoding periodicity in the latent representation. By capturing temporal distance, PSD can effectively learn skills with diverse periods in complex robotic tasks, even with pixel-based observations. We further show that these learned skills achieve high performance on downstream tasks such as hurdling. Moreover, integrating PSD with an existing skill discovery method offers more diverse behaviors, thus broadening the agent’s repertoire. Our code and demos are available at https://jonghaepark.github.io/psd

In this paper, we propose a new and simple control strategy for gait assistance with a hip exoskeleton robot. This controller is based on the time delayed, self-feedback known for stabilizing oscillatory systems under certain conditions. In this controller, there are no separate estimators for the gait phase nor the environment, yet the controller can be generalized to operate under various walking conditions (e.g., stair and ramp walking). We first define a state variable representing the current leg's movement with hip joint angles. A simple assistance control can be described in closed-loop form with the delayed state feedback. By assigning the appropriate time-delay and self-feedback gain, we can generate assistive torques stably under the interaction between human and exoskeleton. The controller provides immediate and smooth assistance to user movement by reflecting the change of leg motion at every control period. The proposed joint-angle-based delayed-feedback assistance controller can operate under various walking speeds and environmental changes (e.g., stairs and ramps) without the need for additional sensors, computational processing, and parameter adjustment. Using a simple leg swing model, we perform a stability analysis under a simplified condition to provide insights into the effects of the time-delayed feedback in oscillatory systems. Then, we experimentally validate the efficacy of the proposed assistance controller by measuring the metabolic energy expenditure for level treadmill walking. We also test and analyze the generated assistive torques and power under the different walking conditions to show the generalizability of the controller.

We propose a simulation framework for gait assistance with ankle pathologies. We first construct the neu-romuscular walking model, then design the parameters for assistance torques for stance and swing legs. The parameter values are determined by performing dynamic optimizations which takes into account the human-exoskeleton interactive dynamics. The simulated energy expenditure and kinematic data are compared with the real data. Case studies involve abnormal gaits with 1) foot drop, 2) foot drop and plantarflexion failure. We evaluate the gait efficiency and walking speed for the different gait types. Our result shows that each gait type should have a different assistance strategy (timing and magnitude) compared to the assistance strategy of a normal gait.

We propose a novel two-tier gait recognizer using a minimal number of mechanical sensors built into a lower limb exoskeleton. The aim of this recognizer is to offer one-step selection of one of the actions ascending, descending, and level walking during five gaits (stair ascent/descent, slope ascent/descent, level walking). The proposed recognizer is executed at the moment of foot contact as estimated by an inertial measurement unit (IMU) on the pelvis without using direct foot sensors. The proposed recognizer selects the gait by using the relations between the angles formed by the hip and knee joints during the last step. As this study constitutes preliminary work for lower limb exoskeletons, we used a hip exoskeleton integrated with two wireless IMUs on the shank to set up the lower limb exoskeleton sensor configuration. Experiments were used to evaluate IMU-based foot contact estimation with respect to a foot sensor based on a force-sensitive resistor (FSR) for stairs, slopes, and level ground. In addition, we evaluated the performance of the proposed two-tier gait recognizer by using two healthy male subjects in various gait environments.

We have developed a hip exoskeleton for seniors with difficulties in walking due to muscle weakness. The exoskeleton is lightweight and moderate in assistance power compared to other hip exoskeletons in the literature. Its controller estimates user gait phase, walking speed, and ground inclinations to generate assistance torque adaptively. To assess the physiological effect of the gait assistance, we compared metabolic energy consumption for 5 adults for walking on a treadmill with and without the exoskeleton at the same speed: the exoskeleton reduced metabolic cost of walking by 13% (p = 0:0024). The step length and the stride time increased under the assistance. Our analysis for the result suggests that the efficiency of hip exoskeletons on saving metabolic energy can be twice as high as that of ankle exoskeletons possibly because muscle-tendon unit in the hip joint is less energy-efficient than in the ankle joint.

In this paper, we propose a novel assistance strategy for stair ascent walking using our robotic hip exoskeleton. Our strategy exploits foot contact event estimated by an inertial measurement unit (IMU) and can detect user intention as well as reflect user preference. In our strategy, a gait cycle is divided into 4 phases and the transitions between the phases are based on events that are unavoidable and can be detected reliably using the sensors available for our exoskeleton. The proposed strategy presents criteria for the initiation and termination of assistance by recognizing user intention, as well as methods to adjust the timing and the amount of assistance corresponding to user preference. When each step starts, the duration of assistance is determined and the torque profile for the whole step is planned ahead. We demonstrate the validity of the proposed strategy for stair ascent and examine its effects on hip joint angle trajectory through experimental results.

This paper presents a simulation framework for pathological gait assistance with a hip exoskeleton. Previously we had developed an event-driven controller for gait assistance [1]. We now simulate (or optimize) the gait assistance in ankle pathologies (e.g., weak dorsiflexion or plantarflexion). It is done by 1) utilizing the neuromuscular walking model, 2) parameterizing assistive torques for swing and stance legs, and 3) performing dynamic optimizations that takes into account the human-robot interactive dynamics. We evaluate the energy expenditures and walking parameters for the different gait types. Results show that each gait type should have a different assistance strategy comparing with the assistance of normal gait. Although we need further studies about the pathologies, our simulation model is feasible to design the gait assistance for the ankle muscle weaknesses.

In this paper, we propose a novel online gait task recognition algorithm for hip exoskeleton. The proposed algorithm provides an automatic and prompt recognition result in just one step based on the relations between both hip joint angles at the moment of foot contact. Gait task recognition is one of the challenges that walking assist devices must address to offer adaptable and reliable assistance to users. However gait task recognition in hip exoskeleton is challenging because the sensors are very limited and fast gait task recognition is required to prevent inadequate assistance and reduce fall risk. Although in general foot contact event can be considered as crucial information during walking, it has not received attention in hip exoskeletons with no sensors corresponding foot force or pressure. In this study, we exploit foot contact event as a critical point to perform gait task recognition in hip exoskeleton. The proposed algorithm suggests a foot contact estimation method without using any foot force or pressure sensors and a rule-based inference system to recognize a new gait task in real time. Results presented from experiments will demonstrate the validity and performance of the proposed algorithm.

This paper presents a control method for walking assist with hip-mounted exoskeleton robots. For modeling a user's current walking motion, a novel finite state machine is first constructed. We divide a walking cycle uniformly using the inevitable zero crossing events. When state transitions occur, we capture the current walking spatio-temporal sensor data as discrete form. By using the sensed hip data as boundary conditions, we also develop a gait primitives based motion reconstruction method. Gait primitives are a form of basis trajectories to represent various joint motions. From those methods we estimate the moment of heel landing with interpolated knee joint motions. Utilizing the user's previous opposite step motion, we predict the positive or negative work intervals of the current step motion. This makes it possible to achieve natural ‘one shot’ assist by driving adapted torques fast. This assist strategy is also effective to enhance gait regularity. The measures of stride time variability are improved by over 30% for the simulated experiment. Various real experimentations demonstrate the feasibility of our approach.

In this paper, we present a modeling and control method for a single-port access robot developed by our robotics group at the Samsung Advanced Institute of Technology. The surgical robot consists of a snake-like 6-degree-of-freedom (DOF) guide tube, two 7-DOF tools, a 3-DOF stereo camera, and a 5-DOF slave arm. The robot is capable of reaching various surgical sites inside the abdominal cavity from a single incision on the body. To estimate the workspace and control the guide tube to a desired location, we first obtain the forward kinematics model of the guide tube and then propose a Cartesian-level controller. The wire actuation mechanism for the tools exhibit nonlinear backlash behavior because of wire compliance and friction between the wire and Teflon-coated conduit. We compensate for the backlash in the tool joints by adding the backlash inverse with smoothing term as a feedforward term.

We propose a control framework for dynamic bipedal locomotion with compliant joints. A novel 3D dynamic walking is achieved by utilizing natural dynamics of the system. It is done by 1) driving robot joints directly with the posture-based state machine and 2) controlling tendon-driven compliant actuators. To enlarge gait's basin attraction for stable walking, we also adaptively plan step-to-step motion and compensate stance/swing motion. Final joint input is described by a superposition of state machine control torques and compensation torques of balancers. Various walking styles are easily generated by composing straight and turning gait-primitives and such walking is effectively able to adapt on various environments. Our proposed method is applied to a torque controlled robot platform, Roboray. Experimental results show that gaits are able to traverse inclined and rough terrains with bounded variations, and the result gaits are human-like comparing the conventional knee bent walkers.

This paper presents a framework to generate dynamic walking for biped robots. A set of self-stable gait primitives is first constructed. It is done by 1) representing parametric gait primitives, 2) utilizing state-dependent torque control, and 3) doing numerical optimization that takes into account the complex multi-body dynamics with frictional contact forces. Dynamic walking to follow the arbitrary path including a curve is then generated online via sequentially composing primitive motions. Results show that dynamic gaits are humanlike and efficient compared to the conventional knee bent walkers. Our proposed method is applied to a torque-controlled, human-sized biped robot platform, `Roboray' which is cable-driven partially for joint compliance. Following a discussion on robot design and control, experimental results are also reported.

We propose a balancing control framework for a torque-controlled biped robot, Roboray. Roboray has two 6 DOF legs and torque sensors are integrated at all the leg joints. It has a new cable-driven joint module as a pitch joint drive, which is highly back-drivable and elastic. Using these hardware characteristics, we propose a new balancing control algorithm. This algorithm is the combination of gravity compensation, virtual gravity control and damping control. A friction compensation technique is also introduced in order to eliminate the nonlinearity of damping and to improve the performance of torque tracking. Our proposed method is applied to a simple inverted pendulum system and Roboray. Experimental results show that these two system keep their balance when they are pushed slightly.

The interplay between robotics and neuromechanics facilitates discoveries in both fields: nature provides roboticists with design ideas, while robotics research elucidates critical features that confer performance advantages to biological systems. Here, we explore a system particularly well suited to exploit the synergies between biology and robotics: high-speed antenna-based wall following of the American cockroach (Periplaneta americana). Our approach integrates mathematical and hardware modeling with behavioral and neurophysiological experiments. Specifically, we corroborate a prediction from a previously reported wall-following template - the simplest model that captures a behavior - that a cockroach antenna-based controller requires the rate of approach to a wall in addition to distance, e.g., in the form of a proportional-derivative (PD) controller. Neurophysiological experiments reveal that important features of the wall-following controller emerge at the earliest stages of sensory processing, namely in the antennal nerve. Furthermore, we embed the template in a robotic platform outfitted with a bio-inspired antenna. Using this system, we successfully test specific PD gains (up to a scale) fitted to the cockroach behavioral data in a "real-world" setting, lending further credence to the surprisingly simple notion that a cockroach might implement a PD controller for wall following. Finally, we embed the template in a simulated lateral-leg-spring (LLS) model using the center of pressure as the control input. Importantly, the same PD gains fitted to cockroach behavior also stabilize wall following for the LLS model.

It may come as no surprise that a simple "planar unicycle" (or "skate") model can successfully follow a wall at high speed under PD-control. What would be surprising is that such a simple control mechanism may underly the control of one of nature's fastest terrestrial insects, the American cockroach. For this paper, we implemented the same controller (up to scale) believed to govern cockroach wall following to successfully control two models of the cockroach: a differential-drive mobile robot with a flexible artificial antenna and a lateral leg spring model with moving center of pressure as the control input. These physical and numerical experiments demonstrate the sufficiency of the cockroach's putative controller in a real-world setting with unmodeled effects, and suggest how the nervous system might guide leg placement in response to sensory stimuli. This hierarchy of models may prove useful in generating prescriptive hypotheses for biological testing and hence elucidating the general principles that underly sensor-guided animal locomotion.

We have developed new methods for the automated assembly of prototype structures and we illustrate them with the construction of a simple one DOF 5mm surgical wrist employing polyester flexures instead of revolute joints. The first step in the structural assembly involves the construction of hollow stainless steel triangular beams that are used for the rigid elements of the structure. It includes the development of a folding fixture to bend stainless steel sheets and the determination of a folding angle sequence by static analysis using a compliant mechanism model. The semi-automatic process of using a millirobot (orthotweezers) to manipulate and assemble the beams and attach the flexures is described in detail. The paper ends with a description of the procedure used to design the wrist.