Abstract Exoskeletons can assist human locomotion in real-life scenarios, but existing tools for decoding locomotion modes (LMs) focus on recognition rather than prediction, which can lead to delayed assistance. This study proposes a long short-term memory (LSTM) neural network to predict five LMs (level-walking, ramp ascent/descent, stair ascent/descent) with greater lead time compared to state-of-the-art methods. We examined the optimal sequence length (SL) for LSTM-based LM prediction, using data from inertial sensors placed on the lower limbs and the lower back, along with a waist-mounted infrared laser. Ten subjects walked in real-life scenarios, both with and without an ankle–foot exoskeleton. Results show that a 1-s SL provides the most advanced and accurate LM prediction, outperforming SLs of 0.6, 0.8, and 1.2 s. The proposed LSTM model achieved an accuracy of 98 ± 0.31%, predicting LMs 0.66 s in advance (for an average stride time of 1.98 ± 0.83 s). Level-walking presented more misclassifications, and the model primarily relied on inertial data over laser input. Overall, these findings demonstrate the LSTM’s strong predictive capability for both assisted and non-assisted walking and independent of which limb executes the transition, supporting its applicability for exoskeleton-assisted locomotion.

Daily motor activities are affected by motor disabilities caused by Parkinson's disease (PD). Monitoring motor disabilities frequently observed in PD is difficult for physicians, as they are limited to the information observed or self-reported during routine consultations, resulting in subjectivity and limited assessment. Thus, it is necessary to evaluate more frequently and objectively, ideally in a continuous manner including daily life tasks. While wearable sensory devices, such as inertial sensors, and their applications are steadily growing, the advances in artificial intelligence studies have revolutionized their ability to extract deeply hidden information for accurate detection and interpretation of motor activities. However, further studies are required, mainly focused on PD. This study aimed to implement a deep learning (DL) based model for recognizing daily motor activities based on inertial data to contribute to PD. The model relied on a convolutional neural network (CNN) architecture, trained and tested on a created custom dataset. We further benchmarked our model against other popular DL frameworks. The dataset included inertial data captured from 18 patients while performing trivial quotidian tasks, such as walking, turning, sitting, and lying. We hypothesized that a DL model based on a CNN architecture could be an appropriate solution for modeling daily motor non-steady and steady-state tasks from a single inertial sensor. We measured an F1 score of 0.906 and an accuracy of 91.1 % on final testing with our optimized CNN model, being the tasks of standing and walking the most accurately recognized by the model. Future challenges should cover exploring attention-based models and increasing the dataset.

This study aims to assess the impact of different visual perturbations on user balance within an immersive Virtual Environment (VE), emphasising ecological validity. Utilising a Virtual Reality headset, the researchers implemented various visual perturbations to discern their effects. The objective was to create a versatile tool featuring animations simulating diverse indoor and outdoor settings within a typical household, providing a range of ecological scenarios for evaluating user balance in the context of these visual disturbances. Participants were exposed to unpredictable visual disturbances within the VE, and their responses were captured using a combination of inertial sensors, electromyography and galvanic skin response. A comprehensive dataset replicating real-world fall scenarios on balance disturbances was collected. Throughout the trials, participants moved around the house, unaware of the timing or nature of visual perturbations. Statistical analysis identified variables with significant mean changes when visual disturbances were introduced. Muscle groups crucial for balance recovery following external disturbances exhibited the most substantial associations, akin to kinematic variables linked to loss of balance. Visual disturbances alone could induce balance disturbances, which were further categorized based on a statistical analysis. This tool offered a versatile platform for assessing balance under various challenging scenarios, facilitating research into balance control and training methods. The dataset generated through this protocol enabled a wide understanding of the impact of visual disturbances on balance and the associated compensatory reactions, shedding light on their potential applications in clinical and rehabilitative settings.

Stimulating the usability and acceptability of canes within the geriatric community is crucial for fall prevention. Clinical evidences support the use of canes to increase the user's balance and reduce the fall risk. Its nonuse is correlated with a higher number of falls. This fateful event draws tragic consequences such as death or injuries that might affect the victim's psychology, reducing their confidence and mobility. However, users tend to put aside canes and canes' acceptability is reduced. To increase their usability and acceptability, appellative and time-effective technology-based strategies, such as automatic fall detection systems, should be incorporated. First, steps toward a fall-related approach demand the ability to identify that a fall event is eminent relying only on information derived from the device. This study proposes and compares two fall event approaches. A machine learning framework determines the best models and features for three classification problems: fall event, fall phase, and fall category. The long short-term memory (LSTM) is the most suitable classifier for fall event and phase classification problems, requiring the first 32 and 17 features, respectively, ranked by the Relief-F and minimum-redundancy maximum-relevancy methods (accuracy >99% and >95%). K-nearest neighbors with the first 40 features ranked by the Relief-F method presented the best performance for fall category classification (accuracy >74%). A finite-state machine (FSM) combined accelerometer and gyroscope's data to detect the cane's fall with a clear proximity to the LSTM performance (accuracy >99%). Both approaches detect a fall before its occurrence with mean lead times higher than 373 (LSTM) and 266 ms (FSM).

Biped robots have not achieved the efficient and harmonious locomotion of the human beings, capable of walking and running on unstructured terrains, with obstacles, holes and slopes. With this in mind, researchers started the development of biomimetic solutions to control the locomotion of biped models. This work presents a new solution of motion control of bipedal robots with adaptable stiffness, by exploring effects of joint stiffness in modulating walking behavior. Further, torque adjustment is achieved through a biomimetic controller that mimics and adjusts the natural dynamics of the robot to the environment. Specifically, the torque adjustment is made using AFOs (adaptive frequency oscillator) to generate the correct equilibrium positions that will be applied to the impedance control that computes the torque of each joint. Results show that the biped model is capable of walking in several types of terrain, including flat terrain, ramps, stairs and flat terrain with obstacles.

In humans, the system that controls several biomechanical tasks has a modular organization (muscle synergies). While the muscle synergies in human walking were already studied deeply, this phenomenon in robotic system is not so explored and could bring simpler controllers requiring a minor number of parameters. So, the purpose of this work is to acquire and interpret synergies from selected joint signals in order to answer some specific research questions regarding the verification of synergistic control of a biped robot in a walking activity, e.g, the number of synergies, their activation profiles, their mechanical role, their quality to reproduce/control walking motions, etc. The selected robotic system was the Darwin-OP robot when walking both with a model based and a bio-inspired controller. Tests were performed with the simulated and real robot. The results appear to suggest that the extraction of synergies are quite useful for the robotic locomotion, since with only 2 synergies, is possible to reconstruct the signals of the studied joints. The biomechanical role for each determined synergy is described based on the functional transformation between sensorimotor signals and biomechanical output. Besides, the robustness of the determined synergies is verified in the locomotor activation pattern across strides and across speeds when the mechanical requirements do not change to a large extent.

Assisted gait monitoring could benefit from the measurement of feet position and orientation. These measurements could also be used to implement intention-based control laws for smart walkers. Several works have been dedicated to the detection of lower limbs. The proposed methods are usually fast, but only detect the position of the legs. Others may even be more complete, but are not adapted to reactive applications. Also, they often use markers attached on the feet, which is unsuitable in daily routine. In this paper, a fast feet position and orientation detection algorithm is proposed. It is based on a camera depth sensor and does not require the use of any marker. The obtained results are compared with a ground truth provided by a motion traking system to experimentally assess the performances of the proposed algorithm.

Quadruped locomotion in irregular and unknown terrains is still a problem to solve. The concept of reflexes is used in this work to contribute for the continuous search of answers about this theme. Biological researches show that spinal reflexes are crucial for a successful locomotion in the most varied terrains, so robotics investigation in this area could be a great advance in the robot's locomotion. In this work, we present a sensory driven reflex controller, capable of generating locomotion in a quadruped compliant robot. This controller is totally dependent on sensory information, so the robot's movements are the result of the robot interactions with the environment. Results show that the proposed controller is capable of generating movements in a flat terrain and is resilient to unexpected perturbations such as a small ramp.

Generating biped locomotion in robotic platforms is hard. It has to deal with the complexity of the tasks which requires the synchronization of several joints, while monitoring stability. Further, it is also expected to deal with the great heterogeneity of existing platforms. The generation of adaptable locomotion further increases the complexity of the task. In this paper, Genetic Programming (GP) is used as an automatic search method for motion primitives of a biped robot, that optimizes a given criterion. It does so by exploring and exploiting the capabilities and particularities of the platform. In order to increase the adaptability of the achieved solutions, feedback pathways were directly included into the evolutionary process through sensory inputs.

The use of the walker in rehabilitation has increased in the past few years. Therapists evaluate patient's rehabilitation by observation and subjective tests. Thus, it is necessary the use of an assistive tool which can measure and quantify the patient's walker-assisted movement and stability, providing an objective clinical assessment. The aim of this study is to detect differences in assisted gait when using the assistive devices (ADs) - crutches, standard walker and rollator (4-wheeled walker) with forearm supports (RFS) - in patients with knee osteoarthritis (KOA) that suffered the surgery - Total Knee Arthroplasty (TKA). Additionally, it is to verify the link between gait parameters and acceleration signals. The evaluation is reached by the use of two 3 axis-accelerometers. The signals extracted from the sensors, at the ankle and trunk, are related to gait events and evaluation of fall risk, respectively. Results show that despite the differences between the signals obtained with the three ADs and with the subjects in this study, it is possible to identify effectively the gait parameters and prove the stability that the RFS provides.

This paper presents an exploratory study in which children with autism interact with ZECA (Zeno Engaging Children with Autism). ZECA is a humanoid robot with a face covered with a material allowing the display of varied facial expressions. The study investigates a novel scenario for robot-assisted play, to help promoting labelling of emotions by children with autism spectrum disorders (ASD). The study was performed during three sessions with two boys diagnosed with ASD. The results obtained from the analysis of the children's behaviours while interacting with ZECA helped us improve several aspects of our game scenario such as the technical specificities of the game and its dynamics, and the experimental setup. The software produced for this study allows the robot to autonomously identify the answers of the child during the session. This automatic identification helped the fluidity of the game and freed the experimenter to participate in triadic interactions with the child. The evaluation of the game scenario that will be used in a future study was the main goal of this pilot study, rather than to quantify and evaluate the performance of the children. Overall, this exploratory study in teaching children about labelling emotions using a humanoid robot embedded in a game scenario demonstrated the possible positive outcomes this child-robot interaction can produce and highlighted the issues regarding data collection and their analysis that will inform future studies.

This paper presents a biped gait optimization system that combines bio-inspired Central Patterns Generators (CPGs) and a multi-objective evolutionary algorithm. CPGs are modeled as autonomous differential equations, that generate the necessary limb movements to perform the walking gait of a biped robot. The search for the best set of CPG parameters is optimized by considering multiple objectives and according to a staged evolution. A sensitivity analysis is used to evaluate the relationship between objectives, objectives and parameters, and allows to determine the functional meanings of the parameters. This resulting functional analysis enables to verify which parameters are relevant to the motor behaviors. The simulation results show the effectiveness of the proposed approach. The different obtained walking gait solutions correspond to different trade-offs between the objectives.

Due to their relatively simple nervous system, insects are an excellent way through which we can investigate how visual information is acquired and processed in order to trigger specific behaviours, as flight stabilization, flying speed adaptation, collision avoidance responses, among others. From the behaviours previously mentioned, we are particularly interested in visually evoked collision avoidance responses. These behaviors are, by necessity, fast and robust, making them excellent systems to study the neural basis of behavior. On the other hand, artificial collision avoidance is a complex task, in which the algorithms used need to be fast to process the captured data and then perform real time decisions. Consequently, neurorobotic models may provide a foundation for the development of more effective and autonomous devices. In this paper, we will focus our attention in the Lobula Giant Movement Detector (LGMD), which is a visual neuron, located in the third layer of the locust optic lobe, that responds selectively to approaching objects, being responsible for trigger collision avoidance maneuvers in locusts. This selectivity of the LGMD neuron to approaching objects seems to result from the dynamics of the network pre-synaptic to this neuron. Tipically, this modelation is done by a conventional Difference of Gaussians (DoG) filter. In this paper, we propose the integration of a different model, an Inversed Difference of Gaussians (IDoG) filter, which preserves the different level of brightness in the captured image, enhancing the contrast at the edges. This change is expected to increase the performance of the LGMD model. Finally, a comparative analysis of both modelations, as well as its effect in the final response of the LGMD neuron, will be performed.

Walkers are commonly seen in hospitals and clinics, but only individuals with fairly good motor function can use them. Hence, walkers should be more adaptive to the various needs of the users. This study includes the conceptual design, implementation and validation of a Smart Walker with a new interface approach integrated. The implementation of the interface is based on two potentiometers and it intends to read the user's movement intentions to command the walker. In addition, this new interface can be adapted to different types of patients through the adjustment of its dynamical characteristics. Based on this real-time identification of user's commands, an approach to the control architecture was developed and it is based on a fuzzy logic algorithm that allows to control the walker's motors and permitsthe user to manipulate the Smart Walker at his own pace. Results with healthy users show the reliability of the proposed device and control architecture to read the user intentions and control the walker accordingly.

The paper reviews a known robot control architecture using nonlinear analysis and control theory viewpoints. The architecture is based on a mesh of dynamic systems and feedthrough maps and is able to drive the robot under temporal constraints. The analysis points to an intuitive, though innovative, conclusion that control architectures can be constructed from a methodological perspective by mixing (i) dynamical systems with fixed points carefully selected to match mission requirements, and (ii) feedthrough maps that perform memoryless transformations on input data. Experiments using the Webots environment are presented to illustrate the ideas developed.

This paper presents a gait multi-objective optimization system that combines bio-inspired Central Patterns Generators (CPGs) and a multi-objective evolutionary algorithm. CPGs are modeled as autonomous differential equations, that generate the necessary limb movement to perform the required walking gait. In order to optimize the walking gait, four conflicting objectives are considered, simultaneously: minimize the body vibration, maximize the velocity, maximize the wide stability margin and maximize the behavioral diversity. The results of NSGA-II for this multi-objective problem are discussed. The effect of the inclusion of a behavioral diversity objective in the system is also studied in terms of the walking gait achieved. The experimental results show the effectiveness of this multi-objective approach. The several walking gait solutions obtained correspond to different trade-off between the objectives.

Most autistic people present some difficulties in developing social behavior, living in their own world. This study has the goal to improve the social life of children with autism with a main focus in promoting their social interaction and communication. It is necessary to call for children's attention and enforce their collaboration, where a robot, LEGO MindStorm, behaves as a mediator/promoter of this interaction. A set of experiments designed to share objects and fulfill simple orders, by the 11 years old autistic child at the time of daily routine work and in-game with the robot, are described. The generalization of the acquired skills by the child in new contexts and environments are also tested. Results are described showing the outcomes of the experiments.

Quadruped locomotion on rough terrain and un-predictable environments is still a challenge, where the concept of Central Pattern Generators (CPG) has brought interesting ideas. In this contribution we present a CPG design based on coupled oscillators, generating the required stepping movements of a limb for omnidirectional motion. Movements are online modulated through small value changes in the CPG's parameters as required to perform the desired omnidirectional locomotion in a quadruped robot. We also present a methodology to modulate the CPG's parameters, reducing the control dimensionality, described in terms of the robot's translational speed, angular velocity and walking orientation. Results show the proposed controller is well suited for the online generation and modulation of the motor patterns required to achieve the desired omnidirectional walking motion.

Postural stability is a requirement for autonomous adaptive legged locomotion. Neurobiological research lead to the idea that there are independent central systems for posture and locomotion, which interact when required.

The head shaking that results from robot locomotion is important because it difficults stable image acquisition and the possibility to rely on that information to act accordingly, for instance, to achieve visually-guided locomotion.

The ability to traverse a wide variety of terrains while walking is basically a requirement for performing useful tasks in our human centric world. In this article, we propose a bio-inspired robotic controller able to generate locomotion and to easily switch between different type of gaits. In order to improve the robot stability and response while locomoting, we adjust both the duty factor and the interlimb phase relationships, according to the velocities. We extend previous work, by applying nonlinear oscillators to generate the rhythmic locomotor movements for a quadruped robot, similarly to the biological counterparts. The generated trajectories are modulated by a drive signal, that modifies the oscillator frequency, amplitude and the coupling parameters among the oscillators, proportionally to the drive signal strength. By increasing the drive signal, locomotion can be elicited and velocity increased while switching to the appropriate gaits. This drive signal can be specified according to sensory information or set a priori. The implementation of the central pattern generator network and the activity modulation layer is shown in simulation and in an AIBO robot.

This article addresses the problem of generating timed trajectories and temporally coordinated movements for two wheeled vehicles, when relatively low-level, noisy sensorial information is used to steer action. The generated trajectories have controlled and stable timing (limit cycle type solutions). Incoupling of sensory information enables sensor driven termination of movement. We build on a previously proposed solution in which timed trajectories and sequences of movements were generated as attractor solutions of dynamic systems. We present a novel system composed of two coupled dynamical architectures that temporally coordinate the solutions of these dynamical systems. The coupled dynamics enable synchronization of the different components providing an independence relatively to the specification of their individual parameters. We apply this architecture to generate temporally coordinated trajectories for two vision-guided mobile robots in a non-structured simulated environment, whose goal is to reach a target within a certain time independently of the environment configuration or the distance to the target. The results illustrate the robustness of the proposed decision-making mechanism and show that the two vehicles are temporal coordinated: if a robot movement is affected by the environment configuration such that it will take longer to reach the target, the control level coordinates the two robots such that they terminate approximately simultaneously

We present an attractor based dynamics that autonomously generates temporally discrete movements and movement sequences stably adapted to changing online sensory information. Autonomous differential equations are used to formulate a dynamical layer with either stable fixed points or a stable limit cycle. A neural competitive dynamics switches between these two regimes according to sensorial context and logical conditions. The corresponding movement states are then converted by simple coordinate transformations into spatial positions of a robot arm. Movement initiation and termination is entirely sensor driven. In this article, the dynamic architecture was changed in order to cope with unreliable sensor information by including this information in the vector field. We apply this architecture to generate timed trajectories for a Puma arm which must catch a moving ball before it falls over a table, and return to a reference position thereafter. Sensory information is provided by a camera mounted on the ceiling over the robot. We demonstrate that the implemented decision-mechanism is robust to noisy sensorial information. Further, a flexible behavior is achieved. Flexibility means that if the sensorial context changes such that the previously generated sequence is no longer adequate, a new sequence of behaviors, depending on the point at which the changed occurred and adequate to the current situation emerges

The timing of movements and of action sequences is important when particular events must be achieved in time-varying environments, avoiding moving obstacles or coordinating multiple robots. However, timing is difficult when it must be compatible with continuous on-line coupling to low-level and often noisy sensory information which is used to initiate and steer action. We extended the Dynamic Approach to Behavior Generation to account for timing constraints. We proposed a solution that uses a dynamical system architecture to autonomously generate timed trajectories and sequences of movements as attractor solutions of dynamic systems. The model consists of a two layer architecture, in which a competitive "neural" dynamics layer controls the qualitative dynamics of a second, "timing" layer. The second layer generates both stable oscillations and stationary states, such that periodic attractors generate timed movement. The first layer controls the switching between the limit cycle and the fixed points, allowing for discrete movements and movement sequences. This model was integrated with another dynamical system without timing constraints. The complete dynamical architecture was demonstrated on a vision-guided mobile robot in real time, whose goal is to reach a target in approximately constant time within a non-structured environment. The obtained results illustrated the stability and flexibility properties of the timing architecture as well as the robustness of the proposed decision-making mechanism.