Reinforcement Learning (RL) has gained significant attention in recent years. As RL software becomes more complex and infiltrates critical application domains, ensuring its quality and correctness becomes increasingly important. An indispensable aspect of software quality/correctness insurance is testing. However, testing RL software faces unique challenges compared to testing traditional software, due to the difficulty on defining the outputs' correctness. This leads to the RL test oracle problem. Current approaches to testing RL software often rely on human oracles, i.e. convening human experts to judge the correctness of RL software outputs. This heavily depends on the availability and quality (including the experiences, subjective states, etc.) of the human experts, and cannot be fully automated. In this paper, we propose a novel approach to design test oracles for RL software by leveraging the Lyapunov stability control theory. By incorporating Lyapunov stability concepts to guide RL training, we hypothesize that a correctly implemented RL software shall output an agent that respects Lyapunov stability control theories. Based on this heuristics, we propose a Lyapunov stability control theory based oracle,$\operatorname{LPEA}(\vartheta, \theta)$, for testing RL software. We conduct extensive experiments over representative RL algorithms and RL software bugs to evaluate our proposed oracle. The results show that our proposed oracle can outperform the human oracle in most metrics. Particularly, LPEA$(\vartheta=100 \%, \theta=75 \%)$outperforms the human oracle by$53.6 \%, 50 \%, 18.4 \%, 34.8 \%, 18.4 \%, 127.8 \%, 60.5 \%, 38.9 \%$, and 31.7 % respectively on accuracy, precision, recall, F1 score, true positive rate, true negative rate, false positive rate, false negative rate, and ROC curve's AUC; and LPEA ($\vartheta=100 \%, \theta=50 \%$) outperforms the human oracle by$48.2 \%, 47.4 \%, 10.5 \%, 29.1 \%$,$10.5 \%, 127.8 \%, 60.5 \%, 22.2 \%$, and 26.0 % respectively on these metrics.

In this article, we present a novel full-pose path planning strategy with degenerate direction avoidance for robot manipulators. Effort seeks to define an efficient multiobjective cost function for finding an optimal path in task space by minimizing the path length, and, meanwhile, maximizing orthogonality to the degenerate direction of the path. Since a robot manipulator by nature is a noncommensurate system, the unit dual quaternion is used to unify the pose representation and formulate the cost function of path length. Additionally, to justify the robot’s ability to maneuver, another cost evaluation of orthogonality is developed on the basis of the analytical Jacobian matrix decomposition. The optimized rapidly exploring random tree (RRT*) is, then, applied to execute the synthesized cost function in planning a path for a six-degree-of-freedom (6-DOF) FANUC-M-20iA industrial robot. By comparing with two relevant path planning methods, simulation analysis results illustrate the superior performance of the proposed path planning approach in terms of path direction and joint change.

In this paper, a novel adaptive sliding mode control scheme with RBF (radial basis function) neural network-based tuning method is proposed for the trajectory tracking of a 6-RSS (Revolute-Spherical-Spherical) parallel robot in Cartesian space. Parallel robot is a highly nonlinear system with closed-chain mechanisms, which poses the major challenges to the controller design. The robust sliding mode controller is developed to deal with system uncertainties such as modeling errors, frictions, and disturbances. With strong adaptation and learning ability, RBF neural network is adopted to identify the parallel robot dynamics, and then the adaptive self-tuning of the control gains in the controller is realized, which is more flexible than manual tuning method and can guarantee the desired results of the changing system. The stability of the controller has been validated using Lyapunov theorem. Simulation results demonstrate that the proposed controller can achieve better tracking performance than the sliding mode controller with fixed control gains.

Common assumptions in most of the previous nonlinear networked leader–follower systems are that the leader is provided as a constant signal and that the controllers are designed for either the joint-space regulation or the translational Cartesian-space motion control. This article addresses the control issue of the delay-induced horizontal shift effect when the leader is moving with a changing speed. A novel mixed-type feedback is introduced to reduce the horizontal shift effect so as to minimize tracking errors experienced by the follower agents. In this article, we also study the complete pose control using a nonsingular terminal sliding mode (NTSM) method. A stability analysis is provided to prove the finite-time boundedness of the tracking error signals. Quantitative evaluations of the multiple effects on the error bound are conducted to facilitate the subsequent control design to improve the performance. Numerical simulation results of a team of two degree-of-freedoms (DOFs) manipulators demonstrate the improved tracking performance of the end effectors with small bounded errors which are affected by network delays, maximum assigned accelerations, and the selection of control gains. The experimental results are provided to demonstrate the performance of the developed controller.

Telerehabilitation with Robotic Exoskeletons is an emerging technology aimed at assisting to restore patients' mobility using a master and slave robotic system. Some of the main challenges for achieving good tracking performance, stability and transparency in telerehabilitation are nonlinearities, uncertain and time-varying parameters in the robot and human models, and communication delays. Additionally, a paramount challenge for this technology is ensuring safe and compliant interaction between the robots and the human operators. This paper presents a novel control approach utilized during unilateral and bilateral teleoperation which address these challenges. An Adaptive Impedance Controller is designed using Lyapunov-based methods for the master exoskeleton while a Proportional-Derivative Impedance Controller is implemented on the slave exoskeleton. Subsequently, a torque limiter technique was implemented on the master side to ensure stability in the presence of time delays. The advantages of these controllers are that they address unknown dynamics, incorporate designed impedance response for rehabilitation applications, and are simple to implement. Simulations for two two-degree-of-freedom robotic exoskeletons are provided to demonstrate the effectiveness of these methods in both passive and assistive telerehabilitation modes, and with time delays.