Current constrained reinforcement learning (RL) methods guarantee constraint satisfaction only in expectation, which is inadequate for safety-critical decision problems. Since a constraint satisfied in expectation remains a high probability of exceeding the cost threshold, solving constrained RL problems with high probabilities of satisfaction is critical for RL safety. In this work, we consider the safety criterion as a constraint on the conditional value-at-risk (CVaR) of cumulative costs, and propose the CVaR-constrained policy optimization algorithm (CVaR-CPO) to maximize the expected return while ensuring agents pay attention to the upper tail of constraint costs. According to the bound on the CVaR-related performance between two policies, we first reformulate the CVaR-constrained problem in augmented state space using the state extension procedure and the trust-region method. CVaR-CPO then derives the optimal update policy by applying the Lagrangian method to the constrained optimization problem. In addition, CVaR-CPO utilizes the distribution of constraint costs to provide an efficient quantile-based estimation of the CVaR-related value function. We conduct experiments on constrained control tasks to show that the proposed method can produce behaviors that satisfy safety constraints, and achieve comparable performance to most safe RL (SRL) methods.

The application requirements of the tendon-sheath mechanism in the field of precision machinery are becoming increasingly extensive. However, the contact friction between the tendon and sheath seriously affects the transmission accuracy. In the case of unavoidable friction, optimizing the tendon transmission path to reduce tension loss and elastic deformation has become an important research direction. In this article, the influence law of the tendon transmission path on the tension and displacement transmission is obtained using the two parameters related to the curvature of the transmission path: total bending angle and equivalent tendon length. Then, based on the optimal control theory and minimum principle, the different transmission path solutions of the minimum tension loss, the minimum tendon deformation, and the coupling of tension and displacement are obtained; the numerical optimization method verifies the correctness of the proposed theory. Finally, an optimal design of a tendon-constrained synchronous rotation mechanism for the manipulator is carried out, and the linkage performance is greatly improved by optimizing the transmission path.

Pattern recognition (PR) methods are commonly utilized in the extraction of motion intentions from myoelectric signals, which is realized by relating several electromyogram (EMG) patterns to specific types of motion. Researchers have reported that the hand-engineering features widely used in PR-based methods can be significantly affected by external confounding factors that diminish their accuracy and robustness in clinical settings. Moreover, since only simple mapping is carried out from the feature space to the task space (involving, for the most part, discrete motion intentions), there is no opportunity to exploit fully the underlying mechanism of synergic neuromuscular control. Inspired by deep learning, we have proposed a novel convolutional neural network (CNN) structure based on the characteristics of raw EMG signals that can effectively decode complex wrist movements with three degrees of freedom (DOF) directly from raw EMG signals rather than relying on hand-engineering features. Our method has the potential to incorporate more information than other models by enlarging the training dataset. We demonstrate here that our method performs significantly better (in terms of R2) than the current state-of-art regression method (i.e., support vector regression), especially when confounding factors are involved. We further found that this CNN-based decoding method can be generalized when multiple healthy subjects are taken into account. For a new subject, our method can provide an appropriate control over 3-DOF cursor movements on a screen even without a specific training.