Reinforcement learning (RL) holds great promise for generating dynamic whole-body motions on humanoid robots, but its real-world application is hindered by the significant sim-to-real gap caused by actuator model mismatch. Most simulators assume torque-controlled or idealized PD actuators, whereas real-world platforms like the NAO robot employ position-controlled joints with hidden, proprietary control loops and nonlinear dynamics. To bridge this gap, we present a simulation-to-reality framework for whole-body motion control that integrates two key components: an equivalent torque-space PID actuator model and a neural network-based inverse kinematics method. First, we perform system identification on the physical robot by analyzing step response data and computing ground-truth joint torques via Lagrangian dynamics, from which we estimate equivalent PID parameters that replicate the real joint dynamics in simulation. Second, we design a learnable inverse kinematics module that maps human motion, represented by end-effector trajectories, into robot-executable joint commands. Using this high-fidelity simulation environment and motion retargeting pipeline, we train a whole-body control policy via RL in IsaacLab and deploy it directly on the physical NAO robot without any fine-tuning. Experimental results demonstrate successful reproduction of diverse human-like motions, including waving, squatting, and bimanual coordination, validating that accurate actuator modeling and structured motion representation are essential for reliable sim-to-real transfer in learning-based humanoid control. (Supplementary video link: https://youtu.be/zLWXgGZQAFk).

Human motion retargeting for humanoid robots, transferring human motion data to robots for imitation, presents significant challenges but offers considerable potential for real-world applications. Traditionally, this process relies on human demonstrations captured through pose estimation or motion capture systems. In this paper, we explore a text-driven approach to obtain imitation motion data more flexibly and simply. To address the inherent discrepancies between the generated motion representations and the kinematic constraints of humanoid robots, we propose an angle signal network based on norm-position and rotation loss (NPR Loss). It generates joint angles, which serve as inputs to a reinforcement learning based whole-body motion control policy. The policy ensures tracking of the generated motions while maintaining the robot’s stability during execution. Our experimental results demonstrate the efficacy of this approach, successfully transferring text-driven human motion to a real humanoid robot NAO.

Parallel micro/nano robots hold great potential in micro-manufacturing/assembly, microsurgery, and precision engineering because of their high precision and stiffness. However, it is challenging to develop a millimeter-scale robot with a large workspace, high bandwidth, and high precision. In this paper, we present the design, fabrication, tests, and potential applications of a piezoelectric planar parallel microrobot. The developed microrobot consists of a parallel mechanism, three amplification mechanisms, and three independently controlled piezoelectric actuators. The microrobot is miniaturized to the millimeter scale through a monolithic integrated manufacturing process, and achieves a dimension of 36$\times$36$\times$34 mm, a weight of 4.9 g, and a static workspace of 33.9 mm$^{2}$. The resonant frequencies reach 55-65 Hz in x and y directions, and 95 Hz in rotation. The microrobot exhibits high positioning accuracy in the trajectory tracking experiment on three different lines, circle, and triangle trajectories at a board bandwidth. Moreover, the microrobot can repeat 50 periodic circle trajectories in one second, with a velocity of 628.3 mm/s and a precision of 10.9$\upmu$m. Furthermore, we conducted three validation experiments to demonstrate the potential applications of the microrobot in tremor compensation for micromanipulation, 3D printing electronics, and minimally invasive surgery.Note to Practitioners—This work is motivated by the need to design a microrobot with a large workspace, high bandwidth, high precision, and compact structure. Such capabilities are essential across various fields of micromanipulation, including micro-manufacturing/assembly, microsurgery, and precision engineering. This paper presents a detailed manufacturing process for this type of microrobot, followed by comprehensive experimental tests. The resonant frequency, workspace, quasi-static and dynamic trajectory tracking experiments of the microrobot are thoroughly tested. The experimental results demonstrate the excellent performance of the microrobot in executing various operational tasks. Additionally, the potential applications in the fields of micromanipulation, 3D printing electronics, and minimally invasive surgery are demonstrated. By assembling various end effectors on the robot platform, the microrobot could have greater potential for application in more fields.

Continuum robots, owing to their inherent compliance, have become essential in endoscopic surgical procedures, such as mucosal ablation. However, the prevalent design of endoscopic manipulators, which typically features only a single active bending segment, often results in limited dexterity and accessibility. Additionally, the incorporation of variable stiffness in these robots has attracted significant interest, with the aim to improve manipulation capabilities in confined spaces. In the paper, we propose a novel variable-length continuum robot with variable stiffness for endoscopic surgery. The robot’s stiffness can be altered either by modifying the catheter’s length or solid-liquid transition of low-melting-point alloy (LMPA). The design and fabrication methods of the robot are meticulously detailed. Additionally, a quasi-static stiffness model along with a learning-based stiffness compensation approach for accurate stiffness estimation are proposed. Leveraging this model, a contact force controller is designed for ablation procedure. The experimental results show that our robot possesses good flexibility and accessibility, making it highly adept at manipulating in confined spaces. Its variable stiffness feature significantly enhances its ability to counteract external disturbance and prevent tip deformation (with a average position change of 1.1mm). Finally, through force control experiments and a surgical demonstration in a gastrointestinal model, we have further validated the robot’s applicability in surgical contexts. Note to Practitioners—This paper proposed a variable-length continuum robot with variable stiffness for endoscopic surgery. The robot can achieve axial elongation and omnidirectional bending motion, having better dexterity and accessibility than traditional medical continuum robots with one active bending segment. The robot’s stiffness can be adjusted by the length changes or solid-liquid transition of low-melting-point alloy (LMPA). Besides, an accurate stiffness model and a contact force controller are proposed for endoscopic ablation surgery. By experimental results, the robot shows high flexibility and accessibility, allowing access to confined spaces for manipulation, and good control accuracy and variable stiffness capability for endoscopic surgery.

Magnetic continuum robots are subject to external magnetic fields and deformed remotely, simplifying the robot’s transmission mechanism and providing it with significant potential for miniaturization and operational flexibility. However, modeling magnetic field distribution generated by permanent magnets is complex and requires time-consuming pre-calibrations. Moreover, it is highly susceptible to environments with ferromagnetic materials, posing significant challenges for the control of magnetic continuum robots. In response, we propose an approach that does not overly focus on the magnetic field distribution but instead directly learns the inverse kinematics of magnetic continuum robots end-to-end. Binding the robot’s configuration to the pose of external magnets, precise control of continuum robots is facilitated. Additionally, we leverage teleoperation techniques to broaden the applicability of this method. By mounting magnets on a robotic arm and directly utilizing the target pose of the external magnet predicted by a multi-layer perceptron (MLP), we achieve the operation and navigation of magnetic continuum robots in complex environments. Experiments demonstrate that the mean control accuracy along the robot using our learning-based inverse kinematics is about half of the robot’s diameter.

Small-scale continuum robot-assisted minimally invasive surgery has received crucial attention due to its smaller incisions and high dexterity. In medical scenarios such as radiofrequency ablation and nasal/throat swab sampling, monitoring and controlling the forces applied to human tissue can help improve the safety and comfort level of the procedure. However, the tip-sensor-based force detection method can barely be deployed due to the miniature size of the continuum robot; meanwhile, the mechanical modeling-based high-precision force estimation cannot be realized on account of the continuum robots’ complex structure with high nonlinear properties. To address the high-precision force estimation challenge for further compliant control during minimally invasive interventions, a learning-based high-precision force estimation method via long short-term memory (LSTM) is proposed in this paper. On this basis, compliance control and high-precision force tracking can be further realized for small-scale continuum robot. The compliance control ensures a smooth and stable transition during the interaction between the robot and the environment, and force tracking can be utilized for maintaining or precisely controlling the force applied to the human tissue. Finally, the contact force sensing and control experiments are carried out on a small-scale continuum robot system prototype, and a demonstration using a human nasal cavity model is conducted. The results validate that the proposed LSTM neural network fits the mechanical model of the continuum robot well with the root mean square error of 3.44mN, and the control method can significantly compensate for the instantaneous impact during contact with an attenuation of 59.3$\%$and rapidly respond to keep the force accurately at the expected value with the mean absolute error of 2.41mNNote to Practitioners—This research is motivated by the increasing number of applications for small-scale continuum robot-assisted minimally invasive interventional surgery, such as radiofrequency ablation, biopsy, and endoscopic submucosal dissection. These surgeries require high-precision contact force sensing and control. However, due to the small size and complex structure of the continuum robot, traditional methods such as installing force sensors and mechanical modeling are not effective. Therefore, this paper utilizes a neural network to infer contact force through more accessible information about the robot, such as the tension of the actuators. This method allows for compliant force control during the dynamic contact between the small-scale continuum robot and human tissue. Experiments conducted on a human nasal cavity model demonstrate that the proposed method can improve the safety and reliability of small-scale continuum robot-assisted minimally invasive surgery.

The Transformer model, which has significantly advanced natural language processing and computer vision, overcomes the limitations of recurrent neural networks and convolutional neural networks. However, it faces challenges with computational efficiency and memory management due to complex computations and variable-length inputs. Despite research efforts, these issues persist. This paper presents a novel optimization of the Transformer model, following an indepth analysis of its computational graph structure. Firstly, we utilize Deep Computing Unit (DCU) as our hardware platform. Secondly, we optimize element-wise and reduction operators through operator fusion and rewriting. Thirdly, we develop a fine-grained memory management algorithm using a greedy strategy. As a result, the training speed of the Transformer model increases by 1.2x - 1.4x without compromising accuracy.

Monocular single-frame 3D human pose estimation (HPE) has garnered significant interest, particularly in the domains of human-computer interaction and human action recognition. Several remarkable works have achieved excellent results in obtaining accurate 2D human poses. Building upon the foundation laid by previous advancements, this paper focuses on 2D-3D lifting. The proposed 2D-3D lifting algorithm in this paper is a Transformer-based model, which demonstrates its unique advantages in processing sequential data. The self-attention mechanism in the Transformer can handle global information without being limited by the receptive field. The Transformer's superior ability to process global information enables it to adaptively learn the relationships between human joints across different human behaviors. However, directly estimating 3D human pose from a single 2D image is a complex task due to depth ambiguity and joint occlusion. This ill-posed problem arises from the limited information available in a 2D image, making it challenging to determine the precise 3D pose. Additionally, occlusions further complicate the accurate estimation of joint positions. This article introduces a Transformer-based network that enhances the algorithm's robustness by utilizing multi-layer dual-stream blocks. One path concatenates the 2D coordinates and the focal length of the camera as input, while the other path takes the difference between the 2D coordinates as input. The two paths undergo an information fusion process at the front end of the block. Experiments conducted on various datasets verify the effectiveness of the algorithm and achieve state-of-the-art performance on Human3.6M and MPI-INF-3DHP benchmarks. Our code will be made publicly available on GitHub.

As a critical part of the 3D human pose estimation (HPE), establishing the 2D-to-3D lifting mapping is limited by depth ambiguity. Most current works generally lack the quantitative analysis of the relative depth expression and the depth ambiguity error expression in lifting mapping, resulting in low prediction efficiency and poor interpretability. To this end, this paper mines and leverages prior geometric knowledge of these expressions based on the pinhole imaging principle, decoupling the 2D-to-3D lifting mapping and simplifying the model training. Specifically, this paper proposes a prior geometric knowledge oriented pose estimation model with two-branch transformer architectures, explicitly introducing high-dimensional prior geometric features to improve model efficiency and interpretability. It converts the regression of spatial coordinates into the prediction of spatial direction vectors between joints to generate multiple feasible solutions further alleviate the depth ambiguity. Moreover, this paper raises a novel non-learning-based absolute depth estimation algorithm based on prior geometric relationship decoupling from relative depth expression for the first time. It establishes multiple independent depth mapping from non-root nodes to the root node to calculate the absolute depth candidate, which is parameter-free, plug-and-play, and interpretable. Experiments show that the proposed pose estimation model achieves state-of-the-art performance on Human 3.6M and MPI-INF-3DHP benchmarks with lower parameters and faster inference speed, and the proposed absolute depth estimation algorithm achieves similar performance to traditional methods without any network parameters. The source code are available athttps://github.com/Humengxian/GKONet.