The rapid development of immersive technologies, such as virtual reality (VR) and mixed reality (MR), presents an opportunity to combine these advancements to achieve embodied intelligence and robotic telemanipulation. We propose a mixed-reality telemanipulation system using hand tracking and digital twins technologies. The system has the characteristics of user friendliness, high level of immersion, naturalness of interaction and multiple degrees of freedom. Moreover, Optimal Resolved Acceleration (ORA) and Resolved Acceleration Control (RAC) are employed to MR remote operation framework. By introducing compliance control, they compensate for the drawbacks of the Resolved - Rate Motion Controller (RRMC), thereby enhancing environmental adaptability and the accuracy of telemanipulation. Consistency and precision testing verified the feasibility of the method, and the accuracy rate reached 94%. The MR remote operation framework enables compliance control. Future efforts will be concentrated on increasing task complexity, boosting system accuracy, and integrating intelligent functions.

This paper looks at how mixed reality can be used for the improvement and enhancement of Chinese acupuncture practice through the introduction of an acupuncture training simulator. A prototype system developed for our study allows practitioners to insert virtual needles using their bare hands into a full-scale 3D representation of the human body with labelled acupuncture points. This provides them with a safe and natural environment to develop their acupuncture skills simulating the actual physical process of acupuncture. It also helps them to develop their muscle memory for acupuncture and better develops their memory of acupuncture points through a more immersive learning experience. We describe some of the design decisions and technical challenges overcome in the development of our system. We also present the results of a comparative user evaluation with potential users aimed at assessing the viability of such a mixed reality system being used as part of their training and development. The results of our evaluation reveal the training system outperformed in the enhancement of spatial understanding as well as improved learning and dexterity in acupuncture practice. These results go some way to demonstrating the potential of mixed reality for improving practice in therapeutic medicine.