As intelligent systems advance rapidly, human-robot collaboration is becoming increasingly important. Ensuring that the intelligent agent's behaviors match human intentions and value preferences is crucial for effective collaboration, which is termed the value alignment problem. Within the Reinforcement Learning (RL) paradigm, value alignment typically relies on pre-designed reward functions, and Cooperative Inverse Reinforcement Learning (CIRL) is often used to model value alignment as a human-robot game. However, existing works often assume that human is perfectly rational, and can fully obtain robot’s belief on human’s preference. To address this limitation, we propose a Particle Filter-based Hierarchical Dynamic Programming algorithm (PFHDP). By modeling the robot's belief state, this algorithm ensures the correct updates of human's estimate of the robot's belief. This allows human to adopt more targeted pedagogical behaviors to guide the robot based on her understanding of the robot's current belief, achieving belief alignment between human and robot and thereby promoting value alignment more effectively. Furthermore, we run experiments to evaluate the proposed method in two cooperative scenarios against some typical benchmark approaches. The experimental results show that our method can strengthen the alignment of belief states between human and robot, leading to enhanced value alignment.

To improve the comfort of surgeons during the use of minimally invasive surgical robots and reduce occupational hazards, the precise puncture robot for vital organs was selected as the research subject. Based on the theory and methods of human-machine ergonomics, combined with JACK software, a simulation analysis was conducted on the comfort, working posture, and visibility of surgeons using existing related products. Design improvements were made based on the analysis results. By comparing and evaluating the simulation experimental data before and after the design changes, it was found that the improved solution at the human-machine ergonomics level helps enhance the comfort of users during the operation process and reduce the risk of occupational diseases among the medical community. This design can serve as a guide and reference for related fields.