Lymphadenectomy generally accompanies various oncology surgeries to remove infected cancer cells. However, there are two limitations in robot-assisted lymphadenectomy: 1) lymph nodes are not visible during operation since they are hidden by the superficial fat layer; 2) intra-operative bleeding may occur during lymph node removal caused by collisions between surgical instruments and delicate blood vessels (arteries or veins) near the lymph nodes. Therefore, we propose a framework integrating augmented reality and virtual fixtures to address these limitations. Augmented reality intra-operatively visualizes the hidden lymph nodes by projecting the corresponding 3D pre-operative model, and virtual fixtures are used to provide force feedback to surgeons to avoid possible collisions when they operate the surgical instruments to resect the lymph nodes surrounding the blood vessel. Ten human subjects were invited to perform an emulated lymphadenectomy based on the da Vinci robot in a dry lab. Experimental results demonstrated that the proposed framework can keep localizing the hidden lymph nodes, and reduce the number of collisions (21% and 48% reduction rates using two different force models compared to the standard setup, respectively) between the instruments and the delicate blood vessel during lymph node resection. It shows the potential to enhance the safety of robot-assisted lymphadenectomy.

Robot-assisted surgery is rapidly developing in the medical field, and the integration of augmented reality shows the potential to improve the operation performance of surgeons by providing more visual information. In this paper, we proposed a markerless augmented reality framework to enhance safety by avoiding intra-operative bleeding, which is a high risk caused by collision between surgical instruments and delicate blood vessels (arteries or veins). Advanced stereo reconstruction and segmentation networks are compared to find the best combination to reconstruct the intra-operative blood vessel in 3D space for registration with the pre-operative model, and the minimum distance detection between the instruments and the blood vessel is implemented. A robot-assisted lymphadenectomy is emulated on the da Vinci Research Kit in a dry lab, and ten human subjects perform this operation to explore the usability of the proposed framework. The result shows that the augmented reality framework can help the users to avoid the dangerous collision between the instruments and the delicate blood vessel while not introducing an extra load. It provides a flexible framework that integrates augmented reality into the medical robotic platform to enhance safety during surgery.