ABSTRACT In this paper, we present an Augmented Reality (AR)‐based application combined with a robotic system for percutaneous renal puncture navigation interaction and demonstrate its technical feasibility. Our system provides an intuitive interaction scheme between the surgeon and the robot without the need for traditional external input devices, and applies an image‐target‐based 3D registration scheme to transform the coordinate system between Hololens2 and the robot without using additional tracking devices. Users can visualize the abdominal puncture phantom and obtain 3D depth information of the lesion site by wearing Hololens2 and control the robot directly using buttons or gestures. To investigate the accuracy and feasibility of the proposed interaction scheme, six subjects were recruited to complete 3D registration alignment accuracy experiments, and puncture positioning accuracy experiments using ultrasound unaided navigation, AR unaided navigation and AR robotic navigation. The results showed that the average alignment error of 3D registration was 3.61 ± 1.05 mm. The average positioning errors of ultrasound freehand navigation, AR freehand navigation and AR robotic navigation were 7.67 ± 2.00 mm, 6.13 ± 1.07 mm and 5.52 ± 0.37 mm, respectively; the average puncture times were 34.86 ± 1.67 s, 22.40 ± 2.07 s, and 29.41 ± 1.37 s.

It's been a long history that most object detection methods obtain objects by using the non-maximum suppression (NMS) and its improved versions like Soft-NMS to remove redundant bounding boxes. We challenge those NMS-based methods from three aspects: 1) The bounding box with highest confidence value may not be the true positive having the biggest overlap with the ground-truth box. 2) Not only suppression is required for redundant boxes, but also confidence enhancement is needed for those true positives. 3) Sorting candidate boxes by confidence values is not necessary so that full parallelism is achievable. In this paper, inspired by belief propagation (BP), we propose the Confidence Propagation Cluster (CP-Cluster) to replace NMS-based methods, which is fully parallelizable as well as better in accuracy. In CP-Cluster, we borrow the message passing mechanism from BP to penalize redundant boxes and enhance true positives simultaneously in an iterative way until convergence. We verified the effectiveness of CP-Cluster by applying it to various mainstream detectors such as FasterRCNN, SSD, FCOS, YOLOv3, YOLOv5, Centernet etc. Experiments on MS COCO show that our plug and play method, without retraining detectors, is able to steadily improve average mAP of all those state-of-the-art models with a clear margin from 0.3 to 1.9 respectively when compared with NMS-based methods.