Surgery is primarily taught through mentoring, where an expert mentor supervises a mentee performing surgery, taking over when necessary. Telementoring systems aim to provide mentees with access to remote mentors, but the physical distance between mentors and mentees poses unique challenges to surgical training. We investigate the underlying needs leading to takeovers in onsite mentoring and assess mentors’ ability to fulfill address these needs remotely using existing telestration tools, namely pointers and drawings on shared views. Through interviews and workshops with expert surgeons, we find that (1) mentors take over to convey gestures related to instrument placement, tissue displacement, force, and movement, (2) mentors gather information about location of tissue, equipment, and instruments, as well as gesture constraints, and (3) surgeons judge telestration insufficient for these needs. Based on this gap between onsite mentoring practices and telementoring tools, we discuss novel tools to address these needs and their evaluation.

All along their careers, surgeons learn through mentoring, operating on patients side-by-side with expert surgeons. However, access to experts is more and more challenging as they are often located far from learners. Telementoring can increase the reach of mentors by supporting mentoring at a distance, but despite their potential, current systems remain rarely institutionalized. We investigate current surgical mentoring practices, to inform future design of telementoring systems in providing support for learning. In a field study, we observe 11 surgeries and conduct follow-up self-confrontation interviews with mentor-mentee pairs. Through Thematic Analysis, we find that the domain of surgery results in mentee's needs for reassurance regarding safety, being spared from workload and risk, both essential to support the acquisition of skills. Second, that mentors understand the situation primarily through the surgical site view but also through inspecting the mentee's posture and the patient's body. Thirdly, that mentors take physical distance as their presence limits mentees' autonomy development. We discuss the limits of current surgical telementoring systems and rethink their role as companions of onsite mentoring.

Although Augmented Reality (AR) has been touted as the future of surgery, its contribution to distributed collaboration such as in surgical teleconsulting has not been articulated. We propose AR-Head Mounted Displays (AR-HMD) to tackle two previously-identified challenges: operating surgeons needing to view and interact with imaging systems that reside away from the operative field, and, their lack of gesturing tools to point and annotate on the shared images and physical environment. We report on a controlled lab experiment where 12 expert gynecology surgeons perform a tumor localisation task guided by a remote radiologist (confederate) via an AR-HMD. We find that bringing the shared images to the place of work reduces the need for clarifications and provides opportunistic access to information when required, and, that pointing and annotating provides opportunities to further support verbal instruction in deictic communication. Our results inform the design of intraoperative AR-HMD systems for surgical telecollaboration.

Videos are a powerful media to learn activities through guided physical training such as surgery, especially when they are produced following human learning models and not as "how-to" videos. However, their success greatly depends on how they are integrated into the extensive curricula of domains where learning occurs through guided practice. In this work, we investigate the impact of integrating video as a learning tool into the learning curricula of surgery. We created a pedagogical video on surgical hysterectomy through a model based on the Conceptual Fields theory (Vergnaud) and performed two rounds of interviews with seven medical residents, who watched the video freely during their residency in gynecology-obstetrics as they trained with experts. We find that videos can complement guided physical training, as they can provide the rationale behind expert action, something that is difficult to explicit during guided training. Still, their linear and static nature limits their integration as true adjuncts. We discuss our vision of moving towards interactive videos created with an ontological approach, developed in a workshop with four expert surgeons, which involves the ability to navigate through levels of information and layers of representations, so that experts can represent information to learners according to pedagogical models that complement their complex and extensive learning curricula.

Robot-Assisted Laparoscopic Surgery (RALS) is now prevalent in operating rooms. This situation requires future surgeons to learn Classic Laparoscopic Surgery (CLS) and RALS simultaneously. Therefore, along with the investigation of the differences in performance between the two techniques, it is essential to study the impact of training in RALS on the skills mastered in CLS. In this article, we study comanipulated RALS (Co-RALS), one of the two designs for RALS, where the human and the robot share the execution of the task. We use a rarely used in Human–Robot Interaction measuring tool: gaze tracking and time recording to measure for the acquisition of skills in CLS when training in Co-RALS or in CLS and time recording to compare the learning curves between Co-RALS and CLS. These metrics allow us to observe differences in Co-RALS and CLS. Training in Co-RALS develops slightly better but not significantly better hand–eye coordination skills and significantly better timewise performance compared with training in CLS alone. Co-RALS enhances timewise performance in laparoscopic surgery on specific types of tasks that require precision rather than depth perception skills compared with CLS. The results obtained enable us to further define the Human–Robot Interaction quality in Co-RALS.

While surgical videos are valuable support material for activities around surgery, their summarization demands great amounts of time from surgeons, resulting in the production of very few videos. We study the practices involving surgical video to motivate and inform the future design of tools for their summarization. Through interviews and observations in a field study, we find that (1) video summaries provide an important support for surgery, being used for self-improvement, education, discussing cases, scientific research, patient communication and as legal resources; (2) video summarization follows a process hindered by the loss of knowledge that originates during recording; and, (3) surgeons develop ad-hoc coordination strategies which involve using the video itself for articulation work, making it both the field of work and coordination artifact. We discuss ways in which tools can facilitate capturing knowledge during live action using these strategies.

Bedside robotic endoscopes render surgeons autonomous from assistants, potentially improving surgical outcome and decreasing costs. Why then have they not been widely adopted? We take a step back and first characterize classic (non-robotic) endoscope use through observations, literature and a domain expert interview. We review the literature on bedside robotic endoscopes and find that existing controls, individually, do not have the power to support both intended and appropriated endoscope uses. We thus explore combining controls to support this diversity of uses. Through an iterative cycle, we design and implement a multimodal and mixed-initiative technique that combines two user controls and one system control. Our evaluations confirm that individual controls do not satisfy the diversity of endoscope uses, and also that our technique indeed does so. Our work highlights the relevance of HCI research in the medical domain through robotic systems.

Robotic-assisted Minimally Invasive Surgery (MIS) is adopted more and more as it overcomes the shortcomings of classic MIS for surgeons while keeping the benefits of small incisions for patients. However, introducing new technology oftentimes affects the work of skilled practitioners. Our goals are to investigate the impacts of telemanipulated surgical robots on the work practices of surgical teams and to understand their cause. We conducted a field study observing 21 surgeries, conducting 12 interviews and performing 3 data validation sessions with surgeons. Using Thematic Analysis, we find that physically separating surgeons from their teams makes them more autonomous, shifts their use of perceptual senses, and turns the surgeon's assistant into the robot's assistant. We open design opportunities for the HCI field by questioning the telemanipulated approach and discussing alternatives that keep surgeons on the surgical field.