Understanding, navigating, and performing goal-oriented actions in Mixed Reality (MR) environments is a challenging task and requires adequate information conveyance about the location of all virtual objects in a scene. Current Head-Mounted Displays (HMDs) have a limited field-of-view where augmented objects may be displayed. Furthermore, complex MR environments may be comprised of a large number of objects which can be distributed in the extended surrounding space of the user. This paper presents two novel techniques for visually guiding the attention of users towards out-of-view objects in HMD-based MR: the 3D Radar and the Mirror Ball. We evaluate our approaches against existing techniques during three different object collection scenarios, which simulate real-world exploratory and goal-oriented visual search tasks. To better understand how the different visualizations guide the attention of users, we analyzed the head rotation data for all techniques and introduce a novel method to evaluate and classify head rotation trajectories. Our findings provide supporting evidence that the type of visual guidance technique impacts the way users search for virtual objects in MR.

Left-right confusion occurs across the entire population and refers to an impeded ability to distinguish between left and right. In medicine this phenomenon is particularly relevant as left and right are always defined with respect to the patient's point of view, i.e. the doctor's right is the patient's left. Traditional anatomy learning resources such as illustrations in textbooks naturally consider this by consistently depicting the anatomy of a patient as seen by an observer standing in front. Augmented Reality Magic Mirrors (MM) are one example of novel anatomy teaching resources and show a user's digital mirror image augmented with virtual anatomy on a large display. As left and right appear to be reversed in such MM setups, similar to real-world physical mirrors, intriguing perceptual questions arise: is a non-reversing MM (NRMM) the more natural choice for the task of anatomy learning and do users even learn anatomy the wrong way with a traditional, reversing MM (RMM)? In this paper, we explore the perceptual differences between an NRMM and RMM design and present the first empirical study comparing these two concepts for the purpose of anatomy learning. Experimental results demonstrate that medical students perform significantly better at identifying anatomically correct placement of virtual organs in an NRMM. However, interaction was significantly more difficult compared to an RMM. We explore the underlying psychological effects and discuss the implications of using an NRMM on user perception, knowledge transfer, and interaction. This study is relevant for the design of future MM systems in the medical domain and lessons-learned can be transferred to other application domains.

Screen-based Augmented Reality (AR) systems can be built as a window into the real world as often done in mobile AR applications or using the Magic Mirror metaphor, where users can see themselves with augmented graphics on a large display. The term Magic Mirror implies that the display shows the users enantiomorph, i.e. the mirror image, such that the system mimics a real-world physical mirror. However, the question arises whether one should design a traditional mirror, or instead display the true mirror image by means of a non-reversing mirror? We discuss the perceptual differences between these two mirror visualization concepts and present a first comparative study in the context of Magic Mirror anatomy teaching.

Medical robotic ultrasound offers potential to assist interventions, ease long-term monitoring and reduce operator dependency. Various techniques for remote control of ultrasound probes through telemanipulation systems have been presented in the past, however not exploiting the potential of fully autonomous acquisitions directly performed by robotic systems. In this paper, a trajectory planning algorithm for automatic robotic ultrasound acquisition under expert supervision is introduced. The objective is to compute a suitable path for covering a volume of interest selected in diagnostic images, for example by prior segmentation. A 3D patient surface point cloud is acquired using a depth camera, which is the sole prerequisite besides the volume delineation. An easily parameterizable path function generates single or multiple parallel scan trajectories capable of dealing with large target volumes. A spline is generated through the preliminary path points and is transferred to a lightweight robot to perform the ultrasound scan using an impedance control mode. The proposed approach is validated via simulation as well as on phantoms and on animal viscera.

Image-guided medical interventions more frequently rely on Augmented Reality (AR) visualization to enable surgical navigation. Current systems use 2-D monitors to present the view from external cameras, which does not provide an ideal perception of the 3-D position of the region of interest. Despite this problem, most research targets the direct overlay of diagnostic imaging data, and only few studies attempt to improve the perception of occluded structures in external camera views. The focus of this paper lies on improving the 3-D perception of an augmented external camera view by combining both auditory and visual stimuli in a dynamic multi-sensory AR environment for medical applications. Our approach is based on Temporal Distance Coding (TDC) and an active surgical tool to interact with occluded virtual objects of interest in the scene in order to gain an improved perception of their 3-D location. Users performed a simulated needle biopsy by targeting virtual lesions rendered inside a patient phantom. Experimental results demonstrate that our TDC-based visualization technique significantly improves the localization accuracy, while the addition of auditory feedback results in increased intuitiveness and faster completion of the task.