A wide range of tools, machines from diverse industries are operated by the precise rotation of physical devices (e.g., knobs, encoders). Literature proved that Mixed Reality can provide rotation guidance by superimposing digital cues onto the real environment. However, existing studies primarily focus on speed rather than precision and lack proven solutions with standard comparison methods. We presented a novel open MR testbed to evaluate MR rotation guidance cue designs. It comprises (i) a 3D-printed structure with a controller mount for MR cue tracking, (ii) an industrial encoder (resolution 1.8º), (iii) an Esp32 microcontroller, (iv) a firmware for USB connection, and (v) a USB foot pedal to confirm rotation, (vi) test software and documentation. The testbed uses available off-the-shelf components, and it is distributed with an open source license, Unity package with cues examples and scripts for running tests with randomized targets and collecting performance data (error count and speed). We evaluated the MR testbed by comparing the Arrow cue (baseline) and two novel designs: Magnifier with a zoomed region and Gestalt leveraging related perception principles. A within-subjects study ($\mathrm{N}=36,180^{\prime \prime}, 18$ọ,$\mathrm{M}=27.58, \pm 3.55$) showed that the Magnifier is the most precise ($p<.001$), while the Gestalt is the fastest ($p<.001$). Moreover, males (higher motor and gaming skills) ($\mathrm{p}<.002, \mathrm{t}=3.89$) resulted in faster ($\mathrm{r}=-0.458, \mathrm{p}<$. 005). Participants who declared with higher gaming skills demonstrated less task load during the experiment ($\mathrm{r}=+0.548, \mathrm{p}<.005$). Overall, the participants slightly preferred the Gestalt (18) over the Magnifier (15), while the Arrow was selected by a few (3). Ultimately, the testbed has proven to be an effective research and evaluation platform for future mixed reality interfaces.

The literature has demonstrated the advantages of Mixed Reality (MR) for product configuration by providing a more engaging and effective end-user experience. While collaborative and remote design tools in MR have been widely explored in previous studies, a noticeable gap remains in the exploration of co-located product configuration for couples. This gap is noteworthy since in many industries, couples (e.g., friends, partners) often make purchasing decisions together in physical retail environments. In this paper, we introduce MR-CoCo, an open MR testbed designed to explore collaborative configurations by co-located couples, both in the role of customers. The testbed is developed in Unity and features: (i) a shared MR space with virtual product 3D model anchoring, (ii) shared visualization of the current configuration, (iii) a versatile UI for selecting configuration areas, (iv) hand gestures for 3D drag and drop of colors and materials from 3D catalog to the product. A case study of the personalization of a sailboat is provided as proof of concept. The user study involved 24 couples (48 participants in total), simulating a purchasing experience and the related configuration using MR-CoCo. We assessed usability through post-experience evaluations, with the System Usability Scale (SUS) and the Co-Presence Configuration Questionnaire (CCQ) to measure collaboration and decision-making. The results demonstrated a high level of usability and perceived quality of collaboration. We also explore guidelines that can be used for remote collaboration applications, enabling configuration across a wide range of industries (e.g., automotive and clothing).

TOTTA outlines the spatial position and rotation guidance of a real/virtual tool (TO) towards a real/virtual target (TA), which is a key task in Mixed reality applications. The task error can have critical consequences regarding safety, performance, and quality, such as surgical implantology or industrial maintenance scenarios. The TOTTA problem lacks a dedicated study and it is scattered in different domains with isolated designs. This work contributes to a systematic review of the TOTTA visual widgets, studying 70 unique designs from 24 papers. TOTTA is commonly guided by the visual overlap -an intuitive, pre-attentive "collimation" feedback- of simple shaped widgets: Box, 3D Axes, 3D Model, 2D Crosshair, Globe, Tetrahedron, Line, Plane. Our research discovers that TO and TA are often represented with the same shape. They are distinguished by topological elements (e.g. edges/vertices/faces), colors, transparency levels, and added. shapes, widget quantity, and size. Meanwhile some designs provide continuous "during manipulation feedback" relative to the distance between TO and TA by text, dynamic color, sonification, and amplified graphical visualization. Some approaches trigger discrete "TA reached feedback" such as color alteration, added sound, TA shape change, and added text. We found the lack of golden standards, including in testing procedures, as current ones are limited to partial sets with different and incomparable setups (different target configurations, avatar, background, etc.). We also found a bias in participants: right-handed, young male, non-color impaired.

Mixed Reality (MR) is proven in the literature to support precise spatial dental drill positioning by superimposing 3D widgets. Despite this, the related knowledge about widget's visual design and interactive user feedback is still limited. Therefore, this study is contributed to by co-designed MR drill tool positioning widgets with two expert dentists and three MR experts. The results of co-design are two static widgets (SWs): a simple entry point, a target axis, and two dynamic widgets (DWs), variants of dynamic error visualization with and without a target axis (DWTA and DWEP). We evaluated the co-designed widgets in a virtual reality simulation supported by a realistic setup with a tracked phantom patient, a virtual magnifying loupe, and a dentist's foot pedal. The user study involved 35 dentists with various backgrounds and years of experience. The findings demonstrated significant results; DWs outperform SWs in positional and rotational precision, especially with younger generations and subjects with gaming experiences. The user preference remains for DWs (19) instead of SWs (16). However, findings indicated that the precision positively correlates with the time trade-off. The post-experience questionnaire (NASA-TLX) showed that DWs increase mental and physical demand, effort, and frustration more than SWs. Comparisons between DWEP and DWTA show that the DW's complexity level influences time, physical and mental demands. The DWs are extensible to diverse medical and industrial scenarios that demand precision.

In this special issue ofIEEE Transactions on Visualization and Computer Graphics (TVCG), we are pleased to present the journal papers from the 22nd IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2023), which will be held as a hybrid conference between October 16 and 20, 2023 in Sydney, Australia. ISMAR continues the over twenty-year long tradition of IWAR, ISMR, and ISAR, and is the premier conference for Mixed and Augmented Reality in the world.

The creation and management of content are among the main open issues for the spread of Augmented Reality. In Augmented Reality interfaces for procedural tasks, a key authoring strategy is chunking instructions and using optimized visual cues, i.e., tailored to the specific information to convey. Nevertheless, research works rarely present rationales behind their choice. This work aims to provide design guidelines for the localization of in-view and not occluded components, which is recurrent information in technical documentation. Previous studies revealed that the most suited visual cues to convey this information are auxiliary models, i.e., abstract shapes that highlight the space region where the component is located. Among them, 3D arrows are widely used, but they may produce ambiguity of information. Furthermore, from the literature, it is unclear how to design auxiliary model shapes and if they are affected by the component shapes. To fill this gap, we conducted two user studies. In the first study, we collected the preference of 45 users regarding the shape, color, and animation of auxiliary models for the localization of various component shapes. According to the results of this study, we defined guidelines for designing optimized auxiliary models based on the component shapes. In the second user study, we validated these guidelines by evaluating the performance (localization time and recognition accuracy) and user experience of 24 users. The results of this study allowed us to confirm that designing auxiliary models following our guidelines leads to a higher recognition accuracy and user experience than using 3D arrows.

Drill tool positioning in dental implantology is a challenging task requiring 5DOF precision as the rotation around the tool axis is not influential. This work improves the quasi-static visual elements of the state-of-the-art with a novel Augmented Collimation Widget (ACW), an interactive tool of position and angle error visualization based on the gestalt reification, the human ability to group geometric elements. The user can seek in a quick, pre-attentive way the collimation of five (three positional and two rotational) error component widgets (ECWs), taking advantage of three key aspects: component separation and reification, error visual amplification, and dynamic hiding of the collimated components. We compared the ACW with the golden standard in a within-subjects (N=30) user test using 32 implant targets, measuring the time, error, and usability. ACW performed significantly better in positional (+19%) and angular (+47%) precision accuracy and with less mental demand (-6%) and frustration (-13%), but with an expected increase in task time (+59%) and physical demand (+64%). The interview indicated the ACW as the main preference and aesthetically more pleasant than GSW, candidating it as the new golden standard for implantology, but also for other applications where 5DOF positioning is key.

Immersive Virtual Reality (IVR) training offers the capability to industrial workers to acquire skills and address complex tasks by immersing them in a safe and controlled virtual environment (VE). However, in the literature, IVR training is mainly based on principles of standardization and efficiency without considering the operators’ well-being. A novel design approach consists of the introduction in the VE of Positive Computing to improve workers’ well-being by applying the Biophilia hypothesis. In this work, we explored the possibility of introducing biophilic elements in a VE training scenario that would support psychological well-being and human potential. However, the introduction of virtual elements not related to the training task may distract operators, impairing their performance. We selected as a training scenario the assembly of a real truck engine. It is accomplished in a workstation, and operators do not interact with the surrounding VE. Therefore, we placed the training area into four different types of VEs: 3D Minimal (MIN), 3D Minimal Biophilic enriched (MIN+BIO), 3D Realistic (REAL), and 3D Realistic Biophilic enriched (REAL+BIO). We compared the MIN and REAL scenarios with the respective biophilic enriched scenarios. The performance of 40 participants was evaluated in terms of completion time, object fixation time, training task accuracy, knowledge accuracy, cognitive load, and user experience. The results revealed that introducing biophilic elements in a VR training environment attracts users’ attention in the idle phase of the training. In contrast, they keep concentrating on the task without worsening their performance during the task accomplishment.

Industrial Augmented Reality (iAR) has demonstrated its advantages to communicate technical information in the fields of maintenance, assembly, and training. However, literature is scattered among different visual assets (i.e., AR visual user interface elements associated with a real scene). In this work, we present a systematic literature review of visual assets used in these industrial fields. We searched five databases, initially finding 1757 papers. Then, we selected 122 iAR papers from 1997 to 2019 and extracted 348 visual assets. We propose a classification for visual assets according to (i) what is displayed, (ii) how it conveys information (frame of reference, color coding, animation), and, (iii) why it is used. Our review shows that product models, text and auxiliary models are, in order, the most common, with each most often used to support operating, checking and locating tasks respectively. Other visual assets are scarcely used. Product and auxiliary models are commonly rendered world-fixed, color coding is not used as often as expected, while animations are limited to product and auxiliary model. This survey provides a snapshot of over 20 years of literature in iAR, useful to understand established practices to orientate in iAR interface design and to present future research directions.

It is our great pleasure to introduce the workshops and tutorials of the 19th IEEE International Symposium on Mixed and Augmented Reality. We will remember this year for the COVID-19 outbreak and the associated decision to migrate the conference into the virtual realm. This decision required a new conference composition, asking for an engaging and innovative format; it also imposes new challenges to workshop and tutorial organizers. Despite this unique situation, we are honored to announce that the ISMAR community members proposed three workshops and seven tutorials, covering various topics addressing Augmented Reality, Virtual Reality, and Mixed Reality.

Technical information presentation is evolving from static contents presented on paper or via digital publishing to real-time context-aware contents displayed via virtual and augmented reality devices. We present a Context-Aware Technical Information Management system (CATIM), that dynamically manages (1) what information as well as (2) how information is presented in an augmented reality interface. CATIM acquires context data about activity, operator, and environment, and then based on these data, proposes a dynamic augmented reality output tailored to the current context. The system was successfully implemented and preliminarily evaluated in a case study regarding the maintenance of a hydraulic valve.

Due to the recent advances in technologies for gesture recognition, midair gestures can be considered the interface of the future in a large number of applications. However, designing effective interfaces with midair gestures is not an easy task because the design is application dependent and it must fulfill many requirements at the same time. Despite the availability of general guidelines in the literature, clear and well-established procedures for the optimal design of midair gesture-based interfaces are, to date, not available and remain an open issue. The main contribution of this paper is a user-centered modular framework, which integrates existing and novel methods. It supports the designer considering multiple aspects including ergonomics, memorability, and specific user requirements tailored to the application scenario. The framework involves three design steps and a final validation step, also supported by dedicated software. We tested with success the proposed framework in an industrial case study, where technicians must easily access technical information by browsing digital manuals during maintenance operations.

We present a novel interaction method for augmented industrial maintenance based on a "magic mirror" interface and virtual motion buttons. The system includes a video camera for object tracking, a video\depth camera for capturing user gestures, a projector for displaying technical instruction to the operator and a LCD monitor providing feedback of the virtual buttons. The operator can trigger maintenance commands by directional swift of the hands in regions sensitive to motion speed and direction. The main advantage of the presented interface is that it can work in realistic industrial conditions: (i) operators wearing gloves, (ii) operators handling tools, (iii) presence of moving machinery and personnel in the background. We measured the performances of the system with a laboratory test and we proved the feasibility with an automotive inspection test case. We calculated an average interaction time below 2 seconds and an error rate lower than 5%. However, we found some performances limitations if the operator is handling tools.

Text readability with augmented reality head-worn displays is critical and at present time, there are no standard guidelines to follow. The readability depends mainly on background lighting, display technology (i.e., OST: optical see-through or VST: video see-through), and text style (e.g., plain text, outline or billboard). In this work, we addressed the readability limits for industrial activities. We experimented the effects of two background illuminances levels (1,000 lx for very fine basic industrial tasks and 4,000 lx for fine machining), two commercially available head-worn display technologies, variable outline widths and contrast polarity of text. We analyzed the performance of 12 subjects by collecting about 3,400 measurements using a specific test application and followed by qualitative interviews. With high illuminances, VST performed better than OST, regardless of contrast polarity and outline width. We found that negative contrast polarity is preferable with VST, and that just a minimum outline (1 px) around black text is optimal. On the contrary, positive contrast polarity should be used with OST and outline is not effective. Therefore, we evaluated the usage limits of the OST by sampling its contrast sensitivity function.

Efficient text visualization in head-worn augmented reality (AR) displays is critical because it is sensitive to display technology, text style and color, ambient illumination and so on. The main problem for the developer is to know the optimal text style for the specific display and for applications where color coding must be strictly followed because it is regulated by laws or internal practices. In this work, we experimented the effects on readability of two head-worn devices (optical and video see-through), two backgrounds (light and dark), five colors (white, black, red, green, and blue), and two text styles (plain text and billboarded text). Font type and size were kept constant. We measured the performance of 15 subjects by collecting about 5,000 measurements using a specific test application and followed by qualitative interviews. Readability turned out to be quicker on the optical see-through device. For the video see-through device, background affects readability only in case of text without billboard. Finally, our tests suggest that a good combination for indoor augmented reality applications, regardless of device and background, could be white text and blue billboard, while a mandatory color should be displayed as billboard with a white text message.

In the fields of industrial design and car manufacturing the creation of 3D curves plays a fundamental role within the design process: it allows the improvement of the visual appeal of artifacts, it enhances ergonomics and the product's commercial competitiveness through product differentiation. When flexibility and intuition are to be privileged it is fundamental to achieve natural, intuitive, mathematically correct, creation and modification of surfaces.The scientific aim of this research is the development of an innovative metaphor to modeling of 3D curves which maintains the natural expertise of the designer. The major contribution of this paper is the capability of the system to create and modify the curve naturally, without mathematical artifices, within the limits set by the use of Bezier curves.The proposed metaphor combines the benefits of two acknowledged techniques referred to as the Digital Tape Drawing and the Eraser Pen, which allow the real-time modification of the curve. The integrated adoption of tangible interfaces and innovative mathematical tools, combined with the adoption of semi-immersive environment and lightweight interaction devices, delivers intuitive curve creation for free-form modeling within the virtual scene. The paper describes the details of the algorithm developed and it highlights its strengths during the styling phase.

Spacedesign is an innovative mixed reality (MR) application addressed to aesthetic design of free form curves and surfaces. It is a unique and comprehensive approach which uses task-specific configurations to support the design workflow from concept to mock-up evaluation and review. The first-phase conceptual design benefits from a workbench-like 3-D display for free hand sketching, surfacing and engineering visualization. Semitransparent stereo glasses augment the pre-production physical prototype by additional shapes, textures and annotations. Both workspaces share a common interface and allow collaboration and cooperation between different experts, who can configure the system for the specific task. A faster design workflow and CAD data consistency can be thus naturally achieved. Tests and collaborations with designers, mainly from automotive industry, are providing systematic feedback for this ongoing research. As far as the authors are concerned, there is no known similar approach that integrates the creation and editing phase of 3D curves and surfaces in virtual and augmented reality (VR/AR). Herein we see the major contribution of our new application.