Abstract Augmented Reality (AR) involves the combination of synthetic and real stimuli, not being restricted to visual cues. For the inclusion of computer-generated sound in AR environments, it is often assumed that the distance attenuation model is the most intuitive and useful system for all users, regardless of the characteristics of the environment. This model reduces the gain of the sound sources as a function of the distance between the source and the listener. In this paper, we propose a different attenuation model not only based on distance, but also considering the listener orientation, so the user could listen more clearly the objects that they are looking at, instead of other near objects that could be out of their field of view and interest. We call this a directional attenuation model. To test the model, we developed an AR application that involves visual and sound stimuli to compare the traditional model versus the new one, by considering two different tasks in two AR scenarios in which sound plays an important role. A total of 38 persons participated in the experiments. The results show that the proposed model provides better workload for the two tasks, requiring less time and effort, allowing users to explore the AR environment more easily and intuitively. This demonstrates that this alternative model has the potential to be more efficient for certain applications.

The visualization of fabrics at microscopic level involves a significant computational cost due to the large number of 3d elements required to display each yarn. Often, these visualizations rely on offline systems that display the final result only after an extended rendering time, which does not allow real-time interaction or visualization. While other models can allow for real-time rendering, they may require high memory usage and preprocessing times. Moreover, threads with metallic wraps used in historical fabrics cannot be represented using existing yarn models. In this paper, we present a yarn-level weaving model based on Sphere Tracing, which enables real-time rendering of historical Jacquard weaves with diverse weft types. The primary goal of this visualization is to analyze and reconstruct weaving structures constructed using these looms to safeguard this valuable cultural heritage. Furthermore, we propose a new level of detail technique, which extends the fundamental Sphere Tracing algorithm. This approach enables the visualization of a complete fabric at varying scales while requiring low memory usage and short preprocessing times.

Robotic-Assisted Surgery (RAS) is beginning to unlock its potential. However, despite the latest advances in RAS, the steep learning curve of RAS devices remains a problem. A common teaching resource in surgery is the use of videos of previous procedures, which in RAS are almost always stereoscopic. It is important to be able to add virtual annotations onto these videos so that certain elements of the surgical process are tracked and highlighted during the teaching session. Including virtual annotations in stereoscopic videos turns them into Mixed Reality (MR) experiences, in which tissues, tools and procedures are better observed. However, an MR-based annotation of objects requires tracking and some kind of depth estimation. For this reason, this paper proposes a real-time hybrid tracking–matching method for performing virtual annotations on RAS videos. The proposed method is hybrid because it combines tracking and stereo matching, avoiding the need to calculate the real depth of the pixels. The method was tested with six different state-of-the-art trackers and assessed with videos of a sigmoidectomy of a sigma neoplasia, performed with a Da Vinci® X surgical system. Objective assessment metrics are proposed, presented and calculated for the different solutions. The results show that the method can successfully annotate RAS videos in real-time. Of all the trackers tested for the presented method, the CSRT (Channel and Spatial Reliability Tracking) tracker seems to be the most reliable and robust in terms of tracking capabilities. In addition, in the absence of an absolute ground truth, an assessment with a domain expert using a novel continuous-rating method with an Oculus Quest 2 Virtual Reality device was performed, showing that the depth perception of the virtual annotations is good, despite the fact that no absolute depth values are calculated.

The SeMap project, funded by the BBVA Foundation, has among its main objectives, to provide semantics to digitised heritage objects, and to make this information more accessible through web visualisation systems that use spatio-temporal maps. This project has had the collaboration and support of the Ministry of Culture and Sport of the Spanish Government, through CERES, the Digital Network of Museum Collections in Spain. This network manages data related to more than 400,000 objects, located in 113 museums of different specialities and from various thematic and geographical areas. These data are currently accessible through a web portal, but there is no public access to a knowledge graph with this information. Thanks to the collaboration of this entity, SeMap granted the access to this information. With the adequate process, this data has been represented by a knowledge graph. Using the resources offered by the Google Maps and Geonames APIs, it has been possible to georeferenced the data with an acceptable level of precision. This fact, has allowed the design and development of a spatio-temporal map visualisation system with a high level of interaction. This paper describes the process of creating a knowledge graph with this information, based on the CIDOC-CRM model, and using the CERES Cultural Heritage vocabularies and thesauri. The technologies and techniques used to visualise its contents are also described. Finally, the results of the evaluation of the project with end users are presented. In order to evaluate the project, an evaluation session based on the System Usability Scale (SUS) has been conducted. Forty-five secondary school students participated in this session, accessing the tool from the classroom and with the devices with which they usually work.

Abstract It is generally accepted that the use of natural interaction provides a positive impact in Virtual Reality (VR) applications. Therefore, it is important to understand what is the best way to integrate and visualize this feature in VR. For this reason, this paper presents a comparative study of the integration of natural hand interaction in two immersive VR systems: a Cave Audio Visual Experience (CAVE) system –where users’ real hands are visible– and a non-see-through Head-Mounted Display (HMD) system –where only a virtual representation of the hands is possible–. In order to test the suitability of using this type of interaction in a CAVE and compare it to an HMD, we raise six research questions related to task performance, usability and perception differences regarding natural hand interaction with these two systems. To answer these questions, we designed an experiment where users have to complete a pick-and-place task with virtual balls and a text-typing task with virtual keyboards. In both systems, the same tracking technology, based on a Leap Motion device, was used. To the best of our knowledge this is the first academic work addressing a comparison of this type. Objective and subjective data were collected during the experiments. The results show that the HMD has a performance, preference and usability advantage over the CAVE with respect to the integration of natural hand interaction. Nevertheless, the results also show that the CAVE system can be, as well, successfully used in combination with an optical hand tracking device.

Abstract 3D modelling of man-made objects is widely used in the cultural heritage sector, among others. It is relevant for its documentation, dissemination and preservation. Related to historical fabrics, weaves and weaving techniques are still mostly represented in forms of 2D graphics and textual descriptions. However, complex geometries are difficult to represent in such forms, hindering the way this legacy is transmitted to new generations. In this paper, we present the design and implementation of SILKNOW’s Virtual Loom, an interactive tool aimed to document, preserve and represent in interactive 3D forms historical weaves and weaving techniques of silk fabrics, dating from the 15th to the 19th centuries. To that end, our tool only requires an image of a historical fabric. Departing from this image, the tool automatically subtracts the design, and allows the user to apply different weaves and weaving techniques. In its current version, the tool embeds five traditional weaving techniques, 39 weaves and six types of yarns, which have been defined thanks to close collaboration of experts in computer graphics, art history and historical fabrics. Additionally, users can change the color of yarns and produce different 3D representations for a given fabric, which are interactive in real time. In this paper, we bring the details of the design and implementation of this tool, focusing on the input data, the strategy to process images, the 3D modelling of yarns, the definition of weaves and weaving techniques and the graphical user interface. In the results section, we show some examples of image analysis in order to subtract the design of historical fabrics, and then we provide 3D representations for all the considered weaving techniques, combining different types of yarns.

Motion-based simulators are used for a variety of applications, such as research, education, entertainment and training. In fact, motion cues are required to achieve the highest regulatory certifications in training vehicle simulators. Nonetheless, the reproduction of self-motion cues presents technological and economic limitations that are not present in the generation of audiovisual cues. For this reason, the generated motion does not generally match the expected one. Therefore, it is necessary to define means to assess the suitability/fidelity of the generated motion cues. After more than 50 years of motion-based vehicle simulation, no mechanism has been universally accepted as the standard solution for the evaluation of motion cues. This paper reviews the mechanisms for obtaining measures of motion fidelity, focusing on those based on objective methods. Since the design of the Objective Motion Cueing Test in 2006, researchers have shown a renewed interest in identifying objective methods to evaluate motion cueing, as the number of works following this approach in recent years reveals. Objective motion fidelity systems allow also performing automatic tuning of MCA by means of optimization techniques, which addresses one of the other main problems of these algorithms. Nevertheless, a universally accepted objective method to assess perceptual motion fidelity in vehicle simulators has not been proposed yet. For this reason, this review work frames and classifies the existing methods. In addition, the authors propose a series of features that an ideal evaluation method for assessing perceptual motion fidelity should include and provide future research guidelines for this complex topic.