Medical imaging plays a crucial role in diagnosis and surgical planning, but accurately merging segmented 3D models with volumetric medical data remains a significant challenge. Traditional surface-based 3D models provide a clear structural representation but may introduce segmentation inaccuracies, whereas Direct Volume Rendering (DVR) maintains all image details but lacks explicit surface definition. We present a novel hybrid rendering algorithm that seamlessly integrates Per-Pixel Linked Lists for accurate transparency sorting of 3D meshes with DVR techniques to ensure a smooth and precise blending of both data types. This approach captures the advantages of both methods—preserving the structural clarity of surface models while maintaining the depth and density information from the medical image. Our algorithm operates in real time and is compatible with virtual reality, enhancing spatial perception through stereoscopic visualisation. By allowing direct interaction with both segmented models and volumetric data, this technique reduces reliance on radiologists’ reports and improves surgical planning. Performance evaluations confirm real-time rendering, and an expert-based usability study further supports the method’s relevance for advanced medical visualisation applications.

The "Physically-Based Synthetic Data for Earth Observation" (SD4EO) project1, initiated in October 2023, aims to integrate physically-based simulation data and artificial intelligence-based data generation tools into Earth Observation applications. Leveraging advancements in physically-based and AI-driven simulation, the project investigates the potential of combining real and simulated data to improve target categorization in AI-driven Earth Observation analytic pipelines. For this purpose, three diverse use cases have been selected: crop field categorization, human settlement categorization, and photovoltaic panel status monitoring. SD4EO project seeks also to expedite innovation in this field and foster collaboration between the Artificial Intelligence, Computer Graphics and Earth Observation scientific communities.

Abstract Augmented Reality (AR) involves the combination of synthetic and real stimuli, not being restricted to visual cues. For the inclusion of computer-generated sound in AR environments, it is often assumed that the distance attenuation model is the most intuitive and useful system for all users, regardless of the characteristics of the environment. This model reduces the gain of the sound sources as a function of the distance between the source and the listener. In this paper, we propose a different attenuation model not only based on distance, but also considering the listener orientation, so the user could listen more clearly the objects that they are looking at, instead of other near objects that could be out of their field of view and interest. We call this a directional attenuation model. To test the model, we developed an AR application that involves visual and sound stimuli to compare the traditional model versus the new one, by considering two different tasks in two AR scenarios in which sound plays an important role. A total of 38 persons participated in the experiments. The results show that the proposed model provides better workload for the two tasks, requiring less time and effort, allowing users to explore the AR environment more easily and intuitively. This demonstrates that this alternative model has the potential to be more efficient for certain applications.

Abstract This paper presents "The Mediterranean Forest," an interactive exhibit designed for the Science Museum of Valencia. The exhibit engages children aged 3–8 by bringing their animal drawings to life in a virtual world. The system allows children to select, paint, scan, and transform drawings into three-dimensional avatars. These virtual animals interact with each other in the environment and with the children in the room. The exhibit is a collaborative space, where users can manipulate the real world, that promotes learning by exploration. The proposed solution integrates various cutting-edge technologies, including a large Virtual Reality projection system with blending techniques, Kinect devices for the interactivity, cheoptics hologram technology for realistic 3D visuals, and AI-driven virtual avatars simulating different behaviors and reacting to the user’s approach. The exhibit was evaluated through a study involving children, with feedback collected using a tailored questionnaire and the results indicated high levels of engagement and enjoyment among the children, with the majority expressing positive experiences. Some questions were not understandable for some children, which should be considered for future formal evaluation. The system's intuitiveness allowed children to navigate and interact with the virtual environment. The exhibit also demonstrates scalability, accommodating large groups of children effectively. Overall, "The Mediterranean Forest" successfully captivates children's attention, fostering creativity and imagination by transforming their drawings into a dynamic virtual world. This study underscores the benefits of using interactive technologies in museum settings to enhance children's learning experiences, encouraging further advancements in this field.

The visualization of fabrics at microscopic level involves a significant computational cost due to the large number of 3d elements required to display each yarn. Often, these visualizations rely on offline systems that display the final result only after an extended rendering time, which does not allow real-time interaction or visualization. While other models can allow for real-time rendering, they may require high memory usage and preprocessing times. Moreover, threads with metallic wraps used in historical fabrics cannot be represented using existing yarn models. In this paper, we present a yarn-level weaving model based on Sphere Tracing, which enables real-time rendering of historical Jacquard weaves with diverse weft types. The primary goal of this visualization is to analyze and reconstruct weaving structures constructed using these looms to safeguard this valuable cultural heritage. Furthermore, we propose a new level of detail technique, which extends the fundamental Sphere Tracing algorithm. This approach enables the visualization of a complete fabric at varying scales while requiring low memory usage and short preprocessing times.

Robotic-Assisted Surgery (RAS) is beginning to unlock its potential. However, despite the latest advances in RAS, the steep learning curve of RAS devices remains a problem. A common teaching resource in surgery is the use of videos of previous procedures, which in RAS are almost always stereoscopic. It is important to be able to add virtual annotations onto these videos so that certain elements of the surgical process are tracked and highlighted during the teaching session. Including virtual annotations in stereoscopic videos turns them into Mixed Reality (MR) experiences, in which tissues, tools and procedures are better observed. However, an MR-based annotation of objects requires tracking and some kind of depth estimation. For this reason, this paper proposes a real-time hybrid tracking–matching method for performing virtual annotations on RAS videos. The proposed method is hybrid because it combines tracking and stereo matching, avoiding the need to calculate the real depth of the pixels. The method was tested with six different state-of-the-art trackers and assessed with videos of a sigmoidectomy of a sigma neoplasia, performed with a Da Vinci® X surgical system. Objective assessment metrics are proposed, presented and calculated for the different solutions. The results show that the method can successfully annotate RAS videos in real-time. Of all the trackers tested for the presented method, the CSRT (Channel and Spatial Reliability Tracking) tracker seems to be the most reliable and robust in terms of tracking capabilities. In addition, in the absence of an absolute ground truth, an assessment with a domain expert using a novel continuous-rating method with an Oculus Quest 2 Virtual Reality device was performed, showing that the depth perception of the virtual annotations is good, despite the fact that no absolute depth values are calculated.

Abstract It is generally accepted that the use of natural interaction provides a positive impact in Virtual Reality (VR) applications. Therefore, it is important to understand what is the best way to integrate and visualize this feature in VR. For this reason, this paper presents a comparative study of the integration of natural hand interaction in two immersive VR systems: a Cave Audio Visual Experience (CAVE) system –where users’ real hands are visible– and a non-see-through Head-Mounted Display (HMD) system –where only a virtual representation of the hands is possible–. In order to test the suitability of using this type of interaction in a CAVE and compare it to an HMD, we raise six research questions related to task performance, usability and perception differences regarding natural hand interaction with these two systems. To answer these questions, we designed an experiment where users have to complete a pick-and-place task with virtual balls and a text-typing task with virtual keyboards. In both systems, the same tracking technology, based on a Leap Motion device, was used. To the best of our knowledge this is the first academic work addressing a comparison of this type. Objective and subjective data were collected during the experiments. The results show that the HMD has a performance, preference and usability advantage over the CAVE with respect to the integration of natural hand interaction. Nevertheless, the results also show that the CAVE system can be, as well, successfully used in combination with an optical hand tracking device.

Abstract 3D modelling of man-made objects is widely used in the cultural heritage sector, among others. It is relevant for its documentation, dissemination and preservation. Related to historical fabrics, weaves and weaving techniques are still mostly represented in forms of 2D graphics and textual descriptions. However, complex geometries are difficult to represent in such forms, hindering the way this legacy is transmitted to new generations. In this paper, we present the design and implementation of SILKNOW’s Virtual Loom, an interactive tool aimed to document, preserve and represent in interactive 3D forms historical weaves and weaving techniques of silk fabrics, dating from the 15th to the 19th centuries. To that end, our tool only requires an image of a historical fabric. Departing from this image, the tool automatically subtracts the design, and allows the user to apply different weaves and weaving techniques. In its current version, the tool embeds five traditional weaving techniques, 39 weaves and six types of yarns, which have been defined thanks to close collaboration of experts in computer graphics, art history and historical fabrics. Additionally, users can change the color of yarns and produce different 3D representations for a given fabric, which are interactive in real time. In this paper, we bring the details of the design and implementation of this tool, focusing on the input data, the strategy to process images, the 3D modelling of yarns, the definition of weaves and weaving techniques and the graphical user interface. In the results section, we show some examples of image analysis in order to subtract the design of historical fabrics, and then we provide 3D representations for all the considered weaving techniques, combining different types of yarns.

This paper describes Intelligent Eye, a mobile phone interactive leisure guide that offers location-based multimedia information. The information offered is related to the user's position, so the main goal of this work is the development of an efficient system to detect where the user is pointing his/her camera at by means of a content-based image retrieval algorithm (CBIR). The CBIR procedure uses color histograms in the HS color space extracted from images, and employs Kullback-Leibler divergence as the similarity measure. Intelligent Eye can be used in a wide range of camera-equipped mobile phones; however, efficiency is improved if GPS data is available. In order to outperform other systems we have made use of a video stream in the classification process instead of using still images. Moreover, the image database can be populated dynamically by means of a feedback procedure with images taken by users. We report preliminary results of the prototype working with real images obtaining a hit classification rate of 96%.