Integrating artificial intelligence (AI) with immersive analytics (IA) represents a promising means of leveraging advanced computational techniques to enhance data visualization and analysis. This study examines the state-of-the-art of AI-IA integration by addressing three key research issues: the significant application domains, the AI techniques used and their combinations, and current challenges and future directions. Results of reviewing 43 relevant studies reveal that AI-IA integration is still in its early stages, as existing research has mainly focused on a limited range of data types and application scenarios. By analyzing the application domains, this systematic literature review supports previous findings of important applications in the fields of education, manufacturing, and healthcare. At the same time, it identifies emerging applications that have progressed from XR and AI domains to AI-IA integration, such as sports events, assistive systems, urban planning, and disaster management. We contribute to extending established visual analytics (VA) pipelines into XR environments with integrated AI techniques. AI techniques are identified as contributing in five ways to this IA pipeline. Our contribution also includes identifying four key challenges and seven opportunities for future exploration. The review concludes that combining AI and IA holds the potential to create innovative applications using advanced AI and immersive visualization techniques. We present an overview of these applications and address key issues for future development.

Abstract In this paper, we investigate the current state and development of personalized smart immersive extended reality environments (PSI-XR). PSI-XR has gained increasing traction across various fields such as education, entertainment, and healthcare, offering customized immersive experiences that address users’ personalized needs. This study performs a systematic literature review by collecting and analyzing related journal and conference papers in the domain. Following a comprehensive search across three databases, which yielded 1276 papers, a refined selection of 94 publications was made to conduct an in-depth analysis of cutting-edge research in the field of PSI-XR. This review focused on examining application domains, relevant technologies, and smart techniques, including artificial intelligence, with particular emphasis on advancements in personalization. The study provides insights into prospective advancements while also identifying the opportunities and challenges in this evolving field. This review is beneficial for both researchers and developers interested in exploring the state-of-the-art personalized perspective in a smart immersive extended reality environment.

Virtual reality, augmented reality, and mixed reality (VR/AR/MR) as information and communication technologies have been recognised and implemented in healthcare in recent years. One of the popular application ways is games, due to the potential benefits of providing an engaging and immersive experience in a virtual environment. This study presents a systematic literature review that evaluates the state-of-the-art on VR/AR/MR game applications in healthcare by collecting and analysing related journal and conference papers published from 2014 through to the first half of 2020. After retrieving more than 3,000 papers from six databases, 88 articles, from both computer science and medicine, were selected and analysed in the review. The articles are classified and summarised based on their (1) publication information, (2) design, implementation, and evaluation, and (3) application. The presented review is beneficial for both researchers and developers interested in exploring current research and future trends in VR/AR/MR in healthcare.

The area of biofeedback interaction has grown over recent years, thanks to the release of more affordable and reliable sensor technology, and the accessibility offered by modern game development tools. This article presents a systematic literature review focusing on how different biofeedback interaction methods have been used for entertainment purposes in video games, between 2008 and 2020. It divides previous contributions in terms of a proposed interaction classification criteria and five different biofeedback methods (with a sixth category combining them): electroencephalography, electrocardiography, eye tracking, electrodermal activity, electromyography, and multi-modal interaction. The review describes the properties, sensor technologies, and the type of data gathered for every included biofeedback method, and presents their respective interaction techniques. It summarizes a set of opportunities and challenges for each included method, based on the results from previous work, and discusses these findings. It also analyzes how these interaction techniques are distributed between different common game genres. The review is beneficial for people interested in biofeedback methods and their potential use for novel interaction techniques in future video games.

Aiming is key for virtual reality (VR) interaction, and it is often done using VR controllers. Recent eye-tracking integrations in commercial VR head-mounted displays (HMDs) call for further research on usability and performance aspects to better determine possibilities and limitations. This paper presents a user study exploring gaze aiming in VR compared to a traditional controller in an "aim and shoot" task. Different speeds of targets and trajectories were studied. Qualitative data was gathered using the system usability scale (SUS) and cognitive load (NASA TLX) questionnaires. Results show a lower perceived cognitive load using gaze aiming and on par usability scale. Gaze aiming produced on par task duration but lower accuracy on most conditions. Lastly, the trajectory of the target significantly affected the orientation of the HMD in relation to the target's location. The results show potential using gaze aiming in VR and motivate further research.

Abstract This paper presents the latest advances and research in Computer Graphics education in a nutshell. It is concerned with topics that were presented at the Education Track of the Eurographics Conference held in Lisbon in 2016. We describe works corresponding to approaches to Computer Graphics education that are unconventional in some way and attempt to tackle unsolved problems and challenges regarding the role of arts in computer graphics education, the role of research‐oriented activities in undergraduate education and the interaction among different areas of Computer Graphics, as well as their application to courses or extra‐curricular activities. We present related works addressing these topics and report experiences, successes and issues in implementing the approaches.

Social interaction between players is an important feature in online games, where text- and voice chat is a standard way to communicate. To express emotions players can type emotes which are text-based commands to play animations from the player avatar. This paper presents a perceptual evaluation which investigates if instead expressing emotions with the face, in real-time with a web camera, is perceived more realistic and preferred in comparison to typing emote-based text commands. A user study with 24 participants was conducted where the two methods to express emotions described above were evaluated. For both methods the participants ranked the realism of facial expressions, which were based on the theory of seven universal emotions stated by American psychologist Paul Ekman: happiness, anger, fear, sadness, disgust, surprise and contempt. The participants also ranked their perceived efficiency of performing the two methods and selected the method they preferred. A significant difference was shown when analyzing the results of ranked facial expression realism. Happiness was perceived as the most realistic in both methods, while disgust and sadness were poorly rated when performed with the face. One conclusion of the perceptual evaluation was that the realism and preference between the methods showed no significant differences. However, participants had higher performance in typing with emotes. Real-time facial capture technology also needs improvements to obtain better recognition and tracking of facial features in the human face.

Selecting color schemes for game objects is an important task. It can be valuable to game designers to know what colors are preferred. Principles of color theory are important to select appropriate colors. This paper presents a perceptual experiment that evaluates some basic principles of color theory applied to game objects to study if a particular combination is preferred. An experiment was conducted with 15 participants who performed a two-alternative forced choice (2AFC) preference experiment using 236 pairs of images each. The pairs were based on color harmonies derived from the colors red, green, and blue. The color harmonies were evaluated against each other and included analogous, complementary, split-complementary, triad, and warm and cool colors. A high and low saturation condition was also included. The color harmonies were applied to an existing game character (avatar) and a new object (cube) to study any potential differences in the results. The initial results show that some color harmonies, in particular triad and split-complementary, were generally preferred over others meaning that it is important to take into account these aspects in game design. Additional results also show that color harmonies with a base in green were not as popular as red and blue color harmonies.

Abstract Rendering realistic illumination effects for complex animated scenes with many dynamic objects or characters is computationally expensive. Yet, it is not obvious how important such accurate lighting is for the overall perceived realism in these scenes. In this paper, we present a methodology to evaluate the perceived fidelity of illumination in scenes with dynamic aggregates, such as crowds, and explore several factors which may affect this perception. We focus in particular on evaluating how a popular spherical harmonics lighting method can be used to approximate realistic lighting of crowds. We conduct a series of psychophysical experiments to explore how a simple approach to approximating global illumination, using interpolation in the temporal domain, affects the perceived fidelity of dynamic scenes with high geometric, motion, and illumination complexity. We show that the complexity of the geometry and temporal properties of the crowd entities, the motion of the aggregate as a whole, the type of interpolation (i.e., of the direct and/or indirect illumination coefficients), and the presence or absence of colour all affect perceived fidelity. We show that high (i.e., above 75%) levels of perceived scene fidelity can be maintained while interpolating indirect illumination for intervals of up to 30 frames, resulting in a greater than three‐fold rendering speed‐up.

No abstract available.

We propose a novel skin shader which translates the simulation of subsurface scattering from texture space to a screen-space diffusion approximation. It naturally scales well while maintaining a perceptually plausible result. This technique allows us to ensure real-time performance even when several characters may appear on screen at the same time. The visual realism of the resulting images is validated using a subjective psychophysical preference experiment. Our results show that, independent of distance and light position, the images rendered using our novel shader have as high visual realism as a previously developed physically-based shader.

High-fidelity image synthesis is the process of computing images that are perceptually indistinguishable from the real world they are attempting to portray. Such a level of fidelity requires that the physical processes of materials and the behavior of light are accurately simulated. Most computer graphics algorithms assume that light passes freely between surfaces within an environment. However, in many applications, we also need to take into account how the light interacts with media, such as dust, smoke, fog, etc., between the surfaces. The computational requirements for calculating the interaction of light with such participating media are substantial. This process can take many hours and rendering effort is often spent on computing parts of the scene that may not be perceived by the viewer. In this paper, we present a novel perceptual strategy for physically based rendering of participating media. By using a combination of a saliency map with our new extinction map (X map), we can significantly reduce rendering times for inhomogeneous media. The visual quality of the resulting images is validated using two objective difference metrics and a subjective psychophysical experiment. Although the average pixel errors of these metric are all less than 1%, the subjective validation indicates that the degradation in quality still is noticeable for certain scenes. We thus introduce and validate a novel light map (L map) that accounts for salient features caused by multiple light scattering around light sources.

Accurate light simulation is of highest importance in archaeological reconstructions when you want to investigate how a site might have looked like in the past. Failure to use the highest fidelity means there is a very real danger of misrepresenting the past. This sketch describes the high fidelity reconstruction and realistic lighting simulation of an ancient Egyptian temple.