Handheld controllers are standard in immersive virtual reality (iVR), but the rise of natural hand-based interactions exposes the limitations of hand gestures, especially for point-and-click tasks with graphical user interfaces (GUI). This shows the need to explore alternative hands-free selection methods. Unlike most studies focusing on the selection task itself, this work evaluates the impact of such methods on multiple dimensions when selections occur alongside another primary task. The tested methods were: head gaze + dwell, leaning, and voice; eye gaze + dwell, leaning, blinking, and voice; and voice-only. Controllers served as the baseline. Methods were further analyzed by pointing and confirming mechanisms. Four dimensions were analyzed: (1) iVR experience, (2) user satisfaction, (3) usability, and (4) efficiency and effectiveness. With 72 participants, results show hands-free methods provide comparable experiences to controllers, suggesting selection methods have a lower impact on the user experience when users focus on a primary task.

Abstract Gamification is increasingly used to make non-game contexts more engaging, yet integrating its elements into professional training applications often requires significant development effort. This work introduces GaToo (from “Gamification Tool”), a modular and reusable gamification plugin built for the Unity engine, designed to simplify this process and reduce the need for custom implementation. The plugin provides a structured set of components, including missions, steps, challenges and achievements, together with a base editor interface that facilitates configuration and management. To ensure broad applicability, it also includes complementary mechanisms that enable integration both in applications still under development and in existing applications, depending on the level of access to their source code. A preliminary study with 15 experienced Unity users was conducted to assess the usability of the plugin interface. The results indicated a score of 82 on the System Usability Scale, considered ’Excellent,’ supporting the ease of use of the plugin editor. By offering a flexible and extensible solution, the plugin allows developers and researchers to concentrate on the design and evaluation of gamification strategies, while minimizing the technical overhead of implementation.

This paper aims to evaluate how different combinations of multisensory stimuli affect the vividness of users’ mental imagery in the context of virtual tourism. To this end, a between-subjects experimental study was conducted with 94 participants, who were allocated to either a positive or a negative immersive virtual environment. The positive environment contained only pleasant multisensory stimuli, whereas the negative contained only unpleasant stimuli. For each of the virtual experiences, a multisensory treasure hunt was developed, where each object found corresponded to a planned combination of stimuli (positive or negative, accordingly). The results showed that positive stimuli involving a higher number of sensory modalities resulted in higher reported vividness. In contrast, when the same multisensory modalities were delivered with negative stimuli, vividness levels decreased - an effect we attribute to potential cognitive overload. Nevertheless, some reduced negative combinations (audiovisual with smell and audiovisual with haptics) remained effective, indicating that olfactory and haptic cues play an important role in shaping users’ vividness of mental imagery, even in negative contexts.

Abstract With the consistent adoption of iVR and growing research on the topic, it becomes fundamental to understand how the perception of Realism plays a role in the potential of iVR. This work puts forwards a hypothesis‐driven theoretical model of how the perception of each multisensory stimulus (Visual, Audio, Haptic and Scent) is related to the perception of Realism of the whole experience (Subjective Realism) and, in turn, how this Subjective Realism is related to Involvement and Presence. The model was validated using a sample of 216 subjects in a multisensory iVR experience. The results indicated a good model fit and provided evidence on how the perception of Realism of Visual, Audio and Scent individually is linked to Subjective Realism. Furthermore, the results demonstrate strong evidence that Subjective Realism is strongly associated with Involvement and Presence. These results put forwards a validated questionnaire for the perception of Realism of different aspects of the virtual experience and a robust theoretical model on the interconnections of these constructs. We provide empirical evidence that can be used to optimise iVR systems for Presence, Involvement and Subjective Realism, thereby enhancing the effectiveness of iVR experiences and opening new research avenues.

Abstract Virtual reality and emotions have become inseparable concepts over the past few years, supported by the increasing number of studies relating them. However, these studies' methodologies are often poorly justified or dependent on the authors' subjective definition of emotion and its classification. Moreover, frequently, these studies only consider two stimuli, specifically audiovisual, despite being known the relevance of including a greater variety of sensory channels to improve the relationship between the individual and the virtual environment. So, to address these gaps, and considering the importance of multisensory stimulation, this paper aims to review the methods and instruments found in the literature regarding the analysis of the users' emotions in virtual reality. Also, we provide an overview of the main limitations of such studies. Little information can be found in the literature regarding the connection between the input stimulus and the users’ emotional responses. This corroborates the difficulty in creating and evaluating immersive virtual experiences when stimulating more than two human senses, typically audiovisual. Nevertheless, we address some clues on the impact of visual, auditory, haptic, smell, and taste elements to trigger specific emotions. Also, we address the association between the research area and the method used. Finally, the main gaps and challenges are discussed. We expect that the combination of these results acts as guidelines for designing richer multisensory virtual experiences. Moreover, we intend to contribute to future research on emotions-based immersive virtual reality by providing a review of the most suitable methodologies and instruments for specific contexts.

How we perceive and experience the world around us is inherently multisensory. Most of the Virtual Reality (VR) literature is based on the senses of sight and hearing. However, there is a lot of potential for integrating additional stimuli into Virtual Environments (VEs), especially in a training context. Identifying the relevant stimuli for obtaining a virtual experience that is perceptually equivalent to a real experience will lead users to behave the same across environments, which adds substantial value for several training areas, such as firefighters. In this article, we present an experiment aiming to assess the impact of different sensory stimuli on stress, fatigue, cybersickness, Presence and knowledge transfer of users during a firefighter training VE. The results suggested that the stimulus that significantly impacted the user's response was wearing a firefighter's uniform and combining all sensory stimuli under study: heat, weight, uniform, and mask. The results also showed that the VE did not induce cybersickness and that it was successful in the task of transferring knowledge.

Immersive Virtual Reality (IVR) provides a platform where the real world can be replicated to a point where users can act and react in the virtual world as they would in reality. However, rendering visual stimuli is computationally heavy. Thus, optimizations must be done to take advantage of computational systems by studying our perception of reality. This study investigated parameters related to light rendering (Global Illumination, Ambient Occlusion, Screen Space Reflections (SSR) and Direct Shadows) in real-time in a virtual replica of a real place using IVR. Participants experienced both virtual and real rooms with only one flashlight and changed the quality settings of the considered parameters so that their sense of reality would be the closest to the one they felt when they experienced the real room. Participants were given a budget to drive them to prioritize what parameters, and their level of quality, are the most important for their sense of reality. Results indicated that participants considered Global Illumination the most important factor, closely followed by Direct Shadows. Ambient Occlusion and Reflections (Screen Space Reflections) were the less prioritized parameters. We conclude that in a lighting setting where only dynamic lights are used, Global Illumination and Direct Shadows should be prioritized over SSR Reflections and Ambient Occlusion when computational power is limited.

Despite advancements in interaction with immersive Virtual Reality (VR) systems, using hand gestures for all interactions still imposes some challenges, especially in interactions with graphical user interfaces that are usually performed with point-and-click interfaces. Therefore, exploring the use of alternative hands-free methods for selection is essential to overcome usability problems and provide natural interaction for users. The results and insights gained from this exploration can lead to enhanced user experiences in VR applications. This study aims to contribute to the literature with the evaluation of the usability of the most commonly used hands-free methods for selection and system control tasks in immersive VR and their impact on standard and validated experience and usability metrics, namely the sense of presence, cybersickness, system usability, workload, and user satisfaction. A Fitts’ selection task was performed using a within-subjects design by nine participants experienced in VR. The methods evaluated were the handheld controllers, the head gaze, eye gaze, and voice commands for pointing at the targets, and dwell time and voice commands to confirm the selections. Results show that the methods provide similar levels of sense of presence and low cybersickness while showing low workload values and high user satisfaction, matching the experience of traditional handheld controllers for non-multimodal approaches. The assisted eye gaze with dwell was the preferred hands-free method and the one with the highest values of usability. Still, developers should minimize the number of gaze movements to reduce fatigue. The evaluation also showed that using a multimodal approach for selections, especially using the voice, decreases user satisfaction and increases users’ frustration.

The use of Virtual Reality (VR) technology to train professionals has increased over the years due to its advantages over traditional training. This paper presents a study comparing the effectiveness of a Virtual Environment (VE) and a Real Environment (RE) designed to train firefighters. To measure the effectiveness of the environments, a new method based on participants' Heart Rate Variability (HRV) was used. This method was complemented with self-reports, in the form of questionnaires, of fatigue, stress, sense of presence, and cybersickness. An additional questionnaire was used to measure and compare knowledge transfer enabled by the environments. The results from HRV analysis indicated that participants were under physiological stress in both environments, albeit with less intensity on the VE. Regarding reported fatigue and stress, the results showed that none of the environments increased such variables. The results of knowledge transfer showed that the VE obtained a significant increase while the RE obtained a positive but non-significant increase (median values, VE: before - 4 after - 7, p = .003; RE: before - 4 after - 5, p = .375). Lastly, the results of presence and cybersickness suggested that participants experienced high overall presence and no cybersickness. Considering all results, the authors conclude that the VE provided effective training but that its effectiveness was lower than that of the RE.

The majority of virtual reality (VR) applications rely on audiovisual stimuli and do not exploit the addition of other sensory cues that could increase the potential of VR. This systematic review surveys the existing literature on multisensory VR and the impact of haptic, olfactory, and taste cues over audiovisual VR. The goal is to identify the extent to which multisensory stimuli affect the VR experience, which stimuli are used in multisensory VR, the type of VR setups used, and the application fields covered. An analysis of the 105 studies that met the eligibility criteria revealed that 84.8 percent of the studies show a positive impact of multisensory VR experiences. Haptics is the most commonly used stimulus in multisensory VR systems (86.6 percent). Non-immersive and immersive VR setups are preferred over semi-immersive setups. Regarding the application fields, a considerable part was adopted by health professionals and science and engineering professionals. We further conclude that smell and taste are still underexplored, and they can bring significant value to VR applications. More research is recommended on how to synthesize and deliver these stimuli, which still require complex and costly apparatus be integrated into the VR experience in a controlled and straightforward manner.

Virtual Reality applications have the goal of transporting their users to a given virtual environment (VE). Thus, Presence is a consensual metric for evaluating the VEs' effectiveness. The present study adapts the Presence Questionnaire (PQ) for the Portuguese-speaking population, maintaining the validity of the contents and concepts, to ascertain the psychometric properties of the instrument.The adaptation to Portuguese was achieved through the standard adaptation process of translation and back-translation process. The sample consisted of 451 individuals (268 males and 183 females). Factor reliability ranged from 0.63 to 0.86. Confirmatory factor analysis produced a theoretical model of 21 items distributed among seven factors, where the covariance between some residual item errors was established. The fit indices obtained were χ2/df=2.077, GFI =0.936, CFI =0.937, RMSEA =0.049, P [RMSEA ≤0.05], MECVI =1.070. Results obtained allowed us to consider that the adapted Portuguese version of the PQ, with 21 items, forms a robust and valid questionnaire whose use is recommended to evaluate Presence in virtual reality research programmes, provided that they use samples of the Portuguese language (Europe).

Abstract Authoring 360 multisensory videos is a true challenge as the authoring tools available are scarce and restrictive. In this paper, we propose an authoring tool with three different authoring interfaces (desktop, immersive, and tangible interface) for creating multisensory 360 videos with the advantage of having a live preview of the multisensory content that is being produced. An evaluation of the three authoring tools having into account gender, system usability, presence, satisfaction, and effectiveness (time to accomplish tasks, number of errors, and number of help requests) is presented. The sample consisted of 48 participants (24 males and 24 females) evenly distributed between the different interfaces (8 males and 8 females for each interface). The results revealed that gender does not have any impact in the studied interfaces regarding all the dependent variables; immersive and tangible interfaces have higher levels of satisfaction than desktop interface as it allows more interaction freedom, and desktop interface have the lowest time to accomplish the tasks because people are more familiar with keyboard and mouse.

The goal for a virtual reality (VR) training system is to enable trainees to acquire all the knowledge they need to perform effectively in a real environment. Such a system should provide an experience so authentic that no further real-world training is necessary, meaning that it is sufficient to train in VR. We evaluate the impact of a haptic thermal stimulus, which is of paramount importance to decision making, on trainees performance and knowledge acquisition. A thermal device was created to deliver the stimulus. As a proof of concept, a procedure from firefighter training is selected, in which sensing the temperature of a door with one's hand is essential. The sample consisted of 48 subjects divided among three experimental scenarios: one in which a virtual thermometer is used (visual stimulus), another in which the temperature is felt with the hand (thermal stimulus) and a third in which both methods are used (visual + thermal stimuli). For the performance evaluation, we measured the total time taken, the numbers of correctly executed procedures and identified neutral planes, the deviation from the target height, and the responses to a knowledge transfer questionnaire. Presence, cybersickness, and usability are measured to evaluate the impact of the haptic thermal stimulus. Considering the thermal stimulus condition as the baseline, we conclude that the significantly different results in the performance among the conditions indicate that the better performance in the visual-only condition is not representative of the real-life performance. Consequently, VR training applications need to deliver the correct stimuli for decision making.

Hands are the most important tool to interact with virtual environments, and they should be available to perform the most critical tasks. For example, a surgeon in VR should keep his/her hands on the instruments and be able to do secondary tasks without performing a disruptive event to the operative task. In this common scenario, one can observe that hands are not available for interaction. The goal of this systematic review is to survey the literature and identify which hands-free interfaces are used, the performed interaction tasks, what metrics are used for interface evaluation, and the results of such evaluations. From 79 studies that met the eligibility criteria, the voice is the most studied interface, followed by the eye and head gaze. Some novel interfaces were brain interfaces and face expressions. System control and selection represent most of the interaction tasks studied and most studies evaluate interfaces for usability. Despite the best interface depending on the task and study, the voice was found to be versatile and showed good results amongst the studies. More research is recommended to improve the practical use of the interfaces and to evaluate the interfaces more formally.

Training firefighters using Virtual Reality (VR) technology brings several benefits over traditional training methods including the reduction of costs and risks. The ability of causing the same level of stress as a real situation so that firefighters can learn how to deal with stress was investigated. An experiment aiming to study the influence that additional stimuli (heat, weight, smell and using personal protective equipment-PPE) have on user's stress level while performing a Virtual Environment (VE) designed to train firefighters was developed. Participants' stress and Heart Rate Variability (HRV) were obtained from electrocardiograms recorded during the experiment. The results suggest that wearing the PPE has the largest impact on user's stress level. The results also showed that HRV was able to evidence differences between two phases of the experiment, which suggests that it can be used to monitor users' quantified reaction to VEs.

Consuming 360 audiovisual content using a Head-Mounted Display (HMD) has become a standard feature for Immersive Virtual Reality (IVR). However, most applications rely only on visual and auditory feedback whereas other senses are often disregarded. The main goal of this work was to study the effect of tactile and olfactory stimuli on participants’ sense of presence and cybersickness while watching a 360 video using an HMD-based IVR setup. An experiment with 48 participants and three experimental conditions (360 video, 360 video with olfactory stimulus, and 360 video with tactile stimulus) was performed. Presence and cybersickness were reported via post-test questionnaires. Statistical analysis showed a significant difference in presence between the control and the olfactory conditions. From the control to the tactile condition, mean values were higher but failed to show statistical significance. Thus, results suggest that adding an olfactory stimulus increases presence significantly while the addition of a tactile stimulus only shows a positive effect. Regarding cybersickness, no significant differences were found across conditions. We conclude that an olfactory stimulus contributes to higher presence and that a tactile stimulus, delivered in the form of cutaneous perception of wind, has no influence in presence. We further conclude that multisensory cues do not affect cybersickness.

Multiple factors can affect presence in virtual environments, such as the number of human senses engaged in a given experience or the extent to which the virtual experience is credible. The purpose of the present work is to study how the inclusion of credible multisensory stimuli affects the sense of presence, namely, through the use of wind, passive haptics, vibration, and scent. Our sample consisted of 37 participants (27 men and 10 women) whose ages ranged from 17 to 44 years old and were mostly students. The participants were divided randomly into 3 groups: Control Scenario (visual and auditory - N = 12), Passive Haptic Scenario (visual, auditory, and passive haptic - N = 13) and Multisensory Scenario (visual, auditory, wind, passive haptic, vibration, and scent - N = 12). The results indicated a significant increase in the involvement subscale when all multisensory stimuli were delivered. We found a trend where the use of passive haptics by itself has a positive impact on presence, which should be the subject of further work.

The present study aims to translate and validate the Temple Presence Inventory (TPI) for the Portuguese context, respecting the maintenance of an equivalent semantics as well as the validity of its contents and concepts. This study also aims to verify the psychometric properties of the instrument (factor validity and internal consistency). The sample consisted of 455 individuals (male = 271, female = 184). The fidelity of the factors varied between 0.5 and 0.84. The confirmatory factor analysis produced a theoretical model with 38 items distributed among eight factors. The covariance between some residual errors of instrument items was considered, and the following fit indices were observed: χ2/df = 2.073; GFI = 0.858; CFI = 0.887; RSMEA = 0.049; AIC = 1527. The results confirm the appropriateness of the version adapted to the Portuguese language of the TPI and that it can be used in research projects aiming to evaluate Presence in the Portuguese-speaking population (Europe).

Recently, there has been a significant progress in the research and development of the high dynamic range (HDR) video technology and the state-of-the-art video pipelines are able to offer a higher bit depth support to capture, store, encode, and display HDR video content. In this paper, we introduce a novel HDR video compression algorithm, which uses a perceptually uniform color opponent space, a novel perceptual transfer function to encode the dynamic range of the scene, and a novel error minimization scheme for accurate chroma reproduction. The proposed algorithm was objectively and subjectively evaluated against four state-of-the-art algorithms. The objective evaluation was conducted across a set of 39 HDR video sequences, using the latest x265 10-bit video codec along with several perceptual and structural quality assessment metrics at 11 different quality levels. Furthermore, a rating-based subjective evaluation (n=40) was conducted with six sequences at two different output bitrates. Results suggest that the proposed algorithm exhibits the lowest coding error amongst the five algorithms evaluated. Additionally, the rate-distortion characteristics suggest that the proposed algorithm outperforms the existing state-of-the-art at bitrates ≥ 0.4 bits/pixel.

Visual coherence between virtual and real objects is a major issue in creating convincing augmented reality (AR) applications. To achieve this seamless integration, actual light conditions must be determined in real time to ensure that virtual objects are correctly illuminated and cast consistent shadows. In this paper, we propose a novel method to estimate daylight illumination and use this information in outdoor AR applications to render virtual objects with coherent shadows. The illumination parameters are acquired in real time from context-aware live sensor data. The method works under unprepared natural conditions. We also present a novel and rapid implementation of a state-of-the-art skylight model, from which the illumination parameters are derived. The Sun's position is calculated based on the user location and time of day, with the relative rotational differences estimated from a gyroscope, compass and accelerometer. The results illustrated that our method can generate visually credible AR scenes with consistent shadows rendered from recovered illumination.

Abstract Since high dynamic range (HDR) displays are not yet widely available, there is still a need to perform a dynamic range reduction of HDR content to reproduce it properly on standard dynamic range (SDR) displays. The most common techniques for performing this reduction are termed tone‐mapping operators (TMOs). Although mobile devices are becoming widespread, methods for displaying HDR content on these SDR screens are still very much in their infancy. While several studies have been conducted to evaluate TMOs, few have been done with a goal of testing small screen displays (SSDs), common on mobile devices. This paper presents an evaluation of six state‐of‐the‐art HDR video TMOs. The experiments considered three different levels of ambient luminance under which 180 participants were asked to rank the TMOs for seven tone‐mapped HDR video sequences. A comparison was conducted between tone‐mapped HDR video footage shown on an SSD and on a large screen SDR display using an HDR display as reference. The results show that there are differences between the performance of the TMOs under different ambient lighting levels and the TMOs that perform well on traditional large screen displays also perform well on SSDs at the same given luminance level.

Abstract In the last decade, an increasing number of techniques have been developed to reproduce high dynamic range imagery on traditional displays. These techniques, known as Tone Mapping Operators (TMOs), have been compared and ranked in different ways according to several image characteristics. However, none of these algorithms has been developed specifically for small screen devices (SSD). In this paper, we present an evaluation of currently used TMOs to show that SSDs with limited size, resolution and colour depth require specific research to find or create an appropriate solution. The research described in this paper is based on psychophysical experiments; using three different types of displays (CRT, LCD and SSD). The obtained results show that rankings obtained are similar for the LCD and CRT but are significantly different for the SSD. Furthermore, these rankings show additionally that some characteristics of TMOs need to be emphasized to obtain better high‐fidelity mapped images for SSDs.

Abstract L‐systems have been used in Computer Graphics, namely for modelling plants, as well as in a few experiments to model urban environments. However, the lack of geospatial awareness is a limitation and in spite of some developments like open L‐systems, that introduced the ability to communicate with the environment, there was a need for more flexibility. This paper presents Geospatial L‐systems, a new extension of L‐systems that incorporates geospatial awareness, and shows an application in the area of expeditious modelling of urban environments. A modelling system, named XL3D, generates virtual urban environments automatically from a XML based document that contains a modelling specification and accesses data sources in an interoperable way. The integration of geospatial L‐systems in this modelling system has increased the potential for automation and the potential to generate virtual urban environments with a higher level of detail and visual fidelity, with a lower level of complexity of the modelling processes. These facts are shown in a case study where a virtual urban environment, taken from an area in the Porto downtown, is generated by this solution.

A map can be a major advantage when exploring unfamiliar environments. With the proliferation of mobile devices, such as PDAs and even mobile phones, the tourist industry is currently exploring the potential of new presentation strategies that will maximize the promotional appeal of tourism in their region. Mobile devices are capable of guiding a tourist when he/she is exploring a city. These mobile devices offer the potential for providing relevant 3D information to enable tourists to locate themselves within the city, rapidly navigate around the unfamiliar environment and explore it interactively. However, the computational resources of current mobile technology prevents the display of full complex 3D content in real-time, and thus selective rendering techniques must be adopted to ensure the viewer is provided with the perceptually most important information at interactive rates. This paper presents a series of experiments which help to identify key features of a scene for users to orientate themselves in that environment. Knowledge of these salient key features enable them to be provided to a user at a high quality while the remainder of the scene can be rendered in a much lower quality, saving significant bandwidth and computing power, without the user being perceptually aware of this difference in quality within the image.

The perception that we have of our world depends on the task we are currently performing in the environment, so if we are driving a car we will pay attention to the objects that are visually important to the task we are performing such as, the road, road signs, other vehicles, etc. The same is true when we explore virtual environments. The creation of high-fidelity 3D maps on mobile devices to aid navigation in urban environments is computationally very expensive, precluding achieving this quality at interactive rates. In this paper we present a case study to show how the human visual system may be exploited, when viewers are undertaking a task, to reduce the overall quality of the displayed image, without the users being aware of this reduction in quality. The displayed images are selectively rendered with the key features used to identify location and orientation in a 3D urban environment produced in high quality and the remainder of the image in low quality.

Finding one's way around an unfamiliar city can be a major challenge. While maps can provide a very good abstract representation of our world, and a simple and efficient way to navigate within that world, they are of little use when, for example, the absence of road signs prevents us from locating where we are on the map. Mobile devices offer the potential for providing relevant 3D information to enable us to locate ourselves, rapidly navigate around an unfamiliar environment and explore it interactively. However, mobile devices are constrained by resources such as bandwidth, storage and small displays. In this paper we investigate which is the most important visual information for position location within an unfamiliar urban environment and show how we can use this knowledge to provide a perceptually high quality 3D virtual environment on existing mobile devices.