Handheld controllers are standard in immersive virtual reality (iVR), but the rise of natural hand-based interactions exposes the limitations of hand gestures, especially for point-and-click tasks with graphical user interfaces (GUI). This shows the need to explore alternative hands-free selection methods. Unlike most studies focusing on the selection task itself, this work evaluates the impact of such methods on multiple dimensions when selections occur alongside another primary task. The tested methods were: head gaze + dwell, leaning, and voice; eye gaze + dwell, leaning, blinking, and voice; and voice-only. Controllers served as the baseline. Methods were further analyzed by pointing and confirming mechanisms. Four dimensions were analyzed: (1) iVR experience, (2) user satisfaction, (3) usability, and (4) efficiency and effectiveness. With 72 participants, results show hands-free methods provide comparable experiences to controllers, suggesting selection methods have a lower impact on the user experience when users focus on a primary task.

Immersive Virtual Reality (IVR) provides a platform where the real world can be replicated to a point where users can act and react in the virtual world as they would in reality. However, rendering visual stimuli is computationally heavy. Thus, optimizations must be done to take advantage of computational systems by studying our perception of reality. This study investigated parameters related to light rendering (Global Illumination, Ambient Occlusion, Screen Space Reflections (SSR) and Direct Shadows) in real-time in a virtual replica of a real place using IVR. Participants experienced both virtual and real rooms with only one flashlight and changed the quality settings of the considered parameters so that their sense of reality would be the closest to the one they felt when they experienced the real room. Participants were given a budget to drive them to prioritize what parameters, and their level of quality, are the most important for their sense of reality. Results indicated that participants considered Global Illumination the most important factor, closely followed by Direct Shadows. Ambient Occlusion and Reflections (Screen Space Reflections) were the less prioritized parameters. We conclude that in a lighting setting where only dynamic lights are used, Global Illumination and Direct Shadows should be prioritized over SSR Reflections and Ambient Occlusion when computational power is limited.

Despite advancements in interaction with immersive Virtual Reality (VR) systems, using hand gestures for all interactions still imposes some challenges, especially in interactions with graphical user interfaces that are usually performed with point-and-click interfaces. Therefore, exploring the use of alternative hands-free methods for selection is essential to overcome usability problems and provide natural interaction for users. The results and insights gained from this exploration can lead to enhanced user experiences in VR applications. This study aims to contribute to the literature with the evaluation of the usability of the most commonly used hands-free methods for selection and system control tasks in immersive VR and their impact on standard and validated experience and usability metrics, namely the sense of presence, cybersickness, system usability, workload, and user satisfaction. A Fitts’ selection task was performed using a within-subjects design by nine participants experienced in VR. The methods evaluated were the handheld controllers, the head gaze, eye gaze, and voice commands for pointing at the targets, and dwell time and voice commands to confirm the selections. Results show that the methods provide similar levels of sense of presence and low cybersickness while showing low workload values and high user satisfaction, matching the experience of traditional handheld controllers for non-multimodal approaches. The assisted eye gaze with dwell was the preferred hands-free method and the one with the highest values of usability. Still, developers should minimize the number of gaze movements to reduce fatigue. The evaluation also showed that using a multimodal approach for selections, especially using the voice, decreases user satisfaction and increases users’ frustration.

The majority of virtual reality (VR) applications rely on audiovisual stimuli and do not exploit the addition of other sensory cues that could increase the potential of VR. This systematic review surveys the existing literature on multisensory VR and the impact of haptic, olfactory, and taste cues over audiovisual VR. The goal is to identify the extent to which multisensory stimuli affect the VR experience, which stimuli are used in multisensory VR, the type of VR setups used, and the application fields covered. An analysis of the 105 studies that met the eligibility criteria revealed that 84.8 percent of the studies show a positive impact of multisensory VR experiences. Haptics is the most commonly used stimulus in multisensory VR systems (86.6 percent). Non-immersive and immersive VR setups are preferred over semi-immersive setups. Regarding the application fields, a considerable part was adopted by health professionals and science and engineering professionals. We further conclude that smell and taste are still underexplored, and they can bring significant value to VR applications. More research is recommended on how to synthesize and deliver these stimuli, which still require complex and costly apparatus be integrated into the VR experience in a controlled and straightforward manner.

The goal for a virtual reality (VR) training system is to enable trainees to acquire all the knowledge they need to perform effectively in a real environment. Such a system should provide an experience so authentic that no further real-world training is necessary, meaning that it is sufficient to train in VR. We evaluate the impact of a haptic thermal stimulus, which is of paramount importance to decision making, on trainees performance and knowledge acquisition. A thermal device was created to deliver the stimulus. As a proof of concept, a procedure from firefighter training is selected, in which sensing the temperature of a door with one's hand is essential. The sample consisted of 48 subjects divided among three experimental scenarios: one in which a virtual thermometer is used (visual stimulus), another in which the temperature is felt with the hand (thermal stimulus) and a third in which both methods are used (visual + thermal stimuli). For the performance evaluation, we measured the total time taken, the numbers of correctly executed procedures and identified neutral planes, the deviation from the target height, and the responses to a knowledge transfer questionnaire. Presence, cybersickness, and usability are measured to evaluate the impact of the haptic thermal stimulus. Considering the thermal stimulus condition as the baseline, we conclude that the significantly different results in the performance among the conditions indicate that the better performance in the visual-only condition is not representative of the real-life performance. Consequently, VR training applications need to deliver the correct stimuli for decision making.

Hands are the most important tool to interact with virtual environments, and they should be available to perform the most critical tasks. For example, a surgeon in VR should keep his/her hands on the instruments and be able to do secondary tasks without performing a disruptive event to the operative task. In this common scenario, one can observe that hands are not available for interaction. The goal of this systematic review is to survey the literature and identify which hands-free interfaces are used, the performed interaction tasks, what metrics are used for interface evaluation, and the results of such evaluations. From 79 studies that met the eligibility criteria, the voice is the most studied interface, followed by the eye and head gaze. Some novel interfaces were brain interfaces and face expressions. System control and selection represent most of the interaction tasks studied and most studies evaluate interfaces for usability. Despite the best interface depending on the task and study, the voice was found to be versatile and showed good results amongst the studies. More research is recommended to improve the practical use of the interfaces and to evaluate the interfaces more formally.

This demo showcases some of the results obtained by the GreenEyes project, whose main objective is to enable visual analysis on resource-constrained multimedia sensor networks. The demo features a multi-hop visual sensor network operated by BeagleBones Linux computers with IEEE 802.15.4 communication capabilities, and capable of recognizing and tracking objects according to two different visual paradigms. In the traditional compress-then-analyze (CTA) paradigm, JPEG compressed images are transmitted through the network from a camera node to a central controller, where the analysis takes place. In the alternative analyze-then-compress (ATC) paradigm, the camera node extracts and compresses local binary visual features from the acquired images (either locally or in a distributed fashion) and transmits them to the central controller, where they are used to perform object recognition/tracking. We show that, in a bandwidth constrained scenario, the latter paradigm allows to reach better results in terms of application frame rates, still ensuring excellent analysis performance.

In a visual sensor network, a large number of camera nodes are able to acquire and process image data locally, collaborate with other camera nodes and provide a description about the captured events. Typically, camera nodes have severe constraints in terms of energy, bandwidth resources and processing capabilities. Considering these unique characteristics, coding and transmission of the pixel-level representation of the visual scene must be avoided, due to the energy resources required. A promising approach is to extract at the camera nodes, compact visual features that are coded to meet the bandwidth and power requirements of the underlying network and devices. Since the total number of features extracted from an image may be rather significant, this paper proposes a novel method to select the most relevant features before the actual coding process. The solution relies on a score that estimates the accuracy of each local feature. Then, local features are ranked and only the most relevant features are coded and transmitted. The selected features must maximize the efficiency of the image analysis task but also minimize the required computational and transmission resources. Experimental results show that higher efficiency is achieved when compared to the previous state-of-the-art.

Nowadays, local feature descriptors have emerged as one of the most promising and powerful visual representation solutions. In fact, with a minimal amount of computational effort, the detection and extraction of visual features can provide reliable and a compact image representation that enables a rich set of image analysis tasks. In this paper, a visual sensor network scenario is considered with energy and bandwidth constraints at each sensor node. In this scenario, low-level binary features can be computed with low complexity and efficient coding schemes can reduce the data rate needed to transmit the features. This paper proposes two binary descriptor coding techniques that exploit the correlation between descriptors of the same image by clustering the extracted descriptors with two solutions: divisive-clustering and agglomerative-clustering. While the former starts with one cluster containing all descriptors which is recursively divided, the latter starts with as many clusters as descriptors which are recursively grouped. The disjoint sets of descriptors are differentially coded and the prediction residual is entropy coded. The experimental results show bitrate savings up to 34% without any impact in the accuracy of the final image retrieval task.

Over the last few years, Parallelism has been gaining increasing importance and multicore processing is now common. Massification of parallelism is driving research and development of novel techniques to overcome current limits of Parallel Computing. However, the scope of parallelization research focuses mainly on ever-increasing performance and much still remains to be accomplished regarding improving productivity in the development of parallel software. This PhD research aims to develop methods and tools to dilute parallel programming complexity and enable nonexpert programmer to fully benefit from a new generation of parallelism-driven programming platforms. Although much work remains to be done to reduce the skill requirements for parallel programming to become within reach of medium-skill programming workforces, it is our belief that this research will help bridge that gap.

Outside of the high-performance computing domain, many applications are irregular in the sense that opportunities to exploit parallelism change throughout the computation, due to the use of complex, pointer-based data structures such as lists and graphs. However, the parallel programming community has relatively little experience in parallelizing irregular applications, and we presently lack a deep understanding of the structure of parallelism and locality in the algorithms that underlie these applications. In this context, irregular algorithms pose a challenging problem to current parallelization methods and techniques.