Abstract Virtual reality (VR) systems aim to create immersive experiences by blending real and virtual worlds. However, accurately perceiving distances in VR remains a challenge due to mismatches between real-world and simulated spatial cues. This study compares the effects of different locomotion methods—teleport, joystick, arms swinging, and real walking—on perceived traveled distance and spatial orientation in VR. An exploratory within-subject user study revealed that while arm swinging was the least preferred method, it yielded the most accurate distance estimations and superior spatial orientation. These findings underscore the importance of locomotion methods in enhancing spatial awareness in VR environments.

Spatial skills, in particular mental rotation, are increasingly linked to STEM achievements and to computational thinking (CT) abilities. Traditional 2D spatial assessments face validity and equity issues from visual ambiguity, occlusion, and missing depth cues, requiring cognitively demanding 2D-to-3D interpretation. These artifacts might contribute to gender disparities, giving males an advantage that may reflect test bias rather than true ability differences. Immersive Virtual Reality (VR) seems to mitigate these issues by providing stereoscopic depth, multi-perspective viewing, realistic lighting, and embodied interaction. We compared spatial performance in VR and traditional 2D assessments using the Virtual Reality Mental Rotation Assessment (VRMRA) in a within-subjects study of 48 adolescents (ages 12-16). VR improved performance, yielding higher accuracy in Mental Rotation Test (MRT)-style tasks (+1.3 items, p < .001), while PSVT:R accuracy did not differ significantly between 2D and VR. We observed that gender effects are task-specific: VR improved the performance of females most in MRT-style tasks, reversing the 2D male advantage, whereas males gained more in PSVT:R-style tasks. Spatial scores correlated positively with CT across both 2D and VR assessments, indicating a link to computational problem-solving. These results suggest that the assessment of spatial skills in VR produces distinct performance patterns with respect to traditional 2D assessment, highlighting the importance of immersive VR for the development of improved tools and its potential implications for inclusive STEM education and talent identification.

Recently, immersive media devices have seen a boost in popularity. However, many problems still remain. Depth perception is a crucial part of how humans behave and interact with their environment. Convergence and accommodation are two physiological mechanisms that provide important depth cues. However, when humans are immersed in virtual environments, they experience a mismatch between these cues. This mismatch causes users to feel discomfort while also hindering their ability to fully perceive object distances. To address the conflict, we have developed a technique that encompasses inverse blurring into immersive media devices. For the inverse blurring, we utilize the classical Wiener deconvolution approach by proposing a novel technique that is applied without the need for an eye-tracker and implemented in a commercial immersive media device. The technique's ability to compensate for the vergence-accommodation conflict was verified through two user studies aimed at reaching and spatial awareness, respectively. The two studies yielded a statistically significant 36% and 48% error reduction in user performance to estimate distances, respectively. Overall, the work done demonstrates how visual stimuli can be modified to allow users to achieve a more natural perception and interaction with the virtual environment.

3D reconstruction is of interest to several fields. However, obtaining the 3D model is usually a time-consuming task that involves manual measurements and reproduction of the object using CAD software, which is not always feasible (e.g. for organic shapes). The necessity of quickly obtaining a dimensionally accurate 3D model of an object has led to the development of several reconstruction techniques, either vision based (with photogrammetry), using laser scanners, or a combination of the two. The contribution of this study is in the analysis of the performances of currently available 3D reconstruction frameworks with the aim of providing a guideline to novice users who may be unfamiliar with 3D reconstruction technologies. We evaluate various software packages on a synthetic dataset representing objects of various shapes and sizes. For comparison, we consider various metrics such as mean errors in the reconstructed cloud point and meshes and reconstruction time. Our results indicate that Colmap produces the best reconstruction.

Due to the Covid-19 pandemic, millions of people had to stay at home for several months and to move their everyday activities online. From smart-working to online learning, also considering entertainment activities, people started to extensively use platforms for remote communication and for sharing video, text, and audio content. Such systems are able to cope with the need of communicating, but the common feeling is that people, and especially students, are missing the physical relationships with classmates and colleagues. In this paper, we analyse two modalities that can be used to give an online University lecture, by focusing on the ‘sense of presence’, i.e. on the ability to produce into students the psychological sense of being in a classroom together with the teacher and their classmates. We consider a standard web-conferencing system, and a Virtual Reality social platform. We also take into account the ‘usability’ factors, to understand if extended reality systems could be, in a near future, a possible alternative to improve distance lectures and meetings.

This paper presents a technique to incorporate spatial blur effects in virtual reality devices. The considered spatial blur is based on foveation and defocus blur: concepts inspired by the human visual system. The proposed technique can be applied to any head-mounted display as a post-processing step. Our foveated depth-of-field method removes intensity leakage artifacts in the transitory regions and works in real-time. We verify the usefulness of our technique by conducting a pilot study on depth perception in virtual environments. In the conducted user study, systems integrated with our blur effect provided a better estimation of object depth in the peripheral regions.

Currently, immersive virtual reality is experienced through head-mounted displays while the user is physically present into a real, cluttered environment. This causes the problem of avoiding dangerous collisions with obstacles in the real environment that are invisible to the user, and also hampers the interaction with real objects. Following the augmented virtuality paradigm, these obstacles should be embedded into the virtual environment. Thus, there is the need of knowing the 3D structure of the real environment to align it with the virtual one. In this paper, we present a method to create a virtual scenario composed of virtual objects having the same spatial occupancy of the corresponding real ones. The real scene is scanned to detect the position and bounding box of objects and obstacles, then virtual elements having similar spatial properties are added to the virtual scene. Two different real environment structure detection and clustering techniques are described and compared, both quantitatively and by considering users’ sense of presence with respect to the standard Chaperone technique. Our results show that the method is a good solution to maintain the real environment awareness while keeping an high level of immersivity and sense of presence.

The human visual system is foveated: we can see fine spatial details in central vision, whereas resolution is poor in our peripheral visual field, and this loss of resolution follows an approximately logarithmic decrease. Additionally, our brain organizes visual input in polar coordinates. Therefore, the image projection occurring between retina and primary visual cortex can be mathematically described by the log-polar transform. Here, we test and model how this space-variant visual processing affects how we process binocular disparity, a key component of human depth perception. We observe that the fovea preferentially processes disparities at fine spatial scales, whereas the visual periphery is tuned for coarse spatial scales, in line with the naturally occurring distributions of depths and disparities in the real-world. We further show that the visual system integrates disparity information across the visual field, in a near-optimal fashion. We develop a foveated, log-polar model that mimics the processing of depth information in primary visual cortex and that can process disparity directly in the cortical domain representation. This model takes real images as input and recreates the observed topography of human disparity sensitivity. Our findings support the notion that our foveated, binocular visual system has been moulded by the statistics of our visual environment.

Grasping is one of the fundamental actions we perform to interact with objects in real environments, and in the real world we rarely experience difficulty picking up objects. Grasping plays a fundamental role for interactive virtual reality (VR) systems that are increasingly employed not only for recreational purposes, but also for training in industrial contexts, in medical tasks, and for rehabilitation protocols. To ensure the effectiveness of such VR applications, we must understand whether the same grasping behaviors and strategies employed in the real world are adopted when interacting with objects in VR. To this aim, we replicated in VR an experimental paradigm employed to investigate grasping behavior in the real world. We tracked participants' forefinger and thumb as they picked up, in a VR environment, unfamiliar objects presented at different orientations, and exhibiting the same physics behavior of their real counterparts. We compared grasping behavior within and across participants, in VR and in the corresponding real world situation. Our findings highlight the similarities and differences in grasping behavior in real and virtual environments.

The depth cue is a fundamental piece of information for artificial and living beings who interact with the surrounding environment in order to handle objects and to avoid obstacles: in such situations, the disparity patterns, which arise when agents fixate objects, are vector fields. We propose a biologically-inspired computational model to estimate dense horizontal and vertical disparity maps by exploiting the cortical paradigms of the primate visual system: in particular, we aim to model the disparity sensitivity of the V1-MT visual pathway. The proposed model is based on a first processing stage composed of a bank of spatial band-pass filters and a static nonlinearity, mimicking complex binocular cells. Then, subsequent pooling stages and decoding strategies allow the model to estimate the vector disparity, after having represented it as a population of MT-like units. We assess the proposed model by using standard benchmarking stereo images, the Middlebury dataset, and specific stereo images that have horizontal and vertical disparities, which characterize the stimuli produced by active vision systems. Moreover, we systemically analyze how the different processing stages affect the model performance, and we discuss their implications for the neural modeling.

The 3D structure of a virtual or augmented scene is affected by distortions that can affect the natural fruition of the experience. Here, we consider the distortions of the spatial layout of virtual scenes, specifically the underestimation of egocentric distances, as experienced by users.The aim of our work is to develop a computer vision based modeling that is able to describe such an underestimation. In particular, we propose that the spatial misperception has origin from the fact that the world is seen by a user through an uncalibrated camera, thus producing a virtual view of a distorted space for him/her. The proposed geometric model is able to explain the experimental evidence of the egocentric distance underestimation that is reported in the literature.

The recognition of complex actions is still a challenging task in Computer Vision especially in daily living scenarios, where problems like occlusion and limited field of view are very common. Recognition of Activity Daily Living (ADL) could improve the quality of life and supporting independent and healthy living of older or/and impaired people by using information and communication technologies at home, at the workplace and in public spaces. This paper proposes to embed spatio-temporal information into ontology models to improve action recognition using visual words. Actions detected by visual words are implemented as Primitive States in the scenario and then used as Components of Composite States to merge them with spatio-temporal patterns that the people display while performing ADLs. In a challenging dataset, such as SmartHome, where a high variance intra-class and low variance inter-class is present, recognition results for some actions improve in precision and recall thanks to spatial information.

We show how a cortical model of early disparity detectors is able to autonomously learn effective control signals in order to drive the vergence eye movements of a binocular active vision system. The proposed approach employs early binocular mechanisms of vision and basic learning processes such as synaptic plasticity and reward modulation. The computational substrate consists of a population of modeled V1 complex cells, that provides a distributed representation of binocular disparity information. The population response also provides a global signal to describe the state of the system and thus its deviation from the desired vergence position. The proposed network, by taking into account the modification of its internal state as a consequence of the action performed, evolves following a differential Hebbian rule. Furthermore, the weights update is driven by an intrinsic signal derived by the overall activity of the population. Exploiting this signal implies a maximization of the population activity itself, thus providing an highly effective reward for the developing of a stable and accurate vergence behaviour. The efficacy of the proposed intrinsic reward signal is comparatively assessed against the ground-truth signal (the actual disparity) providing equivalent results, and thus validating the approach. Experimental tests in a simulated environment demonstrate that the proposed network is able to cope with vergent geometry and thus to learn effective vergence movements for static and moving visual targets in realistic situations.

A method to analyze first-order spatial properties of optical flow is proposed. The approach is based on the use of a set of linear models that dynamically adjust their properties on the basis of context information. These models are generated by a recursive network that takes into account spatial interaction between neighbors. By checking the presence of these models in the optic flow using a multiple model Kalman Filter it is possible to recover information about the coefficients of the affine description and the image motion invariants: divergence, curl and deformation. Reliable estimates of these quantities could help in the analysis of real world complex motion sequences. Experimental results on egomotion estimation and 3D surface reconstruction validate the approach. 1

A perceptual human-machine interface based on visual appearance of hand movements is presented. Gestures are defined as the temporal evolution of 3D poses the of user's hand. Each gesture is described by five temporal sequences corresponding to the time evolution of the stereo depth information at the finger-tips. To this end feature processing directly exploits binocular disparity information considered in its full spatio-temporal dimension. Hidden Markov models are used to represent the statistical properties of the gesture. The high recognition rate obtained by experimental testing on a five gesture alphabet validates the approach.

A cortical model for motion-in-depth selectivity of complex cells in the visual cortex is proposed. The model is based on a time ex(cid:173) tension of the phase-based techniques for disparity estimation. We consider the computation of the total temporal derivative of the time-varying disparity through the combination of the responses of disparity energy units. To take into account the physiological plau(cid:173) sibility, the model is based on the combinations of binocular cells characterized by different ocular dominance indices. The resulting cortical units of the model show a sharp selectivity for motion-in(cid:173) depth that has been compared with that reported in the literature for real cortical cells.