Volumetric video (VV) offers photorealistic 3D capture for immersive educational VR, often created by instructors through live-streamed lessons or prerecorded demonstrations. While enhancing engagement and presence, such instructor-produced content can unintentionally expose sensitive objects, personal information, or biometric identifiers, and may intensify feelings of surveillance. This poster examines these privacy risks in using VV for educational VR and presents a research agenda focused on integrating diminished reality (DR) techniques and real-time 3D scene understanding into VV pipelines to dynamically sanitize environments while balancing realism and privacy.

We present NIER, a video conferencing system that can adaptively maintain a low bitrate (e.g., 10–100 Kbps) with reasonable visual quality while being robust to packet losses. We use key-point-based deep image animation (DIA) as a key building block and address a series of networking and system challenges to make NIER practical. Our evaluations show that NIER significantly outperforms the baseline solutions.

Virtual reality (VR) offers promise in education given its immersive and socially engaging nature, but it can pose challenges for educators when creating VR-specific content. VR videos can function as a new educational tool for VR content creation due to their creation affordability and user-friendliness. However, little empirical research exists on how educators utilize VR videos and associated pedagogy in real classes. Our research employed a contextual inquiry, through in-person interviews and online surveys with 11 instructors to gain actionable insights from envisioned teaching scenarios for VR videos that are informed by actual instructional practices. Our study aims to understand the factors that motivate instructors’ adoption of VR videos, identify challenges educators face when incorporating VR videos into instructional units, and examine pedagogical adjustments when integrating VR videos into teaching. Through empirical evidence, we provide design implications for the development of VR-based learning experiences across diverse educational contexts. Our study also serves as a practical case of how VR can be adopted and integrated into education.

Volumetric videos offer a unique interactive experience and have the potential to enhance social virtual reality and telepresence. Streaming volumetric videos to multiple users remains a challenge due to its tremendous requirements of network and computation resources. In this paper, we develop MuV2, an edge-assisted multi-user mobile volumetric video streaming system to support important use cases such as tens of students simultaneously consuming volumetric content in a classroom. MuV2 achieves high scalability and good streaming quality through three orthogonal designs: hybridizing direct streaming of 3D volumetric content with remote rendering, dynamically sharing edge-transcoded views across users, and multiplexing encoding tasks of multiple transcoding sessions into a limited number of hardware encoders on the edge. MuV2 then integrates the three designs into a holistic optimization framework. We fully implement MuV2 and experimentally demonstrate that MuV2 can deliver high-quality volumetric videos to over 30 concurrent untethered mobile devices with a single WiFi access point and a commodity edge server.

Virtual Reality (VR) has a noteworthy educational potential by providing immersive and collaborative environments. As an alternative but cost-effective way of delivering realistic environments in VR, using 360-degree videos in immersive VR (VR videos) received more attention. Although many studies reported positive learning experiences with VR videos, little is known about how collaborative learning performs on VR video viewing systems. In this study, we implemented two collaborative VR video viewing modes based on the way of group video control, synchronized or shared (Sync mode) and non-synchronized or individual (Non-sync mode) video control, against a conventional VR video viewing setting (Basic mode). We conducted a within-subject study (N = 54) in a lab-simulated remote learning environment. Our results show that collaborative VR video modes (Sync and Non-sync mode) improve users’ learning experiences and collaboration quality, especially with shared video control. Our findings provide directions for designing and employing collaborative VR video tools in online learning environments.

VR has received increased attention as an educational tool and many argue it is destined to influence educational practices, especially with the emergence of the Metaverse. Most prior research on educational VR reports on applications or systems designed for specified educational or training objectives. However, it is also crucial to understand current practices and attitudes across disciplines, having a holistic view to extend the body of knowledge in terms of VR adoption in an authentic setting. Taking a higher-level perception of people in different roles, we conducted a qualitative analysis based on 23 interviews with major stakeholders and a series of participatory design workshops with instructors and students. We identified the stakeholders who need to be considered for using VR in higher education, and highlighted the challenges and opportunities critical for VR current and potential practices in the university classroom. Finally, we discussed the design implications based on our findings. This study contributes a detailed description of current perceptions and considerations from a multi-stakeholder perspective, providing new empirical insights for designing novel VR and HCI technologies in higher education.

The emerging volumetric videos offer a fully immersive, six degrees of freedom (6DoF) viewing experience, at the cost of extremely high bandwidth demand. In this paper, we design, implement, and evaluate Vues, an edge-assisted transcoding system that delivers high-quality volumetric videos with low bandwidth requirement, low decoding overhead, and high quality of experience (QoE) on mobile devices. Through an IRB-approved user study, we build a first-of-its-kind QoE model to quantify the impact of various factors introduced by transcoding volumetric content into 2D videos. Motivated by the key observations from this user study, Vues employs a novel multiview approach with the overarching goal of boosting QoE. The Vues edge server adaptively transcodes a volumetric video frame into multiple 2D views with the help of a few lightweight machine learning models and strategically balances the extra bandwidth consumption of additional views and the improved QoE, indicated by our QoE model. The client selects the view that optimizes the QoE among the delivered candidates for display. Comprehensive evaluations using a prototype implementation indicate that Vues dramatically outperforms existing approaches. On average, it improves the QoE by 35% (up to 85%), compared to single-view transcoding schemes, and reduces the bandwidth consumption by 95%, compared to the state-of-the-art that directly streams volumetric videos.

In this paper, we perform a first comprehensive study of mobile volumetric video streaming. Volumetric videos are truly 3D, allowing six degrees of freedom (6DoF) movement for their viewers during playback. Such flexibility enables numerous applications in entertainment, healthcare, education, etc. However, volumetric video streaming is extremely bandwidth-intensive. We conduct a detailed investigation of each of the following aspects for point cloud streaming (a popular volumetric data format): encoding, decoding, segmentation, viewport movement patterns, and viewport prediction. Motivated by the observations from the above study, we propose ViVo, which is to the best of our knowledge the first practical mobile volumetric video streaming system with three visibility-aware optimizations. ViVo judiciously determines the video content to fetch based on how, what and where a viewer perceives for reducing bandwidth consumption of volumetric video streaming. Our evaluations over real wireless networks (including commercial 5G), mobile devices and users indicate that ViVo can save on average 40% of data usage (up to 80%) with virtually no drop in visual quality.

We conduct to our knowledge a first measurement study of commercial 5G performance on smartphones by closely examining 5G networks of three carriers (two mmWave carriers, one mid-band carrier) in three U.S. cities. We conduct extensive field tests on 5G performance in diverse urban environments. We systematically analyze the handoff mechanisms in 5G and their impact on network performance. We explore the feasibility of using location and possibly other environmental information to predict the network performance. We also study the app performance (web browsing and HTTP download) over 5G. Our study consumes more than 15 TB of cellular data. Conducted when 5G just made its debut, it provides a “baseline” for studying how 5G performance evolves, and identifies key research directions on improving 5G users’ experience in a cross-layer manner. We have released the data collected from our study (referred to as 5Gophers) at https://fivegophers.umn.edu/www20.