In this work, we present L3GS, a framework for efficient 3D scene delivery with 3D Gaussian splats [5]. While 3D Gaussian splats achieve a balance between visual fidelity and rendering efficiency, their massive data size still limits realtime deployment. L3GS addresses this by leveraging scene layering, predictive viewport and bandwidth estimation, and priority-based scheduling to progressively deliver the most essential splats first. This demo will showcase L3GS's 3D scene delivery performance under controlled network conditions and compare it with various baselines.

Traditional 3D content representations include dense point clouds that consume large amounts of data and hence network bandwidth, while newer representations such as neural radiance fields suffer from poor frame rates due to their nonstandard volumetric rendering pipeline. 3D Gaussian splats (3DGS) can be seen as a generalization of point clouds that meet the best of both worlds, with high visual quality and efficient rendering for real-time frame rates. However, delivering 3DGS scenes from a hosting server to client devices is still challenging due to high network data consumption (e.g., 1.5 GB for a single scene). The goal of this work is to create an efficient 3D content delivery framework that allows users to view high quality 3D scenes with 3DGS as the underlying data representation. The main contributions of the paper are: (1) Creating new layered 3DGS scenes for efficient delivery, (2) Scheduling algorithms to choose what splats to download at what time, and (3) Trace-driven experiments from users wearing virtual reality headsets to evaluate the visual quality and latency. Our system for Layered 3D Gaussian Splats delivery (L3GS) demonstrates high visual quality, achieving 16.9% higher average SSIM compared to baselines, and also works with other compressed 3DGS representations. The code is available at https://github.com/mavenslab/layered_3d_gaussian_splats.

Augmented reality (AR) seamlessly overlays virtual objects onto the real world, enabling an exciting new range of applications. Multiple users view and interact with virtual objects, which are replicated and shown on each user’s display. A key requirement of AR is that the replicas should be quickly updated and converge to the same state; otherwise, users may have laggy or inconsistent views of the virtual object, which negatively affects their experience. A second key requirement is that the movements of virtual objects in space should preserve certain integrity properties either due to physical boundaries in the real world, or privacy and safety preferences of the user. For example, a virtual cup should not sink into a table, or a private virtual whiteboard should stay within an office. The challenge tackled in this paper is the coordination of virtual objects with low latency, spatial integrity properties and convergence. We introduce “well-organized” replicated data types that guarantee these two properties. Importantly, they capture a local notion of conflict that supports more concurrency and lower latency. To implement well-organized virtual objects, we introduce a credit scheme and replication protocol that further facilitate local execution, and prove the protocol’s correctness. Given an AR environment, we automatically derive conflicting actions through constraint solving, and statically instantiate the protocol to synthesize custom coordination. We evaluate our implementation, H ambazi , on off-the-shelf Android AR devices and show a latency reduction of 30.5-88.4% and a location staleness reduction of 35.6-75.6%, compared to three baselines, for varying numbers of devices, AR environments, request loads, and network conditions.

World-scale augmented reality (AR) is a form of AR where users move around the real world, viewing and interacting with 3D models at specific locations. However, given the geographical scale of world-scale AR, pre-fetching and storing numerous high-quality 3D models locally on the device is infeasible. For example, it would be impossible to download and store 3D ads from all the storefronts in a city onto a single device. A key challenge is thus deciding which remotely-stored 3D models should be fetched onto the AR device from an edge server, in order to render them in a timely fashion - yet with high visual quality - on the display. In this work, we propose a 3D model retrieval framework that makes intelligent decisions of which quality of 3D models to fetch, and when. The optimization decision is based on quality-compression tradeoffs, network bandwidth, and predictions of which 3D models the AR user is likely to view next. To support our framework, we collect real-world traces of AR users playing a world-scale AR game, and use this to drive our simulation and prediction modules. Our results show that the proposed framework can achieve higher visual quality of the 3D models while missing fewer display deadlines (by 20%) and wasting fewer bytes (by 10x), compared to a baseline approach of pre-fetching models within a fixed distance of the user.

Contact tracing is a key mechanism to help contain the COVID-19 pandemic and other pandemics in the future. In this work, we propose using 5G channel signatures – specifically, mm-Wave channel signatures – to perform contact tracing and infer early sources of the infection. Our network-side approach is motivated by the density of mm-Wave base stations, coupled with the large amount of data about mobile device signals already being collected by cellular operators. We model the contact tracing problem as a graph mining problem, and develop machine learning models to estimate contacts between UEs based on 5G channel signatures such as received power. These contacts are also used to infer the original sources of the infection. Simulations of our proposed method using the ns-3 5G mmWave module suggest that contact can be inferred with a recall of 85% and specificity of 94%. Our infection sources estimation method can accurately rank the most likely infection sources, with the true infection sources lying in the top 25% of the ranked list on average. These methods represent a first step towards network-based contact tracing, and can complement other contact tracing methods to help reduce the spread of disease.

Multi-user augmented reality (AR), where multiple co-located users view a common set of virtual objects, is becoming increasingly popular. For example, Google Just a Line allows multiple users to draw virtual graffiti in the same physical space. Multi-user AR requires network communications in order to coordinate the positions of the virtual objects on each user's display, yet there is currently little understanding of how such apps communicate. In this work, we address this key gap in knowledge by showing that the communicated data directly impacts the latency and positioning of the virtual objects rendered on the users' displays. We develop solutions to these problems that we find along three facets: (1) efficient communication strategies that trade off communication latency for spatial consistency of the virtual objects; (2) a new metric that enables mobile AR devices to update their virtual objects as they move around and observe more of the scene; and (3) a tool to automatically quantify how much the virtual objects' positions inadvertently change in time and space. Our evaluation is performed on Android smartphones running open-source AR. The results show that our system, SPAR, can decrease the latency by up to 55%, while decreasing the spatial inconsistency by up to 60%, compared to baseline methods.

Augmented reality (AR) apps where multiple users interact within the same physical space are gaining in popularity (e.g., shared AR mode in Pokemon Go, virtual graffiti in Google's Just a Line). However, multi-user AR apps running over the cellular network can experience very high end-to-end latencies (measured at 12.5 s median on a public LTE network). To characterize and understand the root causes of this problem, we perform a first-of-its-kind measurement study on both public LTE and industry LTE testbed for two popular multi-user AR applications, yielding several insights: (1) The radio access network (RAN) accounts for a significant fraction of the end-to-end latency (31.2%, or 3.9 s median), resulting in AR users experiencing high, variable delays when interacting with a common set of virtual objects in off-the-shelf AR apps; (2) AR network traffic is characterized by large intermittent spikes on a single uplink TCP connection, resulting in frequent TCP slow starts that can increase user-perceived latency; (3) Applying a common traffic management mechanism of cellular operators, QoS Class Identifiers (QCI), can help by reducing AR latency by 33% but impacts non-AR users. Based on these insights, we propose network-aware and network-agnostic AR design optimization solutions to intelligently adapt IP packet sizes and periodically provide information on uplink data availability, respectively. Our solutions help ramp up network performance, improving the end-to-end AR latency and goodput by ~40-70%.