While 3D Gaussian Splatting (3DGS) has revolutionized static scene representation, the extension to dynamic scene, i.e., 3DGS video (GSV), faces challenges related to reconstruction quality, rendering speed, and storage requirements. The substantial data volume of current GSV poses significant hurdles for streaming applications, particularly in the realm of AR, VR and MR. To tackle these challenges, we introduce 4DGStream, a novel framework that integrates an efficient GSV compression method, Light4D, and a bitrate adaptation streaming strategy, QoSmooth, to ensure smooth playback while maintaining high visual quality. Light4D employs a binarizationassisted spatiotemporal deformation network to model the deformation of Gaussian primitive attributes over time, while a spatiotemporal-aware masking module prunes trivial Gaussians, further enhancing long-term reconstruction quality. To reduce storage, Light4D uses a binary hash grid to model the entropy of attributes for arithmetic coding, with its binary nature allowing efficient entropy modeling via a Bernoulli distribution. These components enable Light4D to improve the FPS/Storage metric by up to 12.4× over SpacetimeGS and 26.4× over 4DGS on the Neu3D dataset, with performance gains exceeding 3× orders of magnitude compared to other NeRF-based state-of-the-art (SOTA) methods. Here, FPS/Storage reflects the balance between rendering speed and data storage. Despite significant model size reductions, Light4D maintains or surpasses the reconstruction quality of 4DGS. Furthermore, QoSmooth provides effective rate control to enhance playback smoothness, reducing bitrate level switches by 61.6% and increasing time-average utility by 26.2%. All these improvements make 4DGStream highly suited for GSV streaming, improving QoE by 36.7% compared to SOTA methods.

The widespread integration of the Internet of Things with sensors like depth-of-field cameras, LiDAR scanners, and eye-tracking infrared sensors, in head-mounted devices, has ushered in a new era of immersive digital experiences. Full-scene volumetric video (VV), a key innovation in this integration, provides a deeply immersive experience by capturing the richness and detail of the 3D world. However, its massive data volume presents significant streaming challenges. While 3D tile-based viewport approaches have been proposed, they struggle to full-scene VV given the small video buffer limitation, high tile segmentation overhead, and lack of full-scene consideration. In this work, inspired by the advancements of implicit neural radiance field (NeRF), we present \({\mathsf{V}^{2}\mathsf{NeRF}}\) , a novel full-scene VV streaming system featured by layered representation. It harmonizes the NeRF with explicit point clouds to represent the static background and dynamic foreground, thereby avoiding large data transfers and achieving photorealistic content representation. To tackle the issues of intensive computation requirements and multiscale adaptation scheduling within \({\mathsf{V}^{2}\mathsf{NeRF}}\) system, we propose a lightweight non-visible background removal method and a two-stage decoupled architecture. In addition, an efficient buffer-aware simulated annealing algorithm is developed, alongside the utilization of a perceptually learned metric, to enhance user experience. We further discuss the concerns about practical development and deployment. Extensive prototype evaluations demonstrate \({\mathsf{V}^{2}\mathsf{NeRF}}\) ’s superior streaming and viewing performance on a wide variety of networks, viewing motions, and scenes. For instance, compared to state-of-the-art approaches, it achieves a 24% increment in perceptual quality, an 83% reduction in rebuffering time, and a 54% enhancement in user experience on average.

Networked 360$^\circ$video has become increasingly popular. Despite the immersive experience for users, its sheer data volume, even with the latest H.266 coding and viewport adaptation, remains a significant challenge to today's networks. Recent studies have shown that integrating deep learning into video coding can significantly enhance compression efficiency, providing new opportunities for high-quality video streaming. In this work, we conduct a comprehensive analysis of the potential and issues in applying neural codecs to 360$^\circ$video streaming. We accordingly present$\mathsf {NETA}$, a synergistic streaming scheme that merges neural compression with traditional coding techniques, seamlessly implemented within an edge intelligence framework. To address the non-trivial challenges in the short viewport prediction window and time-varying viewing directions, we propose implicit-explicit buffer-based prefetching grounded in content visual saliency and bitrate adaptation with smart model switching around viewports. A novel Lyapunov-guided deep reinforcement learning algorithm is developed to maximize user experience and ensure long-term system stability. We further discuss the concerns towards practical development and deployment and have built a working prototype that verifies$\mathsf {NETA}$’s excellent performance. For instance, it achieves a 27% increment in viewing quality, a 90% reduction in rebuffering time, and a 64% decrease in quality variation on average, compared to state-of-the-art approaches.

The rise of 5G communication has transformed the telecom industry for critical applications. With the widespread deployment of 5G base stations comes a significant concern about energy consumption. Key industrial players have recently shown strong interest in incorporating energy storage systems to store excess energy during off-peak hours, reducing costs and partic-ipating in demand response. The fast development of batteries opens up new possibilities, such as the transportation area. An effective method is needed to maximize base station battery utilization and reduce operating costs. In this trend towards next-generation smart and integrated energy-communication-transportation (ECT) infrastructure, base stations are believed to play a key role as service hubs. By exploring the overlap between base station distribution and electric vehicle charging infrastructure, we demonstrate the feasibility of efficiently charging EVs using base station batteries and renewable power plants at the Hub. Our model considers various factors, including base station traffic conditions, weather, and EV charging behavior. This paper introduces an incentive mechanism for setting charging prices and employs a deep reinforcement learning-based method for battery scheduling. Experimental results demonstrate the effectiveness of our proposed ECT-Hub in optimizing surplus energy utilization and reducing operating costs, particularly through revenue-generating EV charging.

Incentive mechanisms in Spatial Crowdsourcing (SC) have been widely studied as they provide an effective way to motivate mobile workers to perform spatial tasks. Yet, most existing mechanisms only involve single tasks, neglecting the presence of complementarity and substitutability among tasks. This limits their effectiveness in practice cases. Motivated by this, we consider task bundles for incentive mechanism design and closely analyze the mutual exclusion effect that arises with task bundles. We then develop a combinatorial incentive mechanism, including three key policies: In the offline case, we propose a combinatorial assignment policy to address the conflict between mutual exclusion and assignment efficiency. We next study the conflict between mutual exclusion and truthfulness, and build a combinatorial pricing policy to pay winners that yields both incentive compatibility and individual rationality. In the online case with unknown workers’ utilities, we present an online combinatorial assignment policy that balances the exploration-exploitation trade-off under the mutual exclusion constraints. Through theoretical analysis and numerical simulations using real-world mobile networking datasets, we demonstrate the effectiveness of the proposed mechanism.

Streaming videos from resource-constrained front-end devices over networks to resource-rich cloud servers has long been a common practice for surveillance and analytics. Most existing live video analytics (LVA) systems, however, have been built over terrestrial networks, limiting their applications during natural disasters and in remote areas that desperately call for real-time visual data delivery and scene analysis. With the recent advent of space networking, in particular, Low Earth Orbit (LEO) satellite constellations such as Starlink, high-speed truly global Internet access is becoming available and affordable. This paper examines the challenges and potentials of LVA over modern LEO satellite networking (LSN). Using Starlink as the testbed, we have carried out extensive in-the-wild measurements to gain insights into its achievable performance for LVA. The results reveal that the uplink bottleneck in today's LSN, together with the volatile network conditions, can significantly affect the service quality of LVA and necessitate prompt adaptation. We accordingly develop StarStream, a novel LSN-adaptive streaming framework for LVA. At its core, StarStream is empowered by a Transformer-based network performance predictor tailored for LSN and a content-aware configuration optimizer. We discuss a series of key design and implementation issues of StarStream and demonstrate its effectiveness and superiority through trace-driven experiments with real-world network and video processing data.

360° videos are gaining popularity, but immersive analytics, particularly in object detection, confront challenges from complex scenes and high data volume. This imposes significant burdens on individual users and resource-limited edge devices. Fortunately, Machine Learning as a Service (MLaaS) offers an economical solution for quick deployment without specific hardware or expertise. However, current MLaaS are mostly 2D image-designated and not optimized for the distinctive characteristics of raw 360° video frames. In this paper, we propose a novel MLaaS-based system to address this challenge. Our solution partitions 360° frames into distortion-free 2D regions with dynamic region of interest prediction. We then present an image-stitching algorithm featuring Skyline representation, seamlessly combining all the 2D regions into a unified frame. This frame is then transmitted to the MLaaS platform, with the detected objects being back-projected to yield the final results. Our experiments demonstrate the superiority of this system over baselines, proving its effectiveness in 360° video object detection tasks.

Immersive full-scene volumetric video (VV) showcases the richness and detail of the 3D world, yet poses significant streaming challenges given its massive data volume. Existing 3D tile-based viewport approaches struggle to effectively adapt to full-scene VV owing to their small video buffer limitation, high tile segmentation overhead, and lack of full-scene consideration.

With the reduced hardware costs of omnidirectional cameras and the proliferation of various extended reality applications, more and more 360° videos are being captured. To fully unleash their potential, advanced video analytics is expected to extract actionable insights and situational knowledge without blind spots from the videos. In this paper, we present OmniSense, a novel edge-assisted framework for online immersive video analytics. OmniSense achieves both low latency and high accuracy, combating the significant computation and network resource challenges of analyzing 360° videos. Motivated by our measurement insights into 360° videos, OmniSense introduces a lightweight spherical region of interest (SRoI) prediction algorithm to prune redundant information in 360° frames. Incorporating the video content and network dynamics, it then smartly scales vision models to analyze the predicted SRoIs with optimized resource utilization. We implement a prototype of OmniSense with commodity devices and evaluate it on diverse real-world collected 360° videos. Extensive evaluation results show that compared to resource-agnostic baselines, it improves the accuracy by 19.8% – 114.6% with similar end-to-end latencies. Meanwhile, it hits 2.0× – 2.4× speedups while keeping the accuracy on par with the highest accuracy of baselines.

Today's electrical grid is experiencing a fast transition toward a smart infrastructure. Modern smart grid is expected to integrate Artificial Intelligence of Things (AIoT)-empowered energy management systems (EMS) to sense, analyze, and optimize the power consumption and QoS of diverse end users. Non-Intrusive Load Monitoring (NILM) plays a key role in this transition, particularly considering that many legacy devices/appliances may not have built-in sensors. Yet most of the NILM solutions rely on large (often impractical) datasets for training. In this paper, we address this challenge through a meta learning-inspired approach, which implements a hierarchical architecture with a “meta-learner” to supervise the training of each appliance. Current EMS also relies on a central controller to access long-term information across all participants, which mismatches their distributed nature, and so often with slow responses. To this end, we develop a deep reinforcement learning based controller to make dynamic decisions for each component in the system. The experiment results based on real-world data sets and simulation data show that applying the meta learning approach can greatly improve the performance of NILM and the QoS of the whole system.

In the 5G era (and the upcoming 6G), mobile edge computing (MEC) has been advocated to serve the massive amount of Internet of Things (IoT) devices by base stations (BSs) and edge data centers (EDCs). Geo-distributed EDCs are generally of much smaller scales as compared to mega data centers and hence of much lower costs, but can have fast response to their users so as to satisfy the demands of real-time applications. As their reliability and availability heavily depend on the electrical power supply, most EDCs are equipped with battery groups as backup power in case of power grid load shedding or outage. In a heterogeneous geo-distributed environment, the QoS of heavily loaded EDCs however can be severely impacted by limited backup power while lightly loaded EDCs may simply waste such precious resources. Moreover, a heavily loaded EDC may suffer from deep discharge of its battery group, which will cause a significant reduction of battery capacity and lifetime. This further aggravates the aforementioned situations should load shedding/outage happen again. In this article, we carefully analyze the workloads in EDCs and classify them into interactive workloads and batch workloads, respectively. We then develop a novel battery allocation framework with smart workload migration for EDCs, which simultaneously protects interactive workloads from being interrupted and minimizes the waiting time of batch workloads. Our extensive evaluations show that our strategies can optimize all the objectives within a limited overall cost as compared to state-of-the-art practical allocation.

Data centers are essential components in the current digital world. The number and scales of data centers have both increased a lot in recent years. The distributed data centers are standing out as a promising solution due to the development of modern applications which need a massive amount of computation resource and strict response requirement. However, compared to centralized data centers, distributed data centers are more fragile when the power supply is unstable. Power constraints or outages because of electrical load shedding or other reasons will significantly affect the service performance of data centers and damage the quality of service (QoS) for customers. Moreover, unlike conventional data centers, distributed data centers are often unattended, so we need a system that can automatically calculate the best workload schedule to maximize profit in such situations. In this paper, we closely investigate the influence of electrical load shedding in distributed data centers and construct a physical model to estimate the relationship among power, heat and workload. We then use queueing theory to approximate the tasks’ response time and aim to minimize the overall response time of tasks by migration. Our extensive evaluations show that our method can improve the response time with more than 9% reduction.

360° videos are becoming one of the major media in recent years, providing immersive experience for viewers with more interactions compared to traditional videos. Most of today’s implementations rely on bulky Head-Mounted Displays (HMDs) or require touch screen operations for interactive display, which are not only expensive but also inconvenient for viewers. In this paper, we demonstrate that interactive 360° video streaming can be done with hints from gaze movement detected by the front camera of today’s mobile devices (e.g., a smartphone). We design a lightweight real-time gaze point tracking method for this purpose. Using only the front camera, our solution detects the users’ faces by a lightweight Haar-like cascaded classifier, measures the user’s face-to-screen distance and sight angle, and then derives the location of the user’s gaze point following a customized triangularity model. We integrate it with streaming module and apply a dynamic margin adaption algorithm to minimize the overall energy consumption for battery-constrained mobile devices. Our experiments on state-of-the-art smartphones show the feasibility of our solution and its energy efficiency toward cost-effective real-time 360° video streaming.