While video sensing, performed by resource-constrained pervasive devices, is a key enabler of many machine intelligence applications, the high energy and bandwidth overheads of streaming video transmission continue to present formidable deployment challenges. Motivated by the recent advancements in deep learning models, this article proposes the usage of a Generative Network-based technique for resource-efficient streaming video compression and transmission. However, we empirically show that while such generative network-based models offer superior compression gains compared to H.265, additional DNN optimization mechanisms are needed to substantially reduce their encoder complexity. Our proposed optimized system, dubbed Pr-Ge-Ne , adopts a carefully pruned encoder-decoder DNN, on the pervasive device, to efficiently encode a latent vector representation of intra-frame relative motion, and then uses a generator network at the decoder to reconstruct the frames by overlaying such motion information to ”animate” an initial reference frame. By evaluating three representative streaming video datasets, we show that Pr-Ge-Ne achieves around \(6\) – \(10\) -fold reduction in video transmission rates (with negligible impact on the accuracy of machine perception tasks) compared to H.265, while simultaneously reducing latency and energy overheads on a pervasive device by \(\sim\) 90% and 15–50%, respectively.

While many pervasive computing applications increasingly utilize real-time context extracted from a vision sensing infrastructure, the high energy overhead of DNN-based vision sensing pipelines remains a challenge for sustainable in-the-wild deployment. One common approach to reducing such energy overheads is the capture and transmission of lower-resolution images to an edge node (where the DNN inferencing task is executed), but this results in an accuracy-vs-energy tradeoff, as the DNN inference accuracy typically degrades with a drop in resolution. In this work, we introduce MRIM, a simple but effective framework to tackle this tradeoff. Under MRIM, the vision sensor platform first executes a lightweight preprocessing step to determine the saliency of different sub-regions within a single captured image frame, and then performs a saliency-aware non-uniform downscaling of individual sub-regions to produce a “mixed-resolution” image. We describe two novel low-complexity algorithms that the sensor platform can use to quickly compute suitable resolution choices for different regions under different energy/accuracy constraints. Experimental studies, involving object detection tasks evaluated traces from two benchmark urban monitoring datasets as well as a prototype Raspberry Pi-based MRIM implementation, demonstrate MRIM’s efficacy: even with unoptimized embedded platform, MRIM can provide system energy savings of 35+% or increase task accuracy by 8+%, over conventional baselines of uniform resolution downscaling or image encoding, while supporting high throughput.

Mobile video chatting has emerged as an important Internet multimedia application that greatly enriches interpersonal communications. Mobile power efficiency is crucial to the service quality and time of video chatting on battery-limited smartphones. However, the power characteristics of the video coding and data communication are highly complex due to the time-varying network conditions and dynamic mobile energy features. This incurs crucial challenges to maintaining the low power dissipation of mobile chatting application while streaming satisfactory-quality videos. To address these challenges, this paper presents a joinT cOding-tranSmission Optimization (TOSO) protocol at application layer that performs machine learning based adaptation of the video bit rate and FEC (Forward Error Correction) coding parameters. By taking advantage of analytical and empirical models characterizing the quality-power relationship, TOSO is able to maximize video quality subject to a specified upper bound of power consumption in mobile chat application. This distinguishing feature prevents the video chat from draining battery too quickly. Moreover, it allows the smartphone operating system or the mobile user to define a desired video chat duration given the remaining battery, avoiding unpleasant conversation disruption due to battery depletion. Extensive experiments based on the Linphone platform and Exata network emulator show that TOSO outperforms baseline approaches by 29.3 percent in power conservation while achieving the same video quality level.