Although on-device video super-resolution enables high-quality live 360-degree streaming on mobile devices, existing methods often waste energy by overlooking perceived visual quality. In this paper, we present EOS, an energy-efficient on-device super-resolution system for mobile omnidirectional video (ODV) live streaming. EOS reduces energy waste by dynamically adjusting super-resolution complexity based on the predicted visual quality of super-resolved frames. This approach raises two challenges: (1) designing an adaptive inference policy that maximizes energy savings while minimizing degradation in Quality-of-Experience (QoE), and (2) developing a method to predict visual quality under the constraints of mobile ODV live streaming. To tackle these challenges, EOS introduces EOS SR and a No-Reference Up-scaling Quality Prediction scheme. EOS SR employs a device-agnostic, scalable deep neural network optimized for mobile devices, with an energy-aware scheduler that jointly selects the optimal super-resolution model and GPU frequency. The No-Reference Upscaling Quality Prediction scheme estimates visual quality across arbitrary viewpoints in real time without requiring high-resolution reference videos. Experiments on commodity smartphones show that EOS reduces average power consumption by 34.6%–49.9% compared to baseline methods, while preserving high visual quality and frame rates.

For highly immersive mobile volumetric video streaming, it is essential to deliver photo-realistic full-scene content with smooth playback. Unlike traditional representations such as point clouds, 3D Gaussian Splatting (3DGS) has gained attention for its ability to represent high-quality full-scene 3D content. However, our preliminary experiments show that existing methods for 3DGS-based videos fail to achieve smooth playback on mobile devices. In this paper, we propose Vega, a 3DGS-based photo-realistic full-scene volumetric video streaming system that ensures real-time playback on mobile devices. The core idea behind Vega's real-time rendering is object-level selective computation, which allocates computational resources to visually important objects to meet strict rendering deadlines. To enable mobile streaming based on the selective computation, Vega addresses two challenges: (1) designing an encoding scheme that optimizes the data size of videos while being compatible with object-level prioritization, and (2) developing a rendering pipeline that efficiently operates on resource-constrained mobile devices. We implemented an end-to-end Vega system, consisting of a streaming server and an Android application. Experimental results on commodity smartphones show that Vega achieves 30 frames per second (FPS) for full-scene volumetric video streaming while maintaining competitive data size and visual quality compared to existing baselines.

Mobile Augmented Reality (AR) applications demand high-quality, real-time visual prediction, including pixel-level depth and semantics, to enable immersive and context-aware user experiences. Recently, Vision Foundation Models (VFMs) offer strong generalization capabilities on diverse and unseen data, supporting scalable mobile AR experiences. However, deploying VFMs on mobile devices is challenging due to computational limitations, particularly in maintaining both prediction accuracy and real-time performance. In this paper, we present ARIA, the first system that enables on-device inference acceleration of a VFM. ARIA employs the heterogeneity of mobile processors through a parallel and selective inference scheme: full-frame prediction is periodically offloaded to a processor with high parallelism capability like GPU, while low-latency updates on dynamic regions are conducted via a specialized accelerator like NPU. Implemented and evaluated using mobile devices, ARIA achieved significant improvements in accuracy and deadline success rate on diverse real-world mobile AR scenarios.

Existing Android systems exhibit energy inefficiency during mobile web browsing due to the lack of awareness of application-level context. Inferring such context from system-level data alone is challenging, but one promising opportunity is using the sequence of task wakeup events, where one task activates another. These sequences show correlation with the type of webpage being used. In this article, we present Ember, a lightweight and responsive power management system for mobile web browsing using only task wakeup sequences. Ember introduces a neural network–based approach to predict optimal CPU clamping values by addressing three key challenges: (1) embedding task names, given as natural-language strings, into meaningful vectors using a Word2Vec-based embedding scheme tailored for task wakeup sequences; (2) minimizing inference overhead with a touch-driven hierarchical inference method that combines lightweight logistic regression with high-accuracy neural networks to balance responsiveness and efficiency; and (3) adapting to within-page interaction dynamics through an interaction-adaptive clamping mechanism that adjusts constraints across different user interaction phases. Implemented on commercial Android smartphones, Ember reduced power consumption by 6.2%–31.2% across a wide range of webpages while maintaining user-perceived quality of experience (QoE).

Sound event detection (SED) based on on-device machine learning (ML) presents considerable energy challenges for ultra-low-power sensor devices. In this paper, we propose DASH, a duration-aware SED system designed for energy-constrained sensor devices in domestic environments. As repeated inferences for continuous sound events lead to unnecessary energy consumption, DASH aims to minimize unnecessary inferences by predicting the duration of sound events. However, the variability of sound event durations across different environments and scenarios poses a major challenge in developing a responsive yet energy-efficient duration-aware SED system. To address this, DASH introduces three key solutions: (1) N-probability distribution-based event duration prediction, which identifies checkpoints where new inferences are likely needed; (2) Affinity-guided event classification, which performs low-energy affinity matching at checkpoints to determine whether ML inference is necessary; and (3) Interrupt blocking-enabling cycle-based device state control, which periodically checks for event presence with minimal energy consumption at non-checkpoint times. We implemented DASH on MSP430-based sensor devices deployed in real home environments. Experimental results demonstrate that DASH reduced energy consumption by approximately 97–98% compared to evaluation baselines, with only a 4.7% error rate.

We propose a cross-device web solution, called Vulture, which distributes graphical user interface (GUI) elements of apps across multiple devices without requiring modifications of web apps or browsers. Several challenges should be resolved to achieve the goals. First, the peer–server configuration should be efficiently established to distribute web resources in cross-device web environments. Vulture exploits an in-browser virtual proxy that runs the web server’s functionality in web browsers using a virtual HTTP scheme and a relevant API. Second, the functional consistency of web apps must be ensured in GUI-distributed environments. Vulture solves this challenge by providing a single-browser illusion with a two-tier document object models (DOM) architecture, which handles view state changes and user input seamlessly in cross-device environments. We implemented Vulture and extensively evaluated the system under various combinations of operating platforms, devices, and network capabilities while running 50 real web apps. The experiment results show that the proposed scheme provides functionally consistent cross-device web experiences by allowing fine-grained GUI distribution. We also confirmed that the in-browser virtual proxy reduces the GUI distribution time and the view change reproduction time by averages of 38.47% and 20.46%, respectively.

The live streaming of omnidirectional video (ODV) on mobile devices demands considerable network resources; thus, current mobile networks are incapable of providing users with high-quality ODV equivalent to conventional flat videos. We observe that mobile devices, in fact, underutilize graphics processing units (GPUs) while processing ODVs; hence, we envisage an opportunity exists in exploiting video super-resolution (VSR) for improved ODV quality. However, the device-specific discrepancy in GPU capability and dynamic behavior of GPU frequency in mobile devices create a challenge in providing VSR-enhanced ODV streaming. In this paper, we propose OmniLive, an on-device VSR system for mobile ODV live streaming. OmniLive addresses the dynamicity of GPU capability with an anytime inference-based VSR technique called Omni SR. For Omni SR, we design a VSR deep neural network (DNN) model with multiple exits and an inference scheduler that decides on the exit of the model at runtime. OmniLive also solves the performance heterogeneity of mobile GPUs using the Omni neural architecture search (NAS) scheme. Omni NAS finds an appropriate DNN model for each mobile device with Omni SR-specific neural architecture search techniques. We implemented OmniLive as a fully functioning system encompassing a streaming server and Android application. The experiment results show that our anytime VSR model provides four times upscaled videos while saving up to 57.15% of inference time compared with the previous super-resolution model showing the lowest inference time on mobile devices. Moreover, OmniLive can maintain 30 frames per second while fully utilizing GPUs on various mobile devices.

Games are energy-intensive applications on mobile devices. Optimizing the energy efficiency of games is hence critical for battery-limited mobile devices. Although the advent of energy-aware scheduling (EAS) integrated in recent devices has provided opportunities for improved energy management, the framework is not specifically tuned for game applications. In this paper, we aim to improve the energy efficiency of game applications running on EAS-enabled mobile devices. To this end, we first analyze the functional characteristics of games, and investigate the source of the energy inefficiency. We then propose a scheme, called System-level Energy-optimization for Game Applications (SEGA), to improve the energy efficiency of games. SEGA governs CPU and GPU power consumption in a tightly coupled manner by employing three key techniques: (1) Lsync-aware GPU DVFS governor, (2) adaptive capacity clamping, and (3) on-demand touch boosting. We implemented SEGA on the latest Android-based smartphones. The evaluation results for 23 popular games showed that SEGA reduced the energy consumption of the Google Pixel 2 XL and Samsung Galaxy S9 Plus smartphones, at the device level, by 6.1–22.3 and 4.0–11.7 percent, respectively, with a quality of service (QoS) degradation of 1.1 and 0.5 percent, on average.

The quality of experience (QoE) is an important issue for users when accessing the web. Although many metrics have been designed to estimate the QoE in the desktop environment, few studies have confirmed whether the QoE metrics are valid in the mobile environment. In this paper, we ask questions regarding the validity of using desktop-based QoE metrics for the mobile web and find answers. We first classify the existing QoE metrics into several groups according to three criteria and then identify the differences between the mobile and desktop environments. Based on the analysis, we ask three research questions and develop a system, called WebMythBusters, for collecting and analyzing mobile web experiences. Through an extensive analysis of the collected user data, we find that (1) the metrics focusing on fast completion or fast initiation of the page loading process cannot estimate the actual QoE, (2) the conventional scheme of calculating visual progress is not appropriate, and (3) focusing only on the above-the-fold area is not sufficient in the mobile environment. The findings indicate that QoE metrics designed for the desktop environment are not necessarily adequate for the mobile environment, and appropriate metrics should be devised to reflect the mobile web experience.

Although work has been conducted on smartphone gaze tracking, the existing techniques are not pervasively used because of their heavy weight and low accuracy. Our preliminary analysis shows that these techniques would work better if their models were trained with data from tablets which have large screens. In this paper, we propose GAZEL, a runtime smartphone gaze-tracking scheme that achieves high accuracy on real devices. The key idea of GAZEL, a tablet-to-smartphone transfer learning, is to train a CNN model with data collected from tablets and then transplant the model to a smartphone. To achieve the goal, we designed a new CNN-based model architecture that is head pose resilient and light enough to operate at runtime. We also exploit implicit calibration to alleviate errors caused by differences in users' visual and device characteristics. The experiment results with commercial smartphones show that GAZEL achieves 27.5% better accuracy on smartphones compared to the state-of-the-art techniques and provides gaze tracking at up to 18 fps which is practically usable at runtime.

Web browsing, previously optimized for the desktop environment, is being fine-tuned for energy-efficient use on mobile devices. Although active attempts have been made to reduce energy consumption, the advent of energy-aware scheduling (EAS) integrated in the recent devices suggests the possibility of a new approach for optimizing energy use by browsers. Our preliminary analysis showed that the existing EAS-enabled system is overly optimized for performance, leading to energy inefficiencies while a web browser is running. In this paper, we analyze the characteristics of web browsers, and investigate the cause of energy inefficiency in EAS-enabled mobile devices. We then propose a system, called WebTune, to improve the energy efficiency of mobile browsers. Exploiting the reinforcement learning technique, WebTune learns the optimal execution speed of the web browser's processes, and adjusts the speed at runtime, thus saving energy and ensuring the quality of service (QoS). WebTune is implemented on the latest Android-based smartphones, and evaluated with Alexa's top 200 websites. The experimental results show that WebTune reduced the device-level energy consumption of the Google Pixel 2 XL and Samsung Galaxy S9 Plus smartphones by 18.7-22.0% and 13.7-16.1%, respectively, without degrading the QoS.

Graphics increasingly play a key role in modern mobile devices. The graphics pipeline requires a close relationship between the CPU and the GPU to ensure energy efficiency and the user's quality of experience (QoE). Our preliminary analysis showed that the current techniques employed to achieve energy efficiency in the Android graphics pipeline are not optimized especially in the frame generation process. In this paper, we aim to improve the energy efficiency of the Android graphics pipeline without degrading the user's QoE. To achieve this goal, we studied the internals of the Android graphics pipeline and observed the energy inefficiency in the existing governing framework of the CPU and GPU. Based on the findings, we propose three techniques for addressing energy inefficiency: (1) aggressively capping the maximum CPU frequency, (2) lowering the CPU frequency by raising the GPU minimum frequency, and (3) allocating the frame rendering-related threads in the energy-efficient CPU cores. These techniques are integrated into a single governing framework, called the GFX Governor, and implemented in the newest Android-based smartphones. Experimental results show that without hampering the user's QoE the average energy consumption of Nexus 6P, Pixel XL, and Pixel 2 XL is reduced at the device level by 24.2%, 18.6%, and 13.7%, respectively, for the 60 chosen applications. We also analyzed the efficacy of the proposed technique in comparison with the state-of-the-art Energy-Aware Scheduling (EAS) implemented in the latest smartphone.