Collaborative human-object interaction shows dynamic and complex movements that require mutual anticipation and continuous adjustment between participants and the shared object. Understanding and modeling such collaborative multi-human object interaction (MHOI) scenarios requires high-quality data acquisition as a foundational step, however, this is a challenging task due to the inherent complexity of MHOI scenarios where human-human and human-object interactions occur simultaneously. Such complexity leads to noisy MHOI captures characterized by several artifacts: contact misalignment between hands and objects, motion jitter and temporal inconsistencies in the captured sequences, and missing or incomplete finger-level articulation details. To address these challenges, we present MOCHI (MOtion Enhancement of Collaborative Human-object Interactions), a two-stage framework for enhancing noisy MHOI data. Our approach first generates physically plausible hand grasps through optimization from noisy body input, producing grasps that are both physically plausible and semantically consistent with the body pose, where these optimized grasps are extended into complete hand-object interaction sequences. Consequently, the full-body motion for all participants are refined through a diffusion-based noise optimization framework that uses single-person motion priors. During the optimization process, we introduce optimization objectives to encode human-object and human-human interaction information within these single-person priors. Experimental results demonstrate the effectiveness of our pipeline across diverse MHOI data, either acquired by existing capture methods or synthesized by generative models. We further show robustness of our system across varying numbers of participants and types of interactions, and demonstrate various applications including keyframe-based MHOI creation and data augmentation through varying object geometries.

We propose a cross-device web solution, called Vulture, which distributes graphical user interface (GUI) elements of apps across multiple devices without requiring modifications of web apps or browsers. Several challenges should be resolved to achieve the goals. First, the peer–server configuration should be efficiently established to distribute web resources in cross-device web environments. Vulture exploits an in-browser virtual proxy that runs the web server’s functionality in web browsers using a virtual HTTP scheme and a relevant API. Second, the functional consistency of web apps must be ensured in GUI-distributed environments. Vulture solves this challenge by providing a single-browser illusion with a two-tier document object models (DOM) architecture, which handles view state changes and user input seamlessly in cross-device environments. We implemented Vulture and extensively evaluated the system under various combinations of operating platforms, devices, and network capabilities while running 50 real web apps. The experiment results show that the proposed scheme provides functionally consistent cross-device web experiences by allowing fine-grained GUI distribution. We also confirmed that the in-browser virtual proxy reduces the GUI distribution time and the view change reproduction time by averages of 38.47% and 20.46%, respectively.

In e-scooter sharing systems, users randomly select and use e-scooters based on inaccurate battery information. This simple rental policy leads to low profitability on two fronts. First, inaccurate battery information causes unexpected device shutdowns, causing negative user experiences and refunds. Second, randomly selected e-scooters increase operation costs for battery management. In this paper, we propose e-scooter recommendation system, EcoRide, which provides accurate battery estimation and profitable e-scooter selection to maximize profitability of sharing systems. To this end, we propose a battery estimation considering four factors, i.e., battery state, temperature, user weight, and road slope, that affect the available battery energy in e-scooter applications. We define a parameter, dynamic voltage threshold (DVT), to represent dynamically changing battery energy, and use it to estimate battery availability. Next, to achieve cost-effective e-scooter selection, we introduce a multi-agent reinforcement learning (MARL)-based technique to learn policies that minimize operation costs. We define sharing system operation as a MARL problem with an objective function based on battery management costs. To cope with unstable training due to a wide service area and multiple requests, a centralized training technique is adopted. The proposed battery estimation and e-scooter selection technique are validated through actual driving tests and a sharing system simulator, respectively. Additionally, our case study using open data from Washington D.C. demonstrates a profit gain of up to 68% with EcoRide.

Optimizing deep neural networks (DNNs) running on resource-constrained devices, such as energy harvesting sensor devices, poses unique challenges due to the limited memory and varying energy conditions. Existing efforts have shown that deploying a multi-exit network mitigates the problem by allowing tradeoffs between accuracy and computational complexity. However, previous works did not fully consider two essential requirements: optimized neural architecture and optimized inference policy. In this paper, we present HarvNet, which comprises two complementary techniques for generating and operating a multi-exit network for energy harvesting devices. First, we provide a neural architecture search scheme, HarvNAS, which configures the best multi-exit architecture while meeting memory and energy constraints. Second, HarvSched learns and constructs the best progressive inference policy with different energy constraints by considering runtime factors, such as the harvesting status and the energy storage level. We implemented HarvNAS using the TensorFlow framework and then implemented and evaluated HarvSched on an MSP430-based sensor device. The evaluation showed that HarvNAS generated a model with up to 2.6%p higher accuracy while saving up to 70% of memory compared to the existing technique, and HarvSched enabled zero-downtime operation of the generated model.

Games are energy-intensive applications on mobile devices. Optimizing the energy efficiency of games is hence critical for battery-limited mobile devices. Although the advent of energy-aware scheduling (EAS) integrated in recent devices has provided opportunities for improved energy management, the framework is not specifically tuned for game applications. In this paper, we aim to improve the energy efficiency of game applications running on EAS-enabled mobile devices. To this end, we first analyze the functional characteristics of games, and investigate the source of the energy inefficiency. We then propose a scheme, called System-level Energy-optimization for Game Applications (SEGA), to improve the energy efficiency of games. SEGA governs CPU and GPU power consumption in a tightly coupled manner by employing three key techniques: (1) Lsync-aware GPU DVFS governor, (2) adaptive capacity clamping, and (3) on-demand touch boosting. We implemented SEGA on the latest Android-based smartphones. The evaluation results for 23 popular games showed that SEGA reduced the energy consumption of the Google Pixel 2 XL and Samsung Galaxy S9 Plus smartphones, at the device level, by 6.1–22.3 and 4.0–11.7 percent, respectively, with a quality of service (QoS) degradation of 1.1 and 0.5 percent, on average.

The quality of experience (QoE) is an important issue for users when accessing the web. Although many metrics have been designed to estimate the QoE in the desktop environment, few studies have confirmed whether the QoE metrics are valid in the mobile environment. In this paper, we ask questions regarding the validity of using desktop-based QoE metrics for the mobile web and find answers. We first classify the existing QoE metrics into several groups according to three criteria and then identify the differences between the mobile and desktop environments. Based on the analysis, we ask three research questions and develop a system, called WebMythBusters, for collecting and analyzing mobile web experiences. Through an extensive analysis of the collected user data, we find that (1) the metrics focusing on fast completion or fast initiation of the page loading process cannot estimate the actual QoE, (2) the conventional scheme of calculating visual progress is not appropriate, and (3) focusing only on the above-the-fold area is not sufficient in the mobile environment. The findings indicate that QoE metrics designed for the desktop environment are not necessarily adequate for the mobile environment, and appropriate metrics should be devised to reflect the mobile web experience.

Unmanned aerial vehicles (UAVs) are often used in mission-critical applications, requiring a critical criterion in flight time. Unfortunately, severe power fluctuations, caused by specific flight patterns, degrade the deliverable capacity of the battery and hamper the flight time. A common approach to mitigating power fluctuations is to employ a hybrid energy storage system using a Li-ion battery with an ultracapacitor (UC). However, the conventional scheme poses inherent problems of low-energy density and power leakage due to the use of the UC and the supplementary hardware required for hybrid storage. In this article, we propose Hydrone, a reconfigurable battery architecture that maximizes the flight time of UAVs, overcoming the previous limitations. Hydrone addresses two key challenges that arise when hybrid energy storage is utilized in UAVs: 1) capacity loss and 2) power leakage. First, the proposed scheme compromises the capacity loss of hybrid storage by using a minimal capacity UC for use as a buffer to counteract the power fluctuations. Second, the power leakage of the hybrid battery is minimized by draining power from the UC only when it is necessary. To this end, the Hydrone architecture provides reconfigurability in hardware and offers two modes of battery operation, i.e., a battery-only mode and a hybrid mode. An appropriate operation is then selected at runtime depending on the flight situation and battery status. To switch modes, we employed a reinforcement learning-based switch control, reflecting the power fluctuation adequately on the flight and battery states. We implemented a hardware prototype to demonstrate the efficiency of Hydrone. Our extensive evaluation shows that the flight time of a UAV is prolonged up to 39% in our experiment setup.

The optimal use of batteries in drones is a critical issue for achieving both reliable operation and maximum flight time. The key is to acquire accurate information about the state of charge (SoC) of the battery in runtime. Drones typically employ series-connected lithium-ion polymer (Li-Po) battery cells, whose SoC is affected by many environmental factors as well as flight patterns. In this article, we propose a scheme, called Optrone, which maximizes the flight time of a drone while safely using the battery. Understanding the implications of the factors affecting the SoC of the drone's battery pack, we propose a three-level SoC, which is a metric for representing the SoC of a battery in runtime. We also provide various operating policies to users to improve the safety and efficiency of operating the drone. We implemented the prototype hardware and software for Optrone, and validated its operation in controlled and real environments. The experimental results in a controlled environment showed that the proposed three-level SoC poses less than 3% error and the operating policies achieved a flight time gain of 19.4%, while guaranteeing battery safety. We also observed a flight time gain of about 10% in real outdoor experiments, where the user rightly adheres to the advised Optrone policy.

Web browsing, previously optimized for the desktop environment, is being fine-tuned for energy-efficient use on mobile devices. Although active attempts have been made to reduce energy consumption, the advent of energy-aware scheduling (EAS) integrated in the recent devices suggests the possibility of a new approach for optimizing energy use by browsers. Our preliminary analysis showed that the existing EAS-enabled system is overly optimized for performance, leading to energy inefficiencies while a web browser is running. In this paper, we analyze the characteristics of web browsers, and investigate the cause of energy inefficiency in EAS-enabled mobile devices. We then propose a system, called WebTune, to improve the energy efficiency of mobile browsers. Exploiting the reinforcement learning technique, WebTune learns the optimal execution speed of the web browser's processes, and adjusts the speed at runtime, thus saving energy and ensuring the quality of service (QoS). WebTune is implemented on the latest Android-based smartphones, and evaluated with Alexa's top 200 websites. The experimental results show that WebTune reduced the device-level energy consumption of the Google Pixel 2 XL and Samsung Galaxy S9 Plus smartphones by 18.7-22.0% and 13.7-16.1%, respectively, without degrading the QoS.

Graphics increasingly play a key role in modern mobile devices. The graphics pipeline requires a close relationship between the CPU and the GPU to ensure energy efficiency and the user's quality of experience (QoE). Our preliminary analysis showed that the current techniques employed to achieve energy efficiency in the Android graphics pipeline are not optimized especially in the frame generation process. In this paper, we aim to improve the energy efficiency of the Android graphics pipeline without degrading the user's QoE. To achieve this goal, we studied the internals of the Android graphics pipeline and observed the energy inefficiency in the existing governing framework of the CPU and GPU. Based on the findings, we propose three techniques for addressing energy inefficiency: (1) aggressively capping the maximum CPU frequency, (2) lowering the CPU frequency by raising the GPU minimum frequency, and (3) allocating the frame rendering-related threads in the energy-efficient CPU cores. These techniques are integrated into a single governing framework, called the GFX Governor, and implemented in the newest Android-based smartphones. Experimental results show that without hampering the user's QoE the average energy consumption of Nexus 6P, Pixel XL, and Pixel 2 XL is reduced at the device level by 24.2%, 18.6%, and 13.7%, respectively, for the 60 chosen applications. We also analyzed the efficacy of the proposed technique in comparison with the state-of-the-art Energy-Aware Scheduling (EAS) implemented in the latest smartphone.

Managing the power consumption of display-related components in mobile devices is difficult because of performance degradation. Therefore, eliminating hidden workloads, such as redundant frames, is preferable, as it directly reduces power without affecting the user experience. Our preliminary study shows that the default launcher of the Android Open Source Project (AOSP) and popular applications, such as Instagram and Pinterest, generate redundant frames. In this paper, we propose a scheme to optimize the power consumption of the smartphone's display-related components by preventing redundant frames generation. By analyzing the frame-generation process, we observe that redundant frame generation is possible in the current Android framework. We then propose a scheme that recognizes and prevents redundant frame generation before actual frame generation (i.e., frame rendering in the GPU). The proposed scheme utilizes a display list, which was introduced in recent Android smartphones for efficient frame generation. We implemented the proposed scheme on Nexus smartphones. On the Nexus 5, the proposed solution reduced the energy of the AOSP default launcher, Instagram, and Pinterest by 40, 35.4, and 39.6 percent, respectively. Furthermore, the experimental results with a general usage scenario showed that our scheme prevented about 35 percent of redundant frame generation with a false-positive rate of 1.8 percent.