Abstract Foot‐ground contact information plays a crucial role in character animation and gait analysis, as it helps accurately simulating realistic movement patterns and understanding the biomechanics of walking. Existing motion datasets do not explicitly include foot‐ground contact information, requiring separate computation or manual annotation. Obtaining accurate foot‐ground contact information typically requires additional sensors such as pressure mats or force plates. Without such devices, estimating contact becomes a highly challenging task. We propose ContactVision, a deep learning framework that detects heel and toe contact states directly from video. Our network is trained in a supervised manner using contact labels derived from motion capture data via ground reaction force estimation. This enables training on existing datasets without the need for additional hardware. We demonstrate the utility of our contact detection network in two downstream tasks: gait motion reconstruction and gait analysis. For animation, we incorporate predicted contact labels into a reinforcement learning framework with a two‐segment foot model, enabling realistic foot articulation behavior. For analysis, we estimate clinically relevant gait parameters such as double and single support times, and validate the accuracy against pressure sensor mat data and prior video‐based methods. Our results show competitive performance in both animation and analysis settings. The code is publicly available at github.com/DaeeYong/ContactVision .

Sound event detection (SED) based on on-device machine learning (ML) presents considerable energy challenges for ultra-low-power sensor devices. In this paper, we propose DASH, a duration-aware SED system designed for energy-constrained sensor devices in domestic environments. As repeated inferences for continuous sound events lead to unnecessary energy consumption, DASH aims to minimize unnecessary inferences by predicting the duration of sound events. However, the variability of sound event durations across different environments and scenarios poses a major challenge in developing a responsive yet energy-efficient duration-aware SED system. To address this, DASH introduces three key solutions: (1) N-probability distribution-based event duration prediction, which identifies checkpoints where new inferences are likely needed; (2) Affinity-guided event classification, which performs low-energy affinity matching at checkpoints to determine whether ML inference is necessary; and (3) Interrupt blocking-enabling cycle-based device state control, which periodically checks for event presence with minimal energy consumption at non-checkpoint times. We implemented DASH on MSP430-based sensor devices deployed in real home environments. Experimental results demonstrate that DASH reduced energy consumption by approximately 97–98% compared to evaluation baselines, with only a 4.7% error rate.

Sound event detection (SED) using lightweight sensor device has recently gained attention as a practical means to capture context and activities especially in domestic environments. However, SED applications running on sensor device are severely constrained by device’s energy capacity. One solution is to offload a portion of inference to server for reducing runtime complexity, i.e., energy consumption, of sensor device. Offloading should consider the trade-off between computation and data transmission costs adequately; more computation on sensor device reduces data to be transmitted and vice versa. To address this challenge, we propose SEDAC (Sound Event Detection with Attention-based audio Compression), a novel technique for split learning in SED that compresses data from sensor device to offload less data. SEDAC compresses the input of SED models, or Mel spectrograms, with minimal computation in sensor device. Rather than directly compressing the input, SEDAC achieves data compression by selectively capturing the key parts of sound events using an attention mechanism. The scheme also modifies an existing loss function and employs knowledge distillation to mitigate potential loss of SED accuracy due to data compression. Our evaluation shows that SEDAC outperforms the state-of-the-art data compressive split learning schemes, up to about 30%. Furthermore, our real-world deployment demonstrates that sensor devices with SEDAC successfully operate with minimal energy and memory overhead.

In e-scooter sharing systems, users randomly select and use e-scooters based on inaccurate battery information. This simple rental policy leads to low profitability on two fronts. First, inaccurate battery information causes unexpected device shutdowns, causing negative user experiences and refunds. Second, randomly selected e-scooters increase operation costs for battery management. In this paper, we propose e-scooter recommendation system, EcoRide, which provides accurate battery estimation and profitable e-scooter selection to maximize profitability of sharing systems. To this end, we propose a battery estimation considering four factors, i.e., battery state, temperature, user weight, and road slope, that affect the available battery energy in e-scooter applications. We define a parameter, dynamic voltage threshold (DVT), to represent dynamically changing battery energy, and use it to estimate battery availability. Next, to achieve cost-effective e-scooter selection, we introduce a multi-agent reinforcement learning (MARL)-based technique to learn policies that minimize operation costs. We define sharing system operation as a MARL problem with an objective function based on battery management costs. To cope with unstable training due to a wide service area and multiple requests, a centralized training technique is adopted. The proposed battery estimation and e-scooter selection technique are validated through actual driving tests and a sharing system simulator, respectively. Additionally, our case study using open data from Washington D.C. demonstrates a profit gain of up to 68% with EcoRide.

The capability of energy harvesting application powered by indoor photovoltaic energy is severely affected by dynamic light environments. Accordingly, accurate understanding of the target environment and deploying energy harvesting sensors is practically very hard. In this article, we propose HarvAR, which manages photovoltaic energy harvesting sensors with mobile augmented reality (AR)-empowered techniques. HarvAR utilizes the error-prone RGBD data of mobile device to construct a digital twin (DT), performing depth error compensation and estimating the optical properties of the target space. Using the DT, the proposed system predicts the harvesting capability with low overhead, and recommends adequate locations for installing or relocating harvesting sensors. We implemented the HarvAR system and evaluated its accuracy and efficiency in three indoor environments. Our experiments show that DT configuration and harvesting prediction can be performed in minutes, compared to over 10 h using existing techniques, and harvesting prediction is provided with less than 20% error.

Energy-neutral operation (ENO) aims to provide near-perpetual device operation using energy harvested from ambient environments. Existing ENO techniques, however, have two key problems. The batteries used in harvesting devices have inherently limited lifespans, and the device experiences a long cold-start time when charging the battery. In this paper, we propose a long-lasting and highly responsive batteryless system, called RENO, to solve the problems that occur in energy harvesting devices. Using a supercapacitor to store energy, RENO maximizes the responsiveness in ENO especially running in dynamic harvesting environments such as indoor light energy harvesting. Combining the intermittent characteristics of power-neutral operation (PNO), RENO allows dual-mode operation of PNO and ENO, depending on the current harvesting capability. The device works as a PNO device when charging the energy storage, solving the ENO cold-start issue, while the harvested energy is efficiently managed with ENO. For this dual-mode operation, RENO provides hardware and software that handle the switch between PNO and ENO effectively. Application developers are provided with a well-defined API, which enables energy-efficient development of applications without detailed knowledge of the target hardware. Using the API, developers simply declare a task to be executed as either PNO or ENO, and the rest is handled by the system. The prototype system is implemented, and its functionality is evaluated in controlled environments. We also validate the proposed system with two real-world applications, proving the efficacy of dual-mode batteryless operation.

Duty cycle management is critical for the energy-neutral operation of batteryless devices. Many efforts have been made to develop an effective duty cycling method, including machine-learning-based approaches, but existing methods can barely handle the dynamic harvesting environments of batteryless devices. Specifically, most machine-learning-based methods require the harvesting patterns to be collected in advance, as well as manual configuration of the duty-cycle boundaries. In this article, we propose a configuration-free duty cycling scheme for batteryless devices, called CTRL, with which energy harvesting nodes tune the duty cycle themselves adapting to the surrounding environment without user intervention. This approach combines reinforcement learning (RL) with a control system to allow the learning algorithm to explore all possible search space automatically. The learning algorithm sets the target State of Charge (SoC) of the energy storage, instead of explicitly setting the target task frequency at a given time. The control system then satisfies the target SoC by controlling the duty cycle. An evaluation based on the real implementation of the system using publicly available trace data shows that CTRL outperforms state-of-the-art approaches, resulting in 40% less frequent power failures in energy-scarce environments while achieving more than ten times the task frequency in energy-rich environments.

Transiently-powered devices rely solely on energy harvesters. Such devices typically use capacitors to store the harvested energy, but recent systems employ supercapacitors to store energy for extended operations. In a supercapacitor-based transiently-powered system, the energy-efficient estimation of the state-of-charge (SoC) of a supercapacitor is critical for practical use of the system, due to its tight energy budget. Conventional voltage-based schemes for capacitors do not provide accuracy in SoC estimation for supercapacitors. Also, the supercapacitor-specific SoC estimation which exploits its charge redistribution characteristics provides an accuracy, but the scheme demands significant overhead, and thus, is not applicable to transiently-powered systems, such as wireless sensor nodes. In this article, we specify three requirements for estimating supercapacitor SoC that should be met to function in transiently-powered systems. We then propose a scheme that meets those requirements. The proposed scheme does not require additional hardware, has a low computation cost, and operates when the system is intermittently powered. Thus, the scheme fits the energy-efficient operation of transiently-powered systems. We implemented the proposed scheme in real hardware and evaluated its functionality and accuracy. The proposed scheme estimated the SoC of supercapacitors with high accuracy, in various configurations and use scenarios, while guaranteeing operations in a typical energy harvesting environment.

Mobile users are always demanding extended availability in their battery use. Together with enlarged battery capacity, fast charging is one approach that provides an improved user experience in battery use. Recently, device manufacturers have been developing a variety of fast charging techniques for mobile devices. However, the existing techniques severely reduce the charging power when the device is in use while charging. We experimentally demonstrate that the primary cause of the reduction in the charge speed during device use is to cope with the performance degradation incurred by heat generation. We then propose an adaptive charging scheme, called Always-on Quick Charging, which enables fast charging especially when the device is in use. The key idea of our approach is to adjust the charging power while ensuring that the heat generated by the charging does not affect the performance. The proposed scheme is implemented in Google's Pixel 2XL smartphone. The experiment with a real-world usage scenario shows that the charging speed of the proposed scheme is up to 2.4 times faster than the default scheme, while preserving device performance.