Deep Brain Stimulation (DBS) is a highly effective treatment for Parkinson's Disease (PD). Recent research uses reinforcement learning (RL) for DBS, with RL agents modulating the stimulation frequency and amplitude. But, these models rely on biomarkers that are not measurable in patients and are only present in brain-on-chip (BoC) simulations. In this work, we present an RL-based DBS approach that adapts these stimulation parameters according to brain activity measurable in vivo. Using a TD3 based RL agent trained on a model of the basal ganglia region of the brain, we see a greater suppression of biomarkers correlated with PD severity, compared to modern clinical DBS implementations. Our agent outperforms the standard clinical approaches in suppressing PD biomarkers while relying on information that can be measured in a real world environment, thereby opening up the possibility of training personalized RL agents specific to individual patient needs.

While distributed, neural-network-based resource controllers represent the state of the art for their ability to cope with the ever-expanding decision space, such approaches suffer from several limitations, like conflicting control decisions and partial observability. These effects can significantly impair the controllers’ learning capabilities and the stability of their control policies, causing substantial performance losses. We are the first to solve this problem employing a centralized training and decentralized control regime to mitigate the aforementioned limitations. Specifically, we design a centralized neural network (critic) that evaluates the behavior of multiple decentralized neural controllers (actors) in a system-wide context. The objective of our proposed technique is to maximize the performance under a temperature constraint through dynamic voltage frequency scaling. The evaluation of our technique shows its superiority over the state of the art, yielding average (peak) performance improvements of 20% (34%), which we consider a breakthrough as the gains are measured on a real-world platform.

Reinforcement learning (RL) holds great promise for adaptively optimizing microprocessor performance under power constraints. It allows for online learning of application characteristics at runtime and enables adjustment to varying system dynamics such as changes in the workload, user preferences or ambient conditions. However, online policy optimization remains resource-intensive, with high computational demand and requiring many samples to converge, making it challenging to deploy to edge devices. In this work, we overcome both of these obstacles and present federated power control using dynamic voltage and frequency scaling (DVFS). Our technique leverages federated RL and enables multiple independent power controllers running on separate devices to collaboratively train a shared DVFS policy, consolidating experience from a multitude of different applications, while ensuring that no privacy-sensitive information leaves the devices. This leads to faster convergence and to increased robustness of the learned policies. We show that our federated power control achieves 57 % average performance improvements over a policy that is only trained on local data. Compared to a state-of-the-art collaborative power control, our technique leads to 22 % better performance on average for the running applications under the same power constraint.

Games are one of the most popular and at the same time most computation intensive and energy consuming class of applications on mobile devices like smartphones and tablets. Dynamic voltage and frequency scaling (DVFS) is a common technique for reducing the processing power. However, highly variable and non-deterministic workload characteristics of mobile games mandate sophisticated workload prediction models to predict low-utilization phases of games during which the processor's frequency can be decreased to save energy. While prior works exhibit significant improvements, one main question is left open: How large is the gap between the developed techniques and the theoretically optimal power manager, i.e., a power manager which exactly knows the future workload and, hence, can select the optimal sequence of frequencies that minimizes the power consumption under given timing constraints. In this paper, we discuss that estimating the savings from such an optimal power manager is non-trivial due to the non-deterministic nature of games and the underlying system. In order to address this, we suggest a statistical model of the optimal power manager using which we estimate the potential savings of popular closed-source games. The results of our work have several implications: We reveal a significant gap between savings obtained from recently proposed game power managers and the theoretically optimum savings (up to 54.4% energy savings are possible). Our work strongly motivates future research endeavors to minimize the gap between the optimum and the existing power managers.

Runtime power management using dynamic voltage and frequency scaling (DVFS) has been extensively studied for video processing applications. But there is only a little work on game power management although gaming applications are now widely run on battery-operated portable devices like mobile phones. Taking a cue from video power management, where PID controllers have been successfully used, they were recently applied to game workload prediction and DVFS. However, the use of hand-tuned PID controller gains on relatively short game plays left open questions on the robustness of the controller and the sensitivity of prediction quality on the choice of the gain values. In this paper, we try to systematically answer these questions. We first show that from the space of PID controller gain values, only a small subset leads to good game quality and power savings. Further, the choice of this set highly depends on the scene and the game application. For most gain values the controller becomes unstable, which can lead to large oscillations in the processor’s frequency setting and thereby poor results. We then study a number of time series models, such as a Least Mean Squares (LMS) Linear Predictor and its generalizations in the form of Autoregressive Moving Average (ARMA) models. These models learn most of the relevant model parameters iteratively as the game progresses, thereby dramatically reducing the complexity of manual parameter estimation. This makes them deployable in real setups, where all game plays and even game applications are not a priori known. We have evaluated each of these models (PID, LMS, and ARMA) for a variety of games—ranging from Quake II to more recent closed-source games such as Crysis, Need for Speed—Shift and World in Conflict—with very encouraging results. To the best of our knowledge, this is the first work that systematically explores (a) the feasibility of manually tuning PID controller parameters for power management, (b) time series models for workload prediction for gaming applications, and (c) power management for closed-source games.

Today high-end cars have extremely complex E/E architectures -- with 50--100 electronic control units (ECUs), connected by communication buses like CAN, FlexRay and Ethernet. They are used to run several (control) applications with many million lines of code. We propose a radically new architecture where all these applications are instead run on a mobile phone being carried by the driver. The car now has a considerably simpler architecture with few or no ECUs, using RF links to connect sensors and actuators to the mobile phone with a powerful multicore processor. We discuss the advantages and challenges and describe a small prototype implementation with an adaptive cruise control application.

Compute intensive games currently represent the class of most popular and at the same time most power consuming applications on mobile phones. To reduce the power consumption of games we have developed a game state specific power management technique. Games typically consist of several states such as the game loading, main menu, in-game menu and gaming state. Each of these states has its specific processing requirements, e.g., the game loading state is likely to be memory bound and menu scenes are less interactive than gaming states and hence do not require high frame rates to satisfy the user's perception. Our game state specific governor (i) recognizes these game states by intercepting and analyzing calls made by the game application to the graphics library, and (ii) exploits these state-specific characteristics to enable power management strategies targeted to these individual states at runtime. Thereby, we achieve significant power savings of up to 50.8% compared to Android's default interactive governor.

While dynamic voltage and frequency scaling (DVFS) based power management has been widely studied for video processing, there is very little work on game power management. Recent work on proportional-integral-derivative (PID) controllers fro predicting game workload used hand-turned PID controller gains on relatively short game plays. This left open questions on the robustness of the PID controller and how sensitive the prediction quality is on the choice of the gain values, especially for long game plays involving different scenarios and scene changes. In this paper we propose a Least Mean Squares (LMS) Linear Predictor, which is a regression model commonly used for system parameter identification. Our results show that game workload variation can be estimated using a linear-in-parameters (LIP) model. This observation dramatically reduces the complexity of parameter estimation as the LMS Linear Predictor learns the relevant parameters of the model iteratively as the game progresses. The only parameter to be tuned by the system designer is the learning rate, which is relatively straightforward. Our experimental results using the LMS Linear Predictor show comparable power savings and game quality with those obtained from a highly-tuned PID controller.