Achieving high energy efficiency is a primary design objective for multi-core systems. Dynamic voltage and frequency scaling (DVFS) is one of the most widely-adopted low-power techniques. In this paper, we present a reinforcement learning-based DVFS control approach to reduce energy consumption under user-specified performance requirements. The learning agent periodically selects the voltage and frequency level for all cores based on observations of their computation intensiveness, memory behaviors as well as synchronization among cores. Experimental results on multiple real applications show that the proposed method can achieve significant energy reduction.

A lot of efforts have been devoted to optically enabled high-performance communication infrastructures for future manycore processors. Silicon photonic network promises high bandwidth, high energy efficiency and low latency. However, the ever-increasing network complexity results in high optical power demands, which stress the optical power delivery and affect delivery efficiency. Facing these challenges, we propose Ring-based Optical Active Delivery (ROAD) system, to effectively manage and efficiently deliver high optical power throughout photonic-electronic hybrid systems. Experimental results demonstrate up to 5.49X energy efficiency improvement compared to traditional design without affecting processor performance.

Energy efficiency has become a critical design metric for high-performance systems. Various power management techniques have been proposed for the processor cores such as dynamic voltage and frequency scaling (DVFS), whereas few solutions consider the power losses suffered on the power delivery system (PDS), despite the fact that they have a significant impact on the overall energy efficiency of the system. With the explosive growth of system complexity and highly dynamic workloads variations, it is also challenging to find the optimal power management policies which can effectively match the power delivery with the power consumption. In addition, process variations (PVs) add heterogeneity to systems and make traditional power management methods less effective. To tackle the above problems, we propose a reinforcement-learning-based Chip-Specific Power co-Management (CSPM) scheme for PV-aware manycore systems. Both PDS and processor cores are jointly adjusted by distributed agents with modular Q-learning to improve the overall energy efficiency of the system. System characteristics are naturally included in the learning process to obtain chip-specific policies. Experimental results show that when applied to PV-aware manycore systems with a hybrid PDS constructed by both on- and off-chip voltage regulators, the proposed method achieves a 60.1% reduction of the overall energy delay product (EDP) of the system, on average, compared to a traditional DVFS approach.

Minimizing energy consumption while satisfying the user-specified performance requirement is a primary design objective for mobile devices. To achieve this, off-the-shelf mobile devices are usually equipped with dynamic voltage and frequency scaling (DVFS)-enabled processors to tradeoff performance for energy reduction through online adjustment of operating points. Recently, reinforcement learning algorithms have been widely studied and show great potential for runtime DVFS control because of their adaptability to the changing environment. However, the rapid evolution of hardware and the ever-growing diversity of mobile applications increase the system complexity dramatically and make it hard for the learning agent to quickly obtain an efficient power management policy. To address this challenge, we propose a decentralized collaborative Q-learning (DCQL)-based approach in this article to solve the DVFS control problem of multicore mobile processors. By exchanging learning experiences and knowledge among multiple devices, DCQL increases the convergence rate of the learning algorithm and improves the quality of the derived power management policy. On each device, the 4-phase action selection strategy is applied for efficient exploration of the action space defined by various power settings. Experimental results on realistic applications show that DCQL can achieve 3.4%-8.0% energy reduction over various existing approaches while providing an average of 3.4× speedup over the state-of-the-art individual learning algorithm. We also show that the proposed approach can scale well with the number of cores through clustered DVFS control.

Rapidly evolving embedded applications continuously demand more functionalities and better performance under tight energy and thermal budgets, and maintaining high energy efficiency has become a significant design challenge for mobile devices. Learning-based methods are adaptive to dynamic conditions and show great potential for runtime power management. However, with the ever-increasing complexity of both hardware and software, it is a challenging issue for a learning agent to explore the state-action space sufficiently and quickly find an efficient management policy. In this paper, we propose a reinforcement learning-based multi-device collaborative power management approach to address this issue. Multiple devices with different runtime conditions can acquire related knowledge during the learning process. Efficient knowledge sharing among these devices can potentially accelerate the learning process and improve the quality of the learned policies. We integrate the proposed method with dynamic voltage and frequency scaling on the multicore processors in mobile devices. Experimental results on realistic applications show that the collaborative power management can achieve up to a $7 \times$ speedup and 10% energy reduction compared with state-of-the-art learning-based approaches.

Rapidly evolving embedded applications continuously demand more functionality and better performance under tight energy and thermal budgets, and maintaining high energy efficiency has become a significant design challenge for mobile devices. Although learning-based runtime power management can adapt to dynamic conditions, it is a challenging issue to quickly find an efficient management policy under ever-increasing hardware and software complexity. In this work, we propose a multi-device collaborative power management approach to address this issue. The collaborative power management periodically shares knowledge among multiple devices to accelerate the learning process and improve the quality of learned policies. We integrate the proposed method with dynamic voltage and frequency scaling (DVFS) on the multicore processors in mobile devices. Experimental results on realistic applications show that the collaborative power management can achieve on average 8x speedup and 10% energy saving compared with state-of-the-art learning-based approaches.

Maintaining high energy efficiency has become a critical design issue for high-performance systems. Many power management techniques have been proposed for the processor cores such as dynamic voltage and frequency scaling (DVFS). However, very few solutions consider the power losses suffered on the power delivery system (PDS), despite the fact that they have a significant impact on the system overall energy efficiency. With the explosive growth of system complexity and highly dynamic workloads variations, it is challenging to find the optimal power management policies which can effectively match the power delivery with the power consumption. To tackle the above problems, we propose a reinforcement learning-based power management scheme for manycore systems to jointly monitor and adjust both the PDS and the processor cores aiming to improve system overall energy efficiency. The learning agents distributed across power domains not only manage the power states of processor cores but also control the on/off states of on-chip VRs to proactively adapt to the workload variations. Experimental results with realistic applications show that when the proposed approach is applied to a large-scale system with a hybrid PDS, it lowers the system overall energy-delay-product (EDP) by 41% than a traditional monolithic DVFS approach with a bulky off-chip VR.

Battery-powered mobile devices have limited energy capacity, urging the development of efficient power management approaches. Reinforcement learning (RL) algorithms are adaptive to the changing environment and have been widely used for runtime power management. Recently, collaborative RL-based approaches have been explored to accelerate the learning process. Nonetheless, existing works usually require a cloud service provider to achieve centralized multi-device knowledge sharing, which suffers from the single point of failure and does not guarantee users' privacy. To address this issue, we propose a decentralized multi-device collaborative power management approach in this work, where devices directly share their knowledge with their trusted neighbors. Experimental results show that the proposed method can achieve an up to 21% energy reduction with a 4× speedup over the individual learning-based approach.

The power delivery system (PDS), which plays a crucial role in guaranteeing the proper functionality of computing systems, has been a serious constraint on performance due to its significant power loss, especially for high-performance many-core processors. As the PDS design is usually optimized to provide power to the target chip at its best performance level, the energy efficiency can be notably degraded when the workload of the processors is highly dynamic. Therefore, dynamically adjusting the PDS to adapt to the run-time chip workloads are expected to be beneficial. In addition, the introduction of on-chip voltage regulators (VRs) has also significantly broadened the design space of PDS. In this paper, we present a workload-aware quantized power management scheme to dynamically manage the PDS in order to improve system energy efficiency. VRs at different stages are scheduled as online or offline based on chip workload estimation and prediction. The simulation results show that with the proposed scheme, a hybrid PDS with on/off-chip VRs can achieve 74.6% overall efficiency on average, 12.7% higher than a conventional PDS with one off-chip VR. The proposed scheme also shows its potential advantage in improving system energy efficiency with large-scale many-core processors and with more advanced processor technology nodes.

Energy-efficiency is becoming increasingly important to modern computing systems with multi-/many-core architectures. Dynamic Voltage and Frequency Scaling (DVFS), as an effective low-power technique, has been widely applied to improve energy-efficiency in commercial multi-core systems. However, due to the large number of cores and growing complexity of emerging applications, it is difficult to efficiently find a globally optimized voltage/frequency assignment at runtime. In order to improve the energy-efficiency for the overall multicore system, we propose an online DVFS control strategy based on core-level Modular Reinforcement Learning (MRL) to adaptively select appropriate operating frequencies for each individual core. Instead of focusing solely on the local core conditions, MRL is able to make comprehensive decisions by considering the running-states of multiple cores without incurring exponential memory cost which is necessary in traditional Monolithic Reinforcement Learning (RL). Experimental results on various realistic applications and different system scales show that the proposed approach improves up to 28% energy-efficiency compared to the recent individual-RL approach.

The power delivery system (PDS) plays a crucial role of guaranteeing the proper functionality of many-core processors. However, as PDS is usually optimized to provide power to the target chip at its best performance level, its energy efficiency can be seriously degraded under highly dynamic workloads, making it a major source of system power losses. On-chip voltage regulators (VR), which are able to achieve fast and fine-grained power control, have been popular choices for PDS implementation and provided design opportunities for improving system energy efficiency. In this paper, we propose the adaptive Quantized Power Management (QPM) scheme to dynamically adjust the PDS with both on-chip and off-chip VRs based on run-time workloads. Experimental results on different applications show that QPM applied on a hybrid PDS with both on/off-chip voltage regulators (VR) achieves 74.1% average overall energy efficiency, 12.3% higher than the conventional PDS with single off-chip VR.

Exploring the design space of the memory hierarchy requires the use of effective methodologies, tools, and models to evaluate different parameter values. Reuse distance is of one of the locality models used in the design exploration and permits analytical cache miss estimation, program characterization, and synthetic trace generation. Unfortunately, the reuse distance is limited to a single locality granularity. Hence, it is not a suitable model for caches with hybrid line sizes, such as sectored caches, an increasingly popular choice for large caches. In this work, we introduce a generalization to the reuse distance, which is able to capture locality seen at multiple granularities. We refer to it as Hierarchical Reuse Distance (HRD). The proposed model has same profiling and synthesis complexity as the traditional reuse distance, and our results show that HRD reduces the average miss rate error on sectored caches by more than three times. In addition, it has superior characteristics in exploring multi-level caches with conventional single line size. For instance, our method increases the accuracy on L2 and L3 by a factor of 4 and converges three orders of magnitude faster.