Compute-in-memory (CIM) has emerged as a promising paradigm for accelerating neural networks (NNs), offering high parallelism and superior energy efficiency. To fully leverage the potential of CIM accelerators for deploying NNs, it is essential to carefully optimize their mapping dataflows, which determine how computations and data are scheduled across CIM resources over space and time. However, the diversity of CIM architectures, with their varying hardware parameters and mapping constraints, makes it challenging to develop a unified framework that efficiently optimizes dataflows across different CIM systems. Most existing CIM dataflow frameworks are tailored to a limited range of architectures or dataflow types, and often neglect CIM-specific features such as weight-update scheduling, which limits their effectiveness and generality. In this work, we propose TriCIM, a general dataflow optimization framework designed to adapt to diverse CIM architectures. We begin by introducing a CIM-capacity-aware formulation that classifies the CIM dataflow space into three distinct regions, model-stationary, layer-stationary, and tile-stationary, based on the available CIM capacity and the model size. For each dataflow region, we identify specific bottlenecks and propose tailored optimization strategies, including load balancing, layer grouping, weight-update scheduling, tiling, and inter-tile ordering, to address region-specific bottlenecks and enhance dataflow efficiency. Our evaluation across various CIM accelerators and representative NN models demonstrates that TriCIM consistently finds optimal dataflows, achieving$1.1\times $to$13.2\times $runtime speedup compared to state-of-the-art frameworks.

Computing-In-memory (CIM) processors have emerged as a promising approach for accelerating deep neural networks (DNNs). To unleash the potential of diverse CIM architectures, it is essential to develop a tailored compiler that is aware of both the architectural parameters of CIM processors (e.g., CIM array size, number of CIM units, on-chip buffer capacity) and their dataflow characteristics (e.g., CIM parallelism, dataflow scheduling, buffer allocation). However, existing CIM compilers primarily focus on hardware parameters, while neglecting the diversity of dataflow patterns across different architectures. In this paper, we propose CIMWise, a general, IREE-based, end-to-end AI compiler for CIM processors. At its core, an auto-tuning framework has been developed, comprising a comprehensive search space (including both hardware parameters and dataflow characteristics), an analytical cost model (integrating an adaptable on-chip memory model), and a two-stage simulated annealing search algorithm, to explore optimal scheduling strategies for the target workload on CIM processors. Additionally, CIMWise leverages IREE’s frontend for graph-level optimization and implements a customized backend to perform operator-level optimization using the proposed auto-tuning framework, enabling seamless end-to-end deployment of neural networks on CIM processors. The proposed CIMWise is validated through actual hardware measurements. Experimental results show that CIMWise achieves up to a 58% reduction in energy consumption and a 19% decrease in latency compared to prior CIM compilers.

Thermal/power issues have become increasingly important with more and more transistors being placed on a single chip. Many dynamic thermal/power management techniques have been proposed to address such issues but they all depend heavily on accurate knowledge of the chip's thermal state during runtime. In this paper, we describe a unified statistical framework for designing an on-chip thermal sensing infrastructure that can be used to track the chip's thermal state at runtime. Specifically, we address the following problems in this statistical framework: 1) sensor placement; 2) sensor data compression; 3) sensor data fusion; and 4) overall interplay. Our methods exploit the correlations between temperatures in different parts of the chip to drive sensor placement, data compression, and data fusion in both noiseless and noisy sensor cases. Our framework is also capable of choosing the appropriate degree of compression for each sensor while accounting for their local space constraints during deployment. The experimental results show that the root-mean-square error of the thermal estimates produced by our sensing infrastructure is on average 35% better than an equivalent system that uses a range-based placement scheme and a uniform compression scheme. It took our methods at most about 9 s to decide the overall solution for placement, compression, and data fusion at the design stage. This demonstrates the effectiveness and applicability of our unified statistical design methodology.

In this paper, we investigate dynamic thermal management (DTM) policies under soft thermal constraint that allow the thermal constraint to be violated occasionally for boosting system performance. First, we investigate soft-constraint DTM using lumped radio control (RC) thermal models. We develop analytical expressions for the optimal core frequency policies that maximize overall performance under soft thermal constraint for both single-core and homogeneous multicore processors. We then generalize the problem to heterogeneous multicore processor and use a more accurate distributed RC thermal model to account for the spatial thermal variation. The generalized problem also takes into account the impact of increased temperature on transistor delay and leakage power. The problem is solved by convex optimization. Experimental results indicate that for a two-core processor, a mere 10 °C increase in the core temperature for 100 s results in about 30% performance gain.

In this paper, we investigate the gate sizing problem and develop techniques for improving the runtime by effectively exploiting the graphics processing unit (GPU) resources. Theoretically, we investigate a randomized cutting plane-based convex optimization technique which is highly parallelizable and can effectively exploit the single instruction multiple data structure imposed by GPUs. In order to further improve the runtime, we also develop GPU-oriented implementation guidelines that exploit the specific structure that convex gate sizing formulations impose. We implemented our method on NVIDIA Tesla 10 GPU and obtain 21× to 400× speedup compared to the MOSEK optimization tool implemented on conventional CPU. The quality of solution of our method is very close to that achieved by MOSEK, since both are optimal.

Multicore system-on-chips (SOCs) rely on runtime thermal monitoring using on-chip thermal sensors for dynamic thermal management (DTM). However, on-chip sensors are highly susceptible to noise due to fabrication randomness,VDDfluctuations, etc. This causes discrepancy between the actual temperature and the one observed by thermal sensor. In this paper, we address the problem of estimating the accurate temperature of on-chip thermal sensor when the sensor reading has been corrupted by noise. We present statistical techniques for the following: 1) when the underlying randomness exhibits jointly-Gaussian characteristics we present the optimal solution for temperature estimation; 2) for close to Gaussian cases we give a heuristic based on Moment Matching; 3) when the underlying randomness is non-Gaussian a hypothesis testing framework is used to predict the sensor temperatures. The previous three techniques are investigated in both single sensor and multisensor scenarios, respectively. The latter tries to estimate the actual temperatures for several sensors simultaneously while exploiting the correlations in temperature and circuit parameters among different sensors. The experiments showed that using our estimation schemes the root mean square (RMS) error can be reduce (with very small runtime overhead) by 71.5% as compared to blindly trusting the sensors to be noise-free.

Many DTM schemes rely heavily on the accurate knowledge of the chip's dynamic thermal state to make optimal performance/temperature trade-off decisions. This information is typically generated using a combination of thermal sensor inputs and various estimation schemes such as Kalman filter. A basic assumption used by such schemes is that the statistical characteristics of the power consumption do not change. This is problematic since such characteristics are heavily application dependent. In this paper, we first present autonomous schemes for detecting the change in the statistical characteristics of power and then propose adaptive schemes for capturing such new statistical parameters dynamically. This could enable accurate temperature estimation during runtime given dynamically changing power statistical states. Our schemes use a combination of hypothesis testing and residual whitening methods and can improve the accuracy by 67% as compared to the traditional non-adaptive schemes.

In this paper, we investigate Dynamic Thermal Management (DTM) policies under soft thermal constraint that allows the thermal constraint to be violated for a user specified period. For single core processor, we develop analytical expression for the optimal frequency policy under the soft constraint such that maximal performance can be extracted. We extend this problem to multi-core processor and provide optimal frequency policy when all cores run at same frequency. We also present LP based approximated formulation that generates frequency policies where each core has separate frequency control and considers leakage power. We use frequency legalization to approximate the frequency into discrete values. Experimental results indicate that 10 degree celsius increase in core temperature for 100sec results in 13% performance gain for single core processor, and 30% performance gain for two-core processor. Without Tmax constraint, the performance improves almost 100% for two-core processor.

Thermal/power issues have become increasingly important with more and more transistors being put on a single chip. Many dynamic thermal/power management techniques have been proposed to address such issues but they all heavily depend on accurate knowledge of the chip's thermal state during runtime. In this paper we describe a unified statistical framework for designing an on-chip thermal sensing infrastructure which can be used to track the chip's thermal state at runtime. Specifically we address the following problems: (1)sensor placement; (2)sensor data compression; (3)sensor data fusion; (4)overall interplay. Our methods exploit the thermal correlation to generate the overall solution in both the noiseless and noisy sensor settings. Our framework is also capable of choosing the appropriate degree of compression for each sensor while accounting for their local space constraints when doing the sensor deployment. The experimental results showed that our infrastructure can improve the temperature estimation accuracy by 27% (on average) as compared to an equivalent system that uses range-based placement and uniform compression. It took our methods about 6.3 seconds to decide the overall solution for placement, compression and data fusion at design stage. This demonstrates the effectiveness and applicability of our unified statistical design methodology.

This article addresses the problem of chip-level thermal profile estimation using runtime temperature sensor readings. We address the challenges of: (a) availability of only a few thermal sensors with constrained locations (sensors cannot be placed just anywhere); (b) random chip power density characteristics due to unpredictable workloads and fabrication variability. Firstly we model the random power density as a probability density function. Given such statistical characteristics and the runtime thermal sensor readings, we exploit the correlation in power dissipation among different chip modules to estimate the expected value of temperature at each chip location. Our methods are optimal if the underlying power density has Gaussian nature. We give a heuristic method to estimate the chip-level thermal profile when the underlying randomness is non-Gaussian. An extension of our method has also been proposed to address the dynamic case. Several speedup strategies are carefully investigated to improve the efficiency of the estimation algorithm. Experimental results indicated that, given only a few thermal sensors, our method can generate highly accurate chip-level thermal profile estimates within a few milliseconds.

Multicore SOCs rely on runtime thermal measurements using on-chip sensors for DTM. In this paper we address the problem of estimating the actual temperature of on-chip thermal sensor when the sensor reading has been corrupted by noise. Thermal sensors are prone to noise due to fabrication randomness, VDD fluctuations etc. This causes discrepancy between actual temperature and the one predicted by thermal sensor. Our experiments estimate this variation to be around 30%. In this paper we present a statistical methodology for predicting the actual temperature for a given sensor reading. We present two techniques: single sensor prediction and multi-sensor prediction. The latter tries to estimate the actual temperature for each sensor (of the many on-chip sensors) simultaneously while exploiting the correlations between temperature and noise of different sensors. When the underlying randomness follows a Gaussian characteristic, we present optimal schemes of estimating the expected temperature. We also present heuristic schemes for the case where the Gaussian assumption fails to hold. The experiments showed that using our estimation schemes the RMS error can be reduce as much as 67% as compared to blindly trusting the sensors to be noise free.

This paper addresses the problem of chip level thermal profile estimation using runtime temperature sensor readings. We address the challenges of a) availability of only a few thermal sensors with constrained locations (sensors cannot be placed just anywhere) b) random on-chip power density characteristics due to unpredictable workloads and fabrication variability. Firstly we model the random power density as a probability density function. Given this random characteristic and runtime thermal sensor readings, we exploit the correlation between power dissipation of different chip modules to estimate the expected value of temperature at each chip location. Our methods are optimal if the underlying power density has Gaussian nature. We also present a heuristic to generate the chip level thermal profile estimates when the underlying randomness is non-Gaussian. Experimental results indicate that our method generates highly accurate thermal profile estimates of the entire chip at runtime using only a few thermal sensors.

A novel technique of visual-speech signal representations on X-ray motion films is proposed. The graphic implementation of the dynamic model is reconstructed from videotaped midsagittal X-ray motion films of the articulators during natural speech production of standard Chinese. An intelligent image processing technique is presented for quantizing the articulatory space and for describing the articulatory movements. The authors process separately counters of the motion picture by means of an edge detection and edge displacement analysis, which leads to a new class of intelligent coding methods capable of achieving compression ratio as high as 64:1. Based on this model a computer-aided instruction system of standard-Chinese language is established, in which the amount of data for 107 syllables is as low as 5 kbytes. The CAI system is briefly introduced.>