We reveal two fundamental insights when deploying inference workloads on intermittent systems that use capacitors as energy buffers. Because of varying structural characteristics and data-independent execution of Deep Neural Network (DNN) models, different capacitor configurations can swing inference performance from the good to the bad. Second, capacitor leakage, which is inevitable in real deployments and yet vastly overlooked in existing literature, makes a whole difference when determining the most efficient capacitor configuration. Both insights hold independently of specific techniques that enable intermittent inference. They form the basis to design LEACS: an off-line technique to determine an efficient energy storage configuration for intermittent inference workloads. LEACS determines the number and size of capacitors based on the nature of the energy source, while accounting for capacitor leakage. We test LEACS together with two state-of-the-art intermittent inference execution techniques and compare its performance with four different baselines across three hardware platforms, seven DNNs models, and three real-world energy sources. Our results indicate that LEACS improves inference throughput by up to 3.4 ×, with an average 1.59 × across the settings we test.

We present a measurement and performance analysis of system-level settings to improve the energy efficiency of Deep Neural Network (DNN) inference on battery-less Internet of Things (IoT) devices. To do so, we deliberately trade a small, controllable reduction in inference accuracy for energy gains. Battery-less IoT devices are severely resource-constrained platforms powered by energy harvesting, where execution becomes intermittent as it alternates between bursts of computation and periods of energy recharge. To survive frequent energy failures, devices persist their system state into non-volatile memories, incurring significant energy costs. We leverage aggressive current scaling offered by Spin-Transfer Torque Magnetic RandomAccess Memory (STT-MRAM) during state writes to reduce energy consumption, intentionally allowing controlled write errors that affect inference outcomes. Through an extensive experimental campaign comprising over 2.2+ trillion data points across 4 microcontroller units (MCUs) and 8 benchmarks, we demonstrate that by tolerating a limited accuracy loss we can obtain up to $40 \%$ energy savings. We release our framework and toolset to foster further research in this emerging design space.

We present INTERCEPT, a compile-time toolchain enabling manifold throughput improvements when running intermittent DNN inference on IoT devices, in exchange of a maximum 1% accuracy loss. Intermittently-computing IoT devices rely on ambient energy harvesting and compute opportunistically, as energy is available. They use NVM to persist intermediate results in anticipation of energy failures. Without requiring changes to existing models and by exploiting the features of STT-MRAM as NVM, INTERCEPT optimizes the placement and configuration of state persistence operations when executing the inference process. This happens off-line with no user intervention, while enforcing a maximum 1% accuracy loss. Our results, obtained across three platforms and six diverse neural networks, indicate that INTERCEPT provides a 40% energy gain in a single inference process, on average. With the same energy budget, this yields a 1.9x throughput speedup.

We explore how hardware approximation techniques can be used to reduce the overhead introduced by state persistence operations in intermittent computing. We do so by exploring the trade-off between energy consumption and quality of the results. We specifically adjust the energy/quality ratio of write operations in Spin Transfer Torque Magnetic Random Access Memory (STT-MRAM) by modifying the current applied during these operations. Our evaluation on a heterogeneus set of benchmarks demonstrates up to ≈50% reduction in the state persistence overhead while maintaining an acceptable output quality.