In the rapidly evolving domain of the Internet of Things (IoT), this study focuses on enhancing the energy efficiency of Deep Learning accelerators implemented on FPGA-based heterogeneous platforms aligned with the principles of sustainable computing. Diverging from the conventional focus on the inference phase, this research introduces innovative optimizations aimed at minimizing the overhead associated with the FPGA configuration phase. Our investigation achieved a remarkable 40.13-fold reduction in configuration energy each time the FPGA is powered on through precise fine-tuning of configuration parameters. Furthermore, the implementation of our Idle-Waiting strategy significantly reduced the overall energy consumption across requests. Under scenarios with regular request periods, the enhanced Idle-Waiting strategy augmented with power-saving methods, outperforms the traditional On-Off strategy in duty-cycle mode for request periods extending up to 499.06 ms. This enhancement is most pronounced at a 40 ms request period, where it increases the system's operational lifetime by a factor of 12.39 within a 4147 J energy budget. Additionally, this paper introduces an adaptive strategy switching approach to manage scenarios with irregular request periods, employing both predefined and learnable threshold methods. This approach is not only more consistently stable than single-strategy methods but also generally outperforms them. Within this approach, our learnable threshold experiences only a 10% performance drop compared to the future-aware strategy and is at least 6% better than using single-strategy methods. These results underscore the significant potential for deploying more energy-efficient and sustainable systems within IoT applications. Future research will explore the application of these power-saving techniques to a broader spectrum of tasks on diverse FPGA platforms.

Long Short-Term Memory Networks (LSTMs) are pivotal in on-device time series analysis for embedded systems, particularly for managing sensor data streams. Yet, their deployment on resource-constrained embedded devices presents notable challenges. In response, we introduce a novel parameterized architecture for LSTM accelerators designed explicitly for embedded Field-Programmable Gate Arrays (FPGAs). Our approach involves strategic design choices, such as employing computationally efficient activation functions and optimizing clock frequency with a pipelined Arithmetic Logic Unit (ALU). These decisions drive our architecture towards enhanced energy efficiency while maintaining adaptability across diverse application scenarios. A key feature of our architecture is its configurable parameters, which allow for tailored optimization through the optional use of Digital Signal Processor Slices for ALUs and the selective implementation of activation functions. Our empirical evaluations conducted on the Spartan-7 XC7S15 FPGA demonstrate the robustness of our methodology, achieving a 2.33× improvement in energy efficiency over previous solutions. Furthermore, our study examines the correlation between memory resource types and energy efficiency across various LSTM model sizes. Impressively, even with a 9× increase in the hidden size of the LSTM cell, our accelerator maintains an energy efficiency of 10.03 GOP/s/W, with only a minor decrease of 14.65%. However, it is critical to note that our current design is not yet optimized for larger FPGA models such as the Spartan-7 XC7S25 and XC7S50. For these models, timing constraints, rather than resource limitations, pose challenges to scaling, highlighting a potential area for future optimization.

Deep learning (DL) is quickly becoming a core technology to process sensor data from IoT devices. Nowadays, data is usually sent to remote Cloud services where GPU-based ML platforms process it. In contrast, our vision is to have IoT devices with local, on-device DL. This can increase privacy and accessibility as well as reduce processing costs. To do so, specialized hardware accelerators are needed to perform ML operations efficiently without high performance CPUs. In this paper we introduce our approach for a generator for ML hardware accelerators in the IoT. We focus specifically on the inference of ML algorithms for a typical class of IoT use cases, i.e., processing time-series data under real-time constraints. Additionally our optimization techniques rely on the co-design of hardware and ML algorithms to explore the most specialized and efficient corners of the design space. In this work we analyze the requirements for our generator, discuss optimization stages and techniques and show a case study based on an LSTM model for traffic flow prediction.

Abstract Artificial intelligence (AI) is an important part of today’s pervasive computing systems. Still, there is no end-to-end system platform that allows to deploy, update, manage and execute AI models in pervasive systems. We propose such a system platform in this paper. Most importantly, we reuse concepts and techniques from twenty years of pervasive computing research on how to enable runtime adaptation and apply it to AI. This allows to specify adaptive AI models that are able to react to a multitude of dynamic changes, e.g. with respect to available devices, networking conditions, but also application requirements and sensor data sources. Developers can optimise their applications iteratively, starting with a generic setup and refining it step by step towards their specific pervasive computing scenario. To show the applicability of our platform, we apply it to two pervasive use cases and evaluate them, achieving up to four times faster inference and three times lower energy consumption compared to a classical AI deployment.