Embedding artificial intelligence onto low-power devices is a challenging task that has been partially overcome by recent advances in machine learning and hardware design. Currently, deep neural networks can be deployed on embedded targets to perform various tasks such as speech recognition, object detection or human activity recognition. However, it is still possible to optimize deep neural networks on embedded devices. These optimizations mainly concern energy consumption, memory and real-time constraints, but also easier deployment at the edge. In addition, there is still a need for a better understanding of what can be achieved for different use cases. This work focuses on the quantization and deployment of deep neural networks on low-power 32-bit micro-controllers. In this article, the quantization method used is based on solving an integer optimization problem derived from the neural network model and concerning the accuracy of the computations and results at each point of the network. We evaluate the performance of our quantization method on a collection of neural networks measuring the analysis time and time-to-solution improvement between the floating- and fixed-point networks, considering a typical embedded platform employing a STM32 Nucleo-144 microcontroller.

In this article, we present a new method for implementing a neural network whose weights are floating-point numbers on a fixed-point architecture. The originality of our approach is that fixed-point formats are computed by solving an integer optimization problem derived from the neural network model and concerning the accuracy of computations and results at each point of the network. Therefore, we can bound mathematically the error between the results returned by the floating-point and fixed-point versions of the network. In addition to a formal description of our method, we describe a prototype that implements it. Our tool accepts the most common neural network layers (fully connected, convolutional, max-pooling, etc.), uses an optimizing SMT solver to compute fixed-point formats and synthesizes fixed-point C code from the Tensorflow model of the network. Experimental results show that our tool is able to achieve performance while keeping the relative numerical error below the given tolerance threshold. Furthermore, the results show that our fixed-point synthesized neural networks consume less time and energy when considering a typical embedded platform using an STM32 Nucleo-144 board.

The implementation of control algorithms typically starts with the development of executable models and prototype implementations, e.g. in C, running on desktop computers before being ported to the target embedded architecture. Often, this latter architecture uses fixed-point arithmetic that differs in terms of accuracy from the floating-point arithmetic used by the desktop computer. In this article, we show that our POPiX tool is capable of automatically transforming floating-point codes into fixed-point ones while preserving the functional properties of the original control algorithms and optimizing resources in terms of memory and power consumption. We experiment POPiX on two widely used algorithms: a PID controller and a Kalman filter. Our experimental results validate, at the functional level, the code generation performed automatically by POPiX.

Precision tuning or customized precision number representations is emerging, in these recent years, as one of the most promising techniques that make it possible to save the resources on the available processors. In contrast to the uni-form precision, mixed precision tuning assigns different finite-precision types to each variable and arithmetic operation of a program and offers many additional optimization opportunities. However, this technique introduces new challenge related to the cost of operations or type conversions which can overload the program execution after tuning. In this article, we extend our tool POP, with efficient ways to limit the number of drawbacks of mixed precision and to achieve best compromise between performance and memory consumption. The results of our evaluation are discussed on a well-known set of tests from the FPBench suite.