Many computer vision tasks use convolutional neural networks (CNNs). These networks have a significant computational cost and complex implementations, in particular on embedded systems. A common way to implement CNNs on integrated circuits is to use low-precision quantized weights and activations instead of de facto floating-point (FP) ones. This is important to reduce the implementation cost. However, this has drawbacks regarding accuracy, and Quantization-Aware Training (QAT) is one of the most popular approaches to mitigate this issue. In this article, we introduce a multiplierless-aware training approach that significantly reduces hardware resource consumption. We propose to incrementally fix weights to their current value based on their implementation cost. To compute this cost, we base our approach on a Multiple Constant Multiplication (MCM) shift-and-add solving technique. With this idea, we show a global implementation cost reduction by around 25% w. r. t. a vanilla QAT approach without hardware usage in the loop. Compared to state-of-the-art multiplierless-aware training methods, the network accuracy of our designs is closer to that of a vanilla QAT baseline.

International audience

Multiple Constant Multiplication (MCM) over integers is a frequent operation arising in embedded systems that require highly optimized hardware. An efficient way is to replace costly generic multiplication by bit-shifts and additions, i.e. a multiplierless circuit. In this work, we improve the state-of-the-art optimal approach for MCM, based on Integer Linear Programming (ILP). We introduce a new low-level hardware cost metric, which counts the number of one-bit adders and demonstrate that it is strongly correlated with the LUT count. This new model permitted us to consider intermediate truncations that permit to significantly save resources when a full output precision is not required. We incorporate the error propagation rules into our ILP model to guarantee a user-given error bound on the MCM results. The proposed ILP models for multiple flavors of MCM are implemented as an open-source tool and, combined with an automatic code generator, provide a complete coefficient-to-VHDL flow. We evaluate our models in extensive experiments, and propose an in- depth analysis of the impact that design metrics have on synthesized hardware.

Many algorithms from digital signal processing, including digital filters or discrete transforms, require the multiplications with several constants. These can be efficiently implemented multiplierless by using additions, subtractions, and bit-shifts. Finding a multiplierless solution with minimal cost is known as the multiple constant multiplication (MCM) problem. Usually, not the full precision is required at the output. The state-of-the-art approaches consist in finding an MCM solution first, and truncating it in a second step. In this work, we solve the MCM problem with minimal number of full adders for truncated outputs. By combining the two steps into a global optimization problem, modeled through mixed-integer linear programming, we are able to reduce the number of full adders by 60% in best cases and by 12% on average. Our method has shown its efficiency on more than 80 instances from literature and permits a fast improvement of state-of-the-art results in most of the cases.