Bayesian Optimization Mixed-Precision Neural Architecture Search (BOMP-NAS) is a method to quantizationaware neural architecture search that leverages both Bayesian optimization and mixed-precision quantization to efficiently search for compact, high performance deep neural networks. It is able to find neural networks that achieve state of the art accuracy with less search time. Compared to the closest related work, BOMP-NAS can find these neural networks in$\mathbf{6}\times$less search time.

Optimizing the efficiency of neural networks is cru-cial for ubiquitous machine learning on the edge. However, it requires specialized expertise to account for the wide variety of applications, edge devices, and deployment scenarios. An attractive approach to mitigate this bottleneck is Neural Architecture Search (NAS), as it allows for optimizing networks for both efficiency and task performance. This work shows that including hyperparameter optimization for training-related parameters alongside NAS enables substantial improvements in efficiency and task performance on a predictive maintenance task. Furthermore, this work extends the combination of NAS and hyperparameter optimization with INT8 quantization to enhance efficiency further. Our combined approach, which we refer to as Quantization-Aware NAS (QA-NAS), allows for further improvements in efficiency on the predictive maintenance task. Consequently, our work shows that QA-NAS is a promising research direction for optimizing neural networks for deployment on resource-constrained edge devices in industrial applications.

Convolutional Neural Networks have become state of the art for many computer vision tasks. However, the size of Neural Networks prevents their application in resource constrained systems. In this work, we present a lossy compression technique for intermediate results of Convolutional Neural Networks. The proposed method offers guaranteed compression rates and additionally adapts to performance requirements. Our experiments with networks for classification and semantic segmentation show, that our method outperforms state-of-the-art compression techniques used in CNN accelerators.

To host intelligent algorithms such as Deep Neural Networks on embedded devices, it is beneficial to transform the data representation of neural networks into a fixed-point format with reduced bit-width. In this paper we present a novel quantization procedure for parameters and activations of pre-trained neural networks. For 8 bit linear quantization, our procedure achieves close to original network performance without retraining and consequently does not require labeled training data. Additionally, we evaluate our method for power-of-two quantization as well as for a two-hot quantization scheme, enabling shift-based inference. To underline the hardware benefits of a multiplierless accelerator, we propose the design of a shift-based processing element.

Hardware accelerators for compute intensive algorithms such as convolutional neural networks benefit from number representations with reduced precision. In this paper, we evaluate and extend a number representation based on power-of-two quantization enabling bit-shift-based processing of multiplications. We found that weights of a neural network can either be represented by a single 4 bit power-of-two value or with two 4 bit values depending on accuracy requirements. We evaluate the classification accuracy of VGG-16 and ResNet50 on the ImageNet dataset with weights represented in our novel number format. To include a more complex task, we additionally evaluate the format on two networks for semantic segmentation. In addition, we design a novel processing element based on bit-shifts which is configurable in terms of throughput (4 bit mode) and accuracy (8 bit mode). We evaluate this processing element in an FPGA implementation of a dedicated accelerator for neural networks incorporating a 32-by-64 processing array running at 250 MHz with 1 TOp/s peak throughput in 8 bit mode. The accelerator is capable of processing regular convolutional layers and dilated convolutions in combination with pooling and upsampling. For a semantic segmentation network with 108.5 GOp/frame, our FPGA implementation achieves a throughput of 7.0 FPS in the 8 bit accurate mode and upto 11.2 FPS in the 4 bit mode corresponding to 760.1 GOp/s and 1,218 GOp/s effective throughput, respectively. Finally, we compare the novel design to classical multiplier-based approaches in terms of FPGA utilization and power consumption. Our novel multiply-accumulate engines designed for the optimized number representation uses 9 % less logical elements while allowing double throughput compared to a classical implementation. Moreover, a measurement shows 25 % reduction of power consumption at same throughput. Therefore, our flexible design offers a solution to the trade-off between energy efficiency, accuracy, and high throughput.

We present a Vector Processing Engine (VPE) designed for the acceleration of Convolutional Neural Networks (CNNs) for semantic segmentation. Most CNN accelerators focus on classification. However, CNNs for semantic segmentation incorporate special layer types. Our accelerator supports not only regular convolutional layers, but also dilated convolutions and convolutions in combination with down- or up-sampling. These features are implemented in dedicated address generators which load the corresponding input vector from an input line buffer. The VPE is designed as a 64x64-array where up to 64 output features and 64 input features of a convolutional layer can be unrolled in parallel. The array has a peak performance of 4.12 TOp/s and achieves 3.85 TOp/s on a CNN for semantic segmentation - resulting in an average utilization of 93 %. The design is prototypically implemented on a Virtex UltraScale+ device with a clock rate of 250 MHz. In addition to the overall architecture, we present the two-hot quantization scheme. A value in two-hot quantization can be regarded as a combination of two power-of-two values. Hence, instead of bulky multipliers, two small bit-shifts are implemented. We design and implement dedicated arithmetic engines for this quantization scheme. Additionally, we evaluate this quantization scheme on the rather complex task of semantic segmentation. We show that the performance of an 8 bit two-hot quantization scheme is marginally lower in comparison to a regular 8 bit fixed-point variant.

Efficient acceleration of Deep Neural Networks is a manifold task. In order to save memory requirements and reduce energy consumption we propose the use of dedicated accelerators with novel arithmetic processing elements which use bit shifts instead of multipliers. While a regular power-of-2 quantization scheme allows for multiplierless computation of multiply-accumulate-operations, it suffers from high accuracy losses in neural networks. Therefore, we evaluate the use of powers-of-arbitrary-log-bases and confirmed their suitability for quantization of pre-trained neural networks. The presented method works without retraining of the neural network and therefore is suitable for applications in which no labeled training data is available. In order to verify our proposed method, we implement the log-based processing elements into a neural network accelerator on an FPGA. The hardware efficiency is evaluated in terms of FPGA utilization and energy requirements in comparison to regular 8-bit-fixed-point multiplier based acceleration. Using this approach hardware resources are minimized and power consumption is reduced by 22.3%.

This paper presents a fundamental result on buffer sizing. Given an interconnection wire with n buffers evenly spaced along the wire, we would like to size all buffers such that the Elmore delay is minimized. It is well known that the problem can be solved by an iterative algorithm which sizes one buffer at a time. However, no closed form solution has ever been reported. In this paper, we derive a closed form buffer sizing function f(x) where f(x) gives the optimal buffer size for the buffer at position x. We show that f(x) can be expressed in terms of the Weierstrass elliptic function p(x) and its derivative p'(x).

The design of suitable configurations for virtual private networks (VPNs) is usually difficult and error-prone. The abstract objectives of design are given by high level policies representing various requirements and the designers are often faced with conflicting requirements. Moreover, it is difficult to find a suitable mapping of high level policies to those low level network configurations which correctly and completely implement the abstract objectives. We apply the approach of model-based management where the system itself as well as the management objectives are represented by graphical object instance diagrams. A combination of tool and libraries supports their interactive construction and automated analysis. The implementation of the approach focuses on VPNs which are based on the Linux IPsec software FreeS/WAN.