The state-of-the-art convolutional neural network accelerators are showing a growing interest in exploiting the bit-level sparsity and eliminating the ineffectual computations of zero bits. However, the excessive redundancy and the irregular distribution of nonzero bits limit the real speedup in the accelerators. To address this, we propose an algorithm-architecture codesign, named structured term pruning (STP), to boost the computation efficiency of neural networks inference. Specifically, we enhance the bit sparsity by guiding the weights toward the value with fewer power-of-two terms. Then, we structure the terms with layer-wise group budgets. Retraining is adopted to recover the accuracy drop. We also design the hardware of the group processing element and the fast signed-digital encoder for efficient implementation of STP networks. The system design of STP is realized with some easy alterations on an input stationary systolic array design. Extensive evaluation results demonstrate that STP can reduce significant inference computation costs, and achieve$2.35\times $computational energy saving for the ResNet18 network on the ImageNet dataset.

Deep Neural Networks (DNNs) have achieved remarkable success in various Artificial Intelligence applications. Quantization is a critical step in DNNs compression and acceleration for deployment. To further boost DNN execution efficiency, many works explore to leverage the input-dependent redundancy with dynamic quantization for different regions. However, the sensitive regions in the feature map are irregularly distributed, which restricts the real speed up for existing accelerators. To this end, we propose an algorithm-architecture co-design, named Structured Dynamic Precision (SDP). Specifically, we propose a quantization scheme in which the high-order bit part and the low-order bit part of data can be masked independently. And a fixed number of term parts are dynamically selected for computation based on the importance of each term in the group. We also present a hardware design to enable the algorithm efficiently with small overheads, whose inference time mainly scales with the precision proportionally. Evaluation experiments on extensive networks demonstrate that compared to the state-of-the-art dynamic quantization accelerator DRQ, our SDP can achieve 29% performance gain and 51% energy reduction for the same level of model accuracy.

Deep neural networks have achieved remarkable advancement in various intelligence tasks. However, the massive computation and storage consumption limit applications on resource-constrained devices. While channel pruning has been widely applied to compress models, it is challenging to reach very deep compressions for such a coarse-grained pruning structure without significant performance degradation. In this article, we propose an acceleration-aware fine-grained channel pruning (AFCP) framework for accelerating neural networks, which optimizes trainable gate parameters by estimating residual errors between pruned and original channels with hardware characteristics. Our fine-grained concept consists of both algorithm and structure levels. Different from existing methods that leverage a predefined pruning criterion, AFCP explicitly considers both zero-out and similar criteria for each channel, and adaptively selects the suitable one via residual gate parameters. For structure level, AFCP adopts a fine-grained channel pruning strategy for residual neural networks and a decomposition-based structure, which further extends the pruning optimization space. Moreover, instead of using theoretical computation costs, such as floating-point operations, we propose the hardware predictor that bridges the gap between realistic acceleration and pruning procedure to guide the learning of pruning, which improves the efficiency of model pruning when deployed on accelerators. Extensive evaluation results demonstrate that AFCP outperforms state-of-the-art methods, and achieves a favorable balance between model performance and computation cost.

With the development of deployment platforms and application scenarios for deep neural networks, traditional fixed network architectures cannot meet the requirements. Meanwhile the dynamic network inference becomes a new research trend. Many slimmable and scalable networks have been proposed to satisfy different resource constraints (e.g., storage, latency and energy). And a single network may support versatile architectural configurations including: depth, width, kernel size, and resolution. In this paper, we propose a novel network architecture reuse strategy enabling dynamic precision in parameters. Since our low-precision networks are wrapped in the high-precision networks like an onion, we name it dynamic precision onion quantization (DPOQ). We train the network by using the joint loss with scaled gradients. To further improve the performance and make different precision network compatible with each other, we propose the precision shift batch normalization (PSBN). And we also propose a scalable input-specific inference mechanism based on this architecture and make the network more adaptable. Experiments on the CIFAR and ImageNet dataset have shown that our DPOQ achieves not only better flexibility but also higher accuracy than the individual quantization.

Data free deep neural network quantization is a practical challenge, since the original training data is often unavailable due to some privacy, proprietary or transmission issues. The existing methods implicitly equate data-free with training-free and quantize model manually through analyzing the weights’ distribution. It leads to a significant accuracy drop in lower than 6-bit quantization. In this work, we propose the data free quantization-aware fine-tuning (DFQF), wherein no real training data is required, and the quantized network is fine-tuned with generated images. Specifically, we start with training a generator from the pre-trained full-precision network with inception score loss, batch-normalization statistics loss and adversarial loss to synthesize a fake image set. Then we fine-tune the quantized student network with the full-precision teacher network and the generated images by utilizing knowledge distillation (KD). The proposed DFQF outperforms state-of-the-art post-train quantization methods, and achieve W4A4 quantization of ResNet20 on the CIFAR10 dataset within 1% accuracy drop.

Depth image-based rendering (DIBR) techniques have recently drawn more attention in various 3D applications. In this paper, a real-time high-quality DIBR system that consists of disparity estimation and view synthesis is proposed. For disparity estimation, a local approach that focuses on depth discontinuities and disparity smoothness is presented to improve the disparity accuracy. For view synthesis, a method that contains view interpolation and extrapolation is proposed to render high-quality virtual views. Moreover, the system is designed with an optimized parallelism scheme to achieve a high throughput, and can be scaled up easily. It is implemented on an Altera Stratix IV FPGA at a processing speed of 45 frames per second for 1080p resolution. Evaluated on selected image sets of the Middlebury benchmark, the average error rate of the disparity maps is 6.02%; the average peak signal to noise ratio and structural similarity values of the virtual views are 30.07 dB and 0.9303, respectively. The experimental results indicate that the proposed DIBR system has the top-performing processing speed and its accuracy performance is among the best of state-of-the-art hardware implementations.

Three dimensional (3D) vision applications have drawn more attention nowadays and many products are entering the mass market. View interpolation is a crucial step to generate intermediate viewpoints from reference images. However, it is still challenging to achieve good performance in both processing speed and image quality for various 3D applications. In this paper, a hardware-compatible view interpolation algorithm is proposed, which can produce high-quality intermediate images by disparity warping and color blending. Moreover, a fully pipelined hardware architecture is designed based on the algorithm. A prototype of the proposed architecture has been implemented on an Altera Stratix-IV FPGA board, achieving 65 frames per second (fps) with a full HD (1920 × 1080) resolution. It is evaluated on the Middlebury benchmark quantitatively, and visual results of real-world images are also provided.

Stereo matching is a crucial step for acquiring depth information from stereo images. However, it is still challenging to achieve good performance in both speed and accuracy for various stereo vision applications. In this paper, a hardware-compatible stereo matching algorithm is proposed; its associated hardware implementation is also presented. The proposed algorithm can produce high-quality disparity maps with the use of mini-census transform, segmentation-based adaptive support weight and effective refinement. Moreover, the proposed implementation is optimized as a fully pipelined and scalable hardware system. The proposed design is evaluated based on the Middlebury benchmarks and the average overall error rate is 6.10%. The experimental results indicate that the accuracy is competitive with some state-of-art software implementations.

Stereo matching is a crucial step to extract depth information from stereo images. However, it is still challenging to achieve good performance in both speed and accuracy for various stereo vision applications. In this paper, a hardware-compatible stereo matching algorithm is proposed and its associated hardware implementation is also presented. The proposed algorithm can produce high-quality disparity maps with the combined use of the mini-census transform, segmentation-based adaptive support weight and effective refinement. Moreover, the proposed architecture is optimized as a fully pipelined and scalable hardware system. Implemented on an Altera Stratix-IV FPGA board, it can achieve 65 frames per second (fps) for 1024 × 768 stereo images and a 64 pixel disparity range. The proposed architecture is evaluated based on the Middlebury benchmarks and the average error rate is 6.56%. The experimental results indicate that the accuracy is competitive with some state-of-the-art software implementations.

Image recognition systems implement a number of processing stages: preprocessing, segmentation and classification. In camera based video processing chains, usually several frame delays are incurred between the moment of capture and the actual availability of the classification results. Hardware architectures for stream based video processing have already been widely employed. In this paper, a new hardware architecture for accelerating the generic task of connected component analysis and object labeling in the segmentation step is presented. The architecture is specifically optimized for very low latency between image component capture by a camera and the detection in hardware. This latency constitutes only a few delay lines, thereby shortening the response time by a few orders of magnitude in comparison to traditional frame-buffer based methods.

This paper presents a real time image warping module implemented in hardware. A look-up table (LUT) based reverse mapping is used to relate the source image to the warped image. Frame buffers or line buffers are often used to temporally store the source image. However these methods do not take the underlying pattern of the reverse mapping coordinates into account. The presented architecture uses an adaptable memory allocation which can change the depth and the position of the line buffer between lines. A real-time stereo rectification use case has been implemented to validate the operation of this module. Depending on the scenario, the memory consumption can be reduced by a factor of two and more. A real-time image warping module for video cameras has been implemented in a single FPGA, without the use of off-chip memories.

This paper presents a real-time trinocular disparity processor.The core module performs a pairwise segmented window matching for both the center-right and center-left image pair as their scaled down image pairs.The resulting cost functions are combined which results into nine different curves.A hierarchical classifier is presented which selects the most promising disparity value using information provided by the calculated cost curves and the pixels spatial neighborhood using a two level classification architecture.The disparity processor has been evaluated with an indoor dataset and with a real-time implementation using an FPGA and three cameras.Special care has been taken to reduce the memory footprint so that the processor doesn't need external memory.

Advanced digital paint systems, based on accurate simulation models for the paint, brush and canvas interaction enable a virtual painting environment familiar to artists, but also allow for new ways of expressivity. The new FluidPaint co-located real-brush-input & canvas-display systems put extremely high demands on the real-time image processing and the reaction time between brush strokes and the rendering of the painting result on the screen. A new Smart Camera System-on-Chip (SoC) architecture is presented enabling real-time input of brush-strokes with real brushes in the FluidPaint digital painting system.

This paper presents a novel binary fully adaptable window for incorporating in a stereo matching System-on-Chip (SoC) architecture. This architecture is fully scalable and parameterizable to allow for custom SoC implementations, as well as rapid prototyping on FPGAs. For each window a binary mask window is generated which selects the supporting pixels in the cost aggregation phase of the SAD algorithm. This selection is performed using color similarity and spatial distance metrics. Hardware resource utilization for a fixed window and an adaptable window cost aggregation is compared based on FPGA logic element use.

In general, logic redundancy tends to degrade design-quality by introducing additional delays in signal propagation, by increasing the gate count or simply by making the resulting hardware untestable. Since they cannot always be avoided, unwanted redundancies have to be first identified and then removed from our designs. In this paper an alternative methodology to identify and remove redundancy is proposed, which is based on a formal, symbolic verification strategy. The formal framework underlying our approach aids in identifying redundancies and allows us to guarantee the correctness of their removal.

VLSI-system design can, in general, be characterized in terms of the step-wise refinement of intermediate solutions. Despite the fact that such refinements usually do not preserve time-scales, current formal verification approaches mostly start from the assumption that both specification and implementation utilize the same scales of time. Realizing the importance of being able to cope with differences in timing granularity, this preliminary paper proposes a symbolic methodology to verify that a low-level finite state machine is a refinement of a high-level finite state machine. To illustrate our approach, the step-wise refinement-and verification-of a simple microprocessor is presented.

We focus on the use of the dynamic time warping (DTW) technique in the signature verification area. The DTW algorithm originates from the field of speech recognition, where it is a highly appreciated component of speaker specific isolated word recognisers. A few years ago the DTW algorithm was successfully introduced in the area of online signature verification. The characteristics of speech recognition and signature verification are however rather different. Starting from these dissimilarities, our objective is to extract an alternative DTW approach that is better suited to the signature verification problem.

The paper presents an online signature verification system, based on 3D force patterns and pen inclination angles, as recorded during signing. The feature extraction mechanism is based on the well known elastic matching technique. In contradiction to previous work in the same area however, we emphasise the importance of the final step in the process: the discrimination based on the extracted features. We show that by choosing the right discrimination approach we are able to improve the quality of the entire verification process drastically. The techniques we compare for discrimination, however, are not specific to signature verification, but should be considered carefully in every process where a classification decision is made from a set of parameters.

A method for the verification of embedded software correctness is presented. A formal model for an actual commercial microprocessor is established. This is done by modeling the instruction set and processor architecture. Embedded software takes the form of the assembly program code to be run on the processor. Specifications are given as CTL temporal logic formulae. The method has been implemented in the SMV model checker and is illustrated by a practical embedded system application: a mouse controller. The inconsistency of the specification and the implementation as an assembly language program as it has been published in the applications book of the manufacturer has been uncovered.

In this paper, we discuss an on-line signature verification system based on dynamic time-warping (DTW). The DTW-algorithm originates from the field of speech recognition, and has been applied successfully in the signature verification area more than once. However, until now, few adaptations have been made in order to take the specific characteristics of signature verification into account. According to us, one of the most important differences is the availability of a rather large number of reference patterns, making it possible to determine which parts of a reference signature are important and which are not. By disconnecting the DTW-stage and the feature extraction process we are able to deal efficiently with this extra amount of information. We demonstrate the benefits of our approach by building and evaluating a complete system.

Functional verification of large digital synchronous circuits with respect to complex arithmetic and decision making algorithms is becoming more and more important. Indeed, today, system complexities are still growing, while time-to-marked is still decreasing. First-time-right circuits can be obtained by adequate checking tools for timing, electrical and behavioural verification. The paper presents new results for automatic functional verification, based on the SFG-tracing methodology, applied on applications for medium and high throughput DSP, such as audio, video and image processing. The verification is formal, complete, efficient and independent. It checks functional correctness across synthesis tasks such as allocation, scheduling, clustering, specific datapaths and controller generation, redundancy removal, buffering, pipelining, retiming, etc.>

A high-speed cryptographic coprocessor is presented. This coprocessor is named Subterranean and can be used for both cryptographic pseudorandom sequence generation (Substream) and cryptographic hashing (Subhash). In Substream mode the chip can be used for stream encryption/decryption under control of a 256-bit key. A cryptographic resynchronization mechanism is provided for fast accessibility of encrypted data by legitimate particles. Application fields include the real-time encryption of digital HDTV signals as well as high speed telecommunication and networking such as ATM. The chip has been fabricated within the INVOMEC/EUROCHIP educational VLSI Design Facilities in MIETEC 2.4 /spl mu/ CMOS technology. Measured samples are operating at encryption/decryption rates of 286 Mb/s and hashing rates of 572 Mb/s. The operation of the chip is demonstrated by a setup showing the real-time encryption and decryption of digitized PAL color composite video signals. The designed cryptographic module can be used as a stand-alone device or embedded as a mega-block in a larger chip.>

A novel solution for the efficiency problems encountered in static timing verification is presented. The LSP algorithm is submitted to a critical analysis. A new hierarchy based approach is presented and its advantages and limitations are highlighted. Finally, some results on real life circuits are presented.>

The SFG-tracing methodology addresses the automatic verification of digital synchronous circuit implementations as specified at the algorithmic level as signal- (SFG) or data flow graphs. The SFG-tracing methodology is a multi-level design verification paradigm that aims at bridging the gap between higher level specifications down to lower level implementations up to the transistor switch level. The concepts of the SFG-tracing methodology are illustrated by the automatic verification of a transistor level implementation of a small chip generated from its high level specification by the Cathedral-II silicon compiler. This application, although simple, includes a datapath, register files, a multi-branch micro coded controller, and additional circuitry as necessary for design for testability measures. This application illustrates the SFG-tracing verification methodology as applied to one member of a partitioned SFG behavioral specification. Experimental results on more complex, completely verified designs of 32000 transistors demonstrate the feasibility of the approach.>

Novel rules to derive the signal flow in a VLSI-design are presented. This method is based on design-style independent logical principles, which makes it suitable for circuits that have not been checked on their electrical correctness. The use of two global principles has an important influence for more complicated designs, but also results in an efficiency penalty which may be kept reasonable by introducing graph reduction rules and heuristics. The different steps in the tagging process are explained and illustrated with examples. The dataflow rules are integrated in the overall DIALOG system to perform electrical verification, and they offer an invaluable help to break up the verification problem in subproblems and to cope with feedback in designs, by introducing the concept of intended unilateral blocks.>

A new approach to the false path problem in timing verifiers is presented. This approach is based on the modeling of both the logic and timing behavior of a circuit. Using the logic propagation conditions associated with each delay, efficient algorithms have been developed to find statically sensitizable paths. These algorithms simultaneously perform a longest path search and a partial verification of the sensitization of the paths. The resulting paths undergo a final and complete sensitization. The algorithms find the longest statically sensitizable path, whose length is a lower bound to the critical path length, and its associated sensitizing input vector. The algorithms can be easily modified to provide an ordered list of all the statically sensitizable paths above a given threshold. An initial analysis of the circuit by the PERT algorithm guides the critical path search and allows pruning of subgraphs that cannot lead to the solution. Results show that these techniques succeed in curbing the combinatorial explosion associated with the longest statically sensitizable path search.>

Several novel methods for the electrical-level simulation of digital VLSI MOS circuits on a shared-memory multiprocessor system are presented. A novel parallel algorithm, the overlapped phases algorithm, for the efficient simulation of circuits containing feedback loops, is presented. The algorithm is based on data flow scheduling and local relaxation of the feedback loops. A novel method for the partitioning of largepass transistor networks is discussed. The method is based on the signal flow direction in the elements. This partitioning allows an efficient simulation of these large networks on a multiprocessor system. Parallel element evaluation and the time segment pipelining method, two methods to increase the performance of the parallel circuit simulator, are explained. Simulation tests with actual circuits show a substantial acceleration for the new methods.>

This paper discusses the kernel implementation issues of a new and more formal approach to electrical verification. Rule based analysis of the transistor network is applied to derive the signal flow direction and to identify control and state nodes. Symbolic analysis of Boolean expressions, capturing all aspects of switch level networks, allows to take into account the logical structure of the network and its environment during verification and guarantees more relevant error reports. The application of the proposed strategy on real life examples, demonstrates its usefulness and allows for a realistic evaluation of the tool.

This paper presents several new methods for the efficient parallel simulation of VLSI circuits that contain feedback loops or “difficult” parts such as arrays, registers and pass-transistor networks. A new parallel algorithm has been developed for the efficient simulation of circuits containing feedback loops. It is based on dataflow scheduling and local relaxation of the loops. For the simulation of large pass-transistor networks a partitioning method is used that is based on signal flow in the elements. Parallel element evaluation and time-segment pipelining are included to increase the performance of the parallel circuit simulator. Simulation tests with actual circuits show a substantial acceleration for the new methods.

This paper describes a new method for solving the false path problem in static timing analysis of acyclic, combinational circuits. The conditions under which a path is false are accurately defined. The fact that these conditions explicitly take into account the dynamic behaviour of the circuit, constitutes the main contribution of the paper. An algorithm for computing the longest dynamically sensitizable paths in an acyclic, combinational circuit is presented.

A description is given of efficient techniques for checking tautology of a Boolean expression, i.e. whether two Boolean expressions are equivalent, taking into account 'don't care' behavior if necessary. If two Boolean expressions appear to be not equivalent, a test case is generated. The tautology checker has been developed to perform functional and logical verification of combinational modules and is integrated in an environment for formal electrical verification. Its efficiency is based on a delicate and optimized interaction between a carefully chosen set of Boolean rewriting rules and a number of heuristics. A number of test cases show its performance as compared to existing tautology checkers. Figures on industrial PLAs are included.>

An environment for the generation of modules is described. It includes tools for interactive design of parameterised procedures describing the structure as well as the topology. For the layout symbolic cells are used which are automatically fitted together as defined by the topology.

CAD tools to support a top-down custom design methodology for integrated digital filters are presented. The methodology is based on a tool-box concept, which makes use of specialised analysis, synthesis and optimization programs at each design level: the network, the architecture and the circuit layout. In this paper, CAD tools for filter synthesis, network optimization and architecture optimization are developed. These tools complement design aids for architecture synthesis, and automatic layout generation (silicon compilers) to create a complete design environment. By combining both synthesis as well as optimization aids at each design level, it is possible to achieve complete automation while retaining efficient use of silicon area, speed and power consumption. Application of these tools to the custom integration of wave digital filters with bit-serial architectures is demonstrated.