This article proposes a hardware accelerator for image feature extraction based on the oriented features from accelerated segment test (FAST) and rotated binary robust independent elementary features (ORB) algorithm. The architecture adopts a hybrid workflow to process three scales in parallel as an octave, and multiple octaves are processed in serial by time-sharing the DRAM. In order to support any image resolution, a block-wise dataflow is used on the serial-parallel architecture. The overlapped data between two blocks is reused. As a result, the on-chip memory is limited to 1.47 Mb, and the DRAM bandwidth is compressed by 33%. Besides, 3×3 nonmaximum suppression with heap sorting is applied to balance keypoint distribution in 2-D. It improves the valid keypoint match ratio by 7.09%. In addition, parallel processing in keypoint detection increases 2× throughout. Moreover, approximate computing and superscalar processing reduce the timing cost in orientation estimation and descriptor generation by 92.78% and 67% respectively. Compared to the non-optimized baseline architecture, the proposed architecture saves the total timing cost by 82.4%. The accelerator is implemented in a Xilinx MPSoC, and it achieves 108 fps on full-HD images at 200 MHz under 873 mW.

Deep neural networks dominate in the machine learning field. However, deploying deep neural networks on mobile devices requires aggressive compression of models due to huge amounts of parameters. An extreme case is to restrict weights to binary values {+1/-1} without much loss of accuracy. This promising method not only reduces hardware overhead of memory and computation, but also improves the performance of network inference. In this work, a flexible architecture for binary weight network acceleration is proposed. The architecture fully exploits the inherent multi-level parallelism of neural networks, resulting in utilization of processing elements over 80% for different layers. In addition, we present efficient data placement and transmission methods in coordination with multi-level parallel processing. The accelerator is implemented using SMIC 40nm technology. It operates at 1.2V and achieves up to 974GOPS/W power efficiency.

This paper proposes a hardware accelerator of Oriented FAST and Rotated BRIEF (ORB) algorithm with its full implementation. The accelerator is optimized in computational logics and memory organizations to achieve real-time performance and good matching accuracy. The architecture consists of a loose-coupled pipeline, in which keypoints are detected in scale-level parallel while descriptors are built in task-level parallel. In this way, the workload in different pipeline stages are balanced. Because of the loose-coupled architecture employed, large amounts of image pyramid data are offloaded to external memory and fetched again. To relieve the external memory bandwidth requirement, image pyramid data are stored every other scale and a shared data-reuse structure is also introduced for patch loading. Besides, we design a score recorder to refine the keypoints in different regions adaptively. As a result, the relatively balanced distribution of keypoints in a frame is in favor of establishing keypoint correspondences. The proposed architecture is implemented on a Zynq-family FPGA with 100MHz and can extract 1000 features in full-HD images at 42fps. An external memory bandwidth of 1.96Gbps and internal memory of 583.4Kbits are suitable for practical embedded applications.

ORB (Oriented FAST and Rotated BRIEF) feature extraction is popular in embedded vision applications like visual navigation due to its higher speed and robustness in many situations. However, feature description in ORB still accesses large amounts of image patches especially when an image pyramid is built. In order to reduce internal memory cost as well as maintain low latency processing, we design a hybrid pipeline architecture for ORB feature extraction. The accelerator combines different levels of computing granularity and migrates image pyramids to external memory. In addition, a data reuse scheme is adopted in descriptor generation to minimize external memory access, and achieve the ability to operate in multiple scales. The synthesis result shows 700kb internal memory cost and 24.5mW low power consumption. Experiments demonstrate 22% bandwidth reduction on average by the data reuse scheme. The system is verified on an FPGA platform and can provide 4000 features per frame, achieving up to 81fps in 1080p resolution at 100MHz frequency.

As are widely used in military and security fields, multiple channels of low-bit-rate vocoders are required to perform on embedded devices efficiently. We propose HAVA, a multicore Application Specific Instruction Set Processor for multichannel low-bit-rate vocoders with real-time performance. To provide both flexibility and efficiency, HAVA integrates two types of processing cores and a shared-memory core on a 2-D-mesh on-chip network. Adopting a single-Instruction Set Architecture heterogeneous multicore architecture, HAVA cuts down the real-time performance requirement of vocoders by over 40% compared with other platforms. By leveraging the on-chip network for intercore communication, HAVA can perform multichannel vocoders with a marginal efficiency loss. The chip implementation of HAVA is finished in a 40-nm CMOS technology and it dissipates 149 mW at 100-MHz operating frequency for four channels of encoders.

As one of the mostly used synchronization schemes in parallel programming on multi-core processors, barrier synchronization has been extensively studied in former research works. In conventional master-slave barrier or tree barrier, usually one centric core is selected to collect barrier arriving messages and to broadcast barrier releasing messages. Unfortunately the barrier core sometimes is deviated from the center location and may lead to worse synchronization efficiency. We propose a hybrid tree-based all-to-all (TAB) barrier for NoC-based many-core processors to relieve performance degradation caused by the off-centered barrier core. Performance of TAB barrier is compared to canonical algorithms and former solution, and almost 20% time is saved during off-centered scenarios with marginal area and power overhead.