Demand for high performance deep learning (DL) inference in software applications is growing rapidly. DL workloads run on myriad platforms, including general purpose processors (CPU), system-on-chip (SoC) with accelerators, graphics processing units (GPU), and neural processing unit (NPU) addin cards. DL software engineers typically must choose between relatively slow general hardware (e.g., CPUs, SoCs) or relatively expensive, large, power-hungry hardware (e.g., GPUs, NPUs). This paper describes Centaur Technology's Ncore, the industry's first high-performance DL coprocessor technology integrated into an x86 SoC with server-class CPUs. Ncore's 4096 byte-wide SIMD architecture supports INT8, UINT8, INT16, and BF16 datatypes, with 20 tera-operations-per-second compute capability. Ncore shares the SoC ring bus for low-latency communication and work sharing with eight 64-bit x86 cores, offering flexible support for new and evolving models. The x86 SoC platform can further scale out performance via multiple sockets, systems, or third-party PCIe accelerators. Ncore's software stack automatically converts quantized models for Ncore consumption and leverages existing DL frameworks. In MLPerf's Inference v0.5 closed division benchmarks, Ncore achieves 1218 IPS throughput and 1.05ms latency on ResNet-50v1.5 and achieves lowest latency of all Mobilenet-V1 submissions (329μs). Ncore yields 23x speedup over other x86 vendor percore throughput, while freeing its own x86 cores for other work. Ncore is the only integrated solution among the memory intensive neural machine translation (NMT) submissions.

Swarming intelligence (SI) is the property of a system whereby the collective behaviors of (unsophisticated) agents interacting locally with their environment cause coherent global patterns to emerge. Because there are no central processing requirements, SI is ideal for parallelization, which lends it well to hardware implementations in many inexpensive processors. Several implementations in hardware that exploit this property for rapid calculation and inexpensive construction are highlighted in this paper to provide a good starting point for developing hardware platforms for SI algorithms.

A large portion of South Africa's elephant population can be found on small wildlife reserves. When confined to enclosed reserves the elephant densities are much higher than observed in the wild. The large nutritional demands and destructive foraging behavior of elephants threaten rare species of vegetation. If conservation management is to protect threatened species of vegetation, knowing how long elephants will stay in one area of the reserve as well as which area they will move to next is essential. The goal of this study is to train a recurrent neural network (RNN) to continuously predict an elephant herd's next position in the Pongola Game Reserve. Accurate predictions would provide a useful tool in assessing future impact of elephant populations on different areas of the reserve. The particle swarm optimization (PSO) algorithm is used to adapt the weights of the neural network. Results are presented to show the effectiveness of RNN-PSO for elephant migration prediction.