The performance of CPU-based and GPU-based systems is often low for PDE codes, where large, sparse, and often structured systems of linear equations must be solved. Iterative solvers are limited by data movement, both between caches and memory and between nodes. Here we describe the solution of such systems of equations on the Cerebras Systems CS-1, a wafer-scale processor that has the memory bandwidth and communication latency to perform well. We achieve 0.86 PFLOPS on a single wafer-scale system for the solution by BiCGStab of a linear system arising from a 7-point finite difference stencil on a 600 × 595 × 1536 mesh, achieving about one third of the machine's peak performance. We explain the system, its architecture and programming, and its performance on this problem and related problems. We discuss issues of memory capacity and floating point precision. We outline plans to extend this work towards full applications.

In the Hierarchical Temporal Memory (HTM) paradigm the effect of overlap between inputs on the activation of columns in the spatial pooler is studied. Numerical results suggest that similar inputs are represented by similar sets of columns and dissimilar inputs are represented by dissimilar sets of columns. It is shown that the spatial pooler produces these results under certain conditions for the connectivity and proximal thresholds at initialization. Qualitative arguments about the learning dynamics of the spatial pooler are then discussed.

In a perfect world, code would only be written once and would run on different devices with high efficiency. A programmer's time would primarily be spent on thinking about the algorithms and data structures, not on implementing them. To a degree, that used to be the case in the era of frequency scaling on a single core. However, due to power limitations, parallel programming has become necessary to obtain performance gains. But parallel architectures differ substantially from each other, often require specialized knowledge, and typically necessitate reimplementation and fine tuning of application code. These slow tasks frequently result in situations where most of the time is spent reimplementing old rather than writing new code. The goal of our research is to find new programming techniques that increase productivity, maintain high performance, and provide abstraction to free the programmer from these unnecessary and time-consuming tasks. However, such techniques usually come at the cost of substantial performance degradation. This paper investigates current approaches to portable accelerator programming, seeking to answer whether they make it possible to combine high efficiency with sufficient algorithm abstraction. It discusses OpenCL as a potential solution and presents three approaches of writing portable code: GPU-centric, CPU-centric and combined. By applying the three approaches to a real-world program, we show that it is at least sometimes possible to run exactly the same code on many different devices with minimal performance degradation using parameterization. The main contributions of this paper are an extensive review of the current state-of-the-art regarding the stated problem and our original approach of addressing this problem with a generalized excessive-parallelism approach.

We are presenting a high-performance GPU implementation of a 2-opt and 3-opt algorithms used to solve the Traveling Salesman Problem. The main idea behind it is to take a route that crosses over itself and reorder it so that it does not. It is a very important local search technique and using GPU to parallelize the search greatly decreases the time needed to find the best edges to be swapped in a route. Our results show, that at least 90% of the time during an Iterative Local Search is spent on the 2-opt itself. Our result show that by using our algorithm for GPU, the time need to find optimal swaps can be decreased approximately 100 times in case of 2-opt compared to a sequential CPU code and more than 220-fold speedup can be observed in case of 3-opt search achieving more than 430 GFLOPS on a single Tesla C2075 GPU.

In this paper we are presenting high performance GPU implementations of the 2-opt and 3-opt local search algorithms used to solve the Traveling Salesman Problem. This type of local search optimization is a very effective and fast method in case of small problem instances. However, the time spent on comparing the graph edges grows significantly with the problem size growing. They are usually a part of global search algorithms such as Iterated Local Search (ILS). Our results showed, that at least 90% of the time during a single ILS run is spent on the local search itself. Therefore we utilized GPU to parallelize the local search and that greatly improved the overall speed of the algorithm. Our results show that the GPU accelerated algorithm finds the optimal swaps approximately 3 to 26 times compared to parallel CPU code using 32 cores, operating at the speed of over 1.5 TFLOPS on a single GeForce GTX 680 GPU. The preliminary experimental studies show that the optimization algorithm using the GPU local search converges 10 to 50 times faster on average compared to the sequential CPU version, depending on the problem size.