In this paper, we evaluate a set of coherence architectures in the context of a 1024-core chip multiprocessor (CMP) tailored to throughput-oriented parallel workloads. Based on our analysis, we develop and evaluate two techniques for scaling coherence to thousand-core CMPs. We find that a broadcast-based probe filtering scheme provides reasonable performance up to 128 cores for some benchmarks, but is not generally scalable. We propose a broadcast-collective network for accelerating probe filter misses, which extends scalability but falls short of supporting 1024 cores. We find that a sparse directory with an invalidate-on-evict policy can work well for many throughput-oriented workloads. However, the on-die structures required to achieve good performance carry a large performance and power overhead. To achieve thousand-core scalability with smaller and less associative sparse directories, we introduce WayPoint, a mechanism that increases directory associativity and capacity dynamically. Using less than 3% of total die area, Way-Point achieves performance within 4% of an infinitely large on-die directory.

Two broad classes of memory models are available today: models with hardware cache coherence, used in conventional chip multiprocessors, and models that rely upon software to manage coherence, found in compute accelerators. In some systems, both types of models are supported using disjoint address spaces and/or physical memories. In this paper we present Cohesion, a hybrid memory model that enables fine-grained temporal reassignment of data between hardware-managed and software-managed coherence domains, allowing a system to support both. Cohesion can be used to dynamically adapt to the sharing needs of both applications and runtimes. Cohesion requires neither copy operations nor multiple address spaces.

This paper presents a task-centric memory model for 1000-core compute accelerators. Visual computing applications are emerging as an important class of workloads that can exploit 1000-core processors. In these workloads, we observe data sharing and communication patterns that can be leveraged in the design of memory systems for future 1000-core processors. Based on these insights, we propose a memory model that uses a software protocol, working in collaboration with hardware caches, to maintain a coherent, single-address space view of memory without the need for hardware coherence support. We evaluate the task-centric memory model in simulation on a 1024-core MIMD accelerator we are developing that, with the help of a runtime system, implements the proposed memory model. We evaluate coherence management policies related to the task-centric memory model and show that the overhead of maintaining a coherent view of memory in software can be minimal. We further show that, while software management may constrain speculative hardware prefetching into local caches, a common optimization, it does not constrain the more relevant use case of off-chip prefetching from DRAM into shared caches.

This paper presents Emmucode, a technique for masking hard faults in modern microprocessors that provides graceful performance degradation. Emmucode employs microcode traces with control flow that replace an original instruction once a fault is detected. Emmucode adds lightweight microarchitectural hardware to assist in correcting hard faults in larger structures, such as SIMD execution units found in contemporary microprocessors, where replication is infeasible. Key challenges in implementing microcode traces include maintaining proper architectural state and the optimization of trace code. We are able to significantly optimize traces by exploiting dynamic trace behavior and by performing minor modifications to the microarchitecture. We find that removing hard to predict branches is important for optimizing traces. Emmucode uses partial predication, new microcode operations, and the full use of the microcode's flexibility and visibility to create fast traces. This paper studies the viability of implementing SIMD floating point arithmetic operations found in modern x86 processors using Emmucode traces. Our results show that for programs with 1 to 5 percent of the dynamic instructions replaced by Emmucode, a graceful performance degradation of only 1.3times to 4times is achievable.

This paper considers Rigel, a programmable accelerator architecture for a broad class of data- and task-parallel computation. Rigel comprises 1000+ hierarchically-organized cores that use a fine-grained, dynamically scheduled single-program, multiple-data (SPMD) execution model. Rigel's low-level programming interface adopts a single global address space model where parallel work is expressed in a task-centric, bulk-synchronized manner using minimal hardware support. Compared to existing accelerators, which contain domain-specific hardware, specialized memories, and/or restrictive programming models, Rigel is more flexible and provides a straightforward target for a broader set of applications.

We present HybridOS, a set of operating system extensions for supporting fine-grained reconfigurable accelerators integrated with general-purpose computing platforms. HybridOS specifically targets the application integration, data movement and communication overheads for a CPU/accelerator model when running a commodity operating system. HybridOS provides a simple API for applications and a well-defined hardware interface for reconfigurable accelerators. The goal is to reduce the difficulty in mapping applications into a CPU/accelerator model compared to an unrestrained FPGA platform while achieving whole-application speedups. HybridOS is integrated into a full Linux distribution running on the embedded processor of an FPGA. Application-specific accelerators are implemented in the reconfigurable fabric of the FPGA that are allocated to user applications running on Linux. We have developed and evaluated four methods for accessing the data buffers required by hardware-accelerated applications using our prototype. The results of our work show the feasibility of our system for a case study, JPEG encoding with two accelerators, and an evaluation of HybridOS for varying data movement requirements that can be used as a guide for future applications developers

Data-parallel co-processors have the potential to improve performance in highly parallel regions of code when coupled to a general-purpose CPU. However, applications often have to be modified in non-intuitive and complicated ways to mitigate the cost of data marshalling between the CPU and the co-processor. In some applications the overheads cannot be amortized and co-processors are unable to provide benefit. The additional effort and complexity of incorporating co-processors makes it difficult, if not impossible, to effectively utilize co-processors in large applications.

This paper argues for an implicitly parallel programming model for many-core microprocessors, and provides initial technical approaches towards this goal. In an implicitly parallel programming model, programmers maximize algorithm-level parallelism, express their parallel algorithms by asserting high-level properties on top of a traditional sequential programming language, and rely on parallelizing compilers and hardware support to perform parallel execution under the hood. In such a model, compilers and related tools require much more advanced program analysis capabilities and programmer assertions than what are currently available so that a comprehensive understanding of the input program's concurrency can be derived. Such an understanding is then used to drive automatic or interactive parallel code generation tools for a diverse set of parallel hardware organizations. The chip-level architecture and hardware should maintain parallel execution state in such a way that a strictly sequential execution state can always be derived for the purpose of verifying and debugging the program. We argue that implicitly parallel programming models are critical for addressing the software development crises and software scalability challenges for many-core microprocessors.