Initially developed for gaming and 3-D rendering, graphics processing units (GPUs) were recognized to be a good fit to accelerate deep learning training. Its simple mathematical structure can easily be parallelized and can therefore take advantage of GPUs in a natural way. Further progress in compute efficiency for deep learning training can be made by exploiting the more random and approximate nature of deep learning work flows. In the digital space that means to trade off numerical precision for accuracy at the benefit of compute efficiency. It also opens the possibility to revisit analog computing, which is intrinsically noisy, to execute the matrix operations for deep learning in constant time on arrays of nonvolatile memories. To take full advantage of this in-memory compute paradigm, current nonvolatile memory materials are of limited use. A detailed analysis and design guidelines how these materials need to be reengineered for optimal performance in the deep learning space shows a strong deviation from the materials used in memory applications.

While it has long been argued that near-threshold (~0.5V) operation of CMOS technologies can dramatically improve power efficiency, widespread application of such low voltage operation to VLSI systems has yet to materialize. This is due in part to practical system workload demands, in which single-thread performance needs can limit strategies to improve parallelizeable throughput performance, but also due to barriers in the ability of supporting hardware to counter variability and reliability concerns while maintaining power efficiency throughout the system. This paper describes the issues on which the realization of near-threshold computing depends to explain why this strategy is not yet pervasive today. However, recent advancements across the spectrum of system design--including heterogeneous architectures, transistor and memory technologies, power delivery, packaging, and I/O--suggest that as the market for throughput performance grows, hardware technologies may soon become available to practically harness the promise of near-threshold operation.

After decades of continuous scaling, further advancement of silicon microelectronics across the entire spectrum of computing applications is today limited by power dissipation. While the trade-off between power and performance is well-recognized, most recent studies focus on the extreme ends of this balance. By concentrating instead on an intermediate range, an ~ 8× improvement in power efficiency can be attained without system performance loss in parallelizable applications-those in which such efficiency is most critical. It is argued that power-efficient hardware is fundamentally limited by voltage scaling, which can be achieved only by blurring the boundaries between devices, circuits, and systems and cannot be realized by addressing any one area alone. By simultaneously considering all three perspectives, the major issues involved in improving power efficiency in light of performance and area constraints are identified. Solutions for the critical elements of a practical computing system are discussed, including the underlying logic device, associated cache memory, off-chip interconnect, and power delivery system. The IBM Blue Gene system is then presented as a case study to exemplify several proposed directions. Going forward, further power reduction may demand radical changes in device technologies and computer architecture; hence, a few such promising methods are briefly considered.

3D integration offers a technology that meets the requirements of the current trend in high performance microprocessors.

Despite generation upon generation of scaling, computer chips have until now remained essentially 2-dimensional. Improvements in on-chip wire delay and in the maximum number of I/O per chip have not been able to keep up with transistor performance growth; it has become steadily harder to hide the discrepancy. 3D chip technologies come in a number of flavors, but are expected to enable the extension of CMOS performance. Designing in three dimensions, however, forces the industry to look at formerly-two- dimensional integration issues quite differently, and requires the re-fitting of multiple existing EDA capabilities.