Tabular datasets are ubiquitous in data science applications. Given their importance, it seems natural to apply state-of-the-art deep learning algorithms in order to fully unlock their potential. Here we propose neural network models that represent tabular time series that can optionally leverage their hierarchical structure. This results in two architectures for tabular time series: one for learning representations that is analogous to BERT and can be pre-trained end-to-end and used in downstream tasks, and one that is akin to GPT and can be used for generation of realistic synthetic tabular sequences. We demonstrate our models on two datasets: a synthetic credit card transaction dataset, where the learned representations are used for fraud detection and synthetic data generation, and on a real pollution dataset, where the learned encodings are used to predict atmospheric pollutant concentrations. Code and data are available at https://github.com/IBM/TabFormer.

The Active Memory Cube (AMC) is a novel near-memory processor that exploits high memory bandwidth and low latency close to DRAM to execute scientific applications in an energy-efficient manner. Its energy efficiency is derived from a combination of its novel scalar-vector data-flow path combined with its simple control-flow path that required the development of a sophisticated compiler, co-designed with the architecture. Such co-design is commonly done using hand-tuned codes for simple kernels that typically do not capture the nuances of real-world applications or reveal the complexities of programming a heterogeneous system. At the same time, an entire application is intractable to an early-stage compiler. In this work we describe a progressive, iterative methodology to the co-design of the compiler and architecture for the AMC using LULESH, a real-world hydrodynamics proxy application. We focus on a procedure that calculates the kinematic variables for domain elements. During the concept phase we looked at simpler kernels directly derived from the procedure, and progressively moved to the entire procedure as the compiler and simulation environment matured. We found this progression from the simpler extracted kernels to the entire procedure useful in gradually exposing new issues in the compiler and architecture. Directly applying optimizations developed for the simpler kernels resulted in poor performance on the proxy application, but after developing new compiler passes, the former optimizations could be applied profitably. Co-design on a proxy application revealed opportunities to refine the micro architecture that were not identified with simple micro benchmarks alone. Development of future accelerators and their programming environments can benefit from a similar iterative co-design through component kernels to entire proxy applications.

Computer memories continue to serve the role that they first served in the electronic discrete variable automatic computer (EDVAC) machine documented by John von Neumann, namely that of supplying instructions and operands for calculations in a timely manner. As technology has made possible significantly larger and faster machines with multiple processors, the relative distance in processor cycles of this memory has increased considerably. Microarchitectural techniques have evolved to share this memory across ever-larger systems of processors with deep cache hierarchies and have managed to hide this latency for many applications, but are proving to be expensive and energy-inefficient for newer types of problems working on massive amounts of data. New paradigms include scale-out systems distributed across hundreds and even thousands of nodes, in-memory databases that keep data in memory much longer than the duration of a single task, and near-data computation, where some of the computation is off-loaded to the location of the data to avoid wasting energy in the movement of data. This paper provides a historical perspective on the evolution of memory architecture, and suggests that the requirements of new problems and new applications are likely to fundamentally change processor and system architecture away from the currently established von Neumann model.

We are at the threshold of an explosion in new data, produced not only by large, powerful scientific and commercial computers, but also by billions of low-power devices. The traditional techniques of processing such information by first storing them in databases and then manipulating and serving them through large computers are becoming too expensive. These complex large systems have a high acquisition cost, but in addition, suffer also from high running costs, especially in power consumption. Both these costs can be contained by recognizing that there is a precision implied by traditional computing that is not needed in the processing of most new types of data. The relaxation of precision can help in the wider exploitation of known energy-efficient modes of computing like throughput computing. More important, such relaxation provides us an opportunity to deploy in the processing of this vast new data the same low-power, low-cost technology that was used to generate the data in the first place. Such energy-efficient circuits suffer from greater unreliability and variability in performance when used in the high-throughput mode, but these problems can be addressed by changing the way we design such systems, changing the nature of the algorithms for such systems, and by modifying the expectation of the quality of results produced by such systems. We have called this the approximate computing paradigm.

Modern processors employ a large amount of hardware to dynamically detect parallelism in single-threaded programs and maintain the sequential semantics implied by these programs. The complexity of some of this hardware diminishes the gains due to parallelism because of longer clock period or increased pipeline latency of the machine.In this paper we propose a processor implementation which dynamically schedules groups of instructions while executing them on a fast simple engine and caches them for repeated execution on a fast VLIW-type engine. Our experiments show that scheduling groups spanning several basic blocks and caching these scheduled groups results in significant performance gain over fill buffer approaches for a standard VLIW cache.This concept, which we call DIF (Dynamic Instruction Formatting), unifies and extends principles underlying several schemes being proposed today to reduce superscalar processor complexity. This paper examines various issues in designing such a processor and presents results of experiments using trace-driven simulation of SPECint95 benchmark programs.

Trace driven simulation is a common technique for evaluating different machine design options. Since the computing resources needed for simulation depend on the size of the trace, it is not always practical to use the complete trace of an application for simulation. This paper proposes a new technique, profile-driven sampling, for obtaining a reduced trace that is representative of the complete trace. Inaccuracy in the prediction of performance using such a reduced trace, is generally less than that from reduced traces using published techniques. For example, the prediction inaccuracy due to profile-driven sampling is 50 percent less than that using periodic time-based sampling. Profile-driven sampling also eliminates sensitivity to sampling parameters, such as sample size and inter-sample interval of periodic window based trace sampling, resulting, in a simple and robust trace generation process.

Misprediction of conditional branches is a major cause for reduced performance in processor implementations with large numbers of functional units. We present a hardware scheme which records the path leading to a conditional branch in order to predict the outcome of the branch instruction more accurately. The proposed scheme is analyzed using instruction traces from integer benchmark programs. The results indicate that knowledge of path information leads to better prediction than knowledge of simply the previous branch outcomes for a given number of history items. The results further show that even for equivalent hardware cost, path-based correlation often outperforms patient-based correlation, especially when history information is periodically destroyed, for example, due to context switches.

This paper presents an efficient technique to analyze finite-state machines to determine an optimal one for branch prediction. It also presents results from using this technique to determine optimal 4-state branch predictors for applications in the SPECS9 benchmark suite running on the IBM RS/6000. The paper concludes that the simple 2-bit counter is the only machine that performs consistently well and close to the optimal over all applications.>

Introduces a technique to enhance the ability of dynamic ILP processors to exploit (speculatively executed) parallelism. Existing branch prediction mechanisms used to establish a dynamic window from which ILP can be extracted are limited in their abilities to: (i) create a large, accurate dynamic window, (ii) initiate a large number of instructions into this window in every cycle, and (iii) traverse multiple branches of the control flow graph per prediction. The authors introduce control flow prediction which uses information in the control flow graph of a program to overcome these limitations. They discuss how information present in the control flow graph can be represented using multiblocks, and conveyed to the hardware using Control Flow Tables and Control Flow Prediction Buffers. They evaluate the potential of control flow prediction on an abstract machine and on a dynamic ILP processing model. The results indicate that control flow prediction is a powerful and effective assist to the hardware in making more informed run time decisions about program control flow.>

Methods are presented for generating bounds on interconnection delays in a combinational network having specified timing requirements at its input and output terminals. An automatic placement program that uses wirability as its primary objective could use these delay bounds to generate length or capacitance bounds for interconnection nets as secondary objectives. Thus, unlike previous timing-driven placement algorithms, the desired performance of the circuit is guaranteed when a wirable placement meeting these objectives is found. Fast algorithms are provided that maximize the delay range, and hence the margin for error in layout, for various types of timing constraint.>

A relaxation approach for producing a placement of transistors in a CMOS cell in a grid layout style from the circuit schematic diagram is described. For a given objective function, the approach leads to optimal results in most of the cases attempted. Unlike previous constructive approaches, this approach is iterative. It is also quite flexible. It can be used for unrestricted circuit types and can handle a variety of other important parameters affecting the wireability of the layout. The procedure is targeted for use in the automatic generation of custom and gate-array cell libraries.>

A simple algorithm to perform global wiring is described. Repeated iterations of the algorithm tend to improve the quality of wiring by rerouting around congested areas. Various parameters can be set to give preference to short routes or to reduce the time taken by the algorithm. The algorithm has been tried out for several master-slice chips containing up to 3500 cells with good results. The technique is easily extended to standard cell chip design. An implementation for global wiring of a structured custom chip design style is described along with results. The technique is adaptable to higher level packaging such as chips on modules or modules on a board.

In the synthesis of digital circuits, one encounters the problem of identifying blocks which have been designed, so that there is no replication in the expensive effort of generating the physical layout of these blocks. We present a model for the synthesis of combinational logic into complex MOS circuits and present a ranking and unranking procedure to characterize the layout of each complex MOS circuit.

A new global wiring algorithm designed for implementation on special purpose physical design machines is described. This algorithm computes more accurate estimates of wiring channel demand and supply than other known algorithms. It also makes better use of this information in determining wire routes. By exploiting the parallel processing capability of an interconnected array of microcomputers, the global wiring is completed effectively and quickly even for large chips.