Transformer-based models have made significant advancements across various domains, largely due to the self-attention mechanism’s ability to capture contextual relationships in input sequences. However, processing long sequences remains computationally expensive for Transformer models, primarily due to theO(n2) complexity associated with self-attention. To address this, sparse attention has been proposed to reduce the quadratic dependency to linear. Nevertheless, deploying the sparse transformer efficiently encounters two major obstacles: 1) Existing system optimizations are less effective for the sparse transformer due to the algorithm’s approximation properties leading to fragmented attention, and 2) the variability of input sequences results in computation and memory access inefficiencies. We present Raptor-T, a cutting-edge transformer framework designed for handling long and variable-length sequences. Raptor-T harnesses the power of the sparse transformer to reduce resource requirements for processing long sequences while also implementing system-level optimizations to accelerate inference performance. To address the fragmented attention issue, Raptor-T employs fused and memory-efficient Multi-Head Attention. Additionally, we introduce an asynchronous data processing method to mitigate GPU-blocking operations caused by sparse attention. Furthermore, Raptor-T minimizes padding for variable-length inputs, effectively reducing the overhead associated with padding and achieving balanced computation on GPUs. In evaluation, we compare Raptor-T’s performance against state-of-the-art frameworks on an NVIDIA A100 GPU. The experimental results demonstrate that Raptor-T outperforms FlashAttention-2 and FasterTransformer, achieving an impressive average end-to-end performance improvement of 3.41X and 3.71X, respectively.

Neural architecture search (NAS) has emerged as a hot topic recently which makes artificial intelligence techniques easier to apply and reduces the demand for experts knowledge by generating deep neural network architectures automatically. However, most existing methods for neural architecture search (NAS) heavily rely on underlying black-box controllers to generate potential candidates of network architectures, and suffer from serious problems of lacking interpretability and search efficiency. In this paper, we propose Disentangled Neural Architecture Search (DNAS), which addresses the two issues by adopting disentangled NAS controller and efficient dense-sampling strategy. Specifically, DNAS learns disentangled factors of network architecture by explicitly encouraging the latent factors to be independent. This approach could not only achieve semantic interpretability but also allow us to conveniently identify the promising regions of representations corresponding to high-performance architectures. We further propose a dense-sampling strategy that conducts targeted architecture search within the promising regions to accelerate the searching process. Our DNAS owns several attractive features: 1) it can successfully learn semantic representations of architectures, including operation selection, skip connections, and layer order; 2) it can speed up the process for neural architecture search more than 13× by using dense-sampling and disentangled factors; 3) it can achieve higher accuracy under less computational cost—DNAS achieves state-of-the-art performance of 94.16% on NASBench-101, and 22.7% top-1 test error on ImageNet with 1.6 GPU-days.

In this paper, we introduce a novel privacy-preserving decentralized collaborative training approach for medical image analysis, based on a multi-Blockchain system. With this approach, researchers from different medical institutions can collaboratively train machine learning models without sharing sensitive patient data. A partial-parameter propagation approach is applied to prevent indirect patient information leakage during model propagation. Based on peer-to-peer communication in a multi-Blockchain system, one training task can benefit from other tasks in other Blockchains, according to their task similarity. Furthermore, a personalized model for each institution is learned after collaborative training. Compared with training models by gathering all participants' datasets, experiment results show that our approach achieves similar performance while protecting sensitive patient data. Transferring the trained model from other Blockchains is also proved to be effective in accelerating model convergence. The personalized model process further improves model accuracy.

This paper introduces a flexible, hybrid emulation platform for processor related emulation. It is based on a modern Xeon server and FPGA. With Intel processors implemented in FPGA and plugged into the Xeon server's processor socket, and with the Xeon BIOS modified to accommodate different cores, this platform is able to boot OS and run applications while interacting with the Xeon server's native hardware components. It enables researchers to do architectural changes in FPGA and quickly evaluate their effect on the whole platform. This platform is an ideal vehicle for research on processor technology, SoC architecture, reconfigurable computing, heterogeneous core architecture, etc.

Growing demand for energy-efficient, high-performance systems has resulted in the growth of innovative heterogeneous computing system architectures that use FPGAs. FPGA-based architectures enable designers to implement custom instruction streams executing on potentially thousands of compute elements. Traditionally, FPGAs have been used as compute elements on PCI devices; however, this does not allow the FPGAs to be co-processors. This paper describes a high-performance system architecture that is based on the Intel® Xeon® platform in which one or more FPGAs, acting as application accelerators, replace one or more processors in a dual/multi-processor (DP/MP) platform. The FPGA is thus connected directly to the Front Side Bus (FSB) and enjoys the same privileges as a processor, i.e., full participation in the coherency protocol, unrestricted access to system memory and to other processors via the high bandwidth, and low latency connection to the FSB. In addition, we also describe a software layer called the "Accelerator Abstraction Layer (AAL)", which provides a uniform, hardware- and/or platform-independent application interface. Applications written on AAL can be ported to multiple platforms that have different types of accelerators and the application does not have to be modified. In addition, the AAL also enables the developer/user to reprogram the FPGA on the fly (analogous to an operating system context switch) thereby utilizing the programmable nature of the FPGA. The resulting hardware/software stack creates a flexible and powerful platform for accelerator innovation and deployment.

Larger last level caches are being considered for bridging the performance gap between the processors and the memory subsystem. It requires much longer simulation time to exercise the whole cache and get accurate evaluation results. In this paper, we motivate the need for a trace-driven hardware/software co-simulation approach to solve this problem. We describe the components of the hardware/software co-simulation: (a) a hardware approach for FSB (front side bus) cycle accurate long trace extraction and (b) a software simulation infrastructure to simulate arbitrary length of traces limited only by the storage system. We compare this hardware/software co-simulation approach to previous approaches (software-only and hardware FPGA-cache simulation) and articulate why our proposed approach is more flexible, more repeatable and sufficiently fast for last-level cache exploration. Evaluation results based on our hardware/software co-simulation infrastructure shows that our approach provides accurate results and shows the importance of timing information in accurate trace-driven simulations. We also demonstrate that it is not adequate to use short traces to get accurate results. Instead, the whole trace for the whole lifecycle of the workload, or at least a long trace (~5 minutes) should be used to capture the real behavior of the workloads.

With the amount of data continuing to grow, extracting "data of interest" is becoming popular, pervasive, and more important than ever. Data mining, as this process is known as, seeks to draw meaningful conclusions, extract knowledge, and acquire models from vast amounts of data. These compute-intensive data-mining applications, where thread-level parallelism can be effectively exploited, are the design targets of future multi-core systems. As a result, future multi-core systems will be required to process terabyte-level workloads. To understand the memory system performance of data-mining applications, this paper presents the use of hardware-software co-simulation to explore the cache design space of several multi-threaded data mining applications. Our study reveals that the workloads are memory intensive, have large working-set sizes, and exhibit good data locality. We find that large DRAM caches can be useful to address their large working-set sizes