Gate bootstrapping is a core primitive that enables arbitrary circuit evaluation in fully homomorphic encryption (FHE), where blind rotation remains the dominant performance bottleneck. In this work, we present a sub-millisecond NTRU-based gate bootstrapping scheme that achieves state-of-the-art performance through coordinated algorithmic, software, and hardware-level optimizations.

Large Language Models (LLM) incur substantial resource costs during inference, driving widespread interest in Parameter-Efficient Fine-Tuning (PEFT) techniques. Among these, LoRA dramatically reduces overhead by updating only a few low-rank adapters. However, Multi-tenant LoRA LLM inference faces challenges from heterogeneous requests and latency-throughput trade-offs. Moreover, inefficient adapter reuse, poor cache management, and non-adaptive batching strategies severely restrict inference efficiency, service quality, resource utilization, and fairness. To address these challenges, we propose CoLoRA—a collaborative scheduling framework for multi-tenant LoRA LLM inference, comprising four core modules: (1) Adaptive Priority Scheduling (APS), which dynamically integrates queue waiting time, adapter residency status, and SLA urgency to compute task priorities; (2) Adapter-Aware Scheduling (AAS), which enhances cache management by prioritizing SLA-critical, frequently used, and fairly shared adapters, thus reducing cold-start latency and fragmentation; (3) Load-Aware Batch Scheduling (LBS), which combines real-time GPU utilization and queue depth to adaptively form batches and coalesce tasks targeting the same adapter, thereby improving parallelism while controlling latency; and (4) Unified Scheduler (US), which periodically gathers system metadata to orchestrate the submodules collaboratively and employs a feedback loop to optimize global strategies online. Evaluation on realistic multi-tenant workloads and popular open-source LLM shows that CoLoRA, compared to conventional baselines, increases overall system throughput by 56.5%, reduces P95 latency of online requests by 34%, and significantly enhances GPU utilization and tenant-level fairness, demonstrating its promise for large-scale inference services.

Fully homomorphic encryption (FHE) enables arbitrary computation over encrypted data (ciphertext) without compromising confidentiality. Within this family, TFHE features versatile bootstrapping mechanisms that is attractive for security-critical applications. However, its prohibitive computational cost severely limits practical deployment. While hardware acceleration is promising, mere compute scaling fails to overcome the inherent barriers. In particular, the combination of limited algorithmic parallelism and inadequate understanding of hardware behaviors prevents full exploitation of the available performance headroom.

Federated deep learning (FDL) is a promising privacy-preserving approach for training deep neural networks on distributed datasets without raw data sharing. But the classical synchronous FDL faces straggler problem: slow trainers severely impede overall efficiency. Inspired by speculative execution techniques in modern processors, this paper proposes SwiftFL, a novel and efficient speculative training system for FDL. Instead of simply waiting for slower trainer, SwiftFL proactively updates the global model with predicted gradients, enabling faster trainers to speculatively initiate the next training round. Furthermore, a gradient compensation technique is proposed to correct mispredicted training without re-training. Finally, to overcome the model-drift problem caused by fast trainers perform more local training rounds, we propose a client selection strategy. This strategy determines whether trainers should perform speculative training by striking a balance between two metrics: model drift degree and local training efficiency. In the evaluation, we compare SwiftFL with four state-of-the-art FDL systems and demonstrate that SwiftFL achieves an average speedup of 6.08× while maintaining consistent final model accuracy.

Moving target defenses are a key proactive technique for defending against adversarial attacks by increasing the dynamics, randomness and diversification of the system, and constructs an ensemble of multiple DNNs to obtain stronger defense. However, their models run noticeably slower on existing DNN accelerators than single-model inference. Moreover, they lack the hardware assistance for supporting scheduling mechanism of moving target defenses.Accordingly, our work is the first to propose a novel acceleration framework for moving target defenses against adversarial attacks, called AegisX, which provides architecture support for accelerating the ensemble of moving target defenses with dynamic grouping and early stop. Our hardware architecture integrates three key innovations: 1) a novel scheduler enabling concurrent operator executions and supporting moving target defense scheduling patterns; 2) a utilization-aware resource allocation strategy that fully exploits temporal and spatial sharing; 3) a hierarchical scheduling mechanism with early stop and an idleness-aware resource borrowing scheme to utilize idle computing cores effectively. Benchmark evaluations reveal substantial gains in throughput and energy efficiency.

Fully Homomorphic Encryption (FHE) is a ground-breaking cryptographic technology that enables computation directly on encrypted data, but its practical adoption continues to be hindered by high computational costs. GPUs have emerged as an increasingly attractive acceleration platform, offering massive parallelism and architectural flexibility to accommodate rapidly evolving FHE algorithms. Despite notable advances, most efforts remain confined to isolated, single-level optimizations, limiting their ability to push performance boundaries. In this paper, we present Peregrine, an efficient GPU-based TFHE acceleration built upon multi-level co-design of external product (EP) operations across parallelism, implementation, and scheduling. First, we propose a synchronization-free key unrolling technique that restructures the execution pipeline via operator decoupling, thereby unlocking greater EP-level parallelism. Second, we consolidate fragmented operators into a matrix-centric execution pattern, yielding a high-arithmetic-intensity kernel that substantially enhances external product efficiency. Third, we propose a hierarchical tiling strategy that reformats ring ciphertexts into the Module structure and schedules polynomiallevel tiles for fine-grained GPU mapping of EP operations. Experimental results show that Peregrine outperforms up to$176.9 \times$and$2.2 \times$over state-of-the-art CPU and GPU baselines, respectively, demonstrating its strong applicability to security-critical workloads.

Fully homomorphic encryption (FHE) enables computations on encrypted data. Existing FHE schemes are primarily categorized into RLWE-based word-wise schemes and LWEbased bit-wise schemes. Efficient combination of different FHE schemes adapted to real-world applications has emerged as a research focus. This paper proposes UniFHE, the first FHE accelerator that supports diverse algebraic structures using general arithmetic units to achieve higher performance. UniFHE is compatible with both RLWE-based and LWE-based FHE schemes without modifications to their original algorithmic designs. To support both finite ring and complex field operations, UniFHE introduces a general arithmetic unit and further constructs core computation structures. To balance on-chip memory demands across different schemes, UniFHE adopts a multi-pipeline architecture for LWE-based schemes. The core functional units for RLWE-based schemes are spliced based on the LWE-based pipelines. Furthermore, an on-chip plaintext encoding mechanism significantly reduces off-chip memory bandwidth demands. Experimental results show that, beyond superior area and energy efficiency, UniFHE delivers up to$13.6 \times$higher performance compared to scheme-specific accelerator combinations. Moreover, in hybrid schemes, UniFHE achieves a$3.2 \times$speedup over the state-of-the-art unified FHE accelerator Trinity.

Abstract Fully Homomorphic Encryption (FHE) enables arbitrary computation on encrypted data without decryption, making it a cornerstone of privacy-preserving outsourcing, such as cloud computing. However, homomorphic operations cause ciphertext noise to grow until decryption fails. The efficient solution is bootstrapping, which refreshes the noise in FHE ciphertexts to sustain arbitrary deep homomorphic evaluation. But in practice, bootstrapping consumes over 50% of total execution time, posing a serious obstacle to FHE adoption. This paper presents a systematic survey of FHE bootstrapping algorithms and their optimizations. We organize existing works into three main paradigms: word-wise bootstrapping for BGV, BFV, and CKKS schemes; bit-wise bootstrapping for FHEW and TFHE schemes; and hybrid bootstrapping, which leverages both word-wise schemes and bit-wise schemes. We analyze the evolution of crucial techniques, highlight latest advances in reducing latency, enhancing parallelism, and controlling noise growth, and compare the advantages and limitations of different schemes. Finally, we discuss emerging research trends.

Abstract Ransomware has become one of the most destructive cyberattacks worldwide in recent years and has caused billions of dollars. Numerous defense mechanisms have been proposed to mitigate the ransomware threat. However, as attack technologies advance, ransomware has rapidly evolved. It employs sophisticated techniques to evade traditional defense mechanisms and has lead to devastating impacts on organizations globally. While many studies have focused on ransomware and its defense, none have provided a comprehensive overview of the ongoing battle between defense mechanisms and the evolving ransomware. They also do not explore the techniques employed by ransomware authors to evade detection. To fill this gap and motivate further research, we conduct an extensive investigation into evasive ransomware, including the common techniques they employ and the efforts researchers have made to counter them. Based on this, we offer a preliminary exploration of potential defense concepts that track multi-level events across different attack stages and leverage the correlations between them to construct the attack flow. This paper helps researchers in the ransomware field gain a comprehensive understanding of evasive ransomware. It also offers insights for future researchers to enhance defense mechanisms to cover the potential threats identified in this study, minimizing the losses caused by evasive ransomware.

Fully Homomorphic Encryption (FHE) enables encrypted data processing on untrusted cloud servers, crucial for privacy-sensitive applications. Despite its potential, performance overheads (about 10, 000× slower) limit adoption. ASIC accelerators outperform GPUs/FPGAs by optimizing specific operations but rely on costly 7nm processes and large on-chip memory, hindering cost-effective deployment. Balancing efficiency with manufacturing constraints remains critical. This paper presents TensorFHE+, a GPU-optimized FHE acceleration framework leveraging Tensor Cores to accelerate Number Theoretic Transform (NTT) operations. Key innovations include: 1) Decomposing CKKS kernels into vector/matrix operations for hardware utilization; 2) Vectorized modulo arithmetic; 3) Data layout optimization for memory efficiency. Evaluated on NVIDIA A100, TensorFHE+ outperforms TensorFHE [1] by 1.44× in average (up to 1.69× on ResNet-20) and surpasses prior GPU implementations [2], [3]. The design also demonstrates compatibility with commercial linear algebra accelerators, enabling efficient FHE deployment.

Cloud AI services have experienced rapid development, raising concerns to privacy protection of cloud tenants. Many proposals have been made to use the NPU Trusted Execution Environment (TEE) to protect AI workloads on the cloud. However, existing designs typically bind NPUs exclusively to a single tenant, preventing multiple tenants from sharing the computing power of the TEE-NPUs.As such, we have designed a novel TEE for discrete NPUs, named NXT (NPU eXtension for Trust), which breaks the exclusive binding architecture and allows multiple tenants to securely share the TEE-NPU cluster. Firstly, we introduced an NPU Trusted Agent (NTA) to the most privileged level of the TEE system to assist in the global scheduling of the TEE-NPU cluster. Secondly, we implemented a flexible isolation mechanism to provide security for multi-tenant fine-grained sharing of NPU resources. Thirdly, we support efficient communication between workloads within TEE-NPUs and with legacy NPUs, accelerating multi-workload collaborative computing. We evaluated NXT by extending gem5 and a cycle-accurate NPU simulator to build a prototype. Results show that NXT improves overall utilization by up to 3.49× and ANTT by up to 7.24×, with only 6.38% average overhead for scheduling and isolation. It also boosts parallel inference performance by 58.3% for GPT-2(XL) and 63.6% for LLaMA-13B compared to static protection schemes.

Robust variants represent a promising method to enhance model robustness against adversarial attacks through exploring diverse neural network architectures. However, the significant computational demand of training multiple variants often restricts adversarial defense techniques to a narrow range of model architectures, thus failing to fully exploit the robustness benefits of architectural variations. In this paper, we first reveal that function-preserving knowledge transfer can significantly speed up adversarial training of different architecture variants. Then, we propose ROBIN, a framework for accelerating adversarially robust multi-variant training. By utilizing the architectural similarities among variants, ROBIN facilitates efficient weight transformation across models via two tensor-level atomic operations, hastening the convergence of multiple variants. Our experiments indicate that ROBIN can accelerate the adversarial training process of various architecture variants by 2.56 × to 4.27 ×, enabling efficient exploration of robust network architectures.

We present ShiftPIR, a single-server Private Information Retrieval (PIR) protocol that gravity shifts both computation and communication overhead from the client to the server, thereby significantly improving overall efficiency. This shift is driven by the growing asymmetry between resource-constrained clients and compute-intensive servers, where server-side tasks can be effectively parallelized and scaled. To achieve this, ShiftPIR introduces a novel request generation method in which the client transmits only a compact plaintext offset derived from pre-uploaded seed ciphertexts. The server then reconstructs the full query ciphertexts using homomorphic rotations, eliminating the need for costly ciphertext generation and transmission on the client side. We further design a highly parallelizable query expansion mechanism that removes data dependencies between ciphertext rotations, enabling efficient GPU-based execution. Our experiments demonstrate that ShiftPIR reduces client-side latency to microseconds while maintaining a communication cost within 4X of the non-private baseline—far outperforming prior protocols with 104 -105 × overhead. Compared to the state-of-the-art protocol YPIR, ShiftPIR achieves up to 26X lower end-to-end latency.

In this paper, taking ReRAM-based PIM accelerators as an example, we present a novel attack framework called Corrosion Hammer, which builds based on the Bit Flip Attack (BFA).Unlike previous BFA methods that require explicit memory fault injection techniques, such as Row Hammer, to modify sensitive bits in the victim Neural Network model, Corrosion Hammer implants the trojan during the hardware-software co-design phase and flips sensitive bits using read disturbance, which is a common noise in ReRAM caused by normal read operations.Furthermore, we explore the impact of inputs on the activation time consumption of the trojan and propose a method to expedite activation using normal input.Our experimental results demonstrate that Corrosion Hammer achieves an extremely covert trojan implantation and activation method, with an adversarial attack success rate of 92.46%.Additionally, using a specially designed method, Trojan activation is 61.98× faster compared to activation in an undisturbed normal operation state.It provides a way to significantly speed up the Trojan activation.

Zero-knowledge proof (ZKP) allows a prover to convince a verifier of the truth of a statement without revealing any secret information. This property is utilized in numerous privacy-preserving applications. However, the huge overhead of proof generation impedes the widespread adoption of ZKP. As a result, many ZKP accelerators have been developed to speed up proof generation. However, existing accelerators are designed at the granularity of core operators and exhibit low hardware resource utilization and limited adaptability. In this paper, we identify the commonality of all computation stages in proof generation at the level of basic finite field arithmetic operations. Based on this insight, we propose LegoZK, a dynamically reconfigurable hardware accelerator for ZKP. LegoZK employs finite field arithmetic units (FAUs) as its fundamental components and integrates these FAUs with a hierarchical on-chip network (NoC). By dynamically configuring the FAUs and the NoC, LegoZK can effectively accelerate the entire proof generation process, achieving higher overall performance. Additionally, for the most time-consuming MSM, this paper proposes a fast, fully pipelined bucket reduction algorithm based on lookup tables, which significantly reduces the latency of MSM. Experimental results demonstrate that LegoZK achieves on average speedup of $31.96 \times$ and $11.30 \times$ in proof generation compared to the state-of-the-art ZKP ASIC accelerator PipeZK and the GPU accelerator GZKP, respectively. And compared to PipeZK, LegoZK achieves $\mathbf{5 0. 1 \%}$ area reduction and $\mathbf{3 7. 7 \%}$ power consumption reduction.

Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning approach that adaptes pre-trained Large Language Models (LLMs) to multi-tenant tasks by generating a variety of LoRA adapters. However, this approach faces significant security challenges and is particularly susceptible to malicious servers stealing model parameters and sensitive data. Existing research on addressing security risks in multi-tenant environments remains constrained and insufficient. This paper explores the security challenges and proposes the LoRATEE framework, which embeds LoRA adapters within a server-side Trusted Execution Environment (TEE) and employs a lightweight One-Time Pad (OTP) encryption mechanism to ensure secure data transmission. Additionally, we design a dynamic LoRA adapter prefetching mechanism to reduce I/O latency. Moreover, a LoRA adapter module equivalence-sharing strategy based on Parameter-Efficient Fine-Tuning (PEFT) and minimalist design principles was introduced to optimize adapters loading. Experimental results show that LoRATEE maintains inference efficiency while securing multi-tenant LoRA LLMs systems.

Ransomware attacks pose significant threats and have caused substantial economic losses across various industries worldwide. Existing defense mechanisms typically focus on detecting ransomware in environments free from interference by other legitimate programs. However, in real-world applications, ransomware often coexists with normal programs, resulting in fragmented behavioral patterns that reduce detection accuracy. To address this issue, we propose a system-level ransomware detection approach, named RanDoctor. This method leverages long-time series analysis to capture the behavioral characteristics of ransomware, thereby improving the comprehensiveness and accuracy of detection. To further enhance system performance, we design the Ranformer model, incorporating the ProbSparse self-attention mechanism and a distillation process. Experimental results demonstrate that the RanDoctor system achieves a detection accuracy of 99.5%, representing a 8.0% improvement over state-of-the-art detection models.

To enhance the robustness and accuracy of the small model, existing approaches combine adversarial training with knowledge distillation, introducing a comprehensive single-teacher model to improve the performance of the student model (small model). However, due to the limited knowledge of a teacher model, it appears "knowledge gain saturation" phenomenon. Therefore, we propose a PMP-Guided Adaptive Multi-Teacher Distillation with Meta-Learning. Pontryagin’s Maximum Principle is employed to solve the issue of inconsistent teaching objectives among teachers causing distinct optimization directions. Meanwhile, Meta-learning-network is designed to tackle the problem of a student struggling to balance the learned knowledge. A series of experiments conducted on public datasets demonstrate that our approach outperforms the state-of-the-art methods against various adversarial attacks.

Hardware prefetchers are essential for hiding memory latency and improving performance in commercial processors. However, recent studies have revealed that they can be exploited to launch side-channel attacks that leak sensitive data, recover cryptographic keys, and break the isolation of trusted execution environments. We observe that such attacks closely resemble classic memory safety violations, including buffer overflows, type confusion, use-after-free, and data race. This paper presents TIPS (Tag AugmentatIon for Prefetcher Security), a lightweight extension to the memory safety mechanisms already deployed in commercial processors. TIPS enhances memory tagging to protect prefetchers by enforcing tag-based array bounds, validating pointer types, associating prefetch patterns with their source threads or cores, and suppressing contentionbased interference. Experiments demonstrate that TIPS incurs less than 1.50 % performance overhead and 0.81 % area cost, while providing strong defense against a broad class of prefetcher side-channel attacks.

With the swift progression of artificial intelligence and deep learning, neural networks have achieved remarkable success in domains such as image recognition, natural language processing, and autonomous driving. However, the proliferation of data scales and the extensive deployment of computational resources have engendered significant privacy concerns for users. In scenarios involving personal sensitive data, the safeguarding of privacy is of utmost importance. Homomorphic encryption technology, particularly the CKKS scheme, is capable of performing computations with minimal computational error while preserving data privacy, and it has been extensively utilized in encrypted neural networks. This paper studies Bit-Flip Attacks (BFAs) on encrypted neural networks under the RNS-CKKS scheme. We empirically analyze the effects of bit flips at different memory locations—covering ciphertext data, model weights, and evaluation keys—and report their observable outcomes (silent misclassification, irregular yet decodable outputs, or computation aborts). Under a realistic threat model where the adversary cannot precisely target bytes nor observe model predictions, BFAs can corrupt results but do not leak additional information. Our findings indicate that key corruption often produces conspicuous anomalies that offer detection potential for defenders.

Fully Homomorphic Encryption (FHE) enables direct computation on encrypted data, providing substantial security advantages in cloud-based modern society.However, FHE suffers from significant computational overhead compared to plaintext computation, hindering its adoption in real-world applications.While many accelerators have been designed to address performance bottlenecks, most do not fully leverage cryptographic optimization technologies, leaving room for further performance enhancements.In this work, we propose FAST, an FHE accelerator incorporating recent cryptographic optimizations, including hoisting technology and the gadget decomposition key-switching method (named KLSS method).We analyze ciphertext level consumption throughout application execution and observe that workload requirements vary significantly with different ciphertext levels for both hybrid and KLSS key-switching methods.Additionally, we note the differing computational precision requirements for these key-switching methods.Based on these observations, we designed a versatile framework that supports multiple key-switching methods during a single application execution and integrates hoisting technology.

Fully Homomorphic Encryption (FHE) is an emerging cryptographic technique for privacy-preserving computation, which enables computations on the encrypted data.Nonetheless, the massive computational demands of FHE prevent its further application to real-world workloads.To tackle this problem, several studies focus on the ASIC-based acceleration for FHE.However, the rapid evolution of FHE algorithms poses challenges to the generality of ASIC accelerator design.By contrast, a number of works rely on GPGPUs for FHE accelerations, due to the high parallelism and flexibility provided by GPGPUs.In this work, we propose a GPGPU-based acceleration solution that supports the Cheon-Kim-Kim-Song (CKKS) scheme by further exploiting Tensor Core(TCU) capabilities.In our study, we * Both author contributed equally to this research.

With the growing use of large language models (LLMs) hosted on cloud platforms to offer inference services, privacy concerns about the potential leakage of sensitive information are escalating. Secure Multi-Party Computation (MPC) is a promising solution to protect the privacy in LLM inference. However, MPC requires frequent inter-server communication, causing high performance overhead. Inspired by the prevalent activation sparsity of LLMs, where most neuron are not activated after non-linear activation functions, we propose an efficient private inference system, Comet. This system employs an accurate and fast predictor to predict the sparsity distribution of activation function output. Additionally, we introduce a new private inference protocol. It efficiently and securely avoids computations involving zero values by exploiting the spatial locality of the predicted sparsity distribution. While this computation-avoidance approach impacts the spatiotemporal continuity of KV cache entries, we address this challenge with a low-communication overhead cache refilling strategy that merges miss requests and incorporates a prefetching mechanism. Finally, we evaluate Comet on four common LLMs and compare it with six state-of-the-art private inference systems. Comet achieves a$1.87\times-2.63\times$speedup and a$1.94\times-2.64\times$communication reduction.

Abstract Fully homomorphic encryption (FHE) enables direct computation on encrypted data without decryption, ensuring data privacy in cloud computing scenarios and preventing the leakage of sensitive information. However, the computational overhead of HE typically exceeds that of plaintext computation by 4 to 5 orders of magnitude, while energy consumption is 5 to 6 orders of magnitude higher. These substantial performance and energy overheads significantly hinder the widespread adoption of FHE. This paper proposed LP-HENN, a novel low-power and energy-efficient FHE accelerator architecture that leverages a RISC-V vector coprocessor and ReRAM crossbar arrays. LP-HENN targets power-constrained application scenarios such as edge devices, aiming to provide highly energy-efficient acceleration support for FHE applications. LP-HENN leverages the collaborative work of the vector processor and ReRAM crossbars, employing optimization strategies to achieve full pipelining and minimize memory access. Furthermore, this paper proposed a parameter selection model for early-stage architecture design, which achieves an optimal balance between performance and energy consumption through the collaborative optimization of multiple parameters. Experimental results show that, for an FHE-based convolutional neural network (HE-CNN) inference application, LP-HENN achieves a 31.82Ã- and 11920.56Ã- improvement in performance and energy efficiency, respectively, compared to CPU. Compared to FxHENN, the state-of-the-art FPGA-based FHE accelerator with high energy efficiency for edge devices, LP-HENN achieves a 2.36Ã- and 10.04Ã- improvement in performance and energy efficiency, respectively. The energy efficiency of LP-HENN is comparable to that of F1, the state-of-the-art ASIC FHE accelerator, while featuring a low power design suitable for edge computing.

Abstract The Resistive Random-Access-Memory (ReRAM) crossbar-based Processing-In-Memory (PIM) accelerator shows great promise in accelerating neural networks (NNs). This technique boasts low energy consumption and exceptional performance in multiplication and accumulations (MAC) operations, making ReRAM-based PIM accelerators an ideal solution for intelligent computing in wearable and low-power mobile devices. However, security concerns related to PIM have not been adequately addressed. In this paper, we present a new attack framework called SolutionName for ReRAM-based PIM accelerators. SolutionName builds upon the Bit Flip Attack (BFA), a weight modification attack that manipulates the NN function by flipping specific bits in the deployed quantized NN model. Unlike previous BFA methods that require explicit memory fault injection techniques, such as Row Hammer, to modify sensitive bits in the victim NN, SolutionName implants the trojan during the hardware-software co-design phase and flips sensitive bits using read disturbance. Read disturbance is a common noise in ReRAM caused by normal read operations. This approach enables the trojan to be activated quietly during normal use, eliminating the need for explicit attacks. Furthermore, we explore the impact of inputs on the activation time of the trojan and propose a method to expedite activation using normal input. Our experimental results demonstrate that SolutionName achieves an extremely covert trojan implantation and activation method, with an adversarial attack success rate of 94.38%. Additionally, with a specially designed method, the trojan activation can be accelerated on average by 61.98 $$\times $$ × , providing controllable activation.

The rapid advancement of Transformer-based large language models (LLMs) is presenting significant challenges for their deployment, primarily due to their enormous parameter sizes and intermediate results, which create a bottleneck in memory capacity for effective inference. Compared to traditional DRAM, Non-Volatile Memory (NVM) technologies such as Resistive Random-Access Memory (RRAM) and Phase-Change Memory (PCM) offer higher integration density, making them promising alternatives. However, before NVM can be widely adopted, its reliability issues, particularly manufacturing defects and endurance faults, must be addressed. In response to the limited memory capacity and reliability challenges of deploying LLMs in NVM, we introduce a novel low-overhead weight-level map, namedWolf.Wolfnot only integrates the addresses of faulty weights to support efficient fault tolerance but also includes the addresses of outlier weights in LLMs. This allows for tensor-wise segmented quantization of both outliers and regular weights, enabling lower-bitwidth quantization. TheWolfframework uses a Bloom Filter-based map to efficiently manage outliers and faults. By employing shared hashes for outliers and faults and specific hashes for faults,Wolfsignificantly reduces the area overhead. Building onWolf, we propose a novel fault tolerance method that resolves the observed issue of clustering critical incorrect outliers and fully leverages the inherent resilience of LLMs to improve fault tolerance capabilities. As a result,Wolfachieves segment-wise INT4 quantization with enhanced accuracy. Moreover,Wolfcan adeptly handle Bit Error Rates as high as$1 {\boldsymbol{\times}} 10^{-2}$without compromising accuracy, in stark contrast to the state-of-the-art approach where accuracy declines by more than 20%.

The wide deployment of Large Language Models (LLMs) has given rise to strong demands for optimizing their inference performance. Today’s techniques serving this purpose primarily focus on reducing latency and improving throughput through algorithmic and hardware enhancements, while largely overlooking their privacy side effects, particularly in a multi-user environment. In our research, for the first time, we discovered a set of new timing side channels in LLM systems, arising from shared caches and GPU memory allocations, which can be exploited to infer both confidential system prompts and those issued by other users. These vulnerabilities echo security challenges observed in traditional computing systems, highlighting an urgent need to address potential information leakage in LLM serving infrastructures. In this paper, we report novel attack strategies designed to exploit such timing side channels inherent in LLM deployments, specifically targeting the Key-Value (KV) cache and semantic cache widely used to enhance LLM inference performance. Our approach leverages timing measurements and classification models to detect cache hits, allowing an adversary to infer private prompts with high accuracy. We also propose a token-by-token search algorithm to efficiently recover shared prompt prefixes in the caches, showing the feasibility of stealing system prompts and those produced by peer users. Our experimental studies on black-box testing of popular online LLM services demonstrate that such privacy risks are completely realistic, with significant consequences. Our findings underscore the need for robust mitigation to protect LLM systems against such emerging threats.

With the widespread deployment of Deep-Learning-as-a-Service, secure multi-party computation-based outsourcing neural network (NN) inference has garnered significant attention for its high-security guarantee. Nevertheless, under the dishonest-majority setting with malicious adversaries, prior secure inference works are still costly in terms of communication and run-time. Additionally, existing outsourcing frameworks impose a substantial client-side design, which leads to obstacles in resource-constrained devices. To address the above challenges, we propose MD-SONIC, an online efficient and maliciously-secure framework for outsourcing NN inference with a dishonest majority. We first construct communication-efficient n-party protocols for the basic primitives such as fixed-point multiplication and most significant bit extraction by combining mask-sharing and TinyOT-sharing with SPD$\mathbb {Z}_{2^{k}}$seamlessly. Then, we build fast secure blocks for the widely used NN operators, including matrix multiplication, ReLU, and Maxpool, on top of our basic primitives. To enable an arbitrary number of users to outsource the secure inference task to n computing servers, we propose a lightweight-client and fast$\Sigma $paradigm named SPIN, stemming from zero-knowledge proofs. Our SPIN can be instantiated into a set of efficient outsourcing protocols over multiple algebraic structures (e.g., finite field and ring). We also conduct extensive evaluations of MD-SONIC on various neural networks. Compared to the work by Damgård et al. (IEEE S&P’19) and MD-ML (USENIX Security’24), we achieve up to$594.4\times $and$45.1\times $online communication improvements, and improve the online execution time by at most$14.3\times $(resp.$20.5\times $) and$1.8\times $(resp.$2.3\times $) in LAN (resp. WAN).

Fully homomorphic encryption (FHE) enables direct computation on encrypted data, making it a crucial technology for privacy protection. However, FHE suffers from significant performance bottlenecks. In this context, GPU acceleration offers a promising solution to bridge the performance gap. Existing efforts primarily focus on single-class FHE schemes, which fail to meet the diverse requirements of data types and functions, prompting the development of hybrid multi-class FHE schemes. However, studies have yet to thoroughly investigate specific GPU optimizations for hybrid FHE schemes. In this paper, we present an efficient GPU-based FHE scheme switching acceleration named Chameleon. First, we propose a scalable NTT acceleration design that adapts to larger CKKS polynomials and smaller TFHE polynomials. Specifically, Chameleon tackles synchronization issues by fusing stages to reduce synchronization, employing polynomial coefficient shuffling to minimize synchronization scale, and utilizing an SM-aware combination strategy to identify the optimal switching point. Second, Chameleon is the first to comprehensively analyze and optimize critical switching operations. It introduces CMux-level parallelization to accelerate LUT evaluation and a homomorphic rotation-free matrixvector multiplication to improve repacking efficiency. Finally, Chameleon outperforms the state-of-the-art GPU implementations by 1.23× in CKKS HMUL and 1.15× in bootstrapping. It also achieves up to 4.87× and 1.51× speedups for TFHE bootstrapping compared to CPU and GPU versions, respectively, and delivers a 67.3× average speedup for scheme switching over CPU-based implementation.

Federated tree-based models are popular in many real-world applications owing to their high accuracy and good interpretability. However, the classical synchronous method causes inefficient federated tree-based model training due to tree node dependencies. Inspired by speculative execution techniques in modern high-performance processors, this paper proposes FTSeir, a novel and efficient speculative federated learning system. Instead of simply waiting, FTSeir optimistically predicts the outcome of the prior tree node. By resolving tree node dependencies with a neural network-based split point predictor, the training tasks of child tree nodes can be executed speculatively in advance via separate threads. This speculation enables cross-layer concurrent training, thus significantly reducing the waiting time. Furthermore, we propose an eager verification mechanism to promptly identify mispredictions, thereby reducing wasted computing resources. On a misprediction, an incomplete rollback is triggered for quick recovery by reusing the output of the mis-speculative training, which reduces computational requirements. We implement FTSeir and evaluate its efficiency in a real-world federated learning setting with six public datasets. Evaluation results demonstrate that FTSeir achieves up to 3.45× and 3.60× speedup over the state-of-the-art gradient boosted decision trees and random forests implementations, respectively.

As a prevalent privacy-preserving technology, Trusted Execution Environment has become widely adopted in numerous commercial processors. Nonetheless, they remain susceptible to various controlled-channel attacks. Untrusted operating systems can deduce enclave secrets by manipulating page tables or observing allocation- or swap-based page faults. In this paper, we propose SecPaging, a novel secure enclave paging mechanism based on hardware-enforced and microcode-supported protection to prevent these attacks. First, enclave PTEs are protected through hardware isolation, preventing privileged attackers from malicious tampering or observations. Second, an Eager-Allocation mechanism is employed to prevent allocation-based controlled-channel attacks. Besides, a Record-Reload mechanism is proposed to prevent swap-based controlled-channel attacks. We simulate SecPaging on real SGX. Experiments demonstrate that controlled channel attacks can be defended with minimal performance overhead.

In the face of huge threats from adversarial attacks, developing an efficient defense mechanism is crucial for deep learning systems. Adversarial ensemble defense method is one of the most effective techniques for defending against adversarial attacks, which constructs ensembles of multiple DNNs to improve model's robustness. However, deploying ensemble defense methods on existing DNN inference systems is inefficient and impractical due to their dynamics and randomness. To this end, we propose an inference system for adversarial ensemble defense called Garrison, which can deliver robust and low-latency predictions using Multi-Instance GPUs. Garrison employs a multi-granularity GPU partitioning strategy, optimizing hardware utilization by capitalizing on the intrinsic heterogeneity of GPUs. It also integrates a reinforcement learning-based scheduling mechanism, enabling random ensemble of diverse defense models to enhance robustness while maintaining bounded latency. Our evaluations show that Garrison can improve adversarial robustness by up to 24.5%, while accelerating ensemble inference by up to 6.6X compared to the state-of-the-art inference framework.

Serverless computing has gained widespread attention, and Trusted Execution Environments (TEEs) are well-suited for safeguarding user privacy. However, the additional startup procedure introduced by TEEs imposes considerable performance overhead on confidential serverless workloads. This paper introduces a novel parallelized enclave startup design, EnTurbo, which eliminates the integrity dependence of the enclave startup procedure, accelerating it while ensuring its security. Additionally, EnTurbo parallelizes the measurement procedure, enabling multi-thread measurement for acceleration with provable security. We evaluate EnTurbo by running confidential serverless workloads on SGX simulation mode. Results show that EnTurbo effectively speeds up enclave serverless by 1.42x-6.48x (SGXv1) and 1.33x-3.76x (SGXv2).

Federated tree-based models are popular in many real-world applications owing to their high accuracy and good interpretability. However, the classical synchronous method causes inefficient federated tree model training due to tree node dependencies. Inspired by speculative execution techniques in modern high-performance processors, this paper proposes SpecFL, a novel and efficient speculative federated learning system. Instead of simply waiting, SpecFL optimistically predicts the outcome of the prior tree node. By resolving tree node dependencies with a split point predictor, the training tasks of child tree nodes can be executed speculatively in advance via separate threads. This speculation enables cross-layer concurrent training, thus significantly reducing the waiting time. Furthermore, we propose a greedy speculation policy to exploit speculative training for deeper inter-layer concurrent training and an eager rollback mechanism for lossless model quality. We implement SpecFL and evaluate its efficiency in a real-world federated learning setting with six public datasets. The evaluation results demonstrate that SpecFL can be 2.08-3.33x and 2.14-3.44x faster than the state-of-the-art GBDT and RF implementations, respectively.

Trusted Execution Environment (TEE) architectures have been deployed in various commercial processors to provide secure environments for confidential programs and data. However, as a relatively new feature against security threats, existing designs still face a number of problems. Exploiting the management vulnerabilities, attackers can disclose secrets via controlled-channel or micro-architecture side-channel attacks. To address these problems, this paper proposes a novel TEE architecture, named HyperTEE. In our architecture, enclave management tasks are decoupled from the original computing subsystem to a dedicated, physically isolated Enclave Manage-ment Subsystem (EMS). A properly architected EMS prevents current management vulnerabilities and offers more secure enclave communication. We implemented the HyperTEE prototype on the FPGA platform. Experiments show that HyperTEE only introduces less than 1% area overhead, and 2.0 % and 1.9 % performance overhead on average for enclaves and non-enclave workloads, respectively.

Fully Homomorphic Encryption (FHE) is crucial for privacy-preserving computing, which allows direct computation on encrypted data. While various FHE schemes have been proposed, none of them efficiently support both arithmetic FHE and logic FHE simultaneously. To address this issue, researchers explore the combination of different FHE schemes within a single application and propose algorithms for the conversion between them. Unfortunately, all prior ASIC-based FHE accelerators are designed to support a single FHE scheme, and none of them supports the acceleration for FHE scheme conversion. This necessitates FHE acceleration systems to integrate multiple accelerators for different schemes, leading to increased system complexity and hindering performance enhancement. In this paper, we present the first multi-modal FHE accelerator based on a unified architecture, which efficiently supports CKKS, TFHE, and their conversion scheme within a single accelerator. To achieve this goal, we first analyze the theoretical foundations of the aforementioned schemes and highlight their composition from a finite number of arithmetic kernels. Then, we investigate the challenges for efficiently supporting these kernels within a unified architecture, which include 1) concurrent support for NTT and FFT, 2) maintaining high hardware utilization across various polynomial lengths, and 3) ensuring consistent performance across diverse arithmetic kernels. To tackle these challenges, we propose a novel FHE accelerator named Trinity, which in-corporates algorithm optimizations, hardware component reuse, and dynamic workload scheduling to enhance the acceleration of CKKS, TFHE, and their conversion scheme. By adaptive select the proper allocation of components for NTT and MAC, Trinity maintains high utilization across NTTs with various polynomial lengths and imbalanced arithmetic workloads. The experiment results show that, for the pure CKKS and TFHE workloads, the performance of our Trinity outperforms the state-of-the- art accelerator for CKKS (SHARP) and TFHE (Morphling) by 1.49 x and 4.23 x, respectively. Moreover, Trinity achieves 919.3 x performance improvement for the FHE-conversion scheme over the CPU-based implementation. Notably, despite the performance improvement, the hardware overhead of Trinity is only 85 % of the summed circuit areas of SHARP and Morphling.

Traditional microkernel-based operating systems are popular in embedded and safety-critical applications due to their advantages in security, reliability, and scalability. In recent years, some OS projects like LionsOS and the Hong-Meng microkernel have focused on developing general-purpose operating systems based on microkernels. However, the strong isolation mechanisms of microkernel architectures often lead to performance degradation when implementing user-space system services. This is particularly evident in file systems, where data may need to be transferred across multiple system services, resulting in redundant data copying. To address this issue, this paper proposes a microkernel file system architecture based on a multi-server memory proxy, which reduces redundant data copying across processes in a multi-server design through centralized shared memory. Additionally, this architecture effectively abstracts the use of Direct Memory Access (DMA), allowing any process to securely and conveniently utilize DMA for zerocopy operations. Considering the high-reliability requirements of system services, we implemented the overall design using the Rust programming language, thus avoiding the security issues typically associated with traditional languages like C. Experimental results indicate that our design can enhance file buffered I/O throughput performance by 43%.

To defend against various adversarial attacks, it is essential to develop a robust and high computing efficiency defence framework. Adversarial ensemble learning is the most effective technique for defending against adversarial example attacks, which constructs ensembles of multiple DNNs with adversarial training to obtain stronger defense. However, ensemble models run noticeably slower on existing DNN accelerators than single-model inference. Deploying ensemble models on the existing DNN accelerators leads to many critical issues such as the underutilization of hardware resources. To tackle emerging challenges, we propose EnsGuard, a dynamic asymmetric multi-core systolic array architecture for adversarial ensemble learning inference to fully exploit both static and dynamic parallelism of ensemble models. Specifically, on the hardware level, we propose a novel instruction set extension and develop efficient architecture components to fully exploit the new hardware abstraction of scattered idle computing cores, and use them to dynamically create on-the-fly Neural Processing Units (fNPUs). Moreover, we propose a computing power recycle mechanism to run on-the-fly models (small models) on fNPUs by carefully orchestrating execution order of ensemble models for maximizing hardware resources and bandwidth utilization. On the software level, EnsGuard adopts an integrated hardware/randomized ensemble co-design optimizer aiming at winning both faster inference and higher adversarial robustness. On top of that, a multi-model mapping method based on decision tree is proposed to enable the interleaving of different DNN executions both spatially and temporally, and mitigate straggler problems. Evaluation with a diverse set of workloads shows significant gains in throughput (4.4×) and energy reduction (3.2×).

Understanding how to defend against adversarial attacks is crucial for ensuring the safety and reliability of these systems in real-world applications. Various adversarial defense methods are proposed, which aim at improving the robustness of neural networks against adversarial attacks by changing the model structure, adding detection networks, and adversarial purification network. However, deploying adversarial defense methods in existing DNN accelerators or defensive accelerators leads to many key issues. To address these challenges, this article proposessDNNGuard, an elastic heterogeneous DNN accelerator architecture that can efficiently orchestrate the simultaneous execution of original (target) DNN networks and thedetectalgorithm or network. It not only supports for dense DNN detect algorithms, but also allows for sparse DNN defense methods and other mixed dense-sparse (e.g., dense-dense and sparse-dense) workloads to fully exploit the benefits of sparsity. sDNNGuard with a CPU core also supports the non-DNN computing and allows the special layer of the neural network, and used for the conversion for sparse storage format for weights and activation values. To reduce off-chip traffic and improve resources utilization, a new hardware abstraction with elastic on-chip buffer/computing resource management is proposed to achieve dynamical resource scheduling mechanism. We propose anextended AI instruction setfor neural networks synchronization, task scheduling and efficient data interaction. Experiment results show that sDNNGuard can effectively validate the legitimacy of the input samples in parallel with the target DNN model, achieving an average 1.42× speedup compared with the state-of-the-art accelerators.

For Trusted Execution Environment (TEE), interrupt-based side-channel attacks are becoming significant threats. Malicious supervisors use interrupts to perform single-step side-channel attacks or to improve the accuracy of existing side-channel attacks. This paper proposes a secure interrupt handle mechanism dedicated to TEE, named ChaosINTC. (1) To prevent frequent interrupts, a dynamic interrupt response delay mechanism delays the interrupt delivery with a variable time. (2) To prevent maliciously modifying ISRs, an interrupt handler protecting mechanism performs isolation and integrity checking. We deployed ChaosINTC on an open-source RISC-V core and evaluated its performance via FPGA. Our design provides strong security with marginal hardware and performance costs.

In the cloud computing era, privacy protection is becoming pervasive in a broad range of applications (e.g., machine learning, data mining, etc). Fully Homomorphic Encryption (FHE) is considered the perfect solution as it enables privacy-preserved computation on untrusted servers. Unfortunately, the prohibitive performance overhead blocks the wide adoption of FHE (about 10, 000× slower than the normal computation). As heterogeneous architectures have gained remarkable success in several fields, achieving high performance for FHE with specifically designed accelerators seems to be a natural choice. Until now, most FHE accelerators have focused on efficiently implementing one FHE operation at a time based on ASIC and with significantly higher performance than GPU and FPGA. However, recent state-of-the-art FHE accelerators rely on an expensive and large on-chip storage and a high-end manufacturing process (i.e., 7nm), which increase the cost of FHE adoption.In this paper, we propose TensorFHE, an FHE acceleration solution based on GPGPU for real applications on encrypted data. TensorFHE utilizes Tensor Core Units (TCUs) to boost the computation of Number Theoretic Transform (NTT), which is the part of FHE with highest time-cost. Moreover, TensorFHE focuses on performing as many FHE operations as possible in a certain time period rather than reducing the latency of one operation. Based on such an idea, TensorFHE introduces operation-level batching to fully utilize the data parallelism in GPGPU. We experimentally prove that it is possible to achieve comparable performance with GPGPU as with state-of-the-art ASIC accelerators. TensorFHE performs 913 KOPS and 88 KOPS for NTT and HMULT (key FHE kernels) within NVIDIA A100 GPGPU, which is 2.61× faster than state-of-the-art FHE implementation on GPGPU; Moreover, TensorFHE provides comparable performance to the ASIC FHE accelerators, which makes it even 2.9× faster than the F1+ with a specific workload. Such a pure software acceleration based on commercial hardware with high performance can open up usage of state-of-the-art FHE algorithms for a broad set of applications in real systems.

Fully homomorphic encryption (FHE) holds great promise as an encryption technology for safeguarding privacy by enabling computations directly on encrypted data. However, FHE encounters significant performance bottlenecks due to the extensive utilization of number theoretic transform (NTT) and its inverse (INTT). Therefore, it is crucial to accelerate NTT to enhance the efficiency of FHE. Conventional NTT implementations rely on mandatory synchronization to maintain data consistency, which leads to two critical problems: excessive synchronizations and unexplored synchronization switching points. This paper presents NTTFusion, an efficient GPU-based NTT acceleration design that focuses on boosting the performance of NTT. (i) To reduce the number of synchronizations, we propose two types of stage fusion methods specifically designed for different polynomial lengths. For small polynomial lengths, we employ the butterfly decomposition approach, while for large polynomial lengths, we leverage the thread aggregation method. (ii) To explore the optimal synchronization switching point, we propose an SM-aware synchronization combination strategy to balance synchronization overhead and hardware utilization. Finally, we conduct experiments on a realistic NVIDIA GPU server and demonstrate that the butterfly decomposition method achieves up to 1.37× speedup compared to the state-of-the-art implementation. Furthermore, the thread aggregation method can yield up to 1.32× speedup for larger polynomial lengths. The optimal synchronization switching point-based NTT, which incorporates thread aggregation, can produce a maximum 1.6× performance boost under a typical large polynomial length.

Cross-core cache timing side-channel attacks, which observe cache access behavior of victims running on different physical cores to infer sensitive information, have become a significant threat. Although the attacks are covert, they cause the attacked cachelines to frequently migrate among cache hierarchies, rendering abnormal traffic. Based on this observation, the proposed scheme PiPoMonitor records cache-memory access traffic and prefetch suspicious lines under attack to interfere with adversaries’ probes. In pursuit of security and performance, PiPoMonitor exploits a Cuckoo filter as the recording structure and introduces two features to it: 1) autonomic deletion and 2) relocation accelerating. The former exponentially increases the uncertainty of record eviction against reverse engineering attacks, while the latter leverages a pipelined architecture to alleviate the impact of intensive filter queries on the memory critical path. PiPoMonitor is not only able to effectively mitigate cross-core cache attacks and defeat sophisticated defense-aware attackers but also induces a negligible performance penalty and acceptable hardware overhead.

Fully Homomorphic Encryption (FHE) enables secure offloading of computations to untrusted cloud servers as it allows computing on encrypted data. However, existing well-known FHE schemes suffer from heavy performance overheads. Thus numerous accelerations based on FPGAs, ASICs, and GPUs have been proposed. Compared to FPGAs and ASICs, GPUs have obvious advantages in productivity and development costs. And also, GPUs have already been widely deployed in commercial cloud or supercomputing centers. Therefore, we present HE-Booster, an efficient GPU-based FHE acceleration design. For single-GPU acceleration, a thorough systematic design is exploited to map five common phases in typical FHE schemes to the GPU parallel architecture. In particular, inspired by the regular architecture of NTT/INTT, a novel inter-thread local synchronization is proposed to exploit thread-level parallelism. For multi-GPU acceleration, we propose a scalable parallelization design that exploitsdata-level parallelismthrough fine-grained data partition under different representations. Finally, experiments on 1 NVIDIA GPU demonstrate that our work outperforms 251.7×, 78.5× and 164.9× than three mainstream CPU-based libraries HElib, SEAL, and PALISADE, and up to 170.5× speedup is obtained compared to the GPU-accelerated library cuHE. What's more, performing 8 homomorphic multiplications on 8 GPUs can deliver up to a 7.66× performance boost compared to a single-GPU implementation.

Speculative execution is a critical technique in modern high performance processors. However, continuously exposed transient execution attacks, including Spectre and Meltdown, disclosed a large attack surface in mispredicted execution. Current state-of-the-art defense strategy blocks all memory accesses that use addresses loaded speculatively. However, propagation of base addresses is common in general applications and we find that more than 60% blocked memory accesses use propagated base rather than offset addresses. Therefore, we propose a novel hardware defense mechanism, named Conditional Address Propagation, to identify safe base addresses through taint tracking and address checking by a History Table. Then, the safe base addresses are allowed to be propagated to retrieve performance. For remaining unsafe addresses, they cannot be propagated for security. We constructed experiments on cycle-accurate Gem5 simulator. Compared to the representative study, STT, our mechanism effectively decreases the performance overhead from 13.27% to 1.92% targeting Spectre-type and 19.66% to 5.23% targeting all-type cache-based transient execution attacks.

Recently exposed vulnerabilities reveal the necessity to improve the security of branch predictors. Branch predictors record history about the execution of different processes, and such information from different processes are stored in the same structure and thus accessible to each other. This leaves the attackers with the opportunities for malicious training and malicious perception. Physical or logical isolation mechanisms such as using dedicated tables and flushing during context-switch can provide security but incur non-trivial costs in space and/or execution time. Randomization mechanisms incurs the performance cost in a different way: those with higher securities add latency to the critical path of the pipeline, while the simpler alternatives leave vulnerabilities to more sophisticated attacks.This paper proposes HyBP, a practical hybrid protection and effective mechanism for building secure branch predictors. The design applies the physical isolation and randomization in the right component to achieve the best of both worlds. We propose to protect the smaller tables with physically isolation based on (thread, privilege) combination; and protect the large tables with randomization. Surprisingly, the physical isolation also significantly enhances the security of the last-level tables by naturally filtering out accesses, reducing the information flow to these bigger tables. As a result, key changes can happen less frequently and be performed conveniently at context switches. Moreover, we propose a latency hiding design for a strong cipher by precomputing the "code book" with a validated, cryptographically strong cipher. Overall, our design incurs a performance penalty of 0.5% compared to 5.1% of physical isolation under the default context switching interval in Linux.

Graphic processing units (GPUs) are becoming an essential computational resource in a widely range of domains. At the same time, the security of GPUs has emerged as a primary concern in security-sensitive applications. Memory corruption attacks are the major threat in computer security. With contemporary discrete GPUs supporting unified address space, which allows GPU kernel functions to access the same address space with CPU host applications, memory bugs (e.g., buffer overflow and use-after-free) in GPUs can be exploited by attackers to maliciously tamping CPU data or even hijacking control flow. However, modern GPUs lack memory protection support against memory corruption attacks concurrently available in CPUs, and as a result suffer from security threats, as we demonstrate.In this paper, we migrate the conventional CPU memory protection mechanisms to GPUs and point out that directly adopting the CPU memory protection design to GPUs incurs significant performance overhead. The key reason is that: (1) fine-grained multithreading property of GPUs leads to high memory traffic for security metadata accessing, which leads to significant memory bandwidth contention between regular application data and security metadata and degrades the GPU performance; (2) shared L2 data cache suffers significant interference when regular data shared cache with metadata, which leads to increased cache miss rate and worsens the bandwidth contention problem.Based on our observation, we propose LAK, a low-overhead runtime memory safety solution for GPUs to provide comprehensive protection against memory corruption attacks. First, to provide a strong security guarantee, LAK employs a lock-and-key mechanism to enforce memory operations to only access allowed memory regions. Second, to mitigate bandwidth contention, which degrades GPU performance, LAK introduces two components: (a) a post-coalescing memory protection unit to reduce memory traffic of security metadata and (b) a highly-threaded dedicated L2 metadata cache to reduce the interference between metadata requests and data requests. Our evaluations show that LAK incurs a 19% performance degradation.

We present a Secure Accelerator Enclave design, which includes heterogeneous accelerator running AI workloads into the protection scope of Trusted Execution Environment, called TACC (Trusted Accelerator). TACC supports dynamic user switching and context clearing of accelerator enclave from the microarchitecture level; The physical isolation of in-package memory (3D chip package) and off-package memory is used to realize the full stack (from hardware to software) isolation of enclave internal running memory and external ciphertext memory; It is also equipped with independent hardware AES-GCM module (including DMA engine) to be responsible for the interaction between internal and external memory. On a FPGA development board containing Xilinx xc7z100-ffg900-2 chip, we implemented two versions of TACC prototypes: FAT (144 multipliers and 48 blockRAMs) and SLIM (36 multipliers and 12 blockRAMs). We deployed and ran the RepVGG inference neural networks on them respectively under different batch sizes. The average overhead of our security mechanism is no more than 1.76%.

Memory corruption attacks have existed for multiple decades, and have become a major threat to computer systems. At the same time, a number of defense techniques have been proposed by research community. With the wide adoption of CPU-based memory safety solutions, sophisticated attackers tend to tamper with system memory via direct memory access (DMA) attackers, which leverage DMA-enabled I/O peripherals to fully compromise system memory. The Input-Output Memory Management Units (IOMMUs) based solutions are widely believed to mitigate DMA attacks. However, recent works point out that attackers can bypass IOMMU-based protections by manipulating the DMA interfaces, which are particularly vulnerable to race conditions and other unsafe interactions.State-of-the-art hardware-supported memory protections rely on metadata to perform security checks on memory access. Consequently, the additional memory request for metadata results in significant performance degradation, which limited their feasibility in real world deployments. For quantitative analysis, we separate the total metadata access latency into DRAM latency, on-chip latency, and cache latency, and observe that the actual DRAM access is less than half of the total latency. To minimize metadata access latency, we propose EMC, a low-overhead heap memory safety solution that implements a tripwire based mechanism on the memory controller. In addition, by using memory controller as a natural gateway of various memory access data paths, EMC could provide comprehensive memory safety enforcement to all memory data paths from/to system physical memory. Our evaluation shows an 0.54% performance overhead on average for SPEC 2017 workloads.

Protecting code pointers (e.g., return address, function pointer) from leakage is desirable from a security perspective. Isolation mechanisms have been the favored candidate to protect code pointers. However, these mechanisms result in significant performance overhead as they need to instrument extra instructions for frequent permission switching or bound checking. In this paper, we propose CPP, a novel Code Pointer-only Memory Page Management to restrict attack-critical operations for code pointers by hardware. Our hardware–software co-design allows CPP mark code pointers at page granularity that requires minor hardware modification. CPP checks the legality of their operations in parallel with instruction execution. We implement a prototype system and our evaluation shows CPP can effectively mitigate the code pointer leakage attacks with less than 2.1% performance overhead.

Federated Learning (FL) is a promising approach to privacy-preserving machine learning. However, recent works reveal that gradients can leak private data. Using trusted SGX-processors for this task yields gradient-preserving but requires to prevent exploitation of any side-channel attacks.

Recently exposed vulnerabilities reveal that branch predictors shared by different processes leave the attackers with the opportunities for malicious training and perception. Instead of flush-based or physical isolation of hardware resources, we want to achieve isolation of the content in these hardware tables with some lightweight processing using randomization as follows. (1) Content encoding. We propose to use hardware-based thread-private random numbers to encode the contents of the branch predictor tables. It achieves a similar effect of logical isolation but adds little in terms of space or time overheads. (2) Index encoding. We propose a randomized index mechanism of the branch predictor. This disrupts the correspondence between the branch instruction address and the branch predictor entry, thus increases the noise for malicious perception attacks. Our analyses using an FPGA-based RISC-V processor prototype and additional auxiliary simulations suggest that the proposed mechanisms incur a very small performance cost while providing strong protection.

Cache side channel attacks obtain victim cache line access footprint to infer security-critical information. Among them, cross-core attacks exploiting the shared last level cache are more threatening as their simplicity to set up and high capacity. Stateful approaches of detection-based mitigation observe precise cache behaviors and protect specific cache lines that are suspected of being attacked. However, their recording structures incur large storage overhead and are vulnerable to reverse engineering attacks. Exploring the intrinsic non-determinate layout of a traditional Cuckoo filter, this paper proposes a space efficient Auto-Cuckoo filter to record access footprints, which succeed to decrease storage overhead and resist reverse engineering attacks at the same time. With Auto-Cuckoo filter, we propose PiPoMonitor to detect Ping-Pong patterns and prefetch specific cache line to interfere with adversaries' cache probes. Security analysis shows the PiPoMonitor can effectively mitigate cross-core attacks and the Auto-Cuckoo filter is immune to reverse engineering attacks. Evaluation results indicate PiPoMonitor has negligible impact on performance and the storage overhead is only 0.37%, an order of magnitude lower than previous stateful approaches.

Due to the wide deployment of deep learning applications in safety-critical systems, robust and secure execution of deep learning workloads is imperative. Adversarial examples, where the inputs are carefully designed to mislead the machine learning model is among the most challenging attacks to detect and defeat. The most dominant approach for defending against adversarial examples is to systematically create a network architecture that is sufficiently robust. Neural Architecture Search (NAS) has been heavily used as the de facto approach to design robust neural network models, by using the accuracy of detecting adversarial examples as a key metric of the neural network’s robustness. While NAS has been proven effective in improving the robustness (and accuracy in general), the NAS-generated network models run noticeably slower on typical DNN accelerators than the hand-crafted networks, mainly because DNN accelerators are not optimized for robust NAS-generated models. In particular, the inherent multi-branch nature of NAS-generated networks causes unacceptable performance and energy overheads.To bridge the gap between the robustness and performance efficiency of deep learning applications, we need to rethink the design of AI accelerators to enable efficient execution of robust (auto-generated) neural networks. In this paper, we propose a novel hardware architecture, NASGuard, which enables efficient inference of robust NAS networks. NASGuard leverages a heuristic multi-branch mapping model to improve the efficiency of the underlying computing resources. Moreover, NASGuard addresses the load imbalance problem between the computation and memory-access tasks from multi-branch parallel computing. Finally, we propose a topology-aware performance prediction model for data prefetching, to fully exploit the temporal and spatial localities of robust NAS-generated architectures. We have implemented NASGuard with Verilog RTL. The evaluation results show that NASGuard achieves an average speedup of 1.74× over the baseline DNN accelerator.

Speculative execution side-channel vulnerabilities such as Spectre reveal that conventional architecture designs lack security consideration. This article proposes a software transparent defense framework, named as Conditional Speculation, against Spectre vulnerabilities found on traditional out-of-order microprocessors. It introduces the concept of security dependence to mark speculative memory instructions which could leak information with potential security risks. More specifically, security-dependent instructions are detected and marked with suspect speculation flags in the Issue Queue. All the instructions can be speculatively issued for execution in accordance with the classic out-of-order pipeline. For those instructions with suspect speculation flags, they are considered as safe instructions if their speculative execution dose not refill new cache lines with unauthorized privilege data. Otherwise, they are considered as unsafe instructions and thus not allowed to execute speculatively. To pursue a balance of performance and security, we investigate two filtering mechanisms, Cache-hit-based Hazard Filter and Trusted Page Buffer-based Hazard Filter to filter out false security hazards. As for true security hazards, we have two approaches to prevent them from changing cache states. One is to block all unsafe access, the other is to fetch them from lower-level caches or memory to a speculative buffer temporarily, and refill them after confirming that they are on the correct execution path. Our design philosophy is to speculatively execute safe instructions to maintain the performance benefits of out-of-order execution while delaying the cache updates for speculative execution of unsafe instructions for security consideration. We evaluate Conditional Speculation in terms of performance, security, and area. The experimental results show that the hardware overhead is marginal and the performance overhead is minimal.

Continuous Attacks are common cross-core cache side-channel attack scenarios that we observed, where adversaries frequently probe-target cache lines in a short time. Under Continuous Attacks, the attacked lines go through multiple load-evict processes between different cache (or memory) hierarchies, exhibiting Ping-Pong patterns. Identifying and obscuring these abnormal patterns effectively interfere with the attacker's probe and mitigate such attacks. Our recent proposal, Ping-Pong regulator (PPR), captures multiple Ping-Pong patterns by counting the reaccesses per cache line and blocks them with different obscuring actions (preload or lock). Although PPR mitigates Continuous Attacks, the added regulator directory (RDir) is vulnerable because it cannot record all cache lines simultaneously. Sophisticated attackers can evict the records of the attacked line from the RDir to avoid triggering defensive actions, thereby bypassing PPR. To improve robustness, we further propose PPR+, which dynamically changes the mapping of physical addresses to RDir locations by encryption and periodically changing keys. This randomness makes it difficult for attackers to evict target entries out of the RDir within a limited time. We show that PPR+ tolerates more than 100 years of attacks, induces negligible performance impacts (improves 0.13%), requires acceptable storage overhead (3.15%), and does not need any software support.

Recent studies show that Deep Neural Networks (DNN) are vulnerable to adversarial samples that are generated by perturbing correctly classified inputs to cause the misclassification of DNN models. This can potentially lead to disastrous consequences, especially in security-sensitive applications such as unmanned vehicles, finance and healthcare. Existing adversarial defense methods require a variety of computing units to effectively detect the adversarial samples. However, deploying adversary sample defense methods in existing DNN accelerators leads to many key issues in terms of cost, computational efficiency and information security. Moreover, existing DNN accelerators cannot provide effective support for special computation required in the defense methods.

Return-oriented programming (ROP) is the major exploitation technique to hijack control flow in the presence of non-executable page protections. ROP can be prohibited by ensuring that ret targets legal position. One method is to check whether the predecessor of the target of a ret is a call to identify the illegal use of return. Performing check at each ret with low performance overhead is challenging. To reduce the performance overhead, prior proposals check at critical API functions or system calls and rely on the OS to identify these events. The goal of this paper is to mitigate ROP attacks while incurring negligible storage and performance overheads, and without relying on OS support.

In this paper, we observed Continuous Attacks are one kind of common side channel attack scenarios, where an adversary frequently probes the same target cache lines in a short time. Continuous Attacks cause target cache lines to go through multiple load-evict processes, exhibiting Ping-Pong Patterns. Identifying and obscuring Ping-Pong Patterns effectively interferes with the attacker’s probe and mitigates Continuous Attacks. Based on the observations, this paper proposes Ping-Pong Regulator to identify multiple Ping-Pong Patterns and block them with different strategies (Preload or Lock). The Preload proactively loads target lines into the cache, causing the attacker to mistakenly infer that the victim has accessed these lines; the Lock fixes the attacked lines’ directory entries on the last level cache directory until they are evicted out of caches, making an attacker’s observation of the locked lines is always the L2 cache miss. The experimental evaluation demonstrates that the Ping-Pong Regulator efficiently identifies and secures attacked lines, induces negligible performance impacts and storage overhead, and does not require any software support.

Siamese network is compute-intensive learning model with growing applicability in a wide range of domains. However, state-of-art deep neural network (DNN) accelerators would not work efficiently for Siamese network, as their designs do not account for the algorithm properties of Siamese network. In this paper, we propose a Siamese network accelerator called SNA, the first Simultaneous Multi-Threading (SMT) hardware architecture to perform Siamese network inference with high performance and energy efficiency. We devise an adaptive inter-model computing resource partition and flexible on-chip buffer management mechanism based on the model parallelism and SMT design philosophy. Our architecture is implemented in Verilog and synthesized in a 65nm technology using Synopsys design tools. We also evaluate it with several typical Siamese networks. Compared to the state-of-art accelerator, on average, the SNA architecture offers 2.1x speedup and 1.48x energy reduction.

With its huge real-world demands, large-scale confidential computing still cannot be supported by today's Trusted Execution Environment (TEE), due to the lack of scalable and effective protection of high-throughput accelerators like GPUs, FPGAs, and TPUs etc. Although attempts have been made recently to extend the CPU-like enclave to GPUs, these solutions require change to the CPU or GPU chips, may introduce new security risks due to the side-channel leaks in CPU-GPU communication and are still under the resource constraint of today's CPU TEE.To address these problems, we present the first Heterogeneous TEE design that can truly support large-scale compute or data intensive (CDI) computing, without any chip-level change. Our approach, called HETEE, is a device for centralized management of all computing units (e.g., GPUs and other accelerators) of a server rack. It is uniquely designed to work with today's data centres and clouds, leveraging modern resource pooling technologies to dynamically compartmentalize computing tasks, and enforce strong isolation and reduce TCB through hardware support. More specifically, HETEE utilizes the PCIe ExpressFabric to allocate its accelerators to the server node on the same rack for a non-sensitive CDI task, and move them back into a secure enclave in response to the demand for confidential computing. Our design runs a thin TCB stack for security management on a security controller (SC), while leaving a large set of software (e.g., AI runtime, GPU driver, etc.) to the integrated microservers that operate enclaves. An enclaves is physically isolated from others through hardware and verified by the SC at its inception. Its microserver and computing units are restored to a secure state upon termination.We implemented HETEE on a real hardware system, and evaluated it with popular neural network inference and training tasks. Our evaluations show that HETEE can easily support the CDI tasks on the real-world scale and incurred a maximal throughput overhead of 2.17% for inference and 0.95% for training on ResNet152.

Continually disclosed vulnerabilities reveal that traditional computer architecture lacks the consideration of security. This article proposes a security-first architecture, with an Active Security Processor (ASP) integrated to conventional computer architectures. To reduce the attack surface of ASP and improve the security of the whole system, the ASP is physically isolated from Computation Processor Units (CPU) with an asymmetric address space, which enables both ASP and CPU to run their operating system and applications independently in their own memory space. Furthermore, the ASP, which has the highest privilege (Super Root) of the whole system, possesses two advantageous features. First, the ASP can efficiently access all CPU resources and collect multi-dimensional information to monitor malicious behaviors, meanwhile, the CPU cannot access the ASP's private resources in any way. Second, instead of being scheduled by CPUs, the ASP can actively manage the security mechanisms employed in either CPUs or the ASP. Based on the security-first architecture, we introduce several typical security tasks running on ASP. With different considerations in terms of system overhead, complexity and performance, we also explore four typical system-level implementations for integrating the ASP to the security-first architecture. The first-generation ASP was designed and implemented based on the 40nm technology, and a security computer system was implemented based on it. Evaluations on this real hardware platform demonstrate that the security-first architecture can protect the system effectively with minor performance impacts on computing workloads.

Deep neural network (DNN) models are widely used for inference in many application scenarios. DNN accelerators are not designed with security in mind, but for higher performance and lower energy consumption. Hence, they are suffering from the security risk of being attacked. The insecure design flaws of existing DNN accelerators can be exploited to recover the structure of DNN model from the plain instructions, thus the runtime environment can be controlled to obtain the weights of DNN model. Furthermore, the structure of DNN model running on the accelerator is acquired by the side channel information and interrupt status register. To protect general DNN accelerator from being attacked by model inversion attack, this paper proposes a secure and general architecture called NPUFort, which guarantees the confidentiality of the parameters of DNN model and mitigates side-channel information leakage. The experimental results demonstrate the feasibility and effectiveness of the secure architecture of DNN accelerators with negligible performance overhead.

Modern processor cores share the last-level cache and directory to improve resource utilization. Unfortunately, such sharing makes the cache vulnerable to cross-core cache side channel attacks. Recent studies show that information leakage through cross-core cache side channel attacks is a serious threat in different computing domains ranging from cloud servers and mobile phones to embedded devices. However, previous solutions have limitations of losing performance, lacking golden standards, requiring software support, or being easily bypassed.

Speculative execution side-channel vulnerabilities such as Spectre reveal that conventional architecture designs lack security consideration. This paper proposes a software transparent defense mechanism, named as Conditional Speculation, against Spectre vulnerabilities found on traditional out-of-order microprocessors. It introduces the concept of security dependence to mark speculative memory instructions which could leak information with potential security risk. More specifically, security-dependent instructions are detected and marked with suspect speculation flags in the Issue Queue. All the instructions can be speculatively issued for execution in accordance with the classic out-of-order pipeline. For those instructions with suspect speculation flags, they are considered as safe instructions if their speculative execution will not refill new cache lines with unauthorized privilege data. Otherwise, they are considered as unsafe instructions and thus not allowed to execute speculatively. To reduce the performance impact from not executing unsafe instructions speculatively, we investigate two filtering mechanisms, Cachehit based Hazard Filter and Trusted Page Buffer based Hazard Filter to filter out false security hazards. Our design philosophy is to speculatively execute safe instructions to maintain the performance benefits of out-of-order execution while blocking the speculative execution of unsafe instructions for security consideration. We evaluate Conditional Speculation in terms of performance, security and area. The experimental results show that the hardware overhead is marginal and the performance overhead is minimal.

Control-flow integrity (CFI) is a general method for preventing code-reuse attacks, which utilize benign code sequences to achieve arbitrary code execution. CFI ensures that the execution of a program follows the edges of its predefined static Control-Flow Graph: any deviation that constitutes a CFI violation terminates the application. Despite decades of research effort, there are still several implementation challenges in efficiently protecting the control flow of function returns (Return-Oriented Programming attacks). The set of valid return addresses of frequently called functions can be large and thus an attacker could bend the backward-edge CFI by modifying an indirect branch target to another within the valid return set. This article proposes RAGuard, an efficient and user-transparent hardware-based approach to prevent Return-Oreiented Programming attacks. RAGuard binds a message authentication code (MAC) to each return address to protect its integrity. To guarantee the security of the MAC and reduce runtime overhead: RAGuard (1) computes the MAC by encrypting the signature of a return address with AES-128, (2) develops a key management module based on a Physical Unclonable Function (PUF) and a True Random Number Generator (TRNG), and (3) uses a dedicated register to reduce MACs’ load and store operations of leaf functions. We have evaluated our mechanism based on the open-source LEON3 processor and the results show that RAGuard incurs acceptable performance overhead and occupies reasonable area.

It is fundamentally challenging to build a secure system atop the current computer architecture. The complexity in software, hardware and ASIC manufacture has reached beyond the capability of existing verification methodologies. Without whole-system verification, current systems have no proven security. It is observed that current systems are exposed to a variety of attacks due to the existence of a large number of exploitable security vulnerabilities. Some vulnerabilities are difficult to remove without significant performance impact because performance and security can be conflicting with each other. Even worse, attacks are constantly evolving, and sophisticated attacks are now capable of systematically exploiting multiple vulnerabilities while remain hidden from detection. Eagering to achieve security hardening of current computer architecture, existing defenses are mostly ad hoc and passive in nature. They are normally developed in responding to specific attacks spontaneously after specific vulnerabilities were discovered. As a result, they are not yet systematic in protecting systems from existing attacks and likely defenseless in front of zero-day attacks. To confront the aforementioned challenges, this paper proposes Security-first Architecture , a concept which enforces systematic and active defenses using Active Security Processors . In systems built based on this concept, traditional processors (i.e., Computation Processors ) are monitored and protected by Active Security Processors. The two types of processors execute on their own physically-isolated resources, including memory, disks, network and I/O devices. The Active Security Processors are provided with dedicated channels to access all the resources of the Computation Processors but not vice versa. This allows the Active Security Processors to actively detect and tackle malicious activities in the Computation Processors with minimum performance degradation while protecting themselves from the attacks launched from the Computation Processors thanks to the resource isolation.

The crowd stampede and terrorist attacks in public areas have now become more serious and dangerous threats due to the rapid increase in the population and scale of cities. Therefore, the analysis of crowd aggregation behavior has been a new research focus in the field of intelligent video surveillance. However, such public area scenes not only contain moving crowd but also contain other types of objects. The sizes of these objects are usually small, which make their appearances quite similar. Moreover, the individuals in a crowd move randomly and often occlude each other. All the above factors make the analysis of crowd aggregation very difficult. In this paper, the authors attempt to solve this problem in three aspects. First, a novel global feature is used to represent the moving crowd. This feature can well describe the spatial and the temporal motion information of points-of-interest. Second, a strategy is adopted to cluster the feature points first and then calculate the collectiveness. This makes the collectiveness computation of individual groups more consistent and effective. Finally, more comprehensive collective crowd descriptors are proposed to provide a detailed description of the crowd status. Based on the proposed descriptor, the authors realize the evolution analysis of the group movement and the crowd abnormal detection. The experiment results show that the proposed method is able to efficiently compute the crowd collectiveness in various public areas and provide a reliable reference for the public safety management.

Consolidated server racks are quickly becoming the standard infrastructure for engineering, business, medicine, and science. Such servers are still designed much in the way when they were organized as individual, distributed systems. Given that many fields rely on big-data analytics substantially, its cost-effectiveness and performance should be improved, which can be achieved by flexibly allowing resources to be shared across nodes. Here we describe Venice, a family of data-center server architectures that includes a strong communication substrate as a first-class resource. Venice supports a diverse set of resource-joining mechanisms that enables applications to leverage non-local resources efficiently. We have constructed a hardware prototype to better understand the implications of design decisions about system support for resource sharing. We use it to measure the performance of at-scale applications and to explore performance, power, and resource-sharing transparency tradeoffs (i.e., how many programming changes are needed). We analyze these tradeoffs for sharing memory, accelerators, and NICs. We find that reducing/hiding latency is particularly important, the chosen communication channels should match the sharing access patterns of the applications, and of which we can improve performance by exploiting inter-channel collaboration.

Consolidated server racks are quickly becoming the backbone of IT infrastructure for science, engineering, and business, alike. These servers are still largely built and organized as when they were distributed, individual entities. Given that many fields increasingly rely on analytics of huge datasets, it makes sense to support flexible resource utilization across servers to improve cost-effectiveness and performance. We introduce Venice, a family of data-center server architectures that builds a strong communication substrate as a first-class resource for server chips. Venice provides a diverse set of resource-joining mechanisms that enables user programs to efficiently leverage non-local resources. To better understand the implications of design decisions about system support for resource sharing we have constructed a hardware prototype that allows us to more accurately measure end-to-end performance of at-scale applications and to explore tradeoffs among performance, power, and resource-sharing transparency. We present results from our initial studies analyzing these tradeoffs when sharing memory, accelerators, or NICs. We find that it is particularly important to reduce or hide latency, that data-sharing access patterns should match the features of the communication channels employed, and that inter-channel collaboration can be exploited for better performance.

The exploding growth of digitalized information has led to the rapid growth of data centers, both in numbers and in size. Cluster has been the dominating system architecture used in most data centers. However, the increasingly diversified data center applications have requirements beyond what the cluster architecture can deliver. For instance, clouding computing requires flexible sharing of all data center resources. Big data applications often need large memory capacity. A few applications can use GPGPU effectively. Existing system might be extended to a certain degree to meet those needs. Those extensions however would often be prohibitively expensive. The paper presents our attempt to design a system using commodity products that can meet the varying needs of many emerging data center applications in a cost-effective way. Our attempt is to create a system by connecting multiple nodes through a PCIe switch and then extend the software stack to support resource sharing among these nodes. In particular, a node can directly use the memory, NIC, and GPGPU of other nodes through the PCIe switch with no or little involvement from other nodes. We build a prototype as our evaluation platform. Our evaluation results indicate that those resources can be shared effectively in many cases. For using remote memory as block device, our prototype system has 5 times bandwidth, 11 times IOPS and 1/12 latency compared with the system connected by 10GigE in average for Orion benchmark; Using remote GPGPU via PCIe switch achieves average 60 times speedup than the case without GPGPU, and the performance loss is also acceptable (its average execution time is 1/3 of local GPGPU) for micro-benchmarks from GPU computing SDK; And using remote NIC via PCIe switch achieves average 95% bandwidth and 1.4 times latency of local NIC in httperf testing. While our prototype system offers multiple benefits, it is not perfect and has a lot room for further optimization and extension. We hope the outcome presented in this paper will encourage more researchers to join us in designing highly efficient and cost-effective servers.

In this work, we implement an ARMv8 function and performance simulator based on gem5 infrastructure, which is the first open source ARMv8 simulator. All the ARMv8 A64 instructions other than SIMD are implemented using gem5 ISA description language. The ARMv8 simulator supports multiple CPU models, multiple memory systems, and McPAT power model.

GPGPU can boost the performance significantly for many compute intensive tasks. However, in datacenter scenarios, not all the applications need GPGPU. Thus, it is not necessary to equip GPGPU in each node considering its high price and infrequent usage demands. By leveraging high bandwidth of emerging interconnect like 100Gb Ethernet, PCI Express or RapidIO etc., it is reasonable to figure out a mechanism to allow nodes to share the remote GPGPUs. In this work, we proposed V-OpenCL, which supports transparently using of remote GPGPU in data center.

In many computing domains, hardware accelerators can improve throughput and lower power consumption, instead of executing functionally equivalent software on the general-purpose micro-processors cores. While hardware accelerators often are stateless, network processing exemplifies the need for stateful hardware acceleration. The packet oriented streaming nature of current networks enables data processing as soon as packets arrive rather than when the data of the whole network flow is available. Due to the concurrence of many flows, an accelerator must maintain and switch contexts between many states of the various accelerated streams embodied in the flows, which increases overhead associated with acceleration. We propose and evaluate dynamic reordering of requests of different accelerated streams in a hybrid on-chip/memory based request queue in order to reduce the associated overhead.

The transistor density of microprocessors continues to increase as technology scales. Microprocessors designers have taken advantage of the increased transistors by integrating a significant number of cores onto a single die. However, a large number of cores are met with diminishing returns due to software and hardware scalability issues and hence designers have started integrating on-chip special-purpose logic units (i.e., accelerators) that were previously available as PCI-attached units. It is anticipated that more accelerators will be integrated on-chip due to the increasing abundance of transistors and the fact that not all logic can be powered at all times due to power budget limits. Thus, on-chip accelerator architectures deserve more attention from the research community. There is a wide spectrum of research opportunities for design and optimization of accelerators. This paper attempts to bring out some insights by studying the data access streams of on-chip accelerators that hopefully foster some future research in this area. Specifically, this paper uses a few simple case studies to show some of the common characteristics of the data streams introduced by on-chip accelerators, discusses challenges and opportunities in exploiting these characteristics to optimize the power and performance of accelerators, and then analyzes the effectiveness of some simple optimizing extensions proposed.