Recent research efforts focus on reducing the computational and memory overheads of Large Language Models (LLMs) to make them feasible on resource-constrained devices. Despite advancements in compression techniques, nonlinear operators like Softmax and Layernorm remain bottlenecks due to their sensitivity to quantization. We propose SoftmAP, a software-hardware co-design methodology that implements an integer-only low-precision Softmax using In-Memory Compute (IMC) hardware. Our method achieves up to three orders of magnitude improvement in the energy-delay product compared to A100 and RTX3090 GPUs, making LLMs more deployable without compromising performance.

Processing In-Memory (PIM) is a data-centric computation paradigm that performs computations inside the memory, hence eliminating the memory wall problem in traditional computational paradigms used in Von-Neumann architectures. The associative processor, a type of PIM architecture, allows performing parallel and energy-efficient operations on vectors. This architecture is found useful in vector-based applications such as Hyper-Dimensional (HDC) Reinforcement Learning (RL). HDC is rising as a new powerful and lightweight alternative to costly traditional RL models such as Deep Q-Learning. The HDC implementation of Q-Learning relies on encoding the states in a high-dimensional representation where calculating Q-values and finding the maximum one can be done entirely in parallel. In this article, we propose to implement the main operations of a HDC RL framework on the associative processor. This acceleration achieves up to \(152.3\times\) and \(6.4\times\) energy and time savings compared to an FPGA implementation. Moreover, HDRLPIM shows that an SRAM-based AP implementation promises up to \(968.2\times\) energy-delay product gains compared to the FPGA implementation.

Mixed-precision Deep Neural Networks (DNNs) provide an efficient solution for hardware deployment, especially under resource constraints, while maintaining model accuracy. Identifying the ideal bit precision for each layer, however, remains a challenge given the vast array of models, datasets, and quantization schemes, leading to an expansive search space. Recent literature has addressed this challenge, resulting in several promising frameworks. This paper offers a comprehensive overview of the standard quantization classifications prevalent in existing studies. A detailed survey of current mixed-precision frameworks is provided, with an in-depth comparative analysis highlighting their respective merits and limitations. The paper concludes with insights into potential avenues for future research in this domain.

This paper summarizes the talks of a special session on the IPF 2.0 project, a collaborative German-US research project that leverages self-awareness principles for the self-management of distributed systems of autonomous multiprocessor systems-on-chip (MPSoCs).

The Information Processing Factory (IPF) utilizes factory management principles to tackle the complexities of integrated embedded systems, ensuring continuous safe operation and optimization at runtime. This paper presents a hardware implementation of IPF that enables dynamic task migration across system resources, ensuring reliability in the face of internal or external failures. We demonstrate the effectiveness of IPF through the efficient migration of tasks in multiprocessor SoCs using a safety-critical pacemaker application as a case study. Despite the additional software and hardware requirements, implementing IPF in a pacemaker results in comparable reliability to dual modular redundancy (DMR) with faster service resumption and improved resource utilization.

In this paper, we propose and evaluate efficient sample point reuse methodologies for Importance Splitting- based yield estimation of memory designs to assess the impact of manufacturing variability induced design center shifts. The proposed methodologies enable yield estimation at almost no additional simulation cost to that incurred due to studying the yield at the nominal design center. In order to unbias the Importance Splitting conditional probabilities with respect to the manufacturing variability centers, we evaluate three techniques that rely on Importance Sampling, center projections and geometric ratios. The geometric-based approach is shown to be most accurate for center shifts up-to three sigma away from the nominal. The proposed reuse methodologies achieve up-to 4 orders of magnitude speedup for the analysis of a 132-D 16nm SRAM design compared to traditional Importance Splitting.

Rare fail event estimation methodologies suffer from inefficiency when dealing with high dimensional design space problems. Importance splitting overcomes this complexity by recursively computing the rare fail probability as a product of larger conditional probabilities in the 1-D performance metric space. Its efficiency, however, drops as the events become rarer. In this work, we propose a novel hybrid Importance Sampling Importance Splitting methodology for purposes of rare fail event estimation of high-dimensional memory designs. In this context, we propose and evaluate two methods for unbiasing the estimate, a geometric ratio-based and an Importance Sampling-based methodology. We demonstrate 3-5X reduction in runtime for both theoretical and 16nm SRAM design applications compared to traditional Importance Splitting approaches.