Diffusion models have emerged as state-of-the-art generative AI models but face significant computational challenges due to their iterative denoising process. While quantization techniques help reduce computation, conventional methods often degrade accuracy, and non-matrix operations remain a latency bottleneck. We propose Picasso, an end-to-end diffusion accelerator featuring the novel Hyper-Precision 8 (HYP8) data type that balances numerical precision and hardware efficiency by extending dynamic range while preserving resolution for near-zero values. Picasso integrates Hyper-Efficient Reconfigurable Arrays (HERA) for matrix operations and Unified Non-Matrix Processing Engines (UNPE) for normalization, softmax, and element-wise computations. Fabricated in a 28-nm CMOS process, Picasso achieves 9.83 TOPS with 4.96 TOPS/W energy efficiency, outperforming prior works by up to$26.8\times $in speed,$2.8\times $in energy efficiency, and$30.5\times $in area efficiency, while maintaining FP16-equivalent generation quality with reduced memory footprint.

The Segment Anything Model (SAM) is a powerful foundation model for image segmentation, showing robust zero-shot generalization through prompt engineering. However, relying on manual prompts is impractical for real-world applications, particularly in scenarios where rapid prompt provision and resource efficiency are crucial. In this paper, we propose the Automation of Prompts for SAM (AoP-SAM), a novel approach that learns to generate essential prompts in optimal locations automatically. AoP-SAM enhances SAM’s efficiency and usability by eliminating manual input, making it better suited for real-world tasks. Our approach employs a lightweight yet efficient Prompt Predictor model that detects key entities across images and identifies the optimal regions for placing prompt candidates. This method leverages SAM’s image embeddings, preserving its zero-shot generalization capabilities without requiring fine-tuning. Additionally, we introduce a test-time instance-level Adaptive Sampling and Filtering mechanism that generates prompts in a coarse-to-fine manner. This notably enhances both prompt and mask generation efficiency by reducing computational overhead and minimizing redundant mask refinements. Evaluations of three datasets demonstrate that AoP-SAM substantially improves both prompt generation efficiency and mask generation accuracy, making SAM more effective for automated segmentation tasks.

This work presents Picasso, an end-to-end diffusion accelerator designed for enhancing the efficiency of diffusion-based machine learning models used in applications such as image and video generation, and inpainting. Picasso introduces a novel hyper-precision 8 (HYP8) data type and a reconfigurable architecture designed to significantly enhance hardware efficiency, providing an extended dynamic range without sacrificing accuracy. It also features a unified engine that streamlines the processing of all non-matrix operations and employs sub-block pipeline scheduling to reduce overall latency. Fabricated in 28nm CMOS technology, this accelerator achieves an energy efficiency of 4.96 TOPS/W and a peak performance of 9.83 TOPS. Compared to previous works, Picasso demonstrates speedups ranging from 8.4× to 26.8× while also improving energy and area efficiency by 1.1× to 2.8× and 3.6× to 30.5×, respectively.

Homomorphic encryption (HE) is a type of cryptography that allows computations to be performed on encrypted data. The technique relies on learning with errors problem, where data is hidden under noise for security. To avoid excessive noise, bootstrapping is used to reset the noise level in the ciphertext, but it requires a large key and is computationally expensive. The fully homomorphic encryption over the torus (TFHE) scheme offers a faster and programmable bootstrapping (PBS) algorithm, which is crucial for many privacy-focused applications. Nonetheless, the current TFHE scheme does not support ciphertext packing, resulting in low-throughput performance. To the best of our knowledge, this is the first work that thoroughly analyzes TFHE bootstrapping, identifies the TFHE acceleration bottleneck in GPUs, and proposes a hardware TFHE accelerator to solve the bottleneck.