The use of KV cache in LLM inference leads to large memory footprint and sub-optimal decoding performance. Prior studies typically address one of these two limitations by either offloading or stage-split inference. In this paper, we explore and reveal the possibility of a joint solution, and propose REPA, a GPU-PIM hybrid system to prototype this idea. We leverage reconfigurable ReRAM PIM to achieve fast KV cache persistence, and balance the requirement of processing speed and memory capacity. To fully unleash the parallelization potential of REPA, we propose optimizations in (1) architecture, (2) data mapping and (3) pipelining: (1) We propose bulk-wise memory instructions and multi-level controllers to enable finer-grained parallelism in the PIM device. (2) We propose locality-aware data mapping to make the best of the aforementioned architectural optimization, and reduce long-range data transfer on chip. (3) We adopt sub-batch pipelining to reduce idleness in batches, and propose transfer overlapping to shadow the KV cache transfer by computation. Experimental results show that REPA exhibits high inference speed, energy efficiency and integratability. It is 1.5--6.5× faster, and 8--10× more efficient than NVIDIA A100. It also outperforms state-of-the-art DRAM PIM systems by up to 1.4× for long context inference. When integrated into existing offloading systems, REPA achieves 1.4--2.0× offloading speed, and 1.2--1.4× end-to-end speedup, showcasing its high potential for fast KV cache offloading and processing.

Abstract Due to the absorption and scattering of underwater light, underwater images often have some phenomena such as colorcast, low contrast, and blurred details. Therefore, an underwater image enhancement method based on entropy weight fusion is proposed. First, white balance processing can effectively correct the blue (green) color appearance of the image. Then the white‐balanced images are converted from RGB space to LAB space, and L channel is processed with improved adaptive gamma correction, and then converted back to RGB space. CLAHE and bilateral filtering are performed in RGB space. The RGB space is converted to HSV space, the V channel is processed by single‐scale retinex algorithm combined with guided filtering, stretching the R channel with s‐cosine curve,and then converted back to RGB space. Finally, the three results are fused by entropy weight to obtain the final enhanced image. Experimental results show that the proposed algorithm can improve the contrast and clarity of underwater images, and effectively remove colorcast.