The removal of rain streaks and raindrops is crucial for enhancing the image visibility and mitigating the weather degradations. However, most existing approaches rely on the paired rainy and clean images, which are challenging to obtain in real-world scenarios. To this end, we propose a novel structure-preserving frequency-regularized text-guided optimal transport (SFTOT) framework, which formulates the unpaired rain streaks and raindrops removal as an optimal transport problem. Specifically, we introduce a structure-preserving transport cost, incorporating the structural similarity constraint to minimize the duality gap between the primal and dual formulations, while preserving the structural details of reconstructed images. Furthermore, by embedding the inherent frequency sparsity of rain streaks and raindrops into the transport cost, we derive a frequency-regularized optimal transport objective, ensuring consistency in frequency distributions between the generated and clean images. Additionally, we employ a pre-trained one-step stable diffusion model as the restoration network, which is fine-tuned using the low-rank adaptation (LoRA) adapters and zero convolutional layers, while integrating the domain-specific text prompts for both degraded and clean images to guide the generation process. Extensive experiments demonstrate that our method surpasses the existing well-performing unpaired learning approaches, achieving notable improvements in both the fidelity and photo-realism.

Brain-inspired spiking neural networks (SNNs) have the capability of energy-efficient processing of temporal information. However, leveraging the rich dynamic characteristics of SNNs and prior works in artificial neural networks (ANNs) to construct an effective object detection model for visual tasks remains an open question for further exploration. To develop a directly-trained , low energy consumption and high-performance multi-scale SNN model, we propose a novel interpretable object detection framework Multi-scale Spiking Detector (MSD). Initially, we propose a spiking convolutional neuron as a core component of the Optic Nerve Nucleus Block (ONNB), designed to significantly enhance the deep feature extraction capabilities of SNNs. ONNB enables direct training with improved energy efficiency, demonstrating superior performance compared to state-of-the-art ANN-to-SNN conversion and SNN techniques. In addition, we propose a Multi-scale Spiking Detection Framework to emulate the biological response and comprehension of stimuli from different objects. Wherein, spiking multi-scale fusion and the spiking detector are employed to integrate features across different depths and to detect response outcomes, respectively. Our method outperforms state-of-the-art ANN detectors, with only 7.8 M parameters and 6.43 mJ energy consumption. MSD obtains the mean average precision (mAP) of 62.0% and 66.3% on COCO and Gen1 datasets, respectively.

Transformer-based Large Language Models (LLMs) have become increasingly important. However, scaling LLMs to longer contexts incurs slow inference speed and high GPU memory consumption for caching key-value (KV) vectors. This paper presents RetrievalAttention, a training-free approach to both accelerate the decoding phase and reduce GPU memory consumption by pre-building KV vector indexes for fixed contexts and maintaining them in CPU memory for efficient retrieval. Unlike conventional KV cache methods, RetrievalAttention integrate approximate nearest neighbor search (ANNS) indexes into attention computation. We observe that off-the-shelf ANNS techniques often fail due to the out-of-distribution (OOD) nature of query and key vectors in attention mechanisms. RetrievalAttention overcomes this with an attention-aware vector index. Our evaluation shows RetrievalAttention achieves near full attention accuracy while accessing only 1-3\% of the data, significantly reducing inference costs. Remarkably, RetrievalAttention enables LLMs with 8B parameters to handle 128K tokens on a single NVIDIA RTX4090 (24GB), achieving a decoding speed of 0.107 seconds per token.

research-article Share on Query Processing on Gaming Consoles Authors: Wei Cui Microsoft Research Asia, CN Microsoft Research Asia, CN 0009-0005-9362-3585View Profile , Qianxi Zhang Microsoft Research Asia, CN Microsoft Research Asia, CN 0000-0002-0646-5365View Profile , Spyros Blanas The Ohio State University, US The Ohio State University, US 0009-0004-2703-7177View Profile , Jesús Camacho-Rodríguez Microsoft, US Microsoft, US 0009-0008-9151-6024View Profile , Brandon Haynes Microsoft Gray Systems Lab, US Microsoft Gray Systems Lab, US 0000-0002-1501-9586View Profile , Yinan Li Microsoft Research, US Microsoft Research, US 0009-0004-5483-2862View Profile , Ravi Ramamurthy Microsoft, USA Microsoft, USA 0000-0002-3484-0038View Profile , Peng Cheng Microsoft Research, CN Microsoft Research, CN 0000-0003-4014-4757View Profile , Rathijit Sen Microsoft, US Microsoft, US 0000-0003-4736-2837View Profile , Matteo Interlandi Microsoft, US Microsoft, US 0000-0002-5756-8321View Profile Authors Info & Claims DaMoN '23: Proceedings of the 19th International Workshop on Data Management on New HardwareJune 2023Pages 86–88https://doi.org/10.1145/3592980.3595313Published:18 June 2023Publication History 0citation191DownloadsMetricsTotal Citations0Total Downloads191Last 12 Months191Last 6 weeks191 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Deep learning training on cloud platforms usually follows the tradition of the separation of storage and computing. The training executes on a compute cluster equipped with GPUs/TPUs while reading data from a separate cluster hosting the storage service. To alleviate the potential bottleneck, a training cluster usually leverages its local storage as a cache to reduce the remote IO from the storage cluster. However, existing deep learning schedulers do not manage storage resources thus fail to consider the diverse caching effects across different training jobs. This could degrade scheduling quality significantly.

Approximate Nearest Neighbor Search (ANNS) on high dimensional vector data is now widely used in various applications, including information retrieval, question answering, and recommendation. As the amount of vector data grows continuously, it becomes important to support updates to vector index, the enabling technique that allows for efficient and accurate ANNS on vectors.