How can we accurately quantize a pre-trained Vision Transformer model? Quantization algorithms compress Vision Transformers (ViTs) into low-bit formats, reducing memory and computation demands with minimal accuracy degradation. However, existing methods rely on uniform precision, ignoring the diverse sensitivity of ViT components to quantization. Metric-based Mixed Precision Quantization (MPQ) is a promising alternative, but previous MPQ methods for ViTs suffer from three major limitations: 1) coarse granularity, 2) mismatch in metric scale across component types, and 3) quantization-unaware bit allocation. In this paper, we propose LampQ (Layer-wise Mixed Precision Quantization for Vision Transformers), an accurate metric-based MPQ method for ViTs to overcome these limitations. LampQ performs layer-wise quantization to achieve both fine-grained control and efficient acceleration, incorporating a type-aware Fisher-based metric to measure sensitivity. Then, LampQ assigns bit-widths optimally through integer linear programming and further updates them iteratively. Extensive experiments show that LampQ provides the state-of-the-art performance in quantizing ViTs pre-trained on various tasks such as image classification, object detection, and zero-shot quantization.

How can we share parameters within large language models to significantly reduce memory costs while preserving accuracy?While parameter sharing is a promising solution to the memory overhead of large language models, existing methods rely on naive grouping and fail to correct sharing-induced discrepancies.We propose an accurate and efficient parameter sharing framework, SharVeT (Similarity-aware sharing with Vector-based Tuning), which performs similarity-based grouping to ensure accurate sharing, allocates parameters adaptively to preserve diversity within each group, and applies lightweight refinement with knowledge distillation to correct sharing-induced discrepancies.Experiments show that SharVeT outperforms existing sharing methods, achieving up to 32.1% lower perplexity and 21.2% higher few-shot reasoning accuracy.

How can we accurately capture users’ diverse interests to provide more relevant recommendations based on their historical interactions? Recent advancements in recommender systems have led to the development of multi-interest recommendation models that attempt to capture the diverse interests of users through multiple interest vectors. While theoretically promising, existing implementations frequently struggle with oversimplifying user interests, where models tend to focus on a single dominant vector and overlook the relationships between multiple interests, failing to represent the full complexity of users’ interests. This limits the models’ ability to truly personalize and diversify the recommendations provided to users. In response to this challenge, we propose BaM (Ba lanced Interest Learning for Multi-interest Recommendation), a versatile training scheme tailored for multi-interest recommendation models that ensures the full utilization of all interest vectors, leading to more effective recommendations. Instead of prioritizing an interest vector with the highest similarity to the ground-truth item for loss computation, BaM exploits a soft-selection approach, ensuring balanced training across multiple interest vectors. Furthermore, BaM trains all interest representations simultaneously through a multi-interest loss function that accounts for the contributions of every interest. This allows for a broader consideration of multiple interest vectors which are also related to the users’ diverse preferences with varying degrees of relevance. Extensive experiments with real-world datasets show that BaM achieves up to 31.43% higher accuracy in sequential recommendation compared to the best competitor, resulting in the state-of-the-art performance.

How can we accurately recommend actions for users to control their devices at home? Action recommendation for smart home has attracted increasing attention due to its potential impact on the markets of Internet of Things (IoT). However, designing an effective action recommender system is challenging because it requires handling context correlations, considering both queried contexts and previous histories of users, and dealing with capricious intentions in history. In this work, we propose SmartSense, an accurate action recommendation method for smart home. For individual action, SmartSense summarizes its device control and temporal contexts in a self-attentive manner, to reflect the importance of the correlation between them. SmartSense then summarizes sequences considering queried contexts in a query-attentive manner to extract the query-related patterns from the sequential actions. SmartSense also transfers the commonsense knowledge from routine data to better handle intentions in action sequences. As a result, SmartSense addresses all three main challenges of action recommendation for smart home, and achieves the state-of-the-art performance giving up to 9.8% higher [email protected] than the best competitor.