During inference for transformer-based large language models (LLM), prefilling is the computation of the key-value (KV) cache for input tokens in the prompt prior to autoregressive generation. For longer input prompt lengths, prefilling will incur a significant overhead on decoding time. In this work, we highlight the following pitfall of prefilling: for batches containing high-varying prompt lengths, significant computation is wasted by the standard practice of padding sequences to the maximum length. As LLMs increasingly support longer context lengths, potentially up to 10 million tokens, variations in prompt lengths within a batch become more pronounced. To address this, we propose prepacking, a simple yet effective method to optimize prefilling computation. To avoid redundant computation on pad tokens, prepacking combines prompts of varying lengths into a sequence and packs multiple sequences into a compact batch using a bin-packing algorithm. It then modifies the attention mask and positional encoding to compute multiple prefilled KV-caches for multiple prompts within a single sequence. On standard curated dataset containing prompts with varying lengths, we obtain a significant speed and memory efficiency improvements as compared to the default padding-based prefilling computation within Huggingface across a range of base model configurations and inference serving scenarios.

Large Language Models (LLMs) are increasingly deployed as chatbots, yet their ability to personalize responses to user preferences remains limited. We introduce PrefEval, a benchmark for evaluating LLMs' ability to infer, memorize and adhere to user preferences in long-context conversational setting. PrefEval comprises 3,000 manually curated user preference and query pairs spanning 20 topics. PrefEval contains user personalization or preference information in both explicit and implicit preference forms, and evaluates LLM performance using a generation and a classification task. With PrefEval, we have evaluated 10 open-sourced and proprietary LLMs in multi-session conversations with varying context lengths up to 100k tokens. We benchmark with various prompting, iterative feedback, and retrieval-augmented generation methods. Our benchmarking effort reveals that state-of-the-art LLMs face significant challenges in following users' preference during conversations. In particular, in zero-shot settings, preference following accuracy falls below 10\% at merely 10 turns (~3k tokens) across most evaluated models. Even with advanced prompting and retrieval methods, preference following still deteriorates in long-context conversations. Furthermore, we show that fine-tuning on PrefEval significantly improves performance. We believe PrefEval serves as a valuable resource for measuring, understanding, and enhancing LLMs' proactive preference following abilities, paving the way for personalized conversational agents.

Recent advancements in mixed-modal generative have opened new avenues for developing unified biomedical assistants capable of analyzing biomedical images, answering complex questions about them, and generating multimodal patient reports. However, existing datasets face challenges such as small sizes, limited coverage of biomedical tasks and domains, and a reliance on narrow sources. To address these gaps, we present MedMax, a large-scale multimodal biomedical instruction-tuning dataset for mixed-modal foundation models. With 1.47 million instances, MedMax encompasses a diverse range of tasks, including interleaved image-text generation, biomedical image captioning and generation, visual chat, and report understanding. These tasks span knowledge across diverse biomedical domains, including radiology and histopathology, grounded in medical papers and YouTube videos. Subsequently, we fine-tune a mixed-modal foundation model on the MedMax dataset, achieving significant performance improvements: a 26% gain over the Chameleon model and an 18.3% improvement over GPT-4o across 12 downstream biomedical visual question-answering tasks. Finally, we introduce a unified evaluation suite for biomedical tasks to guide the development of mixed-modal biomedical AI assistants. We release the code, data, and model at https://mint-medmax.github.io/.

Recent large language models (LLMs) have demonstrated strong reasoning capabilities that benefits from online reinforcement learning (RL). These capabilities have primarily been demonstrated within the left-to-right autoregressive (AR) generation paradigm. In contrast, non-autoregressive paradigms based on diffusion generate text in a coarse-to-fine manner. Although recent diffusion-based large language models (dLLMs) have achieved competitive language modeling performance compared to their AR counterparts, it remains unclear if dLLMs can also leverage recent advances in LLM reasoning. To this end, we propose, a framework to adapt pre-trained masked dLLMs into reasoning models via a combination of supervised finetuning (SFT) and RL. Specifically, we develop and extend techniques to improve reasoning in pretrained dLLMs: (a) we utilize a masked SFT technique to distill knowledge and instill self-improvement behavior directly from existing datasets, and (b) we introduce a novel critic-free, policy-gradient based RL algorithm called diffu-GRPO, the first integration of policy gradient methods to masked dLLMs. Through empirical studies, we investigate the performance of different post-training recipes on multiple mathematical and planning benchmarks. We find that d1 yields the best performance and significantly improves performance of a state-of-the-art dLLM.

Many applications of large language models (LLMs), ranging from chatbots to creative writing, require nuanced subjective judgments that can differ significantly across different groups. Existing alignment algorithms can be expensive to align for each group, requiring prohibitive amounts of group-specific preference data and computation for real-world use cases. We introduce Group Preference Optimization (GPO), an alignment framework that steers language models to preferences of individual groups in a few-shot manner. In GPO, we augment the base LLM with an independent transformer module trained to predict the preferences of a group for the LLM generations. For few-shot learning, we parameterize this module as an in-context autoregressive transformer and train it via meta-learning on several groups. We empirically validate the efficacy of GPO through rigorous evaluations using LLMs with varied sizes on three human opinion adaptation tasks. These tasks involve adapting to the preferences of US demographic groups, global countries, and individual users. Our results demonstrate that GPO not only aligns models more accurately but also requires fewer group-specific preferences and less training and inference computing resources, outperforming existing strategies such as in-context steering and fine-tuning methods.

In-context learning is an emergent paradigm in large language models (LLMs) that enables them to generalize to new tasks and domains by simply prompting these models with a few exemplars without explicit parameter updates. Many attempts have been made to understand in-context learning in LLMs as a function of model scale, pretraining data, and other factors. In this work, we propose a new mechanism to probe and understand in-context learning from the lens of decision boundaries for in-context binary classification. Decision boundaries are straightforward to visualize and provide important information about the qualitative behavior of the inductive biases of standard classifiers. To our surprise, we find that the decision boundaries learned by current LLMs in simple binary classification tasks are often irregularly non-smooth, regardless of task linearity. This paper investigates the factors influencing these decision boundaries and explores methods to enhance their generalizability. We assess various approaches, including training-free and fine-tuning methods for LLMs, the impact of model architecture, and the effectiveness of active prompting techniques for smoothing decision boundaries in a data-efficient manner. Our findings provide a deeper understanding of in-context learning dynamics and offer practical improvements for enhancing robustness and generalizability of in-context learning.

Reinforcement learning presents an attractive paradigm to reason about several distinct aspects of sequential decision making, such as specifying complex goals, planning future observations and actions, and critiquing their utilities. However, the combined integration of these capabilities poses competing algorithmic challenges in retaining maximal expressivity while allowing for flexibility in modeling choices for efficient learning and inference. We present Decision Stacks, a generative framework that decomposes goal-conditioned policy agents into 3 generative modules. These modules simulate the temporal evolution of observations, rewards, and actions via independent generative models that can be learned in parallel via teacher forcing. Our framework guarantees both expressivity and flexibility in designing individual modules to account for key factors such as architectural bias, optimization objective and dynamics, transferrability across domains, and inference speed. Our empirical results demonstrate the effectiveness of Decision Stacks for offline policy optimization for several MDP and POMDP environments, outperforming existing methods and enabling flexible generative decision making.

Neural network used for the LiDAR semantic segmentation task needs the point-wise labeled point clouds for training, which is more expensive than bounding box annotations. Enhancing the diversity of training data through object insertion is an effective method to reduce labeling costs. The existing object insertion methods are mainly divided into two categories. First is “copy” the clusters from a LiDAR frame and “paste” it to other frames or positions. Second is inserting CAD models into the background then using LiDAR simulator to generate laser points of the inserted CAD models. “Copy-paste” method cannot generate realistic scanning lines and shadows, and the CAD models, especially the CAD models of flexible objects, are hard to obtain. We propose an object insertion based data augmentation method which can increase the performance of the semantic segmentation network remarkably. First, an object library is created by using the labeled LiDAR point clouds. Then, these objects are inserted into the LiDAR point clouds dynamically during the training. Finally, the realistic scanning lines and shadows are simulated according to the real LiDAR parameters. The experimental results show that the proposed augmentation method can increase the performance of different semantic segmentation frameworks remarkably.

We present a study to examine one's learning and processing capacity of broadband tactile information, such as that derived from speech. In Study 1, we tested a user's capability to recognize tactile locations and movements on the forearm in the presence of masking stimuli and determined 9 distinguishable tactile symbols. We associated these symbols to 9 phonemes using two approaches, random and articulation associations. Study 2 showed that novice participants can learn both associations. However, performance for retention, construction of words and knowledge transfer to recognize unlearned words was better with articulation association. In study 3, we trained novel participants to directly recognize words before learning phonemes. Our results show that novel users can retain and generalize the knowledge to recognize new words faster when they were directly train on words. Finally, Study 4 examined optimal presentation rate for the tactile symbols without compromising learning and recognition rate.

Our hands-on studio will explore how to create meaningful haptic interactions that engage different areas of the body. Participants will gain an understanding of apparent tactile illusions, a perception of illusory motion between two areas on the body, and apply this knowledge towards generating their own haptic experiences. We will introduce participants to Stereo Haptics, a toolkit used to quickly generate haptic sensations through audio platforms with off the shelf hardware and open source software. The studio begins with an introduction to haptics, haptic technology and the illusions it can help create. Next, participants will begin experimenting with Stereo Haptics and using the toolkit to create dynamic haptic interactions. In the final section, participants will work in groups to design haptic solutions for real-life scenarios. By the end of the studio, participants will have a good understanding of tactile illusions, how to create them, and how they can be applied to enrich tangible and embodied interaction using simple stereo-sound technologies.

Research on children's cognitive development has demonstrated the positive effects of listening to stories. However, traditional story listening is losing its appeal to other entertainment technology such as video games. Hence, there is growing interest in studying the influence of ancillary media such as sound and interactive effects, although haptic sensory input has remained relatively unexploited. We implemented a haptic vest that generates vibrotactile stimulation related to story content to augment story listening. Study 1 showed that 5- and 6-year olds, but not 4-year olds, could associate haptic effects with semantic interpretations. In Study 2, children listened to stories containing elements with or without haptic effects. The 5- and 6-year olds showed better comprehension of the haptically-signaled content in the higher-performance story. The results provide initial evidence that haptic effects can potentially enhance the reading/listening experience of children beyond 4 years.

Handheld and wearable devices frequently engage users with simple haptic feedback, such as alerting, shaking, and pulsating. Here we explored intermanual apparent tactile motion-illusory movement between two hands-as a means to enrich such feedback. A series of psychophysical experiments determined the control space for generating smooth and consistent motion across the hands while users held the device. Experiment 1 calibrated the system and showed that vibrotactile detection thresholds decreased with increasing frequency, with similar trends for both hands. Experiment 2 measured effects of vibrotactile parameters on perceived motion. Both duration and temporal separation of stimuli, but not frequency and amplitude, affected subjective motion ratings. In Experiment 3, subjective ratings showed that stimuli with gradual onsets produced a stronger percept of motion than those with abrupt onsets. Finally, Experiment 4 determined a multimodal factor to match moving visual cues across the screen to moving tactile motion across hands. Our results showed compression of visual duration by the tactile system by a factor of approximately 1/3 at two test frequencies. The results of this research are useful for media designers and developers to generate reliable motion across the hands and integrate haptic motion with visual media.

Despite a long history of use in communication, haptic feedback is a relatively new addition to the toolbox of special effects. Unlike artists who use sound or vision, haptic designers cannot simply access libraries of effects that map cleanly to media content, and they lack even guiding principles for creating such effects. In this article, we make progress toward both capabilities: we generate a foundational library of usable haptic vocabulary and do so with a methodology that allows ongoing additions to the library in a principled and effective way. We define a feel effect as an explicit pairing between a meaningful linguistic phrase and a rendered haptic pattern. Our initial experiment demonstrates that users who have only their intrinsic language capacities, and no haptic expertise, can generate a core set of feel effects that lend themselves via semantic inference to the design of additional effects. The resulting collection of more than 40 effects covers a wide range of situations (including precipitation, animal locomotion, striking, and pulsating events) and is empirically shown to produce the named sensation for the majority of our test users in a second experiment. Our experiments demonstrate a unique and systematic approach to designing a vocabulary of haptic sensations that are related in both the semantic and parametric spaces.