While large language models (LLMs) are increasingly capable of handling longer contexts, recent work has demonstrated that they exhibit the "lost in the middle" phenomenon (Liu et al., 2024) of unevenly attending to different parts of the provided context.This hinders their ability to cover diverse source material in multidocument summarization, as noted in the DI-VERSESUMM benchmark (Huang et al., 2024).In this work, we contend that principled content selection is a simple way to increase source coverage on this task.As opposed to prompting an LLM to perform the summarization in a single step, we explicitly divide the task into three steps-(1) reducing document collections to atomic key points, (2) using determinantal point processes (DPP) to perform select key points that prioritize diverse content, and (3) rewriting to the final summary.By combining prompting steps, for extraction and rewriting, with principled techniques, for content selection, we consistently improve source coverage on the DIVERSESUMM benchmark across various LLMs.Finally, we also show that by incorporating relevance to a provided user intent into the DPP kernel, we can generate personalized summaries that cover relevant source information while retaining coverage.

Conversational Recommendation Systems (CRSs) are a particularly interesting application for out-of-the-box LLMs due to their potential for eliciting user preferences and making recommendations in natural language across a wide set of domains. Somewhat surprisingly, we find however that in such a conversational application, the more questions a user answers about their preferences, the worse the model’s recommendations become. We demonstrate this phenomenon on a previously published dataset as well as two novel datasets which we contribute. We also explain why earlier benchmarks failed to detect this round-over-round performance loss, highlighting the importance of the evaluation strategy we use and expanding upon Li et al. (2023a). We also present preference elicitation and recommendation strategies that mitigate this degradation in performance, beating state-of-the-art results, and show how three underlying models, GPT-3.5, GPT-4, and Claude 3.5 Sonnet, differently impact these strategies. Our datasets and code are available at https://github.com/CtrlVGustavo/A-Flash- in-the-Pan-CRS.

Reading monolithic instructional documents in VR is often challenging, especially when tasks are collaborative. Here we present DocuBits, a novel method for transforming monolithic documents into small, interactive instructional elements. Our approach allows users to:(i) create instructional elements (ii) position them within VR and (iii) use them to monitor and share progress in a multi-user VR learning environment. We describe our design methodology as well as two user studies evaluating how both individual users and pairs of users interact with DocuBits compared to monolithic documents while performing a chemistry lab task. Our analysis shows that, for both studies, DocuBits had substantially higher usability, while decreasing perceived workload $(p \lt 0.001)$. Our collaborative study showed that participants perceived higher social presence, collaborator awareness as well as immersion and presence $(p \lt 0.001)$. We discuss our insights for using text-based instructions to support enhanced collaboration in VR environments.

Deep reading fosters text comprehension, memory, and critical thinking. The growing prevalance of digital reading on mobile interfaces raises concerns that deep reading is being replaced by skimming and sifting through information, but this is currently unmeasured. Traditionally, reading quality is assessed using comprehension tests, which require readers to explicitly answer a set of carefully composed questions. To quantify and understand reading behaviour in natural settings and at scale, however, implicit measures are needed of deep versus skim reading across desktop and mobile devices, the most prominent digital reading platforms. In this paper, we present an approach to systematically induce deep and skim reading and subsequently train classifiers to discriminate these two reading styles based on eye movement patterns and interaction data. Based on a user study with 29 participants, we created models that detect deep reading on both devices with up to 0.82 AUC. We present the characteristics of deep reading and discuss how our models can be used to measure the effect of reading UI design and monitor long-term changes in reading behaviours.

While paper instructions are a mainstream medium for sharing knowledge, consuming such instructions and translating them into activities can be inefficient due to the lack of connectivity with the physical environment. We propose PaperToPlace, a novel workflow comprising an authoring pipeline, which allows the authors to rapidly transform and spatialize existing paper instructions into an MR experience, and a consumption pipeline, which computationally places each instruction step at an optimal location that is easy to read and does not occlude key interaction areas. Our evaluation of the authoring pipeline with 12 participants demonstrates the usability of our workflow and the effectiveness of using a machine learning based approach to help extract the spatial locations associated with each step. A second within-subjects study with another 12 participants demonstrates the merits of our consumption pipeline to reduce context-switching effort by delivering individual segmented instruction steps and offering hands-free affordances.

Sharing annotations encourages feedback, discussion, and knowledge passing among readers and can be beneficial for personal and public use. Prior augmented reality (AR) systems have expanded these benefits to both digital and printed documents. However, despite smartphone AR now being widely available, there is a lack of research about how to use AR effectively for interactive document annotation. We propose Dually Noted, a smartphone-based AR annotation system that recognizes the layout of structural elements in a printed document for real-time authoring and viewing of annotations. We conducted experience prototyping with eight users to elicit potential benefits and challenges within smartphone AR, and this informed the resulting Dually Noted system and annotation interactions with the document elements. AR annotation is often unwieldy, but during a 12-user empirical study our novel structural understanding component allows Dually Noted to improve precise highlighting and annotation interaction accuracy by 13%, increase interaction speed by 42%, and significantly lower cognitive load over a baseline method without document layout understanding. Qualitatively, participants commented that Dually Noted was a swift and portable annotation experience. Overall, our research provides new methods and insights for how to improve AR annotations for physical documents.

Virtual reality (VR) offers the promise of an infinite office and remote collaboration, however, existing interactions in VR do not strongly support one of the most essential tasks for most knowledge workers, reading. This paper presents VRDoc, a set of gaze-based interaction methods designed to improve the reading experience in VR. We introduce three key components: Gaze Select-and-Snap for document selection, Gaze MagGlass for enhanced text legibility, and Gaze Scroll for ease of document traversal. We implemented each of these tools using a commodity VR headset with eye-tracking. In a series of user studies with 13 participants, we show that VRDoc makes VR reading both more efficient (p < 0.01) and less demanding (p < 0.01), and when given a choice, users preferred to use our tools over the current VR reading methods.

Conversations aimed at determining good recommendations are iterative in nature.People often express their preferences in terms of a critique of the current recommendation (e.g., "It doesn't look good for a date"), requiring some degree of common sense for a preference to be inferred.In this work, we present a method for transforming a user critique into a positive preference (e.g., "I prefer more romantic") in order to retrieve reviews pertaining to potentially better recommendations (e.g., "Perfect for a romantic dinner").We leverage a large neural language model (LM) in a fewshot setting to perform critique-to-preference transformation, and we test two methods for retrieving recommendations: one that matches embeddings, and another that fine-tunes an LM for the task.We instantiate this approach in the restaurant domain and evaluate it using a new dataset of restaurant critiques.In an ablation study, we show that utilizing critiqueto-preference transformation improves recommendations, and that there are at least three general cases that explain this improved performance.

We introduce a new budgeted framework for online influence maximization, considering the total cost of an advertising campaign instead of the common cardinality constraint on a chosen influencer set. Our approach models better the real-world setting where the cost of influencers varies and advertizers want to find the best value for their overall social advertising budget. We propose an algorithm assuming an independent cascade diffusion model and edge-level semi-bandit feedback, and provide both theoretical and experimental results. Our analysis is also valid for the cardinality-constraint setting and improves the state of the art regret bound in this case.

We address the problem of planning in an environment with deterministic dynamics and stochastic discounted rewards under a limited numerical budget where the ranges of both rewards and noise are unknown. We introduce PlaTypOOS, an adaptive, robust, and efficient alternative to the OLOP (open-loop optimistic planning) algorithm. Whereas OLOP requires a priori knowledge of the ranges of both rewards and noise, PlaTypOOS dynamically adapts its behavior to both. This allows PlaTypOOS to be immune to two vulnerabilities of OLOP: failure when given underestimated ranges of noise and rewards and inefficiency when these are overestimated. PlaTypOOS additionally adapts to the global smoothness of the value function. PlaTypOOS acts in a provably more efficient manner vs. OLOP when OLOP is given an overestimated reward and show that in the case of no noise, PlaTypOOS learns exponentially faster.

To study naturalistic in-cabin emotion we developed SKYNIVI, a modified open source driving simulator, with scenarios designed to elicit startle and frustration. We target generating these emotions because we believe that by detecting these it will be possible for autonomous vehicles to learn to drive better. We show how to use SKYNIVI to develop datasets that capture naturalistic emotions in drivers and passengers for algorithmic development. We recruited 51 participants as dyads and single drivers to participate in two different scenarios. We show that we were able to evoke hundreds of instances of our target emotions in this cohort and present an analysis of factors we found to impact emotional expression including: scenario design , demographic factors, personality and baseline affect . We find that having a second person in the vehicle impacts observed expressions of emotion even when no difference in baseline affect is reported.

An ideal mobile user interface provides users with just the information they want, when they want it. We believe that sensors in the ambient environment can help automatically showcase this information. In this paper, we describe how we inferred users' favorite lunch stations using indoor location trajectories. We had 109 users participate in our study over an eight month period and we were able to predict their lunch station choices with 85% accuracy using a heuristic algorithm. We describe our system, the data we collected and our post-hoc user assessment.

Automated driving has the potential to reduce the amount of fatal crashes, lighten the burden of commutes, and democratize mobility access to wider populations. But delegation of control to automation is not without issues. One of the foreseen drawbacks is that users might experience negative emotional reactions to unanticipated or unexplainable automated maneuvers. In this paper we present a novel method to induce targeted emotional reactions, frustration and startle, in simulated automated driving environments. We describe the data collection process for 17 driver - passenger dyads and discuss the data labelling method for generating reliable novel emotion datasets. This contribution is a foundational methodology towards expanding emotional understanding in automated vehicles, a critical skill for building long-term trusted experiences.

Natural emotional experiences happen "in the wild" as people are mobile, living their daily lives. To capture these experiences, emotion researchers often give participants smartphone applications with various graphical user interfaces (GUIs) to record how they are feeling, however, there exist few empirical tests that assess the comparative benefits and drawbacks of different GUI designs. This paper presents two empirical evaluations of three types of GUI designs for capturing emotion using both a 10 participant in-lab trial and a 100 participant AMT trial. We define GUI scoring metrics and report on participants' ability to rate real world scenarios and evocative images, respectively, in ways that are consistent with population norms and with respect their own emotion word choices. We additionally report on users preferences for different designs, their perceived ease of use and the average time taken to complete an assessment for the different designs.

What could or should your car be saying about you to other cars or other people on the road? In this paper, we present some preliminary results from a multi-state in-vehicle driver monitoring system and position it with respect to our work in M2M communication. We propose to improve our current motion object tracking algorithm with the addition of a driver state variable, allowing cars to make predictions about other cars' trajectories with information beyond position, velocity and maps. We envision a transportation future where autonomous and semi-autonomous vehicles could be talking about us to our benefit and advanced driver assist would extend beyond the vehicle to a network of connected cars.

This paper describes findings from a three country, twenty-four participant study consisting of two in-home and in-car ethnographic interviews, separated by a month during which participants created videos, and their cars were GPS tracked and their Android smartphone data collected during and surrounding their driving times. We demonstrate how an ethnographic research approach that cross-interrogates data produced by GPS sensors, smart phone application monitoring, ethnographic interviews and participant-produced videos maps out a rich design space for future automotive user interfaces. These findings redefine the design space for automotive user interfaces and interactive vehicular interactions by recording real necessities, joys and pain points that people experience when using their cars.

The in-vehicle experience offers a unique challenge for delivering the right amount of information to the driver at the right time. The level of attention required to successfully manage the driving task is often in variable. An ideal in vehicle information delivery system would deliver content to the driver only during low task demand times, such as waiting at a stop light, when the driver's safety would be minimally compromised. The system would also have to respond to sudden changes in the situation such as driver interruption or distraction and terminate gracefully, allowing the driver to refocus on the driving task. In this paper, we present an embedded natural language processing (NLP) system that delivers speech synthesized summarized text content into tailored time slices. The system is also designed to respond dynamically to interruptions. We anticipate that this system could safely deliver speech synthesized content to drivers and allow them to make the most of their time on the road. We have implemented this system on an Atom Z530 processor with 1GB of RAM, a processor comparable to those found in factory installed In-Vehicle Infotainment (IVI) systems and have evaluated it in a laboratory test using a standard NLP corpus to demonstrate this potential.

This paper presents methods for collecting and analyzing physiological data during real-world driving tasks to determine a driver's relative stress level. Electrocardiogram, electromyogram, skin conductance, and respiration were recorded continuously while drivers followed a set route through open roads in the greater Boston area. Data from 24 drives of at least 50-min duration were collected for analysis. The data were analyzed in two ways. Analysis I used features from 5-min intervals of data during the rest, highway, and city driving conditions to distinguish three levels of driver stress with an accuracy of over 97% across multiple drivers and driving days. Analysis II compared continuous features, calculated at 1-s intervals throughout the entire drive, with a metric of observable stressors created by independent coders from videotapes. The results show that for most drivers studied, skin conductivity and heart rate metrics are most closely correlated with driver stress level. These findings indicate that physiological signals can provide a metric of driver stress in future cars capable of physiological monitoring. Such a metric could be used to help manage noncritical in-vehicle information systems and could also provide a continuous measure of how different road and traffic conditions affect drivers.

The ability to recognize emotion is one of the hallmarks of emotional intelligence, an aspect of human intelligence that has been argued to be even more important than mathematical and verbal intelligences. This paper proposes that machine intelligence needs to include emotional intelligence and demonstrates results toward this goal: developing a machine's ability to recognize the human affective state given four physiological signals. We describe difficult issues unique to obtaining reliable affective data and collect a large set of data from a subject trying to elicit and experience each of eight emotional states, daily, over multiple weeks. This paper presents and compares multiple algorithms for feature-based recognition of emotional state from this data. We analyze four physiological signals that exhibit problematic day-to-day variations: The features of different emotions on the same day tend to cluster more tightly than do the features of the same emotion on different days. To handle the daily variations, we propose new features and algorithms and compare their performance. We find that the technique of seeding a Fisher Projection with the results of sequential floating forward search improves the performance of the Fisher Projection and provides the highest recognition rates reported to date for classification of affect from physiology: 81 percent recognition accuracy on eight classes of emotion, including neutral.

Smart physiological sensors embedded in an automobile afford a novel opportunity to capture naturally occurring episodes of driver stress. In a series of ten ninety minute drives on public roads and highways, ECG, EMG, respiration and skin conductance sensors were used to measure the autonomic nervous system activation. The signals were digitized in real time and stored on the SmartCar's Pentium class computer. Each drive followed a pre-specified route through fifteen different events, from which four stress level categories were created according to the results of the subjects self report questionnaires. In total, 545 one minute segments were classified. A linear discriminant function was used to rank each feature individually based on the recognition performance, and a sequential forward floating selection algorithm was used to find an optimal set of features for recognizing patterns of driver stress. Using multiple features improved performance significantly over the best single feature performance.

Affective signal processing algorithms were developed to allow a digital computer to recognize the affective state of a user who is intentionally expressing that state. This paper describes the method used for collecting the training data, the feature extraction algorithms used and the results of pattern recognition using a Fisher linear discriminant and the leave one out test method. Four physiological signals, skin conductivity, blood volume pressure, respiration and an electromyogram (EMG) on the masseter muscle were analyzed. It was found that anger was well differentiated from peaceful emotions (90%-100%), that high and low arousal states were distinguished (80%-88%), but positive and negative valence states were difficult to distinguish (50%-82%). Subsets of three emotion states could be well separated (75%-87%) and characteristic patterns for single emotions were found.

An "affective wearable" is a wearable system equipped with sensors and tools which enables recognition of its wearer's affective patterns. Affective patterns include expressions of emotion such as a joyful smile, an angry gesture, a strained voice or a change in autonomic nervous system activity such as accelerated head rate or increasing skin conductivity. This paper describes new applications of affective wearables, and presents a prototype which gathers physiological signals and their annotations from its wearer. Results of preliminary experiments of its performance are reported for a user wearing four different sensors and engaging in several natural activities.