What would it mean for a digital assistant to stay with our cursor, rather than as a chatbot in a separate window? By staying near the user’s actions, such assistance promises lightweight, continuous, in-the-flow help, but also introduces unique risks of intrusion. However, little is known about the tasks it is suited for, and the design considerations it entails. This study offers an initial investigation of this space through a multi-stage design inquiry. First, a retrospective think-aloud study with nine participants reveals common inefficiencies in everyday information work where cursor-centric support may be valuable. Building on these observations, we proposed the Fairy Cursor as a design probe and developed a proof-of-concept environment to examine users’ interpretations, preferences and concerns around it. Our findings surfaced key challenges and opportunities in designing cursor-centric assistants, including how users tolerate presence, interpret initiatives, and balance assistance with ongoing engagement. We conclude with design implications for in-the-flow human–AI collaboration and outline directions for future systems.

Recent work identified clarity as one of the top quality attributes that notebook users value, but notebooks lack support for maintaining clarity throughout the exploratory phases of the notebook authoring workflow. We propose always-clear notebook authoring that supports both clarity and exploration, and present a Jupyter implementation called Tidynote. The key to Tidynote is three-fold: (1) a scratchpad sidebar to facilitate exploration, (2) cells movable between the notebook and the scratchpad to maintain organization, and (3) linear execution with state forks to clarify program state. An exploratory study (N=13) of open-ended data analysis tasks shows that Tidynote features holistically promote clarity throughout a notebook’s lifecycle, support realistic notebook tasks, and enable novel strategies for notebook clarity. These results suggest that Tidynote supports maintaining clarity throughout the entirety of notebook authoring.

Visual design instructors often provide multi-modal feedback, mixing annotations with text. Prior theory emphasizes the importance of actionable feedback, where “actionability” lies on a spectrum—from surfacing relevant design concepts to suggesting concrete fixes. How might creativity tools implement annotations that support such feedback, and how does the actionability of feedback impact novices’ process-related behaviors, perceptions of creativity, learning of design principles, and overall outcomes? We introduce VizCrit, a system for providing computational feedback that supports the actionability spectrum, realized through algorithmic issue detection and visual annotation generation. In a between-subjects study (N=36), novices revised a design under one of three conditions: textbook-based, awareness-centered, or solution-centered feedback. We found that solution-centered feedback led to fewer design issues and higher self-perceived creativity compared with textbook-based feedback, although expert ratings on creativity showed no significant differences. We discuss the implications for AI in Creativity Support Tools, including the potential of calibrating feedback actionability to help novices balance productivity with learning, growth, and developing design awareness.

A particular concern with SAE Level 3 automation is the takeover transition from the automated vehicle to the human driver. In response, research has focused on investigating this transition. However, researchers have used a wide range of metrics to measure takeover performance. The lack of consistency in these metrics poses challenges for synthesizing findings. To address this issue, we conducted a systematic literature review of studies published between January 2009 and December 2019, focusing on the takeover performance metrics. Following prior research, we categorize these metrics into two dimensions: timeliness and quality. Additionally, we summarize the scenarios used to elicit takeover requests and analyze the corresponding maneuvers (braking, lane changing, and lane keeping). The results have shown inconsistencies in calculation and naming conventions of takeover performance metrics. Based on these findings, this study proposes several directions for standardizing definitions and terminology, and advancing toward a unified measure of takeover performance.

The overview-detail design pattern, characterized by an overview of multiple items and a detailed view of a selected item, is ubiquitously implemented across software interfaces. Designers often try to account for all users, but ultimately these interfaces settle on a single form. For instance, an overview map may display hotel prices but omit other user-desired attributes. This research instead explores the malleable overview-detail interface, one that end-users can customize to address individual needs. Our content analysis of overview-detail interfaces uncovered three dimensions of variation: content, composition, and layout, enabling us to develop customization techniques along these dimensions. For content, we developed Fluid Attributes, a set of techniques enabling users to show and hide attributes between views and leverage AI to manipulate, reformat, and generate new attributes. For composition and layout, we provided solutions to compose multiple overviews and detail views and transform between various overview and overview-detail layouts. A user study on our techniques implemented in two design probes revealed that participants produced diverse customizations and unique usage patterns, highlighting the need and broad applicability for malleable overview-detail interfaces.

Augmented presentations offer compelling storytelling by combining speech content, gestural performance, and animated graphics in a congruent manner. The expressiveness of these presentations stems from the harmonious coordination of spoken words and graphic elements, complemented by smooth animations aligned with the presenter’s gestures. However, achieving such desired congruence in a live presentation poses significant challenges due to the unpredictability and imprecision inherent in presenters’ real-time actions. Existing methods either leveraged rigid mapping without predefined states or required the presenters to conform to predefined animations. We introduce adaptive presentations that dynamically adjust predefined graphic animations to real-time speech and gestures. Our approach leverages script following and motion warping to establish elastic mappings that generate runtime graphic parameters coordinating speech, gesture, and predefined animation state. Our evaluation demonstrated that the proposed adaptive presentation can effectively mitigate undesired visual artifacts caused by performance deviations and enhance the expressiveness of resulting presentations.

Strategies such as deploying honeypots and utilizing Moving Target Defense can occasionally mislead attackers. However, these methods often struggle to counteract the advanced detection and analysis tactics employed by attackers, particularly when the defenses are based on static probabilities for scenario changes. In response, a sophisticated deceptive defense mechanism named DSHopping (Deception Scenario Hopping) has been meticulously developed, utilizing reinforcement learning. DSHopping intelligently calculates transition probabilities, considering the current network state and available resources, which enhances its effectiveness in deception. The experimental results from the prototype system are remarkable, demonstrating that DSHopping has boosted the capture rate by a substantial 46%. This performance starkly contrasts with the best static transition probability observed in various attack detection scenarios. Ultimately, DSHopping has proven to be crucial in strengthening the system's ability to effectively capture attacks.

Unit visualizations have been widely used in data storytelling within interactive articles and videos. However, authoring data stories that contain animated unit visualizations is challenging due to the tedious, time-consuming process of switching back and forth between writing a narrative and configuring the accompanying visualizations and animations. To streamline this process, we present DataParticles, a block-based story editor that leverages the latent connections between text, data, and visualizations to help creators flexibly prototype, explore, and iterate on a story narrative and its corresponding visualizations. To inform the design of DataParticles, we interviewed 6 domain experts and studied a dataset of 44 existing animated unit visualizations to identify the narrative patterns and congruence principles they employed. A user study with 9 experts showed that DataParticles can significantly simplify the process of authoring data stories with animated unit visualizations by encouraging exploration and supporting fast prototyping.

Video is an effective medium for knowledge communication and learning. Yet active viewing and note-taking from videos remain a challenge. Specifically, during note-taking, viewers find it difficult to extract essential information such as representation, composition, motion, and interactions of graphical objects and narration. Current approaches rely on creating static screenshots, manual clipping, manual annotation and transcription. This is often done by repeatedly pausing and rewinding the video, thus disrupting the viewing experience. We propose VideoSticker, a tool designed to support visual note-taking by extracting expressive content and narratives from videos as ‘object stickers.’ VideoSticker implements automated object detection and tracking, linking objects to the transcript, and supporting rapid extraction of stickers across space, time, and events of interest. VideoSticker’s two-pass approach allows viewers to capture high-level information uninterrupted and later extract specific details. We demonstrate the usability of VideoSticker for a variety of videos and note-taking needs.