While different countries vary in their determination of copyrightability, jurisdictions like the United States currently do not allow an artist to copyright AI-generated content when they do not have creative control.One avenue for an author to support their case for copyright protections over work created with AI may then be to demonstrate their intent to "predict" outputs of the generative AI tool during the creation process, shifting elements of randomness from the AI to the human's own decision-making as much as possible.When this happens, the artist might claim to have expressed their idea with generative AI, and seek copyright protection for their work.We propose that generative AI co-creation tools can support this intention by keeping records of the predictability statistics at each generative AI iteration, and capturing the potential alternate options that can be later assessed for how predictably they matched the prompt.

The dynamic nature of web content poses unique challenges to revisiting past online sensemaking tasks.We investigate the value and usage patterns of a personal web archive that treats consumed information as first-class object in both information re-finding and mental model reconstruction.In this work, we introduce a new system called irchiver that passively captures screenshots of active browser windows, extracts text from screenshots and page sources, CCS Concepts• Human-centered computing → Human computer interaction (HCI); • Information systems → World Wide Web.

Shared gaze, where collaborators can see each other's point of gaze visualized on their screen in real time, is a novel non-verbal mechanism that augments remote collaborations and increases shared awareness and common grounding. While past studies have focused on well-structured tasks and analyzed task performance and efficiency, our study explores the domain of collaborative drawing for recreational purposes and focuses on collaborators' own perceptions. We surveyed 75 users of online collaborative drawing platforms who mostly drew collaboratively for recreation and artistic growth; they reported the importance of communication but also of retaining individual space despite the collaborative setting. Informed by this and prior research on shared gaze, we evaluate collaboration by allowing two collaborators to draw synchronously on a shared canvas and share their point of gaze. We conducted a study with 24 pairs that drew collaboratively under all combinations of shared gaze and voice communication. Combining voice and shared gaze was perceived to reach the best balance between tightly coupled collaboration and parallel individual execution. Shared gaze led to higher spatial awareness and less turn-taking was observed in conditions that shared gaze was present. Surprisingly, many participants found the lack of any communication medium to afford the highest degree of divergent thinking. Our findings provide guidelines for adaptive tools that consider individual preferences as well as the nature of the task to better support remote collaborations that are open-ended and prize creativity.

As messaging becomes increasingly salient to social connection and well-being, there exists a gap in understanding and support for neurodivergent users who may struggle with interpreting tone in computer-mediated communication (CMC). We look to existing accessibility practices among neurodivergent communities to thus propose a system of alternative cues that explicitly flag tone and intent, in hopes of better scaffolding neurodiverse communication styles. Through a mixed-method study of conversational pairs (N=18), we assessed the impact of these tone indicators on participants' experiences of synchronous messaging and social connection, interpreting our findings in the context of disability-focused design for messaging interfaces. We found that these alternative cues lowered the overall cognitive load of live tone interpretation for neurodivergent users, enabling a greater sense of confidence and control over self-presentation. In turn, participants reported more positive and in-depth conversations with their partners. We further observed that some participants typed more slowly, wrote longer messages, and used additional communication cues, like emojis and text stylization, concurrently. Participants' use and feedback on this tool also illuminated social context and collaboration's role in cue ambiguity. Our work subsequently suggests future research and design directions on supporting communicative accessibility via the intentional design of alternative cue systems and co-customization.

Online peer support provides space for individuals to connect with others and seek support. However, while empathy is critical for effective support, studies have found that highly empathetic support on these platforms can be rare. Using data from online peer support platforms, we conducted a mixed-methods analysis to study the factors that lead to support seekers’ perceived empathy. We found that CBT techniques like active listening and reflective restatements, along with fostering a space for exploration, increase perceived empathy, whereas rigid adherence to structure, misalignment of concerns, and lack of emotional validation can contribute to low perceived empathy. In addition, despite the high levels of empathy reported by most support seekers (85%), computational models reported low averaged empathy (1.69/6). Lastly, we propose that empathy is not a quantifiable metric and that future algorithmic empathy measurements require human perspectives.

The message a designer wants to convey plays a pivotal role in directing the design of an infographic, yet most authoring workflows start with creating the visualizations or graphics first without gauging whether they fit the message. To address this gap, we propose Epigraphics, a web-based authoring system that treats an “epigraph” as the first-class object, and uses it to guide infographic asset creation, editing, and syncing. The system uses the text-based message to recommend visualizations, graphics, data filters, color palettes, and animations. It further supports between-asset interactions and fine-tuning such as recoloring, highlighting, and animation syncing that enhance the aesthetic cohesiveness of the assets. A gallery and case studies show that our system can produce infographics inspired by existing popular ones, and a task-based usability study with 10 designers show that a text-sourced workflow can standardize content, empower users to think more about the big picture, and facilitate rapid prototyping.

Efforts to expand the authoring of visual stories beyond the 2D canvas have commonly mapped flat imagery to 3D scenes or objects. This translation requires spatial reasoning, as artists must think in two spaces. We propose PortalInk 1, a tool for artists to craft and export 2.5D graphical stories while remaining in 2D space by using SVG transitions. This is achieved via a parallax effect that generates a sense of depth that can be further explored using pan and zoom interactions. Any canvas position can be saved and linked to in a closed drawn stroke, or “portal,” allowing the artist to create spatially discontinuous, or even infinitely looping visual trajectories. We provide three case studies and a gallery to demonstrate how artists can naturally incorporate these interactions to craft immersive comics, as well as re-purpose them to support use cases beyond drawing such as animation, slide-based presentations, web design, and digital journalism.

As social media continues to grow as a space for emotional self-disclosure, it is important to understand whether self-disclosure acts as a causal factor impacting positive outcomes for users. Thus we developed Chirp, an anonymous social media sandbox space designed to explore the underlying effects of disclosure within online spaces. Users in Chirp are prompted to self-disclose moods and emotions using emojis. Through a between-subjects study among a cohort of first-year undergraduate student users on Chirp, we evaluate the effect of self-disclosure within semi-private online spaces on social support. While Chirp use does not show a significant increase in measured feelings of social support, user responses suggest that self-disclosure in Chirp may provide more social support than typical social media use or simple mood tracking over a two-week period. Our findings indicate that even in pseudo-anonymous, low-bandwidth communication platforms, self-disclosure may cause increased feelings of social support. This work highlights the impact of communication in semi-private online spaces on perceived social support.

Augmented Reality (AR) glasses separate dyadic interactions on different sides of the lens, where the person wearing the glasses (primary user) sees an AR world overlaid on their partner (secondary actor). The secondary actor interacts with the primary user understanding they are seeing both physical and virtual worlds. We use grounded theory to study interaction tasks, participatory design sessions, and in-depth interviews of 10 participants and explore how AR real-time modifications affect them. We observe a power imbalance attributed to the: (1) lack of transparency of the primary user’s view, (2) violation of agency over self-presentation, and (3) discreet recording capabilities of AR glasses. This information asymmetry leads to a negotiation of behaviors to reach a silently understood equilibrium. This paper addresses underlying design issues that contribute to power imbalances in dyadic interactions and offers nuanced insights into the dynamics between primary users and secondary actors.

Messaging is a ubiquitous digital communication medium. It is also a minimal medium of communication because of its inability to convey immediate feedback, tone, facial expressions, hesitations, and pauses, or follow the train of the other person’s thoughts. This paper combines quantitative and qualitative approaches for analyzing richer forms of typing indicators in messaging interfaces, such as showing text as it is typed. By assessing users’ subjective workload and interpreting these findings in the context of users’ experiences, we found that more expressive typing indicators were perceived as “rich in communication”, as they helped people communicate more allowing for closer connections. These indicators also increased users’ perceived co-presence. In addition, our research suggests there may be benefits of designing customized typing indicators for relationship maintenance and task-based communication.

Vector illustrations are object-based, meaning they are composed of strokes that can be filtered individually through textures or animations and transformed without loss of quality. These filters are typically difficult to specify without programming prerequisites. We propose filtered.ink , a full-featured illustration application to construct and explore filters via a node graph interface with a live preview. This turns vector graphics and their filters into a form of vector hypermedia that can be shared and remixed with new users. By examining interactions that occur when crafting, remixing, and using filters for dynamic illustrations through a task-based usability study, we expose new workflow patterns and avenues of expression. The observations result in a user model supported by filtered.ink: see, want, rewant, and remix. In this model, the artist breaks away from traditional notions of illustration, taking advantage of the inherent remixability of the strokes and filters in the vector graphics format.

irchiver is a personal web archive tool for users, which also provided an opportunity for information retrieval researchers to access naturalistic long-term browsing histories. It offers users the ability to capture and search their web archives, which are stored in full-resolution images on their local filesystem. Researchers can request these files, or develop a plugin that triggers when a new image is captured. irchiver runs as a Windows background process using a hook and capture technique that works on all 5 major desktop browsers. The first author has been using irchiver for 12 months, encountering unexpected benefits in archive retrieval, change detection, search functionality, and content recovery.

The amount of text people need to read and understand grows daily. Software defaults, designers, or publishers often choose the fonts people read in. However, matching individuals with a faster font could help them cope with information overload. We collaborated with typographers to (1) select eight fonts designed for digital reading to systematically compare their effectiveness and to (2) understand how font and reader characteristics affect reading speed. We collected font preferences, reading speeds, and characteristics from 252 crowdsourced participants in a remote readability study. We use font and reader characteristics to train FontMART, a learning to rank model that automatically orders a set of eight fonts per participant by predicted reading speed. FontMART’s fastest font prediction shows an average increase of 14–25 WPM compared to other font defaults, without hindering comprehension. This encouraging evidence provides motivation for adding our personalized font recommendation to future interactive systems.

Sharing annotations encourages feedback, discussion, and knowledge passing among readers and can be beneficial for personal and public use. Prior augmented reality (AR) systems have expanded these benefits to both digital and printed documents. However, despite smartphone AR now being widely available, there is a lack of research about how to use AR effectively for interactive document annotation. We propose Dually Noted, a smartphone-based AR annotation system that recognizes the layout of structural elements in a printed document for real-time authoring and viewing of annotations. We conducted experience prototyping with eight users to elicit potential benefits and challenges within smartphone AR, and this informed the resulting Dually Noted system and annotation interactions with the document elements. AR annotation is often unwieldy, but during a 12-user empirical study our novel structural understanding component allows Dually Noted to improve precise highlighting and annotation interaction accuracy by 13%, increase interaction speed by 42%, and significantly lower cognitive load over a baseline method without document layout understanding. Qualitatively, participants commented that Dually Noted was a swift and portable annotation experience. Overall, our research provides new methods and insights for how to improve AR annotations for physical documents.

The severe impact of COVID-19 in the United States has forced many students to replace in-person socialization with online digital contact. In this study, we investigate the mental health impacts associated with this shift by examining properties of online interactions that may affect loneliness and perceived social support. Students were surveyed (N=827) across 97 universities across the US during their first full semester impacted by the COVID-19 pandemic (Fall 2020). Private online interactions (messaging, phone call, video call) were found to have a comparable correlation to social support as face-to-face interactions, but public online interactions (social media) were associated with more negative outcomes. Among private platforms, messaging had the strongest correlation with social support; and daily self-disclosure over messaging yielded social support levels that were 1.21x higher than rarely or never disclosing over this platform. We speculate that factors such as the level of privacy and peoples' feelings of control contributed to disclosure and perceived social support in online platforms.

In our age of ubiquitous digital displays, adults often read in short, opportunistic interludes. In this context of Interlude Reading , we consider if manipulating font choice can improve adult readers’ reading outcomes. Our studies normalize font size by human perception and use hundreds of crowdsourced participants to provide a foundation for understanding, which fonts people prefer and which fonts make them more effective readers. Participants’ reading speeds (measured in words-per-minute (WPM)) increased by 35% when comparing fastest and slowest fonts without affecting reading comprehension. High WPM variability across fonts suggests that one font does not fit all. We provide font recommendations related to higher reading speed and discuss the need for individuation, allowing digital devices to match their readers’ needs in the moment. We provide recommendations from one of the most significant online reading efforts to date. To complement this, we release our materials and tools with this article.

The dominant markup language for Web visualizations-Scalable Vector Graphics (SVG)-is comparatively easy to learn, and is open, accessible, customizable via CSS, and searchable via the DOM, with easy interaction handling and debugging. Because these attributes allow visualization creators to focus on design on implementation details, tools built on top of SVG, such as D3.js, are essential to the visualization community. However, slow SVG rendering can limit designs by effectively capping the number of on-screen data points, and this can force visualization creators to switch to Canvas or WebGL. These are less flexible (e.g., no search or styling via CSS), and harder to learn. We introduce Scalable Scalable Vector Graphics (SSVG) to reduce these limitations and allow complex and smooth visualizations to be created with SVG. SSVG automatically translates interactive SVG visualizations into a dynamic virtual DOM (VDOM) to bypass the browser's slow 'to specification' rendering by intercepting JavaScript function calls. De-coupling the SVG visualization specification from SVG rendering, and obtaining a dynamic VDOM, creates flexibility and opportunity for visualization system research. SSVG uses this flexibility to free up the main thread for more interactivity and renders the visualization with Canvas or WebGL on a web worker. Together, these concepts create a drop-in JavaScript library which can improve rendering performance by 3-9× with only one line of code added. To demonstrate applicability, we describe the use of SSVG on multiple example visualizations including published visualization research. A free copy of this article, collected data, and source code are available as open science at osf.io/ge8wp.

Free-hand interaction enables users to directly create artistic augmented reality content using a smartphone, but lacks natural spatial depth information due to the small 2D display’s limited visual feedback. Through an autobiographical design process, three authors explored free-hand drawing over a total of 14 weeks. During this process, they expanded the design space from a single-display smartphone format to a dual-display smartphone-wearable format (Portalware). This new configuration extends the virtual content from a smartphone to a wearable display and enables multi-display free-hand interactions. The authors documented experiences where 1) the display extends the smartphone’s canvas perceptually, allowing the authors to work beyond the smartphone screen view; 2) the additional perspective mitigates the difficulties of depth perception and improves the usability of direct free-hand manipulation; 3) the wearable use cases depend on the nature of the drawing, such as: replicating physical objects, “in-situ” mixed reality pieces, and multi-planar drawings.

The ubiquity of self-tracking devices and smartphone apps has empowered people to collect data about themselves and try to self-improve. However, people with little to no personal analytics experience may not be able to analyze data or run experiments on their own (self-experiments). To lower the barrier to intervention-based self-experimentation, we developed an app called Self-E, which guides users through the experiment. We conducted a 2-week diary study with 16 participants from the local population and a second study with a more advanced group of users to investigate how they perceive and carry out self-experiments with the help of Self-E, and what challenges they face. We find that users are influenced by their preconceived notions of how healthy a given behavior is, making it difficult to follow Self-E’s directions and trusting its results. We present suggestions to overcome this challenge, such as by incorporating empathy and scaffolding in the system.

Peer review has been used in both online and offline classrooms to inspire creativity, gather feedback, and lessen instructor grading loads, especially for design-based tasks without definitive rubrics. To explore the nuances and quality of peer feedback, we developed UX Factor, a peer grading platform that aims to characterize the behavior of peer reviews and the consistency of the ranking models used to aggregate these reviews. This system harnesses the power of pairwise comparisons to minimize bias and encourage context-driven analysis. We adopted UX Factor in a user interface course of 133 students and teaching assistants (TAs) across 3 different individual design projects over a semester and found that the system was effective in eliciting high-quality feedback. We saw that raters have higher agreement than random preferences, and with at least 15 ratings per submission, a simple average of ratings produced rankings that were consistent to both the raw ratings and other more complex models. These rankings were robust to disagreeable raters and changing class sizes, demonstrating the potential of comparative peer review to match the quality of expert feedback at scale.

The life cycle of a peer-produced dataset follows the phases of growth, maturity, and decline. Paying crowdworkers is a proven method to collect and organize information into structured tables. However, these tabular representations may contain inaccuracies due to errors or data changing over time. Thus, the maturation phase of a dataset can benefit from the additional human examination. One method to improve accuracy is to recruit additional paid crowdworkers to verify and correct errors. An alternative method relies on unpaid contributors, collectively editing the dataset during regular use. We describe two case studies to examine different strategies for human verification and maintenance of in-flux tabular datasets. The first case study examines traditional micro-task verification strategies with paid crowdworkers, while the second examines long-term maintenance strategies with unpaid contributions from non-crowdworkers. Two paid verification strategies that produced more accurate corrections at a lower cost per accurate correction were redundant data collection followed by final verification from a trusted crowdworker and allowing crowdworkers to review any data freely. In the unpaid maintenance strategies, contributors provided more accurate corrections when asked to review data matching their interests. This research identifies considerations and future approaches to collectively improving information accuracy and longevity of tabular information.

Self-experiments allow people to explore what behavioral changes lead to improved health and wellness. However, it is challenging to run such experiments in a scientifically valid way that is also flexible and able to accommodate the realities of daily life. We present a set of design principles for guided self-experiments that aim to lower this barrier to self-experimentation. We demonstrate the value of the principles by implementing them in SleepBandits, an integrated system that includes a smartphone application for sleep experiments. SleepBandits guides users through the steps of a single-case experiment, automatically collecting data from the built-in sensors and user input and calculating and presenting results in real-time. We released SleepBandits to the Google Play Store and people voluntarily downloaded and used it. Based on the data from 365 active users from this in-the-wild study, we discuss opportunities and challenges with the design principles and the SleepBandits system.

Messaging is a common mode of communication, with conversations written informally between individuals. Interpreting emotional affect from messaging data can lead to a powerful form of reflection or act as a support for clinical therapy. Existing analysis techniques for social media commonly use LIWC and VADER for automated sentiment estimation. We correlate LIWC, VADER, and ratings from human reviewers with affect scores from 25 participants. We explore differences in how and when each technique is successful. Results show that human review does better than VADER, the best automated technique, when humans are judging positive affect ($r_s=0.45$ correlation when confident, $r_s=0.30$ overall). Surprisingly, human reviewers only do slightly better than VADER when judging negative affect ($r_s=0.38$ correlation when confident, $r_s=0.29$ overall). Compared to prior literature, VADER correlates more closely with PANAS scores for private messaging than public social media. Our results indicate that while any technique that serves as a proxy for PANAS scores has moderate correlation at best, there are some areas to improve the automated techniques by better considering context and timing in conversations.

In-person user studies show that designers draw inspiration by looking at their peers' work while sketching. To recreate this behavior in a virtual environment, we developed Sketchy, a web-based drawing application where users sketch in virtual rooms and use the "Peek'' functionality to gain ideas from their peers' sketches in real-time. To assess if "Peek'' supports individual creativity through finding inspiration, students from a Human-Computer Interaction class sketched user interface design tasks in two studies. Study 1 compares creativity measures with and without Peek between two groups of students, where self-reports reveal Peek increases satisfaction with their final sketch and better supports individual creativity. Study 2 took place in a large classroom, where 90 students, all with Peek enabled, completed different design tasks. Peeking led students to report an intention to change their sketch 18% of the time in Study 1 and 17% of the time in Study 2. Student designers were influenced by sketches that seem closer to completion, contain more details, and are carefully drawn. They were also about three times more likely to clear their canvas and start over if they found a sketch inspirational. Furthermore, sketches created by students with more sketching and design experience influence less experienced student designers. This work explores the directions and benefits of incorporating digital peeking to support individual creativity within a student designer's classroom experience to create more satisfactory final sketches.

Pan and zoom timelines and sliders help us navigate large time series data. However, designing efficient interactions can be difficult. We study pan and zoom methods via crowd-sourced experiments on mobile and computer devices, asking which designs and interactions provide faster target acquisition. We find that visual context should be limited for low-distance navigation, but added for far-distance navigation; that timelines should be oriented along the longer axis, especially on mobile; and that, as compared to default techniques, double click, hold, and rub zoom appear to scale worse with task difficulty, whereas brush and especially ortho zoom seem to scale better. Software and data used in this research are available as open source.

Smartphone augmented reality (AR) lets users interact with physical and virtual spaces simultaneously. With 3D hand tracking, smartphones become apparatus to grab and move virtual objects directly. Based on design considerations for interaction, mobility, and object appearance and physics, we implemented a prototype for portable 3D hand tracking using a smartphone, a Leap Motion controller, and a computation unit. Following an experience prototyping procedure, 12 researchers used the prototype to help explore usability issues and define the design space. We identified issues in perception (moving to the object, reaching for the object), manipulation (successfully grabbing and orienting the object), and behavioral understanding (knowing how to use the smartphone as a viewport). To overcome these issues, we designed object-based feedback and accommodation mechanisms and studied their perceptual and behavioral effects via two tasks: picking up distant objects, and assembling a virtual house from blocks. Our mechanisms enabled significantly faster and more successful user interaction than the initial prototype in picking up and manipulating stationary and moving objects, with a lower cognitive load and greater user preference. The resulting system---Portal-ble---improves user intuition and aids free-hand interactions in mobile situations.

We examine the relationship between eye gaze and typing, focusing on the differences between touch and non-touch typists. To enable typing-based research, we created a 51-participant benchmark dataset for user input across multiple tasks, including user input data, screen recordings, webcam video of the participant's face, and eye tracking positions. There are patterns of eye movements that differ between the two types of typists, representing glances at the keyboard, which can be used to identify touch-.typed strokes with 92% accuracy. Then, we relate eye gaze with cursor activity, aligning both pointing and typing to eye gaze. One demonstrative application of the work is in extending WebGazer, a real-time web-browser-based webcam eye tracker. We show that incorporating typing behavior as a secondary signal improves eye tracking accuracy by 16% for touch typists, and 8% for non-touch typists.

We introduce a tool within the Code Bubbles development environment that allows for continuous execution as the programmer edits. The tool, SEEDE, shows both the intermediate and final results of execution in terms of variables, control and data flow, output, and graphics. These results are updated as the user edits. The tool can be used to help the user write new code or to find and fix bugs. The tool is explicitly designed to let the user quickly explore the execution of a method along with all the code it invokes, possibly while writing or modifying the code. The user can start continuous execution either at a breakpoint or for a test case. This paper describes the tool, its implementation, and its user interface. It presents an initial user study of the tool demonstrating its potential utility.

We show how mouse interaction log classification can help visualization toolsmiths understand how their tools are used "in the wild" through an evaluation of MAGI - a cancer genomics visualization tool. Our primary contribution is an evaluation of twelve visual analysis task classifiers, which compares predictions to task inferences made by pairs of genomics and visualization experts. Our evaluation uses common classifiers that are accessible to most visualization evaluators: -nearest neighbors, linear support vector machines, and random forests. By comparing classifier predictions to visual analysis task inferences made by experts, we show that simple automated task classification can have up to 73 percent accuracy and can separate meaningful logs from "junk" logs with up to 91 percent accuracy. Our second contribution is an exploration of common MAGI interaction trends using classification predictions, which expands current knowledge about ecological cancer genomics visualization tasks. Our third contribution is a discussion of how automated task classification can inform iterative tool design. These contributions suggest that mouse interaction log analysis is a viable method for (1) evaluating task requirements of client-side-focused tools, (2) allowing researchers to study experts on larger scales than is typically possible with in-lab observation, and (3) highlighting potential tool evaluation bias.

We introduce SearchGazer, a web-based eye tracker for remote web search studies using common webcams already present in laptops and some desktop computers. SearchGazer is a pure JavaScript library that infers the gaze behavior of searchers in real time. The eye tracking model self-calibrates by watching searchers interact with the search pages and trains a mapping of eye features to gaze locations and search page elements on the screen. Contrary to typical eye tracking studies in information retrieval, this approach does not require the purchase of any additional specialized equipment, and can be done remotely in a user's natural environment, leading to cheaper and easier visual attention studies.

Structured datasets are difficult to keep up-to-date since the underlying facts evolve over time; curated data about business financials, organizational hierarchies, or drug interactions are constantly changing. Drafty is a platform that enlists visitors of an editable dataset to become ``user-editors'' to help solve this problem. It records and analyzes user-editors' within-page interactions to construct user interest profiles, creating a cyclical feedback mechanism that enables Drafty to target requests for specific corrections from user-editors. To validate the automatically generated user interest profiles, we surveyed participants who performed self-created tasks with Drafty and found their user interest score was 3.2 higher on data they were interested in versus data they had no interest in. Next, a 7-month live experiment compared the efficacy of user-editor corrections depending on whether they were asked to review data that matched their interests. Our findings suggest that user-editors are approximately 3 times more likely to provide accurate corrections for data matching their interest profiles, and about 2 times more likely to provide corrections in the first place.

We present SleepCoacher, an integrated system implementing a framework for effective self-experiments. SleepCoacher automates the cycle of single-case experiments by collecting raw mobile sensor data and generating personalized, data-driven sleep recommendations based on a collection of template recommendations created with input from clinicians. The system guides users through iterative short experiments to test the effect of recommendations on their sleep. We evaluate SleepCoacher in two studies, measuring the effect of recommendations on the frequency of awakenings, self-reported restfulness, and sleep onset latency, concluding that it is effective: participant sleep improves as adherence with SleepCoacher's recommendations and experiment schedule increases. This approach presents computationally-enhanced interventions leveraging the capacity of a closed feedback loop system, offering a method for scaling guided single-case experiments in real time.

Most user interfaces require the user to focus on one element at a time, but StarCraft 2 is a game where players often control more than a hundred units simultaneously. The game interface provides an optional mechanism called "control groups" that allows players to select multiple units and assign them to a group in order to quickly recall previous selections of units. From an analysis of over 3,000 replays, we show that the usage of control groups is a key differentiator of individual players as well as players of different skill levels---novice users rarely use control groups while experts nearly always do. But players also behave differently in how they use their control groups, especially in time-pressured situations. While certain control group behaviors are common across all skill levels, expert players appear to be better at remaining composed and sustaining control group use in battle. We also qualitatively analyze discussions on web forums from players about how they use control groups to provide context about how such a simple interface mechanic has produced numerous ways of optimizing unit control.

As crowdsourcing has gained prominence in recent years, an increasing number of people turn to popular crowdsourcing platforms for their many uses. Experienced members of the crowdsourcing community have developed numerous systems both separately and in conjunction with these platforms, along with other tools and design techniques, to gain more specialized functionality and overcome various shortcomings. It is unclear, however, how novice requesters using crowdsourcing platforms for general tasks experience existing platforms and how, if at all, their approaches deviate from the best practices established by the crowdsourcing research community. We conduct an experiment with a class of 19 students to study how novice requesters design crowdsourcing tasks. Each student tried their hand at crowdsourcing a real data collection task with a fixed budget and realistic time constraint. Students used Amazon Mechanical Turk to gather information about the academic careers of over 2,000 professors from 50 top Computer Science departments in the U.S. In addition to curating this dataset, we classify the strategies which emerged, discuss design choices students made on task dimensions, and compare these novice strategies to best practices identified in crowdsourcing literature. Finally, we summarize design pitfalls and effective strategies observed to provide guidelines for novice requesters.

When visualizing data with uncertainty, a common approach is to treat uncertainty as an additional dimension and encode it using a visual variable. The effectiveness of this approach depends on how the visual variables chosen for representing uncertainty and other attributes interact to influence the user's perception of each variable. We report a user study on the perception of graph edge attributes when uncertainty associated with each edge and the main edge attribute are visualized simultaneously using two separate visual variables. The study covers four visual variables that are commonly used for visualizing uncertainty on line graphical primitives: lightness, grain, fuzziness, and transparency. We select width, hue, and saturation for visualizing the main edge attribute and hypothesize that we can observe interference between the visual variable chosen to encode the main edge attribute and that to encode uncertainty, as suggested by the concept of dimensional integrality. Grouping the seven visual variables as color-based, focus-based, or geometry-based, we further hypothesize that the degree of interference is affected by the groups to which the two visual variables belong. We consider two further factors in the study: discriminability level for each visual variable as a factor intrinsic to the visual variables and graph-task type (visual search versus comparison) as a factor extrinsic to the visual variables. Our results show that the effectiveness of a visual variable in depicting uncertainty is strongly mediated by all the factors examined here. Focus-based visual variables (fuzziness, grain, and transparency) are robust to the choice of visual variables for encoding the main edge attribute, though fuzziness has stronger negative impact on the perception of width and transparency has stronger negative impact on the perception of hue than the other uncertainty visual variables. We found that interference between hue and lightness is much greater than that between saturation and lightness, though all three are color-based visual variables. We also found a compound relationship between discriminability level and the degree of dimensional integrality. We discuss the generalizability and limitation of the results and conclude with design considerations for visualizing graph uncertainty derived from these results, including recommended choices of visual variables when the relative importance of data attributes and graph tasks is known.

This paper introduces two novel algorithms for learning behaviors from human-provided rewards. The primary novelty of these algorithms is that instead of treating the feedback as a numeric reward signal, they interpret feedback as a form of discrete communication that depends on both the behavior the trainer is trying to teach and the teaching strategy used by the trainer. For example, some human trainers use a lack of feedback to indicate whether actions are correct or incorrect, and interpreting this lack of feedback accurately can significantly improve learning speed. Results from user studies show that humans use a variety of training strategies in practice and both algorithms can learn a contextual bandit task faster than algorithms that treat the feedback as numeric. Simulated trainers are also employed to evaluate the algorithms in both contextual bandit and sequential decision-making tasks with similar results.

In order to be useful in real-world situations, it is critical to allow non-technical users to train robots. Existing work has considered the problem of a robot or virtual agent learning behaviors from evaluative feedback provided by a human trainer. That work, however, has treated feedback as a numeric reward that the agent seeks to maximize, and has assumed that all trainers will provide feedback in the same way when teaching the same behavior. We report the results of a series of user studies that indicate human trainers use a variety of approaches to providing feedback in practice, which we describe as different “training strategies.” For example, users may not always give explicit feedback in response to an action, and may be more likely to provide explicit reward than explicit punishment, or vice versa. If the trainer is consistent in their strategy, then it may be possible to infer knowledge about the desired behavior from cases where no explicit feedback is provided. We discuss a probabilistic model of human-provided feedback that can be used to classify these different training strategies based on when the trainer chooses to provide explicit reward and/or explicit punishment, and when they choose to provide no feedback. Additionally, we investigate how training strategies may change in response to the appearance of the learning agent. Ultimately, based on this work, we argue that learning agents designed to understand and adapt to different users' training strategies will allow more efficient and intuitive learning experiences.

How do video game skills develop, and what sets the top players apart? We study this question of skill through a rating generated from repeated multiplayer matches called TrueSkill. Using these ratings from 7 months of games from over 3 million players, we look at how play intensity, breaks in play, skill change over time, and other games affect skill. These analyzed factors are then combined to model future skill and games played; the results show that skill change in early matches is a useful metric for modeling future skill, while play intensity explains eventual games played. The best players in the 7-month period, who we call "Master Blasters", have varied skill patterns that often run counter to the trends we see for typical players. The data analysis is supplemented with a 70 person survey to explore how players' self-perceptions compare to the gameplay data; most survey responses align well with the data and provide insight into player beliefs and motivation. Finally, we wrap up with a discussion about hiding skill information from players, and implications for game designers.

Past studies of user behavior in Web search have correlated eye-gaze and mouse cursor positions, and other lines of research have found cursor interactions to be useful in determining user intent and relevant parts of Web pages. However, cursor interactions are not all the same; different types of cursor behavior patterns exist, such as reading, hesitating, scrolling and clicking, each of which has a different meaning. We conduct a search study with 36 subjects and 32 search tasks to determine when gaze and cursor are aligned, and thus when the cursor position is a good proxy for gaze position. We study the effect of time, behavior patterns, user, and search task on the gaze-cursor alignment, findings which lead us to question the maxim that "gaze is well approximated by cursor." These lessons inform an experiment in which we predict the gaze position with better accuracy than simply using the cursor position, improving the state-of-the-art technique for approximating visual attention with the cursor. Our new technique can help make better use of large-scale cursor data in identifying how users examine Web search pages.

Players can change the rules of a multi-person game to experience a different gameplay mechanic, add thematic color, or fine-tune its balance. To better understand game variants, we use a grounded approach to analyze 62 variants for Texas Hold'em, a popular card game, and a follow-up case-study of 91 variants of Halo 2, a popular video game. We study their development and examine whether lessons from Texas Hold'em apply to a constrained system such as Halo 2. We discover video gamers' reliance on 'honor rules', rules dependent on the cooperative spirit of its players. We develop a theory of 'necessity' in rule adoption, showing players' sensitivity to the impact of one change on the whole game. In solving game-design problems, adjustments drawn from a set of 'canned' rule changes address common problems with familiar solutions. We find a complex interplay between who can play and what rules are chosen. Our findings have implications for game designers and for variants in non-game contexts.

Web search components such as ranking and query suggestions analyze the user data provided in query and click logs. While this data is easy to collect and provides information about user behavior, it omits user interactions with the search engine that do not hit the server; these logs omit search data such as users' cursor movements. Just as clicks provide signals for relevance in search results, cursor hovering and scrolling can be additional implicit signals. In this work, we demonstrate a technique to extend models of the user's search result examination state to infer document relevance. We start by exploring recorded user interactions with the search results, both qualitatively and quantitatively. We find that cursor hovering and scrolling are signals telling us which search results were examined, and we use these interactions to reveal latent variables in searcher models to more accurately compute document attractiveness and satisfaction. Accuracy is evaluated by computing how well our model using these parameters can predict future clicks for a particular query. We are able to improve the click predictions compared to a basic searcher model for higher ranked search results using the additional log data.

Smartphones are convenient, but their small screens make searching, clicking, and reading awkward. Thus, perusing product reviews on a smartphone is difficult. In response, we introduce RevMiner - a novel smartphone interface that utilizes Natural Language Processing techniques to analyze and navigate reviews. RevMiner was run over 300K Yelp restaurant reviews extracting attribute-value pairs, where attributes represent restaurant attributes such as sushi and service, and values represent opinions about the attributes such as fresh or fast. These pairs were aggregated and used to: 1) answer queries such as "cheap Indian food", 2) concisely present information about each restaurant, and 3) identify similar restaurants. Our user studies demonstrate that on a smartphone, participants preferred RevMiner's interface to tag clouds and color bars, and that they preferred RevMiner's results to Yelp's, particularly for conjunctive queries (e.g., "great food and huge portions"). Demonstrations of RevMiner are available at revminer.com.

Understanding the impact of individual and task differences on search result page examination strategies is important in developing improved search engines. Characterizing these effects using query and click data alone is common but insufficient since they provide an incomplete picture of result examination behavior. Cursor- or gaze-tracking studies reveal richer interaction patterns but are often done in small-scale laboratory settings. In this paper we leverage large-scale rich behavioral log data in a naturalistic setting. We examine queries, clicks, cursor movements, scrolling, and text highlighting for millions of queries on the Bing commercial search engine to better understand the impact of user, task, and user-task interactions on user behavior on search result pages (SERPs). By clustering users based on cursor features, we identify individual, task, and user-task differences in how users examine results which are similar to those observed in small-scale studies. Our findings have implications for developing search support for behaviorally-similar searcher cohorts, modeling search behavior, and designing search systems that leverage implicit feedback.

Today's Web browsers allow users to open links in new windows or tabs. This action, which we call 'branching', is sometimes performed on search results when the user plans to eventually visit multiple results. We detect branching behavior on a large commercial search engine with a client-side script on the results page. Two-fifths of all users spawned new tabs on search results in the timeframe of our study; branching usage varied with different query types and vertical. Both branching and backtracking are viable methods for visiting multiple search results. To understand user search strategies, we treat multiple result clicks following a query as ordered events to understand user search strategies. Users branching in a query are more likely to click search results from top to bottom, while users who backtrack are less likely to do so; this is especially true for queries involving more than two clicks. These findings inform an experiment in which we take a popular click model and modify it to account for the differing user behavior when branching. By understanding that users continue examining search results before viewing a branched result, we can improve the click model for branching queries.

Video games are primarily designed for the players. However, video game spectating is also a popular activity, boosted by the rise of online video sites and major gaming tournaments. In this paper, we focus on the spectator, who is emerging as an important stakeholder in video games. Our study focuses on Starcraft, a popular real-time strategy game with millions of spectators and high level tournament play. We have collected over a hundred stories of the Starcraft spectator from online sources, aiming for as diverse a group as possible. We make three contributions using this data: i) we find nine personas in the data that tell us who the spectators are and why they spectate; ii) we strive to understand how different stakeholders, like commentators, players, crowds, and game designers, affect the spectator experience; and iii) we infer from the spectators' expressions what makes the game entertaining to watch, forming a theory of distinct types of information asymmetry that create suspense for the spectator. One design implication derived from these findings is that, rather than presenting as much information to the spectator as possible, it is more important for the stakeholders to be able to decide how and when they uncover that information.

Understanding how people interact with search engines is important in improving search quality. Web search engines typically analyze queries and clicked results, but these actions provide limited signals regarding search interaction. Laboratory studies often use richer methods such as gaze tracking, but this is impractical at Web scale. In this paper, we examine mouse cursor behavior on search engine results pages (SERPs), including not only clicks but also cursor movements and hovers over different page regions. We: (i) report an eye-tracking study showing that cursor position is closely related to eye gaze, especially on SERPs; (ii) present a scalable approach to capture cursor movements, and an analysis of search result examination behavior evident in these large-scale cursor data; and (iii) describe two applications (estimating search result relevance and distinguishing good from bad abandonment) that demonstrate the value of capturing cursor data. Our findings help us better understand how searchers use cursors on SERPs and can help design more effective search systems. Our scalable cursor tracking method may also be useful in non-search settings.

Web search engines respond to a query by returning more results than can be reasonably reviewed. These results typically include the title, link, and snippet of content from the target link. Each result has the potential to be useful or useless and thus reviewing it has a cost and potential benefit. This paper studies the behavior of a rational agent in this setting, whose objective is to maximize the probability of finding a satisfying result while minimizing cost. We propose two similar agents with different capabilities: one that only compares result snippets relatively and one that predicts from the result snippet whether the result will be satisfying. We prove that the optimal strategy for both agents is a stopping rule: the agent reviews a fixed number of results until the marginal cost is greater than the marginal expected benefit, maximizing the overall expected utility. Finally, we discuss the relationship between rational agents and search users and how our findings help us understand reviewing behaviors.

Search engines return ranked lists of Web pages in response to queries. These pages are starting points for post-query navigation, but may be insufficient for search tasks involving multiple steps. Search trails mined from toolbar logs start with a query and contain pages visited by one user during post-query navigation. Implicit endorsements from many trails can enhance result ranking. Rather than using trails solely to improve ranking, it may also be worth providing trail information directly to users. In this paper, we quantify the benefit that users currently obtain from trail-following and compare different methods for finding the best trail for a given query and each top-ranked result. We compare the relevance, topic coverage, topic diversity, and utility of trails selected using different methods, and break out findings by factors such as query type and origin relevance. Our findings demonstrate value in trails, highlight interesting differences in the performance of trailfinding algorithms, and show we can find best-trails for a query that outperform the trails most users follow. Findings have implications for enhancing Web information seeking using trails.

Search trails mined from browser or toolbar logs comprise queries and the post-query pages that users visit. Implicit endorsements from many trails can be useful for search result ranking, where the presence of a page on a trail increases its query relevance. Follow-ing a search trail requires user effort, yet little is known about the benefit that users obtain from this activity versus, say, sticking with the clicked search result or jumping directly to the destination page at the end of the trail. In this paper, we present a log-based study estimating the user value of trail following. We compare the relevance, topic coverage, topic diversity, novelty, and utility of full trails over that provided by sub-trails, trail origins (landing pages), and trail destinations (pages where trails end). Our findings demonstrate significant value to users in following trails, especially for certain query types. The findings have implications for the design of search systems, including trail recommendation systems that display trails on search result pages.

Users frequently modify a previous search query in hope of retrieving better results. These modifications are called query reformulations or query refinements. Existing research has studied how web search engines can propose reformulations, but has given less attention to how people perform query reformulations. In this paper, we aim to better understand how web searchers refine queries and form a theoretical foundation for query reformulation. We study users' reformulation strategies in the context of the AOL query logs. We create a taxonomy of query refinement strategies and build a high precision rule-based classifier to detect each type of reformulation. Effectiveness of reformulations is measured using user click behavior. Most reformulation strategies result in some benefit to the user. Certain strategies like add/remove words, word substitution, acronym expansion, and spelling correction are more likely to cause clicks, especially on higher ranked results. In contrast, users often click the same result as their previous query or select no results when forming acronyms and reordering words. Perhaps the most surprising finding is that some reformulations are better suited to helping users when the current results are already fruitful, while other reformulations are more effective when the results are lacking. Our findings inform the design of applications that can assist searchers; examples are described in this paper.

We explore the use of abstracted screenshots as part of a new help interface. Graphstract, an implementation of a graphical help system, extends the ideas of textually oriented Minimal Manuals to the use of screenshots, allowing multiple small graphical elements to be shown in a limited space. This allows a user to get an overview of a complex sequential task as a whole. The ideas have been developed by three iterations of prototyping and evaluation. A user study shows that Graphstract helps users perform tasks faster on some but not all tasks. Due to their graphical nature, it is possible to construct Graphstracts automatically from pre-recorded interactions. A second study shows that automated capture and replay is a low-cost method for authoring Graphstracts, and the resultant help is as understandable as manually constructed help.