While machine learning is a powerful tool for the analysis and classification of complex real-world datasets, it is still challenging, particularly for developers with limited expertise, to incorporate this technology into their software systems. The first step in machine learning, data labeling, is traditionally thought of as a tedious, unavoidable task in building a machine learning classifier. However, in this paper, we argue that it can also serve as the first opportunity for developers to gain insight into their dataset. Through a Label-and-Learn interface, we explore visualization strategies that leverage the data labeling task to enhance developers' knowledge about their dataset, including the likely success of the classifier and the rationale behind the classifier's decisions. At the same time, we show that the visualizations also improve users' labeling experience by showing them the impact they have made on classifier performance. We assess the visualizations in Label-and-Learn and experimentally demonstrate their value to software developers who seek to assess the utility of machine learning during the data labeling process.

There is always something new to learn about feature-rich software, even for experienced users. This paper focuses on a specific type of learning activity that we refer to as ad libitum exploration. Based on an interview study with 11 experienced software users, we define ad libitum exploration as the process of routinely seeking new software knowledge, without necessarily having a specific problem to solve. To support this activity, we designed Switter, an alternative Twitter client embedded in a replica of Photoshop's user interface. Given a tweet referencing a tutorial, Switter highlights the interface elements mentioned in the tutorial in the interface replica. Switter also allows users to filter tweets by clicking tools in the interface replica. Through a weeklong field study with nine Photoshop experts, we found that Switter supports a range of software learning objectives, from focused exploration targeting known weaknesses, to the discovery of novel command combinations.

Crowdsourcing systems are designed to elicit help from humans to accomplish tasks that are still difficult for computers. How to motivate workers to stay longer and/or perform better in crowdsourcing systems is a critical question for designers. Previous work have explored different motivational frameworks, both extrinsic and intrinsic. In this work, we examine the potential for curiosity as a new type of intrinsic motivational driver to incentivize crowd workers. We design crowdsourcing task interfaces that explicitly incorporate mechanisms to induce curiosity and conduct a set of experiments on Amazon's Mechanical Turk. Our experiment results show that curiosity interventions improve worker retention without degrading performance, and the magnitude of the effects are influenced by both personal characteristics of the worker and the nature of the task.

The names, icons, and tooltips of commands in feature-rich software are an important source of guidance when locating and selecting amongst commands. Unfortunately, these cues can mislead users into believing that a command is appropriate for a given task, when another command would be more appropriate, resulting in wasted time and frustration. In this paper, we present command disambiguation techniques that inform the user of alternative commands before, during, and after an incorrect command has been executed. To inform the design of these techniques, we define categories of false-feedforward errors caused by misleading interface cues, and identify causes for each. Our techniques are the first designed explicitly to solve this problem in feature-rich software. A user study showed enthusiasm for the techniques, and revealed their potential to play a key role in learning of feature-rich software.

User comments posted to popular online tutorials constitute a rich additional source of information for readers, yet current designs for displaying user comments on tutorial webpages do little to support their use. Instead, comments are separated from the tutorial content they reference and tend to be ordered according to post date. We propose and evaluate the TaggedComments system, a new approach to displaying comments that users post to online tutorials. Using tags supplied by commenters, TaggedComments seeks to enhance the role of user comments by 1) improving their visibility, 2) allowing users to personalize their use of the comments according to their particular information needs, and 3) providing direct access to potentially helpful comments from the tutorial content. A laboratory evaluation with 16 participants shows that, in comparison to the standard comment layout, TaggedComments significantly improves users' subjective impressions of comment utility when interacting with Photoshop tutorials.

Users often make continued and sustained use of online resources to complement use of a desktop application. For example, users may reference online tutorials to recall how to perform a particular task. While often used in a coordinated fashion, the browser and desktop application provide separate, independent mechanisms for helping users find and re-find task-relevant information. In this paper, we describe InterTwine, a system that links information in the web browser with relevant elements in the desktop application to create interapplication information scent. This explicit link produces a shared interapplication history to assist in re-finding information in both applications. As an example, InterTwine marks all menu items in the desktop application that are currently mentioned in the front-most web page. This paper introduces the notion of interapplication information scent, demonstrates the concept in InterTwine, and describes results from a formative study suggesting the utility of the concept.

Classroom response systems (CRS) have been shown to dramatically improve a variety of learning outcomes in science and engineering education. Their adoption in research university classrooms has been very slow. This paper postulates a contributing factor to the lack of widespread adoption of CRS in that context and introduces a new CRS designed to mitigate the identified factor. The CRS, NetClick, is built on the premise that the requirement of authoring new content prevents many Professors from using existing CRS. After describing the operation and advantages of NetClick to other CRS, we describe initial feedback from Computer Science Professors on its use for converting existing teaching slides into interactive content.

Handwriting recognizers occasionally misinterpret digital ink input, requiring users to compare their ink input and the recognizer output to identify and correct errors. Technologies like Anoto pens can make this error discovery and correction task more difficult, because verification of recognizer output may occur many hours after data input and may involve the verification of many documents. In this paper, we explore the design space for error discovery aids geared toward "out-of-the-moment" verification. We present three discovery techniques, a visual proximity technique, a multimodal technique, and a character manipulation technique, and analyze the performance of our techniques. Experimental results show that the visual proximity technique outperforms all others in number of errors caught, and is also significantly better than a control technique. This paper is the first experimental study of techniques that aid in the discovery of handwriting recognition errors.

Computer Algebra Systems and matrix-based mathematics packages provide sophisticated functionality to assist with mathematical problem solving. However, despite their widespread adoption, little work in the human–computer interaction community has examined the extent to which these computational tools support expert problem solving. In this article, we report findings from a qualitative study comparing and contrasting the work practices and software use of practicing researchers in mathematics and engineering who share the goal of developing and defending new mathematical formulations. Our findings indicate that although computational tools are used by both groups to support their work, current mathematics software plays a relatively minor, somewhat untrusted role in the process. Our data suggest that five primary factors limit the applicability of current mathematics software to expert work practices: (a) a lack of transparency in how current software derives its computed results; (b) the lack of clearly defined operational boundaries indicating whether the system can meaningfully operate on the user's input (whether expressions or data); (c) the need for free-form two-dimensional input to support annotations, diagrams, and in-place manipulation of objects of interest; (d) the potential for transcription problems when switching between physical and computational media; and (e) the need for collaboration, particularly in early stages of problem solving. Each of these issues suggests a concrete direction for future improvement of mathematics software for experts. These findings also have more general implications for the design of computational systems intended to support complex problem solving.

This paper presents a recognizer for identifying references to user interface components in online documentation. The recognizer first extracts phrases matching a list of known components, then employs a classifier to reject coincidental matches. We describe why this seemingly straightforward problem is challenging, then show how informal conventions in documentation writing can be leveraged to perform classification. Using the features identified in this paper, our approach achieves an average F1 score of 0.81, and can correctly distinguish between actual command references and coincidental matches in 93.7% of test cases.

This paper introduces the PICL, the portable in-circuit learner. The PICL explores the possibility of providing standalone, low-cost, programming-by-demonstration machine learning capabilities to circuit prototyping. To train the PICL, users attach a sensor to the PICL, demonstrate example input, then specify the desired output (expressed as a voltage) for the given input. The current version of the PICL provides two learning modes, binary classification and linear regression. To streamline training and also make it possible to train on highly transient signals (such as those produced by a camera flash or a hand clap), the PICL includes a number of input inferencing techniques. These techniques make it possible for the PICL to learn with as few as one example. The PICL's behavioural repertoire can be expanded by means of various output adapters, which serve to transform the output in useful ways when prototyping. Collectively, the PICL's capabilities allow users of systems such as the Arduino or littleBits electronics kit to quickly add basic sensor-based behaviour, with little or no programming required.

People routinely rely on Internet search engines to support their use of interactive systems: they issue queries to learn how to accomplish tasks, troubleshoot problems, and otherwise educate themselves on products. Given this common behavior, we argue that search query logs can usefully augment traditional usability methods by revealing the primary tasks and needs of a product's user population. We term this use of search query logs CUTS - characterizing usability through search. In this paper, we introduce CUTS and describe an automated process for harvesting, ordering, labeling, filtering, and grouping search queries related to a given product. Importantly, this data set can be assembled in minutes, is timely, has a high degree of ecological validity, and is arguably less prone to self-selection bias than data gathered via traditional usability methods. We demonstrate the utility of this approach by applying it to a number of popular software and hardware systems.

Collaborative tagging mechanisms are integral to social computing applications in a variety of domains. Their expected benefits include simplified retrieval of digital content, as well as enhanced ability of a community to makes sense of the shared content. We examine the impact of collaborative tagging in context of nutrition management. In a controlled experiment we asked individuals to assess the nutritional value of meals based on photographic images and observed the impact of different types of tags and tagging mechanisms on individuals nutritional sensemaking. The results of the study show that tags enhance individuals' ability to remember the viewed meals. However, we found that some types of tags can be detrimental to sensemaking, rather than supporting it. These findings stress the importance of tagging vocabularies and suggest a need for expert moderation of community sensemaking.

This paper introduces query-feature graphs, or QF-graphs. QF-graphs encode associations between high-level descriptions of user goals (articulated as natural language search queries) and the specific features of an interactive system relevant to achieving those goals. For example, a QF-graph for the GIMP graphics manipulation software links the query "GIMP black and white" to the commands "desaturate" and "grayscale." We demonstrate how QF-graphs can be constructed using search query logs, search engine results, web page content, and localization data from interactive systems. An analysis of QF-graphs shows that the associations produced by our approach exhibit levels of accuracy that make them eminently usable in a range of real-world applications. Finally, we present three hypothetical user interface mechanisms that illustrate the potential of QF-graphs: search-driven interaction, dynamic tooltips, and app-to-app analogy search.

Both handwriting recognition systems and their users are error prone. Handwriting recognizers make recognition errors, and users may miss those errors when verifying output. As a result, it is common for recognized documents to contain residual errors. Unfortunately, in some application domains (e.g. health informatics), tolerance for residual errors in recognized handwriting may be very low, and a desire might exist to maximize user accuracy during verification. In this paper, we present a technique that allows us to measure the performance of a user verifying recognizer output. We inject artificial errors into a set of recognized handwritten forms and show that the rate of injected errors and recognition errors caught is highly correlated in real time. Systems supporting user verification can make use of this measure of user accuracy in a variety of ways. For example, they can force users to slow down or can highlight injected errors that were missed, thus encouraging users to take more care.

This paper presents results from a study examining perceptions and practices of usability in the free/open source software (FOSS) community. 27 individuals associated with 11 different FOSS projects were interviewed to understand how they think about, act on, and are motivated to address usability issues. Our results indicate that FOSS project members possess rather sophisticated notions of software usability, which collectively mirror definitions commonly found in HCI textbooks. Our study also uncovered a wide range of practices that ultimately work to improve software usability. Importantly, these activities are typically based on close, direct interpersonal relationships between developers and their core users, a group of users who closely follow the project and provide high quality, respected feedback. These relationships, along with positive feedback from other users, generate social rewards that serve as the primary motivations for attending to usability issues on a day-to-day basis. These findings suggest a need to reconceptualize HCI methods to better fit this culture of practice and its corresponding value system.

Research indicates that less than 2% of the population reads license agreements during software installation [12]. To address this problem, we developed textured agreements, visually redesigned agreements that employ factoids, vignettes, and iconic symbols to accentuate information and highlight its personal relevance. Notably, textured agreements accomplish these goals without requiring modification of the underlying text. A between-subjects experimental study with 84 subjects indicates these agreements can significantly increase reading times. In our study, subjects spent approximately 37 seconds on agreement screens with textured agreements, compared to 7 seconds in the plain text control condition. A follow-up study examined retention of agreement content, finding that median scores on a comprehension quiz increased by 4 out of 16 points for textured agreements. These results provide convincing evidence of the potential for textured agreements to positively impact software agreement processes.

We present a new survey metric, the Creativity Support Index (CSI) that is designed to help researchers and designers evaluate the level of creativity support provided by various systems or interfaces. We initially employed a top-down literature-based approach to develop a beta version of the Creativity Support Index (Beta CSI). We discuss our usage of the Beta CSI in three different studies and what we learned from those deployments. We also present the results from an extensive creativity vocabulary study (n=300), which revealed a set of orthogonal creativity factors. This led to the current version of the CSI presented in this paper. Initial results from these formative evaluations suggest the value of this tool in assessing and comparing creativity support tools at points in time and longitudinally.

Computer Algebra Systems (CAS) provide sophisticated functionality to assist with mathematical problem solving. Despite their widespread adoption, however, little work in the HCI community has examined the extent to which these computational tools support domain experts. In this paper, we report findings from a qualitative study investigating the work practices and tools of nine mathematicians in a research setting. Counter to our expectations, our data suggests that computational tools play only a minor role in their workflow, with the limited use of CAS owing primarily to four factors: (1) the need for transparency in CAS's reasoning to explain computed results; (2) the problem of rigidity and formality in CAS's input/output style dialogue; (3) the need for 2D input to support a wide range of annotations, diagrams, and in-place manipulation of objects of interest; and (4) the need for collaboration, particularly in early stages of problem solving. While grounded in the study of mathematicians, these findings (particularly the first) have implications for the design of computational systems intended to support complex problem solving.

No abstract available.

Today's handheld gaming systems allow players to engage in multiplayer games via ad-hoc, wireless networking. They are also now sufficiently commonplace that it is possible to study how portability and ad-hoc wireless networking have affected the social gaming practices of owners of these systems. In this paper, we report findings from a qualitative study investigating the collocated multiplayer gaming practices of Nintendo DS owners. Based on interviews of nine DS owners and observations of three organized gaming events, we identified three major themes surrounding the social, multiplayer gaming practices of Nintendo DS users: renegade gaming, or the notion that users reappropriate contexts traditionally hostile to game play; pragmatic and social barriers to the formation of ad-hoc pick-up games, despite a clear desire for multiplayer, collocated gaming; and private gaming spheres, or the observation that the handheld device's form factor creates individual, privatized gaming contexts within larger social contexts. These findings lead to a set of implications for the design of future handheld gaming systems.

Open source projects are gradually incorporating usability methods into their development practices, but there are still many unmet needs. One particular need for nearly any open source project is data that describes its user base, including information indicating how the software is actually used in practice. This paper presents the concept of open instrumentation, or the augmentation of an open source application to openly collect and publicly disseminate rich application usage data. We demonstrate the concept of open instrumentation in ingimp, a version of the open source GNU Image Manipulation Program that has been modified to collect end-user usage data. ingimp automatically collects five types of data: The commands used, high-level user interface events, overall features of the user's documents, summaries of the user's general computing environment, and users' own descriptions of their planned tasks. In the spirit of open source software, all collected data are made available for anyone to download and analyze. This paper's primary contributions lie in presenting the overall design of ingimp, with a particular focus on how the design addresses two prominent issues in open instrumentation: privacy and motivating use.

This paper introduces kinematic templates, an end-user tool for defining content-specific motor space manipulations in the context of editing 2D visual compositions. As an example, a user can choose the "sandpaper" template to define areas within a drawing where cursor movement should slow down. Our current implementation provides templates that amplify or dampen the cursor's speed, attenuate jitter in a user's movement, guide movement along paths, and add forces to the cursor. Multiple kinematic templates can be defined within a document, with overlapping templates resulting in a form of function composition. A template's strength can also be varied, enabling one to improve one's strokes without losing the human element. Since kinematic templates guide movements, rather than strictly prescribe them, they constitute a visual composition aid that lies between unaided freehand drawing and rigid drawing aids such as snapping guides, masks, and perfect geometric primitives.

Collocation preferences represent the commonly used expressions, idioms, and word pairings of a language. Because collocation preferences arise from consensus usage, rather than a set of well-defined rules, they must be learned on a case-by-case basis, making them particularly challenging for non-native speakers of a language. To assist non-native speakers with these parts of a language, we developed AwkChecker, the first end-user tool geared toward helping non-native speakers detect and correct collocation errors in their writing. As a user writes, AwkChecker automatically flags collocation errors and suggests replacement expressions that correspond more closely to consensus usage. These suggestions include example usage to help users choose the best candidate. We describe AwkChecker's interface, its novel methods for detecting collocation errors and suggesting alternatives, and an early study of its use by non-native English speakers at our institution. Collectively, these contributions advance the state of the art in writing aids for non-native speakers.

This paper introduces Jump, a prototype computer vision-based system that transforms paper-based architectural documents into tangible query interfaces. Specifically, Jump allows a user to obtain additional information related to a given architectural document by framing a portion of the drawing with physical brackets. The framed area appears in a magnified view on a separate display and applies the principle of semantic zooming to determine the appropriate level of detail to show. Filter tokens can be placed on the paper to modify the digital presentation to include information not on the original drawing itself, such as electrical, mechanical, and structural information related to the given space. These filter tokens serve as tangible sliders in that their relative location on the paper controls the degree to which their information is blended with the original document. To address the issue of recognition errors, Jump introduces the notion of a reflection window, or an inset window that serves to reproduce Jump's current interpretation of the visual scene. The system's overall design is informed by a set of in situ studies of architectural technologists and formative evaluations with the same group.

Creativity support tools is a research topic with high risk but potentially very high payoff. The goal is to develop improved software and user interfaces that empower users to be not only more productive but also more innovative. Potential users include software and other engineers, diverse scientists, product and graphic designers, architects, educators, students, and many others. Enhanced interfaces could enable more effective searching of intellectual resources, improved collaboration among teams, and more rapid discovery processes. These advanced interfaces should also provide potent support in hypothesis formation, speedier evaluation of alternatives, improved understanding through visualization, and better dissemination of results. For creative endeavors that require composition of novel artifacts (e.g., computer programs, scientific papers, engineering diagrams, symphonies, artwork), enhanced interfaces could facilitate exploration of alternatives, prevent unproductive choices, and enable easy backtracking. This U.S. National Science Foundation sponsored workshop brought together 25 research leaders and graduate students to share experiences, identify opportunities, and formulate research challenges. Two key outcomes emerged: (a) encouragement to evaluate creativity support tools through multidimensional in-depth longitudinal case studies and (b) formulation of 12 principles for design of creativity support tools.

The complexity of many problems necessitates creating and exploring multiple, alternative solutions. However, current user interfaces do not cleanly support creating alternatives at a time when they are likely to be discovered: as users interactively modify data. This paper presents Parallel Paths, a novel model of interaction that facilitates generating, manipulating, and comparing alternative solutions. In contrast to existing approaches such as automated history capture tools, Parallel Paths emphasizes the active, simultaneous development of multiple, alternative solutions. We demonstrate this model of interaction in Parallel Pies, a user interface mechanism developed for image manipulation tasks that allows users to: easily create solution alternatives as they interact with a command; embed the alternatives in the same workspace; manipulate the alternatives independently or simultaneously as if they were the same object; and perform side-by-side comparisons of each. Results from an initial evaluation are presented, along with implications for future designs.

The creative process requires experimentation, the exploration of variations, and the continual evaluation of one's progress. While these processes are frequently non-linear and iterative, modern user interfaces do not explicitly support these practices, and instead impose a linear progression through tasks that is a poor fit for creative pursuits. In this paper we use data from three case studies, and draw upon Sch$#246;n's theory of reflection-in-action to illustrate specific deficiencies in current user interfaces when used in creative endeavors. We then develop a set of guidelines for user interface design and demonstrate their application in three designs intended to support tasks in the domain of image manipulation.

We introduce Side Views, a user interface mechanism that provides on-demand, persistent, and dynamic previews of commands. Side Views are designed to explicitly support the practices and needs of expert users engaged in openended tasks. In this paper, we summarize results from field studies of expert users that motivated this work, then discuss the design of Side Views in detail. We show how Side Views' design affords their use as tools for clarifying, comparing, and contrasting commands; generating alternative visualizations; experimenting without modifying the original data (i.e., "what-if" tools); and as tools that support the serendipitous discovery of viable alternatives. We then convey lessons learned from implementing Side Views in two sample applications, a rich text editor and an image manipulation application. These contributions include a discussion of how to implement Side Views for commands with parameters, for commands that require direct user input (such as mouse strokes for a paint program), and for computationally-intensive commands.

Historically one of the visions for human-computer symbiosis has been to augment human intelligence and extend people's cognitive abilities. In this paper, we present two visually-based systems to enhance a person's ability to flexibly control their pace while engaged in a cognitively demanding activity. In these investigations, we explore pacing interfaces that minimize the cognitive demands for assessing a current pace, provide ambient cues that can be quickly interpreted without incurring significant interruption from the current task, and place knowledge in the world to flexibly support different pacing strategies. Evaluation of our pacing interfaces shows that technology can successfully support pacing.