Abstract People are becoming increasingly sophisticated in their ability to navigate information spaces using search, hyperlinks, and visualization. But, mobile phones preclude the use of multiple coordinated views that have proven effective in the desktop environment (e.g., for business intelligence or visual analytics). In this work, we propose to model information as multivariate heterogeneous networks to enable greater analytic expression for a range of sensemaking tasks while suggesting a new, list‐based paradigm with gestural navigation of structured information spaces on mobile phones. We also present a mobile application, called Orchard, which combines ideas from both faceted search and interactive network exploration in a visual query language to allow users to collect facets of interest during exploratory navigation. Our study showed that users could collect and combine these facets with Orchard, specifying network queries and projections that would only have been possible previously using complex data tools or custom data science.

Across the world, policy initiatives are being developed to engage children with computer programming and computational thinking. Diversity and inclusion has been a strong force in this agenda, but children with disabilities have largely been omitted from the conversation. Currently, there are no age appropriate tools for teaching programming concepts and computational thinking to primary school children with visual disabilities. We address this gap through presenting the design and implementation of Torino, a tangible programming language for teaching programming concepts to children age 7–11 regardless of level of vision. In this paper, we: (1) describe the design process done in conjunction with children with visual disabilities; (2) articulate the design decisions made; and (3) report insights generated from an evaluation with 10 children with mixed visual abilities that considers how children are able to trace (read) and create (write) programs with Torino. We discuss key design trade-offs: (1) readability versus extensibility; and (2) size versus liveness. We conclude by reflecting upon how an inclusive design approach shaped the final result.

The application of modern NLP and ML techniques to large-scale datasets can generate implicit graphs that are so densely connected as to be unusable when rendered as node-link diagrams. We present a two-stage approach to extracting usable, map-like layouts from large, dense input graphs. This approach uses edge-cutting strategies based on node and edge metrics to reduce a graph to a skeletal structure showing only essential relationships, before filling in the resulting communities to create dense but usable layouts. Through a case study on a 145k-document adversarial health communication dataset, we show that each edge-cutting strategy has advantages and disadvantages, and that the appropriate choice of strategy depends on the data, user, and task.

Technical Support Scams delivered through malicious webpages and linked advertisements are an endemic problem on the web. To avoid detection, scammers systematically vary the URLs, web page designs, and phone numbers of scam delivery. We describe a web-scale pipeline powered by Cloud AI services that continuously detects and links evidence of such scams. By integrating this evidence in a graph structure and exposing it for forensic analysis in a user interface, we enable investigators and law enforcement partners to track the evolving scope and sophistication of scam operations. This approach automates and scales investigative tradecraft that contributed to major enforcement actions by the FTC in 2016. From 2016-2018, it helped reduce consumer exposure to such scams by 5 percent.

As Visual Analytics (VA) research grows and diversifies to encompass new systems, techniques, and use contexts, gaining a holistic view of analytic practices is becoming ever more challenging. However, such a view is essential for researchers and practitioners seeking to develop systems for broad audiences that span multiple domains. In this paper, we interpret VA research through the lens of Activity Theory (AT)-a framework for modelling human activities that has been influential in the field of Human-Computer Interaction. We first provide an overview of Activity Theory, showing its potential for thinking beyond tasks, representations, and interactions to the broader systems of activity in which interactive tools are embedded and used. Next, we describe how Activity Theory can be used as an organizing framework in the construction of activity typologies, building and expanding upon the tradition of abstract task taxonomies in the field of Information Visualization. We then apply the resulting process to create an activity typology for Visual Analytics, synthesizing a wide range of systems and activity concepts from the literature. Finally, we use this typology as the foundation of an activity-centered design process, highlighting both tensions and opportunities in the design space of VA systems.

While exertion games facilitate, and benefit from, social play, most exertion games merely support players acting independently. To help designers explore the richness of social play in exertion games, we present the design dimension “bodily interplay” that gives critical focus to how players’ bodies interact with one another. We offer two broad categories of bodily interplay— parallel and interdependent play —to explain how exertion games can facilitate independent and offensive/defensive-type experiences. These categories can be applied to both the physical and virtual space, and by looking at all permutations of these categories, we articulate four ways of coupling the spaces: comparative, actuated, derived, and projected coupling. This article illustrates the inspirational power of the dimensions by applying them to the analysis of four exertion games. Altogether, we articulate a vocabulary that can guide designers in the creation of social exertion games, helping players profit from the many benefits of exertion.

Notes can be taken in a linear or nonlinear way. Previous work suggests that nonlinear note taking is advantageous in terms of sense-making and long-term recall. However, previous studies also reveal that the combination of divided attention and time pressure make realtime notetaking a challenge. In this paper, we propose a new hybrid workflow inheriting advantages from both linear and nonlinear notetaking approaches. Our resulting HyNote (Hybrid Notetaking) system uses statistical parsing of linear raw notes to facilitate concepts mapping, allowing users to smoothly switch between linear and nonlinear approaches with low effort and time costs. Results from our preliminary study of HyNote show that users can easily map concepts in realtime and achieve superior understanding of lecture contents in a video learning task compared with using the traditional linear Notepad application.

We explore design opportunities for using interactive technologies to enrich group fitness exercises, such as group spinning and swimming, in which an instructor guides a workout program and members synchronously perform a shared physical activity. As a case study, we investigate group fitness swimming. The design challenge is to coordinate a large group of people by considering trade-offs between social awareness and information overload. Our resulting group fitness swimming game, SwimTrain, allows a group of people to have localized synchronous interactions over a virtual space. The game uses competitive and cooperative phases to help group members acquire group-wide awareness. The results of our user study showed that SwimTrain provides socially-enriched swimming experiences, motivates swimmers to follow a training regimen and exert more intensely, and allows strategic game play dealing with skill differences among swimmers. Consequently, we propose several practical considerations for designing group fitness exergames.

Prolonged inactivity in office workers is a well-known contributor to various diseases, such as obesity, diabetes, and cardiovascular dysfunction. In recent years, active workstations that incorporate physical activities such as walking and cycling into the workplace have gained significant popularity, owing to the accessibility of the workouts they offer. While their efficacy is well documented in medical and physiological literature, research regarding the user experience of such systems has rarely been performed, despite its importance for interactive systems design. As a case study, we focus on active workstations that incorporate under desk elliptical trainers, and conduct controlled experiments regarding work performance and a four week long field deployment to explore user experience with 13 participants. We investigate how such workouts influence work performance, when and why workers work out during working hours, and the general feelings of workers regarding usage. Our experimental results indicate that while work performance is not influenced, the cognitive load of tasks critically influences workout decisions. Active workstations were alternatively used as mood enhancers, footrests, and for fidgeting, and there exist unique social and technical aspects to be addressed, such as noise issues and space constraints. Our results provide significant implications for the design of active workstations and interactive workplaces in general.

The Slide and Canvas metaphors are two ways of helping people create visual aids for oral presentations. Although such physical metaphors help both authors and audiences make sense of material, they also constrain authoring in ways that can negatively impact presentation delivery. In this article, we derive heuristics for the design of presentation media that are independent of any underlying physical metaphors. We use these heuristics to craft a new kind of presentation medium called SlideSpace—one that combines hierarchical outlines, content collections, and design rules to automate the real-time, outline-driven synthesis of hybrid Slide-Canvas visuals. Through a qualitative study of SlideSpace use, we validate our heuristics and demonstrate that such a hybrid presentation medium can combine the advantages of existing systems while mitigating their drawbacks. Overall, we show how a heuristic design approach helped us challenge entrenched physical metaphors to create a fundamentally digital presentation medium with the potential to transform the activities of authoring, delivering, and viewing presentations.

The unique aquatic nature of swimming makes it difficult to use social or technical strategies to mitigate the tediousness of monotonous exercises. In this study, we propose the use of a smartphone-based multiplayer exergame named MobyDick . MobyDick is designed to be played while swimming, where a team of swimmers collaborate to hunt down a virtual monster. To this end, we take into account both human factors and technical challenges under swimming contexts. First, we perform a comparative analysis of a variety of wireless networking technologies in the aquatic environment and identify various technical constraints on wireless networking. Second, we develop a swimming activity recognition system to enable precise and real-time game inputs. Third, we devise a multiplayer game design by employing the unique interaction mode viable in an underwater environment, where the abilities of human communication are highly limited. Finally, we prototype MobyDick on waterproof off-the-shelf Android phones, and we deploy it in real swimming pool environments ( n = 8). Our qualitative analysis of user interview data reveals certain unique aspects of multiplayer swimming games.

Good alignment and repetition of objects across presentation slides can facilitate visual processing and contribute to audience understanding. However, creating and maintaining such consistency during slide design is difficult. To solve this problem, we present two complementary tools: (1) StyleSnap, which increases the alignment and repetition of objects by adaptively clustering object edge positions and allowing parallel editing of all objects snapped to the same spatial extent; and (2) FlashFormat, which infers the least-general generalization of editing examples and applies it throughout the selected range. In user studies of repetitive styling task performance, StyleSnap and FlashFormat were 4-5 times and 2-3 times faster respectively than conventional editing. Both use a mixed-initiative approach to improve the consistency of slide decks and generalize to any situations involving direct editing across disjoint visual spaces.

The present study used a flat keyboard without moving keys and enabled with haptic keyclick feedback to examine the effect of haptic keyclick feedback on touch typing performance. We investigated, with well-controlled stimuli and a within-participant design, how haptic keyclick feedback might improve typing performance in terms of typing speed, typing efficiency and typing errors. Of the three kinds of haptic feedback we tested, all increased typing speed and decreased typing errors compared to a condition without haptic feedback. We did not find significant differences among the types of haptic feedback. We also found that auditory keyclick feedback alone is not as effective as haptic keyclick feedback, and the addition of auditory feedback to haptic feedback does not lead to any significant improvement in typing performance. We also learned that global haptic keyclick feedback simulated through local keyclick feedback on each key (as opposed to haptic feedback all over the keyboard) might have the additional and unexpected benefit of helping a typist to locate keys on a keyboard. Furthermore, the participants preferred auditory or haptic keyclick feedback to no feedback, and haptic feedback restricted to the typing finger alone is preferred to that over a larger area of the keyboard.

Advances in sensing technologies have led to research into exertion games that support physically effortful experiences. Despite the existence of theoretical frameworks that can be used to analyze such exertion experiences, there are few tools to support the hands-on practice of exertion game design. To address this, we present a set of design cards based on the "Exertion Framework", grounded in our experience of creating exertion games for over a decade. We present results demonstrating the value and utility of these Exertion Cards based on our studies of their use in three workshops held over seven sessions with 134 design students and experts. We also articulate lessons learned from transforming a theoretical framework into a design tool that aims to support designers in their practice.

Once upon a time, people told stories unencumbered by slides. What modern presentations gain through visual slide support, however, is often at the expense of storytelling. We present TurningPoint, a probe to investigate the potential use of narrative-driven talk planning in slideware. Our study of TurningPoint reveals a delicate balance between narrative templates focusing author attention in ways that save time, and fixating attention in ways that limit experimentation.

Rehearsal is a critical component of preparing to give an oral presentation, yet it is frequently abbreviated, performed in ways that are inefficient or ineffective, or simply omitted. We conducted an exploratory study to understand the relationship between the theory and practice of presentation rehearsal, classifying our qualitative results into five themes to motivate more structured rehearsal support deeply integrated in slide presentation software. In a within-subject study (N=12) comparing against participants' existing rehearsal practices, we found that our resulting PitchPerfect system significantly improved overall presentation quality and content coverage as well as provided greater support for content mastery, time management, and confidence building.

Managing time while presenting is challenging, but mobile devices offer both convenience and flexibility in their ability to support the end-to-end process of setting, refining, and following presentation time targets. From an initial HCI-Q study of 20 presenters, we identified the need to set such targets per 'zone' of consecutive slides (rather than per slide or for the whole talk), as well as the need for feedback that accommodates two distinct attitudes to time management. These findings led to the design of TalkZones, a mobile application for timing support. When giving a 20-slide, 6m40s rehearsed but interrupted talk, 12 participants who used TalkZones registered a mean overrun of only 8s, compared with 1m49s for 12 participants who used a regular timer. We observed a similar 2% overrun in our final study of 8 speakers giving rehearsed 30-minute talks in 20 minutes. Overall, we show that TalkZones can encourage presenters to advance slides before it is too late to recover, even under the adverse timing conditions of short and shortened talks.

Supporting physical exertion is a growing trend in digital technology design. However, most experiences focus on bodily actions in which participants act independently of each other. In contrast, we focus on virtual body-to-body interactions between multiple participants, inspired by combat-oriented sports such as boxing that highlight the need to act while avoiding reciprocal bodily action. Mediating such body-to-body interactions with technology is challenging, particularly when participants are not co-located. Prior systems have often involved a mixture of novel physical interfaces and interactions through virtual avatars. This paper charts a design space for virtual combat play experiences and offers a set of design dimensions and recommendations for future systems. We draw on our experiences of designing and evaluating Remote Impact - a boxing-style exertion game involving aggressive bodily interaction with a large force-sensing projection surface. By expanding our knowledge of mediated exertion with an understanding of combat interactions we extend the social experience space of exertion play.

Taxonomies are a useful and ubiquitous way of organizing information. However, creating organizational hierarchies is difficult because the process requires a global understanding of the objects to be categorized. Usually one is created by an individual or a small group of people working together for hours or even days. Unfortunately, this centralized approach does not work well for the large, quickly changing datasets found on the web. Cascade is an automated workflow that allows crowd workers to spend as little at 20 seconds each while collectively making a taxonomy. We evaluate Cascade and show that on three datasets its quality is 80-90% of that of experts. Cascade has a competitive cost to expert information architects, despite taking six times more human labor. Fortunately, this labor can be parallelized such that Cascade will run in as fast as four minutes instead of hours or days.

Games that engage both mind and body by targeting users' kinesthetic intelligence have the potential to transform the activity of learning across a wide variety of domains. To investigate this potential in the context of second language learning, we have developed SpatialEase: a Kinect game for the body-based learning of language that is grounded in space and motion. In this game, learners respond to audio commands in the second language by moving their bodies in space, while a game mechanic based on distributed cued-recall supports learning over time. Our comparison of SpatialEase with the popular Rosetta Stone software for learner of Mandarin Chinese showed similar learning gains over a single session and generated several key implications for the future design of mixed-modality learning systems.

Presentations are a crucial form of modern communication, yet there is a dissonance between everyday practices with presentation tools and best practices from the presentation literature. We conducted a grounded theory study to gain a better understanding of the activity of presenting, discovering the potential for a more dynamic, automated, and story-centered approach to prototyping slide presentations that are themselves dynamic in their ability to help presenters rehearse and deliver their story. Our prototype tool for dynamic presentation prototyping, which we call HyperSlides, uses a simple markup language for the creation of hierarchically structured scenes, which are algorithmically transformed into hyperlinked slides of a consistent and minimalist style. Our evaluation suggests that HyperSlides helps idea organization, saves authoring time, creates aesthetic layouts, and supports more flexible rehearsal and delivery than linear slides, at the expense of reduced layout control and increased navigation demands.

Presentation authoring is an important activity, but often requires the secondary task of collecting the information and media necessary for both slides and speech. Integration of implicit search and peripheral displays into presentation authoring tools may reduce the effort to satisfy not just active needs the author is aware of, but also latent needs that she is not aware of until she encounters content of perceived value. We develop SidePoint, a peripheral panel that supports presentation authoring by showing concise knowledge items relevant to the slide content. We study SidePoint as a technology probe to examine the benefits and issues associated with peripheral knowledge panels for presentation authoring. Our results show that peripheral knowledge panels have the potential to satisfy both types of needs in ways that transform presentation authoring for the better.

Exercising with others, such as jogging in pairs, can be socially engaging. However, if exercise partners have different fitness levels then the activity can be too strenuous for one and not challenging enough for the other, compromising engagement and health benefits. Our system, Jogging over a Distance, uses heart rate data and spatialized sound to create an equitable, balanced experience between joggers of different fitness levels who are geographically distributed. We extend this prior work by analyzing the experience of 32 joggers to detail how specific design features facilitated, and hindered, an engaging and balanced exertion experience. With this knowledge, we derive four dimensions that describe a design space for balancing exertion experiences: Measurement, Adjustment, Presentation and Control. We also present six design tactics for creating balanced exertion experiences described by these dimensions. By aiding designers in supporting participants of different physical abilities, we hope to increase participation and engagement with physical activity and facilitate the many benefits it brings about.

Learning a second language is hard, especially when the learner's brain must be retrained to identify sounds not present in his or her native language. It also requires regular practice, but many learners struggle to find the time and motivation. Our solution is to break down the challenge of mastering a foreign sound system into minute-long episodes of "microtraining" delivered through mobile gaming. We present the example of Tip Tap Tones - a mobile game with the purpose of helping learners acquire the tonal sound system of Mandarin Chinese. In a 3-week, 12-user study of this system, we found that an average of 71 minutes' gameplay significantly improved tone identification by around 25%, regardless of whether the underlying sounds had been used to train tone perception. Overall, results suggest that mobile microtraining is an efficient, effective, and enjoyable way to master the sounds of Mandarin Chinese, with applications to other languages and domains.

Flashcard systems typically help students learn facts (e.g., definitions, names, and dates), relying on intense initial memoriztion with subsequent tests delayed up to days later. This approach does not exploit the short, sparse, and mobile opportunities for microlearning throughout the day, nor does it support learners who need the motivation that comes from successful study sessions. In contrast, our MemReflex system of adaptive flashcards gives fast-feedback by retesting new items in quick succession, dynamically scheduling future tests according to a model of the learner's memory. We evaluate MemReflex across three user studies. In the first two studies, we demonstrate its effectiveness for both audio and text modalities, even while walking and distracted. In the third study of second-language vocabulary learning, we show how MemReflex enhanced learner accuracy, confidence, and perceptions of control and success. Overall, the work suggests new directions for mobile microlearning and "micro activities" in general.

Learning a new language is hard, but learning to use it confidently in conversations with native speakers is even harder. From our field research with language learners, with support from Cognitive Psychology and Second Language Acquisition, we argue for the value of contextual microlearning in the many breaks spread across different places and throughout the day. We present a mobile application that supports such microlearning by leveraging the location-based service Foursquare to automatically provide contextually relevant content in the world's major cities. In an evaluation of Mandarin Chinese learning, a four-week, 23-user study spanning Beijing and Shanghai compared this contextual system to a system based on word frequency. Study sessions with the contextual version lasted half as long but occurred in twice as many places as sessions with the frequency version, suggesting a complementary relationship between the two approaches.

Exertion games require investing physical effort. The fact that such games can support physical health is tempered by our limited understanding of how to design for engaging exertion experiences. This paper introduces the Exertion Framework as a way to think and talk about Exertion Games, both for their formative design and summative analysis. Our Exertion Framework is based on the ways in which we can conceive of the body investing in game-directed exertion, supported by four perspectives on the body (the Responding Body, Moving Body, Sensing Body and Relating Body) and three perspectives on gaming (rules, play and context). The paper illustrates how this framework was derived from prior systems and theory, and presents a case study of how it has been used to inspire novel exertion interactions.

Nowadays it has become increasingly common for family members to be distributed in different time zones. These time differences pose specific challenges for communication within the family and result in different communication practices to cope with them. To gain an understanding of current challenges and practices, we interviewed people who regularly communicate with immediate family members living in other time zones. We report primary findings from the interviews, and identify design opportunities for improving the experience of cross time zone family communication.

Exertion activities, such as jogging, require users to invest intense physical effort and are associated with physical and social health benefits. Despite the benefits, our understanding of exertion activities is limited, especially when it comes to social experiences. In order to begin understanding how to design for technologically augmented social exertion experiences, we present "Jogging over a Distance", a system in which spatialized audio based on heart rate allowed runners as far apart as Europe and Australia to run together. Our analysis revealed how certain aspects of the design facilitated a social experience, and consequently we describe a framework for designing augmented exertion activities. We make recommendations as to how designers could use this framework to aid the development of future social systems that aim to utilize the benefits of exertion.

The majority of Tangible User Interfaces (TUIs) consist of rigid objects that are either held in the hands, or arranged relative to each other on a horizontal or vertical surface. In this paper we consider the design space of TUIs that can be created by moving parts of an assembly relative to each other -- creating articulated tangible interfaces. An analytic approach to this design space allows us to identify the potential applications and trade-offs of TUIs that include mechanically articulated parts.

Tangible User Interfaces (TUIs) are commonly accepted as those in which the configuration of physical objects embodies digital system state, providing "graspable" digital media that can be manipulated in the focus of users' attention. In this paper we offer an alternative perspective on the use of tangibility in interaction, in which meaning is created not through precise manipulations of a computationally-interpreted spatial syntax, but through imprecise interactions with independently meaningful, digitally-augmented physical tokens. Users are free to arrange such tokens around the periphery of their workspace, away from their normal centre of attention, ready to selectively and fluidly engage them in loosely related, dispersed episodes of use. We call this concept "peripheral tangible interaction", and in this paper we describe both our analytic approach to designing a personal desktop TUI supporting such an interaction style, and user responses to its analytically-inspired features during extended deployment in a real office context.

In the context of tangibility, mobile phones are rapidly becoming sensor-rich handheld computers with the potential to take better advantage of our physical capabilities and our lifetime of experiences interacting both in and with the world around us. In this paper, we analyse four different ways in which mobiles can be used to represent and control digital information, showing that each resulting interaction style is characterized by a unique coordination of the user's attention and two hands in relation to the mobile device. We present our analysis in terms of a framework that can be used to critically examine future schemes of bimanual interaction with mobile phones.

Project plans fail to reflect the real work situation if they represent infrequently updated, high level objectives. We propose a tangible interface that allows team members to update and share their progress in real time through peripheral task management