Blind and low-vision (BLV) employees in mixed-visual ability teams often encounter information (e.g., PDFs, diagrams) in inaccessible formats. To enable teamwork, teams must transform these representations by modifying or re-creating them into accessible forms. However, these transformations are frequently overlooked, lack infrastructural support, and cause additional labour. To design systems that move beyond one-off accommodations to effective mixed-ability collaboration, we need a deeper understanding of the representations, their transformations and how they occur. We conducted a week-long diary study with follow-up interviews with 23 BLV and sighted professionals from five legal, non-profit, and consulting teams, documenting 36 transformation cases. Our analysis characterizes how teams perform representational transformations for accessibility: how they are triggered proactively or reactively, how they simplify or enhance, and four common patterns in which workers coordinate with each other to address representational incompatibility. Our findings uncover opportunities for designing systems that can better support mixed-visual ability work.

Generative AI (GenAI) is becoming a widely adopted learning support tool for both students and instructors, as it offers benefits such as personalized tutoring and scaffolded learning. However, recent research highlights potential drawbacks such as overreliance and metacognitive issues, especially in novice programmers. Most prior work focuses on introductory programming courses, and important questions remain about the underlying mechanisms behind the negative effects of GenAI and if findings can be generalized when students learn more advanced computer science concepts. To address this gap, we conducted a mixed-methods study comparing student interactions with GenAI to two traditional learning supports in a second-year algorithms course: algorithm visualization (AV) and human live tutoring (LT). Twelve students participated in three 90-minute study sessions focusing on sorting, tree, and graph algorithms. We recorded gaze and interaction data, and each session concluded with a test assessing their conceptual understanding of the topic. Our analysis classifies when during the problem-solving process participants sought help, and compares the interaction patterns across the three learning supports. Although GenAI produced a larger increase in self-efficacy compared to live tutoring, it was associated with noticeably lower results in learning outcomes. We found that participants did not use algorithm visualizations effectively, faced usage barriers when using GenAI to learn advanced topics, and that live tutoring yielded the highest learning outcomes.

When a reader encounters a word in English, they split the word into smaller orthographic units in the process of recognizing its meaning. For example, "rough", when split according to phonemes, is decomposed as r-ou-gh (not as r-o-ugh or r-ough), where each group of letters corresponds to a sound. Since there are many ways to segment a group of letters, this constitutes a computational operation that has to be solved by the reading brain, many times per minute, in order to achieve the recognition of words in text necessary for reading. In English, the irregular relationships between groups of letters and sounds, and the wide variety of possible groupings make this operation harder than in more regular languages such as Italian. If this segmentation takes a significant amount of time in the process of recognizing a word, it is conceivable that providing segmentation information in the text itself could help the reading process by reducing its computational cost. In this paper we explore whether and how different visual interventions from the visualization literature could communicate segmentation information for reading and word recognition. We ran a series of pre-registered lexical decision experiments with 192 participants that tested five main types of visual segmentations: outlines, spacing, connections, underlines and color. The evidence indicates that, even with a moderate amount of training, these visual interventions always slow down word identification, but each to a different extent (between 32.7ms-color technique-and 70.7ms-connection technique). These findings are important because they indicate that, at least for typical adult readers with a moderate amount of specific training in these visual interventions, accelerating the lexical decision task is unlikely. Importantly, the results also offer an empirical measurement of the cost of a common set of visual manipulations of text, which can be useful for practitioners seeking to visualize alongside or within text without impacting reading performance. Finally, the interaction between typographically encoded information and visual variables presented unique patterns that deviate from existing theories, suggesting new directions for future inquiry.

Finding a particular object in a display is important for viewers in many visualizations, for example, when reacting to brushing or to a highlighted object. This can be enabled by making the target object different in one of the visual variables that determine the object's appearance; for example, by changing its color or size. Certain interpretations of the visual search literature have promoted the view that using visual variables such as hue-often labeled as preattentive-would make the target object automatically "popout," implying that an object can be located almost instantly, regardless of the number of objects in the display. In this paper we present a study that serves as a bridge between the extensive visual search literature and visualization, establishing empirical base measurements for the localization task. By testing displays with up to hundreds of objects, we are able to show that none of the common visual variables is immune to the increase in the number of objects. We also provide the first empirically informed comparisons between visual variables for this task in the context of visualization, and show how different visual variables have varying robustness with respect to two additional dimensions: the location of the target and the overall visual arrangement (layout). A free copy of this paper and all supplemental materials are available on our online repository: https://osf.io/z68ak/overview.

Data tables are one of the most common ways in which people encounter data. Although mostly built with text and numbers, data tables have a spatial layout and often exhibit visual elements meant to facilitate their reading. Surprisingly, there is an empirical knowledge gap on how people read tables and how different visual aids affect people's reading of tables. In this work, we seek to address this vacuum through a controlled study. We asked participants to repeatedly perform four different tasks with four table representation conditions (plain tables, tables with zebra striping, tables with cell background color encoding cell value, and tables with in-cell bars with lengths encoding cell value). We analyzed completion time, error rate, gaze-tracking data, mouse movement and participant preferences. We found that color and bar encodings help for finding maximum values. For a more complex task (comparison of proportional differences) color and bar helped less than zebra striping. We also characterize typical human behavior for the four tasks. These findings inform the design of tables and research directions for improving presentation of data in tabular form.

Sonification serves as a powerful tool for data accessibility, especially for people with vision loss. Among various modalities, speech is a familiar means of communication similar to the role of text in visualization. However, speech-based sonification is underexplored. We introduce SpeechTone, a novel speech-based mark for data sonification and extension to the existing Erie declarative grammar for sonification. It encodes data into speech attributes such as pitch, speed, voice and speech content. We demonstrate the efficacy of SpeechTone through three examples.

Diagrams often appear as node-link representations in contexts such as taxonomies, mind maps and networks in textbooks. Despite their pervasiveness, they present accessibility challenges for blind and low-vision people. To address this challenge, we introduce Touch-and-Audio-based Diagram Access (TADA), a tablet-based interactive system that makes diagram exploration accessible through musical tones and speech. We designed TADA informed by an interview study with 15 participants who shared their challenges and strategies with diagrams. TADA enables people to access a diagram by: i) engaging in open-ended touch-based explorations, ii) searching for nodes, iii) navigating between nodes and iv) filtering information. We evaluated TADA with 25 participants and found it useful for gaining different perspectives on diagrammatic information.

Learning to program poses significant challenges, not only to learners but also teaching challenges to instructors. Several previous approaches to facilitate learning to program or analyze algorithms have employed visuals and visualizations, but they are limited in interactivity and in the ability of instructors and learners to customize the visuals to their learning goals. In this work, we present the design and implementation of P-Inti, an interactive constructive learning aid designed to allow learners to use a visual canvas to explore aspects of the state, control flow and execution of a program or algorithm. Instructors can also use the tool to generate interactive visual explanations for code and support quick switching between different algorithms.

Applications that use multiple devices and surfaces provide new opportunities for innovative interaction -- but despite the wide variety of research that has been carried out on multi-surface systems, building these kinds of applications is still difficult. In particular, multi-surface apps that use interactive visual workspaces are complicated because current tools do not provide low-level access to a connected and interactive graphical canvas that is shown on different devices. This difficulty limits the explorations that designers and developers can carry out within the multi-surface design space. To address this problem, we have developed WAMS -- an open-source web-based toolkit that provides several programming abstractions for building visual-workspace applications across multiple surfaces. WAMS uses three main concepts -- a virtual visual workspace, views onto that workspace, and graphical workspace objects -- and provides support for connecting multiple devices, creating and manipulating objects, managing and laying out views, and handling events from multiple surfaces. WAMS simplifies the development of a wide variety of applications including composite display configurations, shared-workspace groupware, systems that place different UI elements onto different devices, and bring-your-own-device applications. We describe WAMS's main abstractions and concepts, provide several examples that show the breadth of the approach, and assess the toolkit in terms of effectiveness, coverage, extensibility, and integration with existing practices and tools.

In this paper, we present Demystify, a general tool for creating human-interpretable step-by-step explanations of how to solve a wide range of pen and paper puzzles from a high-level logical description. Demystify is based on Minimal Unsatisfiable Subsets (MUSes), which allow Demystify to solve puzzles as a series of logical deductions by identifying which parts of the puzzle are required to progress. This paper makes three contributions over previous work. First, we provide a generic input language, based on the Essence constraint language, which allows us to easily use MUSes to solve a much wider range of pen and paper puzzles. Second, we demonstrate that the explanations that Demystify produces match those provided by humans by comparing our results with those provided independently by puzzle experts on a range of puzzles. We compare Demystify to published guides for solving a range of different pen and paper puzzles and show that by using MUSes, Demystify produces solving strategies which closely match human-produced guides to solving those same puzzles (on average 89% of the time). Finally, we introduce a new randomised algorithm to find MUSes for more difficult puzzles. This algorithm is focused on optimised search for individual small MUSes.

Essay-writing is a complex, cognitively demanding activity. Essay-writers must synthesise source texts and original ideas into a textual essay. Previous work found that writers produce better essays when they create effective intermediate representations. Diagrams, such as concept maps and argument maps, are particularly effective. However, there is insufficient knowledge about how people use these intermediate representations in their essay-writing workflow. Understanding these processes is critical to inform the design of tools to support workflows incorporating intermediate representations. We present the findings of a study, in which 20 students planned and wrote essays. Participants used a tool that we developed, Write Reason, which combines a free-form mapping interface with an essay-writing interface. This let us observe the types of intermediate representations participants built, and crucially, the process of how they used and moved between them. The key insight is that much of the important cognitive processing did not happen within a single representation, but instead in the processes that moved between multiple representations. We label these processes ‘representational transformations’. Our analysis characterises key properties of these transformations: cardinality, explicitness, and change in representation type. We also discuss research questions surfaced by the focus on transformations, and implications for tool designers.

The advent of variable font technologies---where typographic parameters such as weight, x-height and slant are easily adjusted across a range---enables encoding ordinal, interval or ratio data into text that is still readable. This is potentially valuable to represent additional information in text labels in visualizations (e.g., font weight can indicate city size in a geographical visualization) or in text itself (e.g., the intended reading speed of a sentence can be encoded with the font width). However, we do not know how different parameters, which are complex variations of shape, are perceived by the human visual system. Without this information it is difficult to select appropriate parameters and mapping functions that maximize perception of differences within the parameter range. We provide an empirical characterization of seven typographical parameters of Latin fonts in terms of absolute perception and just noticeable differences (JNDs) to help visualization designers to choose typographic parameters for visualizations that contain text, as well as support typographers and type designers when selecting which levels of these parameters to implement to achieve differentiability between normal text, emphasized text and different headings.

Deep learning algorithms increasingly support automated systems in areas such as human activity recognition and purchase recommendation. We identify a current trend in which data is transformed first into abstract visualizations and then processed by a computer vision deep learning pipeline. We call this VisuaLization As Intermediate Representation (VLAIR) and believe that it can be instrumental to support accurate recognition in a number of fields while also enhancing humans’ ability to interpret deep learning models for debugging purposes or for personal use. In this paper we describe the potential advantages of this approach and explore various visualization mappings and deep learning architectures. We evaluate several VLAIR alternatives for a specific problem (human activity recognition in an apartment) and show that VLAIR attains classification accuracy above classical machine learning algorithms and several other non-image-based deep learning algorithms with several data representations.

The ability to understand, process and evaluate arguments made by others and ourselves is important in many personal and professional spheres, such as political debates. Analysis typically appears in written form, but a growing number of tools support analysis through diagram-based graphical representations. These UIs might support better argument analysis because arguments have non-linear structures that are difficult to convey through linear text. However, there is little empirical evidence on the advantages or mechanisms that might make graph UIs superior to traditional textual documents. We ran and analyzed a study with twenty participants who used text and graph editors to analyze political debates. Our findings demonstrate the tradeoffs between the two approaches and explain key mechanisms that support the analysis in both media.

Dynamic networks can be challenging to analyze visually, especially if they span a large time range during which new nodes and edges can appear and disappear. Although it is straightforward to provide interfaces for visualization that represent multiple states of the network (i.e., multiple timeslices) either simultaneously (e.g., through small multiples) or interactively (e.g., through interactive animation), these interfaces might not support tasks in which disjoint timeslices need to be compared. Since these tasks are key for understanding the dynamic aspects of the network, understanding which interactive visualizations best support these tasks is important. We present the results of a series of laboratory experiments comparing two traditional approaches (small multiples and interactive animation), with a more recent approach based on interactive timeslicing. The tasks were performed on a large display through a touch interface. Participants completed 24 trials of three tasks with all techniques. The results show that interactive timeslicing brings benefit when comparing distant points in time, but less benefits when analyzing contiguous intervals of time.

2D scalar data fields are often represented as heatmaps because color can help viewers perceive structure without having to interpret individual digits. Although heatmaps and color mapping have received much research attention, there are alternative representations that have been generally overlooked and might overcome heatmap problems. For example, color perception is subject to context-based perceptual bias and high error, which can be addressed through representations that use digits to enable more accurate value reading. We designed a series of three experiments that compare five techniques: a regular table of digits (Digits), a state-of-the-art heatmap (Color), a heatmap with an interactive tooltip showing the value under the cursor (Tooltip), a heatmap with the digits overlapped over it (DigitsColor), and FatFonts. Data analysis from the three experiments, which test locating values, finding extrema, and clustering tasks, show that overlapping digits on color (DigitsColor) offers a substantial increase in accuracy (between 10 and 60 percent points of improvement over the plain heatmap (Color), depending on the task) at the cost of extra time when locating extrema or forming clusters, but none when locating values. The interactive tooltip offered a poor speed-accuracy tradeoff, but participants preferred it to the plain heatmap (color) or digits-only (Digits) representations. We conclude that hybrid color-digit representations of scalar data fields could be highly beneficial for uses where spatial resolution and speed are not the main concern.

Problems such as timetabling or personnel allocation can be modeled and solved using discrete constraint programming languages. However, while existing constraint solving software solves such problems quickly in many cases, these systems involve specialized languages that require significant time and effort to learn and apply. These languages are typically text-based and often difficult to interpret and understand quickly, especially for people without engineering or mathematics backgrounds. Visualization could provide an alternative way to model and understand such problems. Although many visual programming languages exist for procedural languages, visual encoding of problem specifications has not received much attention. Future problem visualization languages could represent problem elements and their constraints unambiguously, but without unnecessary cognitive burdens for those needing to translate their problem's mental representation into diagrams. As a first step towards such languages, we executed a study that catalogs how people represent constraint problems graphically. We studied three groups with different expertise: non-computer scientists, computer scientists and constraint programmers and analyzed their marks on paper (e.g., arrows), gestures (e.g., pointing) and the mappings to problem concepts (e.g., containers, sets). We provide foundations to guide future tool designs allowing people to effectively grasp, model and solve problems through visual representations.

Virtual reality (VR) strives to replicate the sensation of the physical environment by mimicking people's perceptions and experience of being elsewhere. These experiences are of-ten mediated by the objects and tools we interact with in the virtual world (e.g., a controller). Evidence from psychology posits that when using the tool proficiently, it becomes em-bodied (i.e., an extension of one's body). There is little work,however, on how to measure this phenomenon in VR, andon how different types of tools and controllers can affect the experience of interaction. In this work, we leverage cognitive psychology and philosophy literature to construct the Locus-of-Attention Index (LAI), a measure of tool embodiment. We designed and conducted a study that measures readiness-to-hand and unreadiness-to-hand for three VR interaction techniques: hands, a physical tool, and a VR controller. The study shows that LAI can measure differences in embodiment with working and broken tools and that using the hand directly results in more embodiment than using controllers.

Network data that changes over time can be very useful for studying a wide range of important phenomena, from how social network connections change to epidemiology. However, it is challenging to analyze, especially if it has many actors, connections or if the covered timespan is large with rapidly changing links (e.g., months of changes with changes at second resolution). In these analyses one would often like to compare many periods of time to others, without having to look at the full timeline. To support this kind of analysis we designed and implemented a technique and system to visualize this dynamic data. The Dynamic Network Plaid (DNP) is designed for large displays and based on user-generated interactive timeslicing on the dynamic graph attributes and on linked provenance-preserving representations. We present the technique, interface and the design/evaluation with a group of public health researchers investigating non-suicidal self-harm picture sharing in Instagram.

Full-coverage displays can place visual content anywhere on the interior surfaces of a room (e.g., a weather display near the coat stand). In these settings, digital artefacts can be located behind the user and out of their field of view - meaning that it can be difficult to notify the user when these artefacts need attention. Although much research has been carried out on notification, little is known about how best to direct people to the necessary location in room environments. We designed five diverse attention-guiding techniques for full-coverage display rooms, and evaluated them in a study where participants completed search tasks guided by the different techniques. Our study provides new results about notification in full-coverage displays: we showed benefits of persistent visualisations that could be followed all the way to the target and that indicate distance-to-target. Our findings provide useful information for improving the usability of interactive full-coverage environments.

Previous research has identified trade-offs when it comes to designing visualization tools. While constructive "bottom-up' tools promote a hands-on, user-driven design process that enables a deep understanding and control of the visual mapping, automated tools are more efficient and allow people to rapidly explore complex alternative designs, often at the cost of transparency. We investigate how to design visualization tools that support a user-driven, transparent design process while enabling efficiency and automation, through a series of design workshops that looked at how both visualization experts and novices approach this problem. Participants produced a variety of solutions that range from example-based approaches expanding constructive visualization to solutions in which the visualization tool infers solutions on behalf of the designer, e.g., based on data attributes. On a higher level, these findings highlight agency and granularity as dimensions that can guide the design of visualization tools in this space.

As display environments become larger and more diverse - now often encompassing multiple walls and room surfaces - it is becoming more common that users must find and manipulate digital artifacts not directly in front of them. There is little understanding, however, about what techniques and devices are best for carrying out basic operations above, behind, or to the side of the user. We conducted an empirical study comparing two main techniques that are suitable for full-coverage display environments: mouse-based pointing, and ray-cast 'laser' pointing. Participants completed search and pointing tasks on the walls and ceiling, and we measured completion time, path lengths and perceived effort. Our study showed a strong interaction between performance and target location: when the target position was not known a priori the mouse was fastest for targets on the front wall, but ray-casting was faster for targets behind the user. Our findings provide new empirical evidence that can help designers choose pointing techniques for full-coverage spaces.

The emergence of tools that support fast-and-easy visualization creation by non-experts has made the benefits of InfoVis widely accessible. Key features of these tools include attribute-level operations, automated mappings, and visualization templates. However, these features shield people from lower-level visualization design steps, such as the specific mapping of data points to visuals. In contrast, recent research promotes constructive visualization where individual data units and visuals are directly manipulated. We present a qualitative study comparing people's visualization processes using two visualization tools: one promoting a top-down approach to visualization construction (Tableau Desktop) and one implementing a bottom-up constructive visualization approach (iVoLVER). Our results show how the two approaches influence: 1) the visualization process, 2) decisions on the visualization design, 3) the feeling of control and authorship, and 4) the willingness to explore alternative designs. We discuss the complex trade-offs between the two approaches and outline considerations for designing better visualization tools.

New interaction techniques, like multi-touch, tangible interaction, and mid-air gestures often promise to be more intuitive and natural; however, there is little work on how to measure these constructs. One way is to leverage the phenomenon of tool embodiment---when a tool becomes an extension of one's body, attention shifts to the task at hand, rather than the tool itself. In this work, we constructed a framework to measure tool embodiment by incorporating philosophical and psychological concepts. We applied this framework to design and conduct a study that uses attention to measure readiness-to-hand with both a physical tool and a virtual tool. We introduce a novel task where participants use a tool to rotate an object, while simultaneously responding to visual stimuli both near their hand and near the task. Our results showed that participants paid more attention to the task than to both kinds of tool. We also discuss how this evaluation framework can be used to investigate whether novel interaction techniques allow for this kind of tool embodiment.

iVoLVER, the Interactive Visual Language for Visualization Extraction and Reconstruction, is a web-based pen-and-touch interface that graphically supports construction of interactive visualizations. iVoLVER is designed to enable data extraction from different types of artifacts (e.g., photos) and to use that data to generate original representations of that data. People can create visualizations from data that is not structured in traditional formats without the need of textual programming or sitting at their desk. This demonstration shows how iVoLVER visualizations are constructed and also demonstrates the possible uses of iVoLVER in several contexts.

Using real time eye tracking, gaze-contingent displays can modify their content to represent depth (e.g., through additional depth cues) or to increase rendering performance (e.g., by omitting peripheral detail). However, there has been no research to date exploring how gaze-contingent displays can be leveraged for manipulating perceived color. To address this, we conducted two experiments (color matching and sorting) that manipulated peripheral background and object colors to influence the user's color perception. Findings from our color matching experiment suggest that we can use gaze-contingent simultaneous contrast to affect color appearance and that existing color appearance models might not fully predict perceived colors with gaze-contingent presentation. Through our color sorting experiment we demonstrate how gaze-contingent adjustments can be used to enhance color discrimination. Gaze-contingent color holds the promise of expanding the perceived color gamut of existing display technology and enabling people to discriminate color with greater precision.

We present the design and implementation of iVoLVER, a tool that allows users to create visualizations without textual programming. iVoLVER is designed to enable flexible acquisition of many types of data (text, colors, shapes, quantities, dates) from multiple source types (bitmap charts, webpages, photographs, SVGs, CSV files) and, within the same canvas, supports transformation of that data through simple widgets to construct interactive animated visuals. Aside from the tool, which is web-based and designed for pen and touch, we contribute the design of the interactive visual language and widgets for extraction, transformation, and representation of data. We demonstrate the flexibility and expressive power of the tool through a set of scenarios, and discuss some of the challenges encountered and how the tool fits within the current infovis tool landscape.

Proposed as “data analysis anywhere, anytime, from anything”, Opportunistic Information Visualization (Opportu-Vis) [1] seeks to provide analytical support in scenarios where the data of interest is not explicitly available and has to be retrieved from digital artifacts that are not traditionally used as data sources. Examples include raster images, web pages, vector files, and photographs. This showpiece presents how iVoLVER, the Interactive Visual Language for Visualization Extraction and Reconstruction, provides support in such settings. We briefly describe the overall construction approach of the tool in scenarios where different digital artifacts are used to compose interactive visuals. All of this becomes possible by using the data extraction capabilities of iVoLVER together with the elements of its visual language.

Displaying 3D objects on horizontal displays can cause problems in the way that the virtual scene is presented on the 2D surface; inappropriate choices in how 3D is represented can lead to distorted images and incorrect object interpretations. We present four experiments that test 3D perception. We varied projection geometry in three ways: type of projection (perspective/parallel), separation between the observer's point of view and the projection's center (discrepancy), and the presence of motion parallax (with/without parallax). Projection geometry had strong effects different for each task. Reducing discrepancy is desirable for orientation judgments, but not for object recognition or internal angle judgments. Using a fixed center of projection above the table reduces error and improves accuracy in most tasks. The results have far-reaching implications for the design of 3D views on tables, in particular, for multi-user applications where projections that appear correct for one person will not be perceived correctly by another.

We present A Delicate Agreement, an interactive art installation designed to intrigue viewers by offering them an unfolding story that is endlessly fascinating. To achieve this, we set our story in the liminal space of an elevator, and populated this elevator with a set of unique characters. Viewers watch the story unfold through peepholes in the elevator's doors, where in turn their gaze can trigger changes in the storyline. This storyline's interactive response was created via a complex adaptive system using simple rules based on Goffman's performance theory.

With new computer technologies portable devices are rapidly approaching the dimensions and characteristics of traditional pen and paper-based tools. Text and graphic documents are now commonly viewed using small tablet computers. We conducted a study with small groups of participants to better understand how paper-based text and graphics are used by small collaborative groups as compared to how these groups make use of documents presented on a digital tablet with digital styluses. Our results indicate that digital tools, as compared to paper tools, can affect the levels of verbal communication and participant gaze engagement with other group members. Additionally, we observed how participants spatially arranged paper-based and digital tools during collaborative group activities, how often they switched from digital to paper, and how they still prefer paper overall.

Over the past decade, multi-touch surfaces have become commonplace, with many researchers and practitioners describing the benefits of their natural, physical-like interactions. We present a pair of studies that empirically investigates the psychophysical effects of direct interaction with both physical and virtual artefacts. We use the phenomenon of Kinesthetic Figural After Effects-a change in understanding of the physical size of an object after a period of exposure to an object of different size. Our studies show that, while this effect is robustly reproducible when using physical artefacts, this same effect does not manifest when manipulating virtual artefacts on a direct, multi-touch tabletop display. We contribute quantitative evidence suggesting a psychophysical difference in our response to physical vs. virtual objects, and discuss future research directions to explore measurable phenomena to evaluate the presence of physical-like changes from virtual on-screen objects.

Blur in images can create the sensation of depth because it emulates an optical property of the eye; namely, the limited depth of field created by the eye's lens. When the human eye looks at an object, this object appears sharp on the retina, but objects at different distances appear blurred. Advances in gaze-tracking technologies enable us to reproduce dynamic depth of field in regular displays, providing an alternative way of conveying depth. In this paper we investigate gaze-contingent depth of field as a method to produce realistic 3D images, and analyze how effectively people can use it to perceive depth. We found that GC DOF increases subjective perceived realism and depth and can contribute to the perception of ordinal depth and distance between objects, but it is limited in its accuracy.

Collaborative tabletop systems can employ direct touch, where people's real arms and hands manipulate objects, or indirect input, where people are represented on the table with digital embodiments. The input type and the resulting embodiment dramatically influence tabletop interaction: in particular, the touch avoidance that naturally governs people's touching and crossing behavior with physical arms is lost with digital embodiments. One result of this loss is that people are less aware of each others' arms, and less able to coordinate actions and protect personal territories. To determine whether there are strategies that can influence group interaction on shared digital tabletops, we studied augmented digital arm embodiments that provide tactile feedback or movement alterations when people touched or crossed arms. The study showed that both augmentation types changed people's behavior (people crossed less than half as often) and also changed their perception (people felt more aware of the other person's arm, and felt more awkward when touching). This work shows how groupware designers can influence people's interaction, awareness, and coordination abilities when physical constraints are absent.

Rotations performed with the index finger and thumb involve some of the most complex motor action among common multi-touch gestures, yet little is known about the factors affecting performance and ergonomics. This note presents results from a study where the angle, direction, diameter, and position of rotations were systematically manipulated. Subjects were asked to perform the rotations as quickly as possible without losing contact with the display, and were allowed to skip rotations that were too uncomfortable. The data show surprising interaction effects among the variables, and help us identify whole categories of rotations that are slow and cumbersome for users.

We studied the memorability of free-form gesture sets for invoking actions. We compared three types of gesture sets: user-defined gesture sets, gesture sets designed by the authors, and random gesture sets in three studies with 33 participants in total. We found that user-defined gestures are easier to remember, both immediately after creation and on the next day (up to a 24% difference in recall rate compared to pre-designed gestures). We also discovered that the differences between gesture sets are mostly due to association errors (rather than gesture form errors), that participants prefer user-defined sets, and that they think user-defined gestures take less time to learn. Finally, we contribute a qualitative analysis of the tradeoffs involved in gesture type selection and share our data and a video corpus of 66 gestures for replicability and further analysis.

In tabletop work with direct input, people avoid crossing each others' arms. This natural touch avoidance has important consequences for coordination: for example, people rarely grab the same item simultaneously, and negotiate access to the workspace via turn-taking. At digital tables, however, some situations require the use of indirect input (e.g., large tables or remote participants), and in these cases, people are often represented with virtual arm embodiments. There is little information about what happens to coordination and reaching when we move from physical to digital arm embodiments. To gather this information, we carried out a controlled study of tabletop behaviour with different embodiments. We found dramatic differences in moving to a digital embodiment: people touch and cross with virtual arms far more than they do with real arms, which removes a natural coordination mechanism in tabletop work. We also show that increasing the visual realism of the embodiment does not change behaviour, but that changing the thickness has a minor effect. Our study identifies important design principles for virtual embodiments in tabletop groupware, and adds to our understanding of embodied interaction in small groups.

A transmogrifier is a novel interface that enables quick, on-the-fly graphic transformations. A region of a graphic can be specified by a shape and transformed into a destination shape with real-time, visual feedback. Both origin and destination shapes can be circles, quadrilaterals or arbitrary shapes defined through touch. Transmogrifiers are flexible, fast and simple to create and invite use in casual InfoVis scenarios, opening the door to alternative ways of exploring and displaying existing visualizations (e.g., rectifying routes or rivers in maps), and enabling free-form prototyping of new visualizations (e.g., lenses).

In this paper we explore numeric typeface design for visualization purposes. We introduce FatFonts, a technique for visualizing quantitative data that bridges the gap between numeric and visual representations. FatFonts are based on Arabic numerals but, unlike regular numeric typefaces, the amount of ink (dark pixels) used for each digit is proportional to its quantitative value. This enables accurate reading of the numerical data while preserving an overall visual context. We discuss the challenges of this approach that we identified through our design process and propose a set of design goals that include legibility, familiarity, readability, spatial precision, dynamic range, and resolution. We contribute four FatFont typefaces that are derived from our exploration of the design space that these goals introduce. Finally, we discuss three example scenarios that show how FatFonts can be used for visualization purposes as valuable representation alternatives.

An increasing number of interactive displays of very different sizes, portability, projectability and form factors are starting to become part of the display ecosystems that we make use of in our daily lives. Displays are shaped by human activity into an ecological arrangement and thus an ecology. Each combination or ecology of displays offer substantial promise for the creation of applications that effectively take advantage of the wide range of input, affordances, and output capability of these multi-display, multi-device and multi-user environments. Although the last few years have seen an increasing amount of research in this area, knowledge about this subject remains under explored, fragmented, and cuts across a set of related but heterogeneous issues. This workshop brings together researchers and practitioners interested in the challenges posed by infrastructure and design.

Attaching a large external display can help a mobile device user view more content at once. This paper reports on a study investigating how different configurations of input and output across displays affect performance, subjective workload and preferences in map, text and photo search tasks. Experimental results show that a hybrid configuration where visual output is distributed across displays is worst or equivalent to worst in all tasks. A mobile device-controlled large display configuration performs best in the map search task and equal to best in text and photo search tasks (tied with a mobile-only configuration). After conducting a detailed analysis of the performance differences across different UI configurations, we give recommendations for the design of distributed user interfaces.

In the real world, touch based interaction relies on haptic feedback (e.g., grasping objects, feeling textures). Unfortunately, such feedback is absent in current tabletop systems. The previously developed Haptic Tabletop Puck (HTP) aims at supporting experimentation with and development of inexpensive tabletop haptic interfaces in a do-it-yourself fashion. The problem is that programming the HTP (and haptics in general) is difficult. To address this problem, we contribute the Haptictouch toolkit, which enables developers to rapidly prototype haptic tabletop applications. Our toolkit is structured in three layers that enable programmers to: (1) directly control the device, (2) create customized combinable haptic behaviors (e.g., softness, oscillation), and (3) use visuals (e.g., shapes, images, buttons) to quickly make use of these behaviors. In our preliminary exploration we found that programmers could use our toolkit to create haptic tabletop applications in a short amount of time.

Abstract Detail‐in‐context lens techniques can be useful for exploring visualizations of data spaces that are too large or have too much detail to fit in regular displays. For example, by bending the space in the right way we can bring together details from two separate areas for easy comparison while roughly keeping the context that situates each area within the global space. While these techniques can be powerful tools, they also introduce distortions that need to be understood, and often the tools have to be disabled in order to have access to the undistorted data. We introduce the undistort lens, a complement to existing distortion‐based techniques that provides a local and separate presentation of the original geometry without affecting any distortion‐based lenses currently used in the presentation. The undistort lens is designed to allow interactive access to the underlying undistorted data within the context of the distorted space, and to enable a better understanding of the distortions. The paper describes the implementation of a generic back‐mapping mechanism that enables the implementation of undistort lenses for arbitrary distortion based techniques, including those presented in the lens literature. We also provide a series of use‐case scenarios that demonstrate the situations in which the technique can complement existing lenses.

Indirect input techniques allow users to quickly access all parts of tabletop workspaces without the need for physical access; however, indirect techniques restrict the available social cues that are seen on direct touch tables. This reduced awareness results in impoverished coordination; for example, the number of conflicts might increase since users are more likely to interact with objects that another person is planning to use. Conflicts may also arise because indirect techniques reduce territorial behavior, expanding the interaction space of each collaborator. In this paper, we introduce three new tabletop coordination techniques designed to reduce conflicts arising from indirect input, while still allowing users the flexibility of distant object control. Two techniques were designed to promote territoriality and to allow users to protect objects when they work near their personal areas, and the third technique lets users set their protection levels dynamically. We present the results of an evaluation, which shows that people prefer techniques that automatically provide protection for personal territories, and that these techniques also increase territorial behavior.

Multi-display environments (MDEs) are now becoming common, and are becoming more complex, with more displays and more types of display in the environment. One crucial requirement specific to MDEs is that users must be able to move objects from one display to another; this cross-display movement is a frequent and fundamental part of interaction in any application that spans two or more display surfaces. Although many cross-display movement techniques exist, the differences between MDEs—the number, location, and mixed orientation of displays, and the characteristics of the task they are being designed for—require that interaction techniques be chosen carefully to match the constraints of the particular environment. As a way to facilitate interaction design in MDEs, we present a taxonomy that classifies cross-display object movement techniques according to three dimensions: the referential domain that determines how displays are selected, the relationship of the input space to the display configuration, and the control paradigm for executing the movement. These dimensions are based on a descriptive model of the task of cross-display object movement. The taxonomy also provides an analysis of current research that designers and researchers can use to understand the differences between categories of interaction techniques.

Multi-monitor displays and multi-display environments are now common. Cross-display cursor movement, in which a user moves the pointer from one display to another, occurs frequently in these settings. There are several techniques for supporting this kind of movement, and these differ in the way that they deal with displayless space (the physical space between displays). Stitching is the method used by most operating systems; in this technique, the cursor jumps from the edge of one display directly into the next display. In contrast, Mouse Ether maps the motor space of the mouse exactly to the physical space of the displays, meaning that the cursor has to travel across displayless space until it reaches the next display. To determine which of these approaches is best for cross-display movement, we carried out a study comparing Stitching, Mouse Ether, and a variant of Mouse Ether with Halo for off-screen feedback. We found that Stitching is equivalent to or faster than any variant of Mouse Ether, and that Halo improves Ether’s performance (but not enough to outperform Stitching). Results also indicate that the larger the gap between displays, the longer the targeting takes – even for Stitching. These findings provide valuable guidance for practitioners and raise new interesting questions for research.

The interaction techniques that are used in tabletop groupware systems (such as pick-and-drop or pantograph) can affect the way that people collaborate. However, little is known about these effects, making it difficult for designers to choose appropriate techniques when building tabletop groupware. We carried out an exploratory study to determine how several different types of interaction techniques (pantograph, telepointers, radar views, drag-and-drop, and laser beam) affected coordination and awareness in two tabletop tasks (a game and a storyboarding activity). We found that the choice of interaction technique significantly affected coordination measures, performance measures, and preference - but that the effects were different for the two different tasks. Our study shows that the choice of tabletop interaction technique does indeed matter, and provides insight into how tabletop systems can better support group work.

Multi-display environments compose displays that can be at different locations from and different angles to the user; as a result, it can become very difficult to manage windows, read text, and manipulate objects. We investigate the idea of perspective as a way to solve these problems in multi-display environments. We first identify basic display and control factors that are affected by perspective, such as visibility, fracture, and sharing. We then present the design and implementation of E-conic, a multi-display multi-user environment that uses location data about displays and users to dynamically correct perspective. We carried out a controlled experiment to test the benefits of perspective correction in basic interaction tasks like targeting, steering, aligning, pattern-matching and reading. Our results show that perspective correction significantly and substantially improves user performance in all these tasks.

The rapid increase in display sizes and resolutions has led to the re-emergence of many pen-based interaction systems like tabletop and wall display environments. Pointing in these environments is an important task, but techniques have not exploited the manipulation of control and display parameters to the extent seen in desktop environments. We have overcome these in the design of a new pen-based interaction technique -- Bubble Radar. Bubble Radar allows users to reach both specific targets and empty space, and supports dynamic switching between selecting and placing. The technique is based on combining the benefits of a successful pen-based pointing technique, the Radar View, with a successful desktop object pointing technique -- the Bubble Cursor. We tested the new technique in a user study and found that it was significantly faster than existing techniques, both for overall pointing and for targeting specific objects.

Multi-display environments and smart meeting rooms are now becoming more common. These environments build a shared display space from variety of devices: tablets, projected surfaces, tabletops, and traditional monitors. Since the different display surfaces are usually not organized in a single plane, traditional schemes for stitching the displays together can cause problems for interaction. However, there is a more natural way to compose display space -- using perspective. In this paper, we develop interaction techniques for multi-display environments that are based on the user's perspective on the room. We designed the Perspective Cursor, a mapping of cursor to display space that appears natural and logical from wherever the user is located. We conducted an experiment to compare two perspective-based techniques, the Perspective Cursor and a beam-based technique, with traditional stitched displays. We found that both perspective techniques were significantly faster for targeting tasks than the traditional technique, and that Perspective Cursor was the most preferred method. Our results show that integrating perspective into the design of multi-display environments can substantially improve performance.

Recent advances in multi-user collaboration have seen a proliferation of interaction techniques for moving digital objects from one device to another. However, little is known about how these techniques work in realistic situations, or how they compare to one another. We conducted a study to compare the efficiency of six techniques for moving objects from a tablet to a tabletop display. We compared the techniques in four different distance ranges and with three movement directions. We found that techniques like the Radar View and Pick-and-Drop, that have a control-to-display ratio of 1, are significantly faster for object movement than techniques that have smaller control-to-display ratios. We also found that using spatial manipulation of objects was faster than pressure-based manipulation.