Visions for the future of computing, such as those on Ubiquitous Computing or Tangible Interfaces, are highly cited and frequently used in teaching. Yet, we know little about the practical value of these visions for research on Human-Computer Interaction (HCI) or how HCI researchers engage with them individually and collectively. To address this gap, we conducted a survey with 172 HCI researchers. We identified key benefits and pitfalls as well as specific uses of visions by researchers. Researchers appreciate how visions guide us, drive us, and initiate new fields. Simultaneously, researchers acknowledge how visions create hype, restrict our creativity, and make us disregard real-world problems. Based on these insights, we derive tensions related to the pursuit of visions and discuss critical reading practices. Our paper offers a metascientific account of visions in the HCI field along with tools for critical reflection when engaging with them.

Data visualization literacy is vital for preparing students for higher education in data-driven fields, yet instructions and teaching materials for high school remain limited. To address this gap, we applied participatory design (PD) and cooperative prototyping (CP) with high school teachers and data visualization experts as equal partners. Over a period of 16 months, we conducted an interview and four iterative workshops to identify teaching challenges and co-create solutions. This process produced three prototype sets, culminating in a 17-step teaching unit on the CoTinker platform for computational thinking. The unit uses a music dataset and a developed visualization tool to engage students in collaboratively creating data visualizations through hands-on learning while introducing visualization concepts, visual storytelling, and data ethics. Our work demonstrates how PD and CP can bridge research and practice, and contributes a replicable framework for integrating data visualization into high school by use of co-designed tools grounded in real-world classrooms.

A truly anytime, anywhere approach to immersive analytics is predicated on the ability for users to author and explore visualizations in an immersive fashion and without a desktop computer. In this paper, we present a generalizable visual programming technique for immersive authoring, exploration, and presentation of data visualizations through direct manipulation, and implement it in a web-based augmented reality (AR) system. In a preregistered user study, we use this implementation to explore how analysts author and use data visualizations in immersive AR accessed using a head-mounted display (HMD). The study focuses on the authoring technique, spatial organization strategies, and sensemaking and presentation practices in such immersive analytics environments for HMD-based AR. Our findings suggest that participants particularly liked the direct manipulation of our approach, and that participants' spatial organization is not based on physical landmarks in the room.

We introduce Datamancer, a wearable device enabling bimanual gesture interaction across multi-display ubiquitous analytics environments. Datamancer addresses the gap in gesture-based interaction within data visualization settings, where current methods are often constrained by limited interaction spaces or the need for installing bulky tracking setups. Datamancer integrates a finger-mounted pinhole camera and a chest-mounted gesture sensor, allowing seamless selection and manipulation of visualizations on distributed displays. By pointing to a display, users can acquire the display and engage in various interactions, such as panning, zooming, and selection, using both hands. Our contributions include (1) an investigation of the design space of gestural interaction for physical ubiquitous analytics environments; (2) a prototype implementation of the Datamancer system that realizes this model; and (3) an evaluation of the prototype through demonstration of application scenarios, an expert review, and a user study.

Extended Reality (XR) head-mounted displays (HMDs) coupled with standard mouse and keyboard hardware present a unique opportunity to rethink traditional desktop productivity. In this paper, we propose WindowSpace, an XR-based window manager for placing 2D windows in immersive 3D space while using a traditional mouse and keyboard. WindowSpace is a web-based system supporting consumer-level HMD-AR that can display native 2D windows from standard operating systems using a native desktop streaming server based on a standards-compliant optimized WebRTC protocol. To facilitate interacting with these native applications, we investigate the design space for effectively controlling the cursor between windows arranged in immersive 3D space. We propose three pointing techniques in our design space: an off-screen cursor (EtherXR), a head-tracking based pointer (Face&Jump), and a novel technique based on switching screens by extending the mouse trajectory (BallisticXR). Our empirical study revealed that EtherXR performs slightly better overall; however, BallisticXR showed better accuracy. Notably, Face&Jump yielded slower and less accurate performance, suggesting that full gaze assistance may be necessary to support this approach effectively.

We introduce DashSpace, a live collaborative immersive and ubiquitous analytics (IA/UA) platform designed for handheld and head-mounted Augmented/Extended Reality (AR/XR) implemented using WebXR and open standards. To bridge the gap between existing web-based visualizations and the immersive analytics setting, DashSpace supports visualizing both legacy D3 and Vega-Lite visualizations on 2D planes, and extruding Vega-Lite specifications into 2.5D. It also supports fully 3D visual representations using the Optomancy grammar. To facilitate authoring new visualizations in immersive XR, the platform provides a visual authoring mechanism where the user groups specification snippets to construct visualizations dynamically. The approach is fully persistent and collaborative, allowing multiple participants-whose presence is shown using 3D avatars and webcam feeds-to interact with the shared space synchronously, both co-located and remotely. We present three examples of DashSpace in action: immersive data analysis in 3D space, synchronous collaboration, and immersive data presentations.

Computing education is becoming increasingly important in high school curricula, but there are challenges in teaching computing, including teachers' professional development and the development of appropriate learning tools. Participatory design and cooperative prototyping offer opportunities for identifying challenges in computing education and ways for teachers to be involved in the design of new learning tools. This study describes the process of codesigning a learning tool to overcome some of these challenges for computing education in high school, involving teachers, developers, and researchers as collaborators. The study investigating how cooperative prototyping and participatory design can lead to mutual learning about challenges in teaching computing. We prototyped learning activities and the learning tool, CoTinker, in collaboration between six teachers, two developers, and four researchers in four subcases and implemented them in classrooms with a total of 83 students. We show how challenges such as students' collaboration and transfer of knowledge in computing education were overcome by applying the prototypes in computing education classrooms in high school.

Webstrates are web substrates, a practical realization of shareable dynamic media under which distributability, shareability, and malleability are fundamental software principles. Webstrates blur the distinction between application and document in a way that enables users to share, repurpose, and refit software across a variety of domains, but its reliance on a central server constrains its use; it is at odds with personal and collective control of data; and limits applications to the web. We extend the fundamental principles to include interoperability and sovereignty over data and propose MyWebstrates, an implementation of Webstrates on top of a new, lower-level substrate for synchronization built around local-first software principles. MyWebstrates registers itself in the user’s browser and function as a piece of local software that can selectively synchronise data over sync servers or peer-to-peer connections. We show how MyWebstrates extends Webstrates to enable offline collaborative use, interoperate between Webstrates on non-web technologies such as Unity, and maintain personal and collective sovereignty over data. We demonstrate how this enables new types of applications of Webstrates and discuss limitations of this approach and new challenges that it reveals.

How can we let users adapt video-based meetings as easily as they rearrange furniture in a physical meeting room? We describe a design space for video conferencing systems that includes a five-step “ladder of tailorability,” from minor adjustments to live reprogramming of the interface. We then present Mirrorverse and show how it applies the principles of computational media to support live tailoring of video conferencing interfaces to accommodate highly diverse meeting situations. We present multiple use scenarios, including a virtual workshop, an online yoga class, and a stand-up team meeting to evaluate the approach and demonstrate its potential for new, remote meetings with fluid transitions across activities.

This paper proposes a new approach for online UI adaptation that aims to overcome the limitations of the most commonly used UI optimization method involving multiple objectives: weighted sum optimization. Weighted sums are highly sensitive to objective formulation, limiting the effectiveness of UI adaptations. We propose ParetoAdapt, an adaptation approach that uses online multi-objective optimization with a posteriori articulated preferences—that is, articulation of preferences after the optimization has concluded—to make UI adaptation robust to incomplete and inaccurate objective formulations. It offers users a flexible way to control adaptations by selecting from a set of Pareto optimal adaptation proposals and adjusting them to fit their needs. We showcase the feasibility and flexibility of ParetoAdapt by implementing an online layout adaptation system in a state-of-the-art 3D UI adaptation framework. We further evaluate its robustness and run-time in simulation-based experiments that allow us to systematically change the accuracy of the estimated user preferences. We conclude by discussing how our approach may impact the usability and practicality of online UI adaptations.

Computational media describes a vision of software, which, in contrast to application-centric software, is (1) malleable, so users can modify existing functionality, (2) computable, so users can run custom code, (3) distributable, so users can open documents across different devices, and (4) shareable, so users can easily share and collaborate on documents. Over the last ten years, the Webstrates and Codestrates projects aimed at realizing this vision of computational media. Webstrates is a server application that synchronizes the DOM of websites. Codestrates builds on top of Webstrates and adds an authoring environment, which blurs the use and development of applications. Grounded in a chronology of the development of Webstrates and Codestrates, we present eight tensions that we needed to balance during their development. We use these tensions as an analytical lens in three case studies and a game challenge in which participants created games using Codestrates. We discuss the results of the game challenge based on these tensions and present key takeaways for six of them. Finally, we present six lessons learned from our endeavor to realize the vision of computational media, demonstrating the balancing act of weighing the vision against the pragmatics of implementing a working system.

Most modern applications are immutable and turn-key despite the acknowledged benefits of empowering users to modify their software. Writing extensible software remains challenging, even for expert programmers. Reprogramming or extending existing software is often laborious or wholly blocked, requiring sophisticated knowledge of application architecture or setting up a development environment. We present Varv, a programming model representing reprogrammable interactive software as a declarative data structure. Varv defines interactive applications as a set of concepts that consist of a schema and actions. Applications in Varv support incremental modification, allowing users to reprogram through addition and selectively suppress, modify, or add behavior. Users can define high-level concepts, creating an abstraction layer and effectively a domain-specific language for their application domain, emphasizing reuse and modification. We demonstrate the reprogramming and collaboration capabilities of Varv in two case studies and illustrate how the event engine allows for extensive tooling support.

In January 2021, WhatsApp announced an update to their privacy policy, sparking an outcry that saw millions of users install other messaging apps such as Telegram and Signal. This presented a rare opportunity to study users’ experiences when trying to leave the world’s most popular communication app. We conducted surveys in February and May with 1525 WhatsApp users from Mexico, Spain, South Africa, and the United Kingdom. Over a quarter wanted to switch at least part of their communication to other apps, but 74% of them failed to do so. By May, 27% had increased their use of other apps, and only 16% used WhatsApp less. Beyond network effects, users struggled with making informed choices of alternative apps and with differences in their design and functionality. We suggest messaging interoperability as an approach to alleviate switching costs and discuss implications for HCI research and competition regulation of digital services.

Computational thinking (CT) has been recognized as an important part of K-12 education in many countries and is often described in term relating to computer science both in concepts and context. However, the unique practice of computer scientists collaborating on problem solving is often overlooked in CT education in high school. We believe that collaboration is a key aspect of modern computing, and propose to integrate collaborative CT activities into existing high school education. The aim is to prepare students better for exploring computational approaches to high school disciplines as they are practiced in the professional world. Our research question is: how can we design tools for supporting students' collaboration while teaching CT in high school? A new computational environment, CoTinker, has been developed. The tool uses a cross-device approach and foster students' verbal communication and collaboration in groups. The tool was tested in a Biology class. Students' computational thinking (CT) was assessed through learning activities and students' communication was analyzed by audio recordings, of students, imbedded in CoTinker and by focus group interviews with students. Preliminary findings showed students being able to collaborate and solve CT tasks in groups independent of teacher or researchers. Results also indicated that students were able to use, navigate, and critically reflect upon the CoTinker environment.

Computing today happens across multiple devices, applications, users, organizational units, and in the rest of the world outside. Groups and communities come together for different reasons and operate within contexts that may differ from dominant modes of production and consumption. With a foundation in activity theoretical HCI, we develop the concept of collective artifact ecologies. This concept enables us to identify struggles of collective use of computational devices today, delimiting collective artifact ecologies in order to study and explain how they develop and overlap. Through an analysis of three empirical cases, we illustrate the notion of collectives and how they face challenges in establishing, maintaining and negotiating their artifact ecologies. This paper, therefore, contributes a theoretical foundation for analyzing groups and communities as collectives, with a particular emphasis on the multiple tools and artifacts they use. To serve as a starting point for further engagement with these concepts, we provide guiding questions to support the understanding of collective artifact ecologies.

In hybrid meetings, multiple co-located participants communicate with remote participants through video. But video communication inhibits non-verbal cues, and this often causes remote participants to feel excluded. To address this issue, we built MirrorBlender: a What-You-See-Is-What-I-See video-conferencing system for blending, repositioning, and resizing mirrors. Mirrors here denote shared video feeds of people and screens. In a qualitative study of MirrorBlender with three hybrid meeting sessions, we found that the shared control of mirrors supported users in negotiating a blended interpersonal space. Moreover, it enabled diverse acts of inclusion of remote participants. In particular, remote participants brought attention to themselves by manipulating the position, scale, and translucency of their camera and screen feeds. Participants also embodied and leveraged their mirror images for deictic gestures and playful interactions. Based on these findings, we discuss new opportunities for supporting video-mediated collaboration.

Programming by Demonstration (PbD) is a well-known technique that allows non-programmers to describe interactivity by performing examples of the expected behavior, but it has not been extensively explored for AR. We present Rapido, a novel early-stage prototyping tool to create fully interactive mobile AR prototypes from non-interactive video prototypes using PbD. In Rapido, designers use a mobile AR device to record a video prototype to capture context, sketch assets, and demonstrate interactions. They can demonstrate touch inputs, animation paths, and rules to, e.g., have a sketch follow the focus area of the device or the user’s world-space touches. Simultaneously, a live website visualizes an editable overview of all the demonstrated examples and infers a state machine of the user flow. Our key contribution is a method that enables designers to turn a video prototype into an executable state machine through PbD. The designer switches between these representations to interactively refine the final interactive prototype. We illustrate the power of Rapido’s approach by prototyping the main interactions of three popular AR mobile applications.

In this pictorial, we investigate how a virtual collection can put down its roots in a physical space and integrate into the local community. We present PLACED, a place-centric digital service that supports participation and community-production of knowledge in library events. We illustrate how PLACED has been deployed and used at a local public library over a six-month period. We examine the community-produced virtual collection that grew out of this library event with a focus on its placeness.

We propose using virtual reality (VR) as a design tool for sketching and simulating spatially-aware interactive spaces. Using VR, designers can quickly experience their envisioned spaces and interactions by simulating technologies such as motion tracking, multiple networked devices, or unusual form factors such as spherical touchscreens or bezel-less display tiles. Design ideas can be rapidly iterated without restrictions by the number, size, or shape and availability of devices or sensors in the lab. To understand the potentials and challenges of designing in VR, we conducted a user study with 12 interaction designers. As their tool, they used a custom-built virtual design environment with finger tracking and physics simulations for natural interactions with virtual devices and objects. Our study identified the designers' experience of space in relation to their own bodies and playful design explorations as key opportunities. Key challenges were the complexities of building a usable yet versatile VR-based "World Editor".

The popularity of computational notebooks heralds a return of software as computational media rather than turn-key applications. We believe this software model has potential beyond supporting just the computationally literate. We studied a biomolecular nano-design lab that works on a current frontier of science - RNA origami - whose researchers depend on computational tools to do their work, yet are not trained as programmers. Using a participatory design process, we developed a computational labbook to concretise what computational media could look like, using the principles of computability, malleability, shareability, and distributability suggested by previous work. We used this prototype to co-reflect with the nanoscientists about how it could transform their practice. We report on the computational culture specific to this research area; the scientists' struggles managing their computational environments; and their subsequent disempowerment yet dependence. Lastly, we discuss the generative potential and limitations of the four design principles for the future of computational media.

We investigate how to design community technologies for public events. We do so with a focus on technologies that give rise to new forms of participation and knowledge co-production in public libraries. Specifically, we deployed a digital service at a major public library during its four-week creative workshop series. The system offered an alternative way for people to work together as a community, to go beyond achieving individual goals, and to contribute to the achievement of public goals (e.g., building community bookshelves). We report on how the system has reconfigured physical spaces and afforded new social practices in the library. We propose Computational Alternatives as a fruitful approach for gaining situated, nuanced insights into a technology's possible adoption. We offer key insights in the form of computational alternatives vignettes -- grounded stories that encapsulate sociotechnical implications of technology, pointing to plausible alternative futures.

Programming assignments in computer science courses are often processed in pairs or groups of students. While working together, students face several shortcomings in today's software: The lack of real-time collaboration capabilities, the setup time of the development environment, and the use of different devices or operating systems can hamper students when working together on assignments. Text processing platforms like Google Docs solve these problems for the writing process of prose text, and computational notebooks like Google Colaboratory for data analysis tasks. However, none of these platforms allows users to implement interactive applications. We deployed a web-based literate programming system for three months during an introductory course on application development to explore how collaborative programming practices unfold and how the structure of computational notebooks affect the development. During the course, pairs of students solved weekly programming assignments. We analyzed data from weekly questionnaires, three focus groups with students and teaching assistants, and keystroke-level log data to facilitate the understanding of the subtleties of collaborative programming with computational notebooks. Findings reveal that there are distinct collaboration patterns; the preferred collaboration pattern varied between pairs and even varied within pairs over the course of three months. Recognizing these distinct collaboration patterns can help to design future computational notebooks for collaborative programming assignments.

Despite sophisticated technologies for representational fidelity in hybrid meetings, in which co-located and remote participants collaborate via video or audio, meetings are still often disrupted by practical problems with trying to include remote participants. In this paper, we use micro-analysis of three disruptive moments in a hybrid meeting from a global software company to unpack blended technological and conversational practices of inclusion and exclusion. We argue that designing truly valuable experiences for hybrid meetings requires moving from the traditional, essentialist, and perception-obsessed user-centered design approach to a phenomenological approach to the needs of meetings themselves. We employ the metaphor of 'configuring the meeting' to propose that complex ecologies of people, technology, spatial, and institutional organization must be made relevant in the process of design.

Designing interfaces or applications that move beyond the bounds of a single device screen enables new ways to engage with digital content. Research addressing the opportunities and challenges of interactions with multiple devices in concert is of continued focus in HCI research. To inform the future research agenda of this field, we contribute an analysis and taxonomy of a corpus of 510 papers in the cross-device computing domain. For both new and experienced researchers in the field we provide: an overview, historic trends and unified terminology of cross-device research; discussion of major and under-explored application areas; mapping of enabling technologies; synthesis of key interaction techniques spanning across multiple devices; and review of common evaluation strategies. We close with a discussion of open issues. Our taxonomy aims to create a unified terminology and common understanding for researchers in order to facilitate and stimulate future cross-device research.

We present Vistribute, a framework for the automatic distribution of visualizations and UI components across multiple heterogeneous devices. Our framework consists of three parts: (i) a design space considering properties and relationships of interactive visualizations, devices, and user preferences in multi-display environments; (ii) specific heuristics incorporating these dimensions for guiding the distribution for a given interface and device ensemble; and (iii) a web-based implementation instantiating these heuristics to automatically generate a distribution as well as providing interaction mechanisms for user-defined adaptations. In contrast to existing UI distribution systems, we are able to infer all required information by analyzing the visualizations and devices without relying on additional input provided by users or programmers. In a qualitative study, we let experts create their own distributions and rate both other manual distributions and our automatic ones. We found that all distributions provided comparable quality, hence validating our framework.

Situated sketching and enactment aim at grounding designs in the spatial, social and cultural practices of a particular place. This is particularly relevant when designing for public places in which human activities are open-ended, multi-faceted, and difficult to anticipate, such as libraries, train stations, or commercial areas. In order to investigate situated sketching and enactment, we developed Ébauche. It enables designers to collaboratively sketch interfaces, distribute them across multiple displays and enact use cases. We present the lessons learned from six situated sketching and enactment workshops on public displays with Ébauche. And we present the results of a controlled study with 8 pairs of designers who used paper and Ébauche. We present the various ways in which participants leveraged the place, and how paper or Ébauche influenced the integration of their designs in the place. Looking at the design outcomes, our results suggest that paper leads to broader exploration of ideas and deeper physical integration in the environment. Whereas Ébauche leads to more refined sketches and more animated enactments.

We present Videostrates, a concept and a toolkit for creating real-time collaborative video editing tools. Videostrates supports both live and recorded video composition with a declarative HTML-based notation, combining both simple and sophisticated editing tools that can be used collaboratively. Videostrates is programmable and unleashes the power of the modern web platform for video manipulation. We demonstrate its potential through three use scenarios: collaborative video editing with multiple tools and devices; orchestration of multiple live streams that are recorded and broadcast to a popular streaming platform; and programmatic creation of video using WebGL and shaders for blue screen effects. These scenarios only scratch the surface of Videostrates' potential, which opens up a design space for novel collaborative video editors with fully programmable interfaces.

Abstract Analyzing complex data is a non‐linear process that alternates between identifying discrete facts and developing overall assessments and conclusions. In addition, data analysis rarely occurs in solitude; multiple collaborators can be engaged in the same analysis, or intermediate results can be reported to stakeholders. However, current data‐driven communication tools are detached from the analysis process and promote linear stories that forego the hierarchical and branching nature of data analysis, which leads to either too much or too little detail in the final report. We propose a conceptual design for integrated data‐driven reporting that allows for iterative structuring of insights into hierarchies linked to analytic provenance and chosen analysis views. The hierarchies become dynamic and interactive reports where collaborators can review and modify the analysis at a desired level of detail. Our web‐based InsideInsights system provides interaction techniques to annotate states of analytic components, structure annotations, and link them to appropriate presentation views. We demonstrate the generality and usefulness of our system with two use cases and a qualitative expert review.

Visualization tools are often specialized for specific tasks, which turns the user's analytical workflow into a fragmented process performed across many tools. In this paper, we present a component model design for data visualization to promote modular designs of visualization tools that enhance their analytical scope. Rather than fragmenting tasks across tools, the component model supports unification, where components-the building blocks of this model-can be assembled to support a wide range of tasks. Furthermore, the model also provides additional key properties, such as support for collaboration, sharing across multiple devices, and adaptive usage depending on expertise, from creating visualizations using dropdown menus, through instantiating components, to actually modifying components or creating entirely new ones from scratch using JavaScript or Python source code. To realize our model, we introduce VISTRATES, a literate computing platform for developing, assembling, and sharing visualization components. From a visualization perspective, Vistrates features cross-cutting components for visual representations, interaction, collaboration, and device responsiveness maintained in a component repository. From a development perspective, Vistrates offers a collaborative programming environment where novices and experts alike can compose component pipelines for specific analytical activities. Finally, we present several Vistrates use cases that span the full range of the classic "anytime" and "anywhere" motto for ubiquitous analysis: from mobile and on-the-go usage, through office settings, to collaborative smart environments covering a variety of tasks and devices.

Sticky notes are ubiquitous in design processes because of their tangibility and ease of use. Yet, they have well-known limitations in professional design processes, as documentation and distribution are cumbersome at best. This paper compares the use of sticky notes in ideation with a remediated digital sticky notes setup. The paper contributes with a nuanced understanding of what happens when remediating a physical design tool into digital space, by emphasizing focus shifts and breakdowns caused by the technology, but also benefits and promises inherent in the digital media. Despite users' preference for creating physical notes, handling digital notes on boards was easier and the potential of proper documentation make the digital setup a possible alternative. While the analogy in our remediation supported a transfer of learned handling, the users' experiences across technological setups impact their use and understanding, yielding new concerns regarding cross-device transfer and collaboration.

Application-centric computing dominates human-computer interactions, yet the concept of an application is ambiguous and the impact of its ubiquity underexplored. We unpack "the application" through the lens of non-standard knowledge work: freelance, self-employed, and fixed-term contract workers who create knowledge in collaboration with a wide variety of stakeholders on a per-project basis. Based on interviews with fourteen participants we describe how: i) their economic value is intertwined with data and skills related to specific applications; ii) their access to this value is systematically jeopardised in collaboration due to the different application practices, preferences, and proficiencies of other stakeholders; and iii) they mitigate the costs of this compromise through cross-application collaboration strategies. We trace these experiences to common characteristics of applications, such as update processes, interface symmetries, application-document relationships, and operating system and hardware dependencies. By empirically and analytically focusing on "the application", we reveal the implications of the current application-centric computing paradigm and discuss how variations within this model create qualitatively different human-computer interactions.

Developing cross-device multi-user interfaces (UIs) is a challenging problem. There are numerous ways in which content and interactivity can be distributed. However, good solutions must consider multiple users, their roles, their preferences and access rights, as well as device capabilities. Manual and rule-based solutions are tedious to create and do not scale to larger problems nor do they adapt to dynamic changes, such as users leaving or joining an activity. In this paper, we cast the problem of UI distribution as an assignment problem and propose to solve it using combinatorial optimization. We present a mixed integer programming formulation which allows real-time applications in dynamically changing collaborative settings. It optimizes the allocation of UI elements based on device capabilities, user roles, preferences, and access rights. We present a proof-of-concept designer-in-the-loop tool, allowing for quick solution exploration. Finally, we compare our approach to traditional paper prototyping in a lab study.

PolarTrack is a novel camera-based approach to detecting and tracking mobile devices inside the capture volume. In PolarTrack, a polarization filter continuously rotates in front of an off-the-shelf color camera, which causes the displays of observed devices to periodically blink in the camera feed. The periodic blinking results from the physical characteristics of current displays, which shine polarized light either through an LC overlay to produce images or through a polarizer to reduce light reflections on OLED displays. PolarTrack runs a simple detection algorithm on the camera feed to segment displays and track their locations and orientations, which makes PolarTrack particularly suitable as a tracking system for cross-device interaction with mobile devices. Our evaluation of PolarTrack's tracking quality and comparison with state-of-the-art camera-based multi-device tracking showed a better tracking accuracy and precision with similar tracking reliability. PolarTrack works as standalone multi-device tracking but is also compatible with existing camera-based tracking systems and can complement them to compensate for their limitations.

A sizable part of HCI research on cross-device interaction is driven by the vision of users conducting complex knowledge work seamlessly across multiple mobile devices. This is based on the Weiserian assumption that people will be inclined to distribute their work across multiple ``pads' if such are available. We observed that this is not the reality today, even when devices were in abundance. We present a study with 24 participants in 12 dyads completing a collaborative visualization task with up to six tablets. They could choose between three different visualization types to answer questions about economic data. Tasks were designed to afford simultaneous use of tablets, either with linked or independent views. We found that users typically utilized only one tablet per user. A quantitative and qualitative analysis revealed a ``legacy bias' that introduced barriers for using more tablets and reduced the overall benefit of multi-device visualization.

We introduce Codestrates, a literate computing approach to developing interactive software. Codestrates blurs the distinction between the use and development of applications. It builds on the literate computing approach, commonly found in interactive notebooks such as Jupyter notebook. Literate computing weaves together prose and live computation in the same document. However, literate computing in interactive notebooks are limited to computation and it is challenging to extend their user interface, reprogram their functionality, or develop stand-alone applications. Codestrates builds literate computing capabilities on top of Webstrates and demonstrates how it can be used for (i) collaborative interactive notebooks, (ii) extending its functionality from within itself, and (iii) developing reprogrammable applications.

Discussions on what makes user interfaces "natural" or "intuitive" have led researchers to apply Fauconnier and Turner's theory of Conceptual Blends to explain how users rely on familiar and real-world concepts when they learn to use new digital technologies -- as a blend of experiences from the --physical and the "digital" world. This pursuit has multiple challenges of which we address four: The continuous dynamic development of experiences; the multiplicity and complexity involved; the distinction between "real" and "virtual" experiences, and finally applying descriptive concepts predictively. Based on our background in activity theoretical HCI we discuss two cases to nuance the discussion of conceptual blends and HCI. We provide an understanding of conceptual blends beyond one-to-one static blends, and immediately recognizable concepts. We focus on multiplicity, dynamics and learning, and in that we provide a more advanced methodological scaffolding of analyses of conceptual blends, hence we propose that designers need to seed blends in design.

Cross-device collaboration with tablets is an increasingly popular topic in HCI. Previous work has shown that tablet-only collaboration can be improved by an additional shared workspace on an interactive tabletop. However, large tabletops are costly and need space, raising the question to what extent the physical size of shared horizontal surfaces really pays off. In order to analyse the suitability of smaller-than-tabletop devices (e.g. tablets) as a low-cost alternative, we studied the effect of the size of a shared horizontal interactive workspace on users' attention, awareness, and efficiency during cross-device collaboration. In our study, 15 groups of two users executed a sensemaking task with two personal tablets (9.7") and a horizontal shared display of varying sizes (10.6", 27", and 55"). Our findings show that different sizes lead to differences in participants' interaction with the tabletop and in the groups' communication styles. To our own surprise we found that larger tabletops do not necessarily improve collaboration or sensemaking results, because they can divert users' attention away from their collaborators and towards the shared display.

In this workshop, we will review and discuss open issues, technical challenges and conceptual models for multi-device spatial or proxemic interaction. We aim to bring together researchers, students and practitioners working on technical infrastructures, studies and designs of spatial in-terfaces, or domain specific multi-device applications that use space as a unit of analysis. We focus specifically on analysing how such interfaces, tools and tracking technolo-gy can be deployed "in the wild". The workshop will facili-tate knowledge exchange about the current state of spatial and proxemic interactive systems, identify application do-mains and enabling technologies for cross-surface interac-tions in the wild, and establish a research community to develop effective strategies for successful design of cross-device interactions.

We revisit Alan Kay's early vision of dynamic media that blurs the distinction between documents and applications. We introduce shareable dynamic media that are malleable by users, who may appropriate them in idiosyncratic ways; shareable among users, who collaborate on multiple aspects of the media; and distributable across diverse devices and platforms. We present Webstrates, an environment for exploring shareable dynamic media. Webstrates augment web technology with real-time sharing. They turn web pages into substrates, i.e. software entities that act as applications or documents depending upon use. We illustrate Webstrates with two implemented case studies: users collaboratively author an article with functionally and visually different editors that they can personalize and extend at run-time; and they orchestrate its presentation and audience participation with multiple devices. We demonstrate the simplicity and generative power of Webstrates with three additional prototypes and evaluate it from a systems perspective.

In this paper we introduce the notion of techsona, as a possible counterpart design instrument to personas. We use a case study to illustrate a design process where techsonas help pinpoint technological ideas and alternatives and analyze these systematically. While a persona captures “imagine a user…”, a scenario “imagine a situation…” the techsona gives the missing piece of “imagine a technology…” We show how a recently developed activity theoretical model can serve both as a framing for techsonas and as an analytical interface between personas and techsonas. The paper discusses the potentials and problems of the techsona and concludes that the techsona truly helps with a missing piece in persona and scenario-based design.

This paper raises the question of why electronic whiteboards are not ubiquitous. The paper provides a design-oriented analysis of traditional as well as electronic whiteboards in the context of collaborative and individual activities. We offer a novel perspective on whiteboards for collaborative activity based on a survey of the electronic whiteboard literature, a series of interviews with users of traditional whiteboards, and concepts rooted in Activity Theory. We identify a number of characteristics of the non-electronic whiteboard that are important to understand and preserve in the design of electronic whiteboard systems. Most importantly, we argue that the strength of non-electronic whiteboards is a combination of their simplicity and stability as well as a discontinuity between material on and outside of the whiteboard. We argue that the non-electronic whiteboard has uses and properties, which will require an electronic substitute to differ fundamentally in design compared to our traditional personal computing devices as well as most designs seen today. We present a set of themes for design of future electronic whiteboard systems that emphasize limitations as a main design principle. We conclude with three principles for design: The idea of installation rather than application; the principle of supplementing rather than replacing; and finally the principle of embracing and enhancing discontinuities.

Public Wifi hotspots in cafes and public places are based on wireless local area network technology (WLAN). In contrast to the common understanding of connecting directly to the internet when connecting to a Wifi hotspot, we are proposing to bring the original notion of connecting to a local network back to the fore. By hooking into the act of a user connecting to a local network, we see a design space emerge that allows us to spur various (off-and on-line) activities in a hyperlocal context. We propose to (re-)explore this notion of the locality of networks in the age of the ubiquitous Wifi hotspot in cafes, bars, community centers, and other (semi-)public places in order to facilitate co-located activities for such varied purposes as fostering local community, civic participation, sociality in general, and entertainment. We propose a network locality that builds on local infrastructure (WLAN) and combines personal devices (mobile phones) with stationary interactive surface technology (wall displays and tables) to facilitate social on-and off-line interactions in local settings. In this position paper, we outline technological opportunities associated with this idea as well as envisioned usage scenarios in different settings.

This paper presents a novel middleware for developing flexible interactive multi-surface applications. Using a scenario-based approach, we identify the requirements for this type of applications. We then introduce Substance, a data-oriented framework that decouples functionality from data, and Shared Substance, a middleware implemented in Substance that provides powerful sharing abstractions. We describe our implementation of two applications with Shared Substance and discuss the insights gained from these experiments. Our finding is that the combination of a data-oriented programming model with middleware support for sharing data and functionality provides a flexible, robust solution with low viscosity at both design-time and run-time.

Although devices of all shapes and sizes currently dominate the technological landscape, human–computer interaction (HCI) as a field is not yet theoretically equipped to match this reality. In this article we develop the human–artifact model, which has its roots in activity theoretical HCI. By reinterpreting the activity theoretical foundation, we present a framework that helps addressing the analysis of individual interactive artifacts while embracing that they are part of a larger ecology of artifacts. We show how the human–artifact model helps structuring the understanding of an artifact's action-possibilities in relation to the artifact ecology surrounding it. Essential to the model is that it provides four interconnected levels of analysis and addresses the possibilities and problems at these four levels. Artifacts and their use are constantly developing, and we address development in, and of, use. The framework needs to support such development through concepts and methods. This leads to a methodological approach that focuses on new artifacts to supplement and substitute existing artifacts. Through a design case, we develop the methodological approach and illustrate how the human–artifact model can be applied to analyze present artifacts and to design future ones. The model is used to structure such analysis and to reason about findings while providing leverage from activity theoretical insights on mediation, dialectics, and levels of activity.

This paper addresses interaction in multi-surface environments and questions whether the current application-centric approaches to user interfaces are adequate in this context, and presents an alternative approach based on instrumental interaction. The paper presents the VIGO (Views, Instruments, Governors and Objects) architecture and describes a prototype implementation. It then illustrates how to apply VIGO to support distributed interaction. Finally, it demonstrates how a classical Ubicomp interaction technique, Pick-and-Drop, can be easily implemented using VIGO.