This paper investigates how people use physical space, free from digital constraints, to organize thoughts and ideas. Such spatial arrangements commonly support sensemaking, idea generation, and data understanding. We collected over 300 publicly shared images capturing these activities collected from real-world contexts. Our visual analysis identifies a small set of recurring spatial patterns such as clusters, grids as well as the techniques people use to build and adapt them, such as color coding, directional lines, and selective rule-following. We discuss how these practices can inform the design of more expressive and adaptable digital spatially-aware tools that better support the diverse ways people think with space.

The uses of handheld mobile devices are diverse, yet interaction is not; touchscreens are the singular primary interface on most mobiles. Touch interaction has usability issues (e.g., the “fat fingers problem”) which impair the fine control of small interface elements, such as when working with text. Beyond text entry, this includes tasks like placing the in-text cursor (caret), text selection, and copy/paste. Current solutions for touch usability issues do not address complex uses like working with text. We propose deformable interaction, specifically bend, added alongside touch to support working with text on mobile. We explore this through a study of BendAide, a novel deformable 3D printed case for mobiles that adds bend interaction to the device. We found that people perceive different advantages between bend and touch and that they will alternate between these inputs based on task demands and their personal abilities. Adding alternate input options to mobile could reduce the complexity of on-display interfaces and interactions and give people more choice in how they use their devices.

Integrating physical learning materials with mobile device applications may have benefits for early childhood learning. We present three techniques for creating a hybrid tangible-augmented reality (T-AR) enabling technology platform. This platform enables researchers to develop applications that use readily available physical learning materials, such as letters, numbers, symbols or shapes. The techniques are visual marker-based; computer-vision and machine-learning; and capacitive touches. We describe details of implementation and demonstrate these techniques through a use case of a reading tablet app that uses wooden/plastic letters for input and augmented output. Our comparative analysis revealed that the machine-learning technique most flexibly sensed different physical letter sets but had variable accuracy impacted by lighting and tracking lag at this time. Lastly, we demonstrate how this enabling technology can be generalized to a variety of early learning apps through a second use case with physical numbers.

Science-Fiction (Sci-Fi) movies have long been a frontier in showcasing futuristic computer interfaces and their associated interactions. Unconstrained by technological limitations, they are free to depict the most imaginative systems, including augmenting objects attributes that are not yet possible in reality. We present a case study on Sci-Fi movies where tangible objects are part of these systems, and examine how they illustrate Tangible User Interfaces (TUIs) concepts. We provide three examples of tangible systems and one that deviates considerably (holographic system), and analyze them using a well-established interaction model (MCRpd). We found that TUIs in movies exhibit various levels of the model's characteristics and demonstrate an inclusive and diverse context through combining interaction modalities and catering to audience needs. We argue that these aspects provide valuable lessons and implications in designing future TUIs and hope to broaden the design space by initiating discussions on the fascinating worlds in Sci-Fi movies.

EmotoTent is an interactive socio-emotional learning system developed in response to escalating levels of violence, inequality and marginalization in schools seen in the early 21st Century. The system is inspired by advances in biosensing wearables, tattoo displays, brain sensors, robotic agents, artificial intelligence (AI), gestural interaction and 3D holographic displays. By 2030, technological advances will enable us to prototype and investigate questions related to experiential and embodied emotional learning; emotion-based human-computer interaction, affective biosensing, empathetic AI agents, and 3D interactive holographic environments. We envision EmotoTent as a modular, emotion-sensing Holodeck. In the EmotoTent program children learn and practice emotion regulation and empathy with peers, pets and a robotic dog agent in ways that are experiential, embodied and playful. We propose EmotoTent as a core element of a K-6 socio-emotional learning curriculum designed to improve school culture through the enhancement of children's ability to regulate emotions and interact with human and non-human species with empathy and compassion. Enhancing these qualities has been shown to lead to reductions in violence and bullying, racism, gender inequality and other forms of marginalization. We predict that the EmotoTent socio-emotional learning program will improve school cultures and create a foundation for children's lifelong well-being.

The majority of mobile applications use built-in touchscreens and/or accelerometers to provide direct ways for user inputs. Yet, the need to manipulate the device itself (e.g. touch, tilt) poses usability issues such as occlusion and inaccuracy. To address these issues, research proposed using the built-in magnetometer and magnets to facilitate around-device interactions. However, there is little evaluation in how this technique impacts performance and user experience beyond simple docking tasks. To fill this gap, we explored the mobile gameplay context by implementing an interface that uses rotatory gestures from a magnetic ring as input, and compared two control mappings (angular and linear) with touch and tilt in a usability study using a mobile game. We found that rotatory gestures with the ring, when mapped to angular controls, were on par with touch and superior over tilt, and engendered greater gameplay experience and sense of mapping. Based on our findings, we discuss implications of using this technique for gameplay, as well as other applications.

Digital adaptation of physical games often includes complete digitization resulting in the replacement of physical movements with virtual counterparts using input devices. We believe that augmenting the game by adding digital elements while also preserving physical movements can enhance player engagement. We present Re-Twist, a digitally augmented version of Twister. We introduce the element of time and score in Re-Twist by using a pressure-sensitive Twister mat that communicates with a projected screen. To investigate the effect of digital augmentation, we conducted a comparative study between the original Twister and Re-Twist. 81% of 16 participants preferred Re-Twist over the original Twister because of increased competition and urgency created by digital augmentation. We discuss the effect of digital augmentation on the competition, social, and challenge aspect of the game. This can guide new ways of game design by relooking at similar augmentation of other traditional games.

When compared to mainstream touch/button-centric devices, deformable devices enable a more organic and tangible way for human-computer interaction. This studio provides participants an opportunity to have hands-on experience in fabricating controllers that use various deformation inputs (e.g., bending, stretching), and promote novel interaction mechanisms using hand gestures. Participants will learn different types of deformation inputs and create their own sensors. They will work in groups to design deformable controllers using both the sensors they make in-session and/or pre-built ones, to afford novel hand gestural inputs beyond conventional touches and clicks. The main objectives of this studio are to facilitate exchange of experience in fabricating deformable sensors/materials, and to foster creativity in using such controllers as inputs with hand gestures.

We present our initial work on using a magnetic ring as a tangible input control to support around-device interactions for mobile devices, aiming to address touchscreen issues such as occlusion and imprecision, at the same time provides tangibility to interaction. This input mechanism allows users to use the surface on which the mobile device is placed as an extended input space, with a rotatory motion as the main form of interaction. Our technique requires no calibration, no modifications to the mobile device, and no external power for the ring, which also functions as an accessory item (a finger ring) when not in use. We discuss our design criteria, prototype implementation and illustrative applications, and directions for future work.

Bend gestures can be used as a form of Around Device Interaction to address usability issues in touchscreen mobile devices. Yet, it is unclear whether bend gestures can be easily learned and memorized as control schema for games. To answer this, we built a novel deformable smartphone case that detects bend gestures at its corners and sides, and created PaperNinja, a mobile game that uses bends as input. We conducted a study comparing the effect of three pre-game training levels on learnability and memorability: no training, training of the bend gestures only, and training of both the bend gestures and their in-game action mapping. We found that including gesture-mapping positively impacted the initial learning (faster completion time and fewer gestures performed), but had a similar outcome as no training on memorability, while the gestures-without-mapping led to a negative outcome. Our findings suggest that players can learn bend gestures by discovery and training is not essential.

Recent work has explored how on-the-wrist devices can be augmented with displays, sensors, and aesthetics to produce a new genre of wearable, digital jewelry. Most often, these devices are augmented with hardware (e.g., touch-screens, gyroscopes) to receive wearer input. Breakthroughs in sensing materials allow novel inputs such as squeeze, bend, and stretch, which may be more suited to interaction with a discreet wearable.

Large interactive displays presenting useful content responding to users' actions are becoming ubiquitous in public places such as museums and campuses. Being an integral part of the surrounding environment, a major design challenge is to first attract passersby's attention and entice them to interact. One promising approach is to incorporate proxemics parameters to guide how the content are visually presented. In this paper we describe the design of a field experiment using user-display distances in determining how visual content was displayed, and its deployment at a university campus with frequent foot traffic. We also offer some selection criteria for reference when conducting a field experiment with public large interactive displays.

We describe the design, development, and deployment of two scientific outreach games to support an open house event at an Institute for Quantum Computing (IQC) by an 11-member team of academics and practitioners over a one-year period. We overview our collaborative design process, the development of design goals aligned with facilitating scientific outreach during a one-day open house, and report on how successfully our final prototypes achieved those goals. We then reflect on implications for the use of interactive surfaces and spaces as a platform for scientific outreach, and in particular, the effectiveness of large displays as 'honey pots', the immaturity of technologies for creating interactive spaces, and extending the reach of outreach activities through technology.

We describe the design, development, and deployment of a scientific outreach game to support an open house event at an Institute for Quantum Computing (IQC) by an 11-member team of academics and practitioners over a one-year period. We overview our design process, the development of design goals that aligned with facilitating scientific outreach during a one-day open house, and report on how successfully our final prototype achieved those goals. We then reflect on the effectiveness of large displays as `honey pots' and extending the reach of outreach activities through technology. This demonstration accompanies our ISS 2016 paper.

A known challenge of designing large public interactive displays is to create an interface that attracts a passerby's attention and communicates its interactivity. However, typical "in-the-wild" field study methods of assessing public display design solutions require costly system implementation and deployment, creating challenges for assessing early stage design concepts. Such studies also limit the amount of experimental control researchers have over the environment, limiting the precision of results. To address these issues, we developed a complementary laboratory-based study methodology that employs experimental deception to assess the ability of an interface design solution to attract a passerby's attention. Our methodology enables more rigorous control of confounding factors, study of early-stage prototypes, and requires minimal setup. We used this methodology to assess existing visual design solutions for drawing attention and enticing interaction, compare our results to previous studies, and reflect on the benefits and limitations of this assessment approach.

During collaborative gameplay players make use of various methods to become aware of the overall game status, develop strategies, and convey information to other players. Efficient and effective use of these methods is essential, especially during fast-paced time-critical collaborative games such as first-person shooters. This paper presents an observational study aimed to understand the communication channels and awareness cues used by players during gameplay to collaboratively achieve the game objectives. The study revealed that players utilize a variety of unconventional communication channels and awareness cues in both the physical and virtual environments to compensate for the inability to use commonly available collaborative human interaction mechanisms, such as eye gaze and gesturing, during gameplay. Players tended to use only auditory cues from their partner in the physical environment, while relying heavily on interacting with their partner through the virtual environment, using a variety of central and peripheral cues to maintain awareness during gameplay. Implications of these findings are discussed and recommendations for improving the quality of gameplay are provided.

In wireless sensor networks, estimating nodal positions is important for routing efficiency and location-based services. Traditional techniques based on precise measurements are often expensive and power-inefficient, while approaches based on landmarks often require bandwidth-inefficient flooding and hence are not scalable for large networks. In this paper, we propose and investigate a cost-effective and distributed algorithm to accurately estimate nodal positions for wireless sensor networks. In our algorithm, a node only needs to identify and exchange information with a certain number of neighbors (around 30) in its proximity in order to estimate its relative nodal position accurately. For location-identification, only a small number of nodes (around 10) are needed to have additional GPS capabilities to accurately estimate the absolute position of every node in the network. Our algorithm is shown to have fast convergence with low estimation error, even for large networks.