Choreographic reflection relies on iterative dialogue, where dancers and choreographers refine movement through embodied demonstration and shared feedback in studio rehearsal. With the shift to video, this exchange becomes constrained: annotations detach from the body, gestures lose spatial grounding, and subtle variations are difficult to capture. Advances in markerless motion capture enable 3D reconstruction from rehearsal video, allowing past recordings to be re-materialized for embodied interaction in XR. We present DanXeReflect, an XR system that transforms flat video into a virtual studio where movements appear as interactive avatars. Users can re-enact poses to search sequences, perform alternative revisions alongside originals, and attach annotations directly to body parts. A study with choreographers and dancers shows how these embodied interactions reposition spatio-temporal data as collaborative anchors, extending reflective dialogue beyond co-located rehearsal into asynchronous, distributed practice.

Fingerspelling is a critical part of American Sign Language (ASL) recognition and has become an accessible optional text entry method for Deaf and Hard of Hearing (DHH) individuals. In this paper, we introduce SpellRing, a single smart ring worn on the thumb that recognizes words continuously fingerspelled in ASL. SpellRing uses active acoustic sensing (via a microphone and speaker) and an inertial measurement unit (IMU) to track handshape and movement, which are processed through a deep learning algorithm using Connectionist Temporal Classification (CTC) loss. We evaluated the system with 20 ASL signers (13 fluent and 7 learners), using the MacKenzie-Soukoref Phrase Set of 1,164 words and 100 phrases. Offline evaluation yielded top-1 and top-5 word recognition accuracies of 82.45% (9.67%) and 92.42% (5.70%), respectively. In real-time, the system achieved a word error rate (WER) of 0.099 (0.039) on the phrases. Based on these results, we discuss key lessons and design implications for future minimally obtrusive ASL recognition wearables.

Our hands serve as a fundamental means of interaction with the world around us. Therefore, understanding hand poses and interaction contexts is critical for human-computer interaction (HCI). We present EchoWrist, a low-power wristband that continuously estimates 3D hand poses and recognizes hand-object interactions using active acoustic sensing. EchoWrist is equipped with two speakers emitting inaudible sound waves toward the hand. These sound waves interact with the hand and its surroundings through reflections and diffractions, carrying rich information about the hand’s shape and the objects it interacts with. The information captured by the two microphones goes through a deep learning inference system that recovers hand poses and identifies various everyday hand activities. Results from the two 12-participant user studies show that EchoWrist is effective and efficient at tracking 3D hand poses and recognizing hand-object interactions. Operating at 57.9 mW, EchoWrist can continuously reconstruct 20 3D hand joints with MJEDE of 4.81 mm and recognize 12 naturalistic hand-object interactions with 97.6% accuracy.

In this paper, we introduce EyeEcho, a minimally-obtrusive acoustic sensing system designed to enable glasses to continuously monitor facial expressions. It utilizes two pairs of speakers and microphones mounted on glasses, to emit encoded inaudible acoustic signals directed towards the face, capturing subtle skin deformations associated with facial expressions. The reflected signals are processed through a customized machine-learning pipeline to estimate full facial movements. EyeEcho samples at 83.3 Hz with a relatively low power consumption of 167mW. Our user study involving 12 participants demonstrates that, with just four minutes of training data, EyeEcho achieves highly accurate tracking performance across different real-world scenarios, including sitting, walking, and after remounting the devices. Additionally, a semi-in-the-wild study involving 10 participants further validates EyeEcho’s performance in naturalistic scenarios while participants engage in various daily activities. Finally, we showcase EyeEcho’s potential to be deployed on a commercial-off-the-shelf (COTS) smartphone, offering real-time facial expression tracking.

In this paper, we present GazeTrak, the first acoustic-based eye tracking system on glasses. Our system only needs one speaker and four microphones attached to each side of the glasses. These acoustic sensors capture the formations of the eyeballs and the surrounding areas by emitting encoded inaudible sound towards eyeballs and receiving the reflected signals. These reflected signals are further processed to calculate the echo profiles, which are fed to a customized deep learning pipeline to continuously infer the gaze position. In a user study with 20 participants, GazeTrak achieves an accuracy of 3.6° within the same remounting session and 4.9° across different sessions with a refreshing rate of 83.3 Hz and a power signature of 287.9 mW. Furthermore, we report the performance of our gaze tracking system fully implemented on an MCU with a low-power CNN accelerator (MAX78002). In this configuration, the system runs at up to 83.3 Hz and has a total power signature of 95.4 mW with a 30 Hz FPS.

Seams are areas of overlapping fabric formed by stitching two or more pieces of fabric together in the cut-and-sew apparel manufacturing process. In SeamPose, we repurposed seams as capacitive sensors in a shirt for continuous upper-body pose estimation. Compared to previous all-textile motion-capturing garments that place the electrodes on the clothing surface, our solution leverages existing seams inside of a shirt by machine-sewing insulated conductive threads over the seams. The unique invisibilities and placements of the seams afford the sensing shirt to look and wear similarly as a conventional shirt while providing exciting pose-tracking capabilities. To validate this approach, we implemented a proof-of-concept untethered shirt with 8 capacitive sensing seams. With a 12-participant user study, our customized deep-learning pipeline accurately estimates the relative (to the pelvis) upper-body 3D joint positions with a mean per joint position error (MPJPE) of 6.0 cm. SeamPose represents a step towards unobtrusive integration of smart clothing for everyday pose estimation.

Design activities, such as brainstorming or critique, often take place in open spaces combining whiteboards and tables to present artefacts. In co-located settings, peripheral awareness enables participants to understand each other’s locus of attention with ease. However, these spatial cues are mostly lost while using videoconferencing tools. Telepresence robots could bring back a sense of presence, but controlling them is distracting. To address this problem, we present ReMotion, a fully automatic robotic proxy designed to explore a new way of supporting non-collocated open-space design activities. ReMotion combines a commodity body tracker (Kinect) to capture a user’s location and orientation over a wide area with a minimally invasive wearable system (NeckFace) to capture facial expressions. Due to its omnidirectional platform, ReMotion embodiment can render a wide range of body movements. A formative evaluation indicated that our system enhances the sharing of attention and the sense of co-presence enabling seamless movement-in-space during a design review task.

We present EchoSpeech, a minimally-obtrusive silent speech interface (SSI) powered by low-power active acoustic sensing. EchoSpeech uses speakers and microphones mounted on a glass-frame and emits inaudible sound waves towards the skin. By analyzing echos from multiple paths, EchoSpeech captures subtle skin deformations caused by silent utterances and uses them to infer silent speech. With a user study of 12 participants, we demonstrate that EchoSpeech can recognize 31 isolated commands and 3-6 figure connected digits with 4.5% (std 3.5%) and 6.1% (std 4.2%) Word Error Rate (WER), respectively. We further evaluated EchoSpeech under scenarios including walking and noise injection to test its robustness. We then demonstrated using EchoSpeech in demo applications in real-time operating at 73.3mW, where the real-time pipeline was implemented on a smartphone with only 1-6 minutes of training data. We believe that EchoSpeech takes a solid step towards minimally-obtrusive wearable SSI for real-life deployment.

Recent research in robotic proxies has demonstrated that one can automatically reproduce many non-verbal cues important in co-located collaboration. However, they often require a symmetrical hardware setup in each location. We present the VRoxy system, designed to enable access to remote spaces through a robotic embodiment, using a VR headset in a much smaller space, such as a personal office. VRoxy maps small movements in VR space to larger movements in the physical space of the robot, allowing the user to navigate large physical spaces easily. Using VRoxy, the VR user can quickly explore and navigate in a low-fidelity rendering of the remote space. Upon the robot’s arrival, the system uses the feed of a 360 camera to support real-time interactions. The system also facilitates various interaction modalities by rendering the micro-mobility around shared spaces, head and facial animations, and pointing gestures on the proxy. We demonstrate how our system can accommodate mapping multiple physical locations onto a unified virtual space. In a formative study, users could complete a design decision task where they navigated and collaborated in a complex 7.5m x 5m layout using a 3m x 2m VR space.

Collaborative design activities are often centered around physical artifacts. Depending on the design activity, this can be the model of a building, paper crafts, carving artwork, or a new circuit to be debugged and evaluated. In a typical setting, collaborators are seated around a table and divide their attention between the design artifact under review, at least one laptop supporting measurements and information foraging, and of course their collaborators. Although these activities involve complex sets of tools and configurations, people can easily work together when they are present in the same space. This is because the physical presence of a partner affords peripheral awareness to inform where the partner's attention is and what they are doing. This peripheral awareness allows collaborators to coordinate actions and manage coupling to achieve a shared task. For example, it is quite easy to know when your partner switches their focus from a breadboard to you as a request to start a face to face discussion.

On-skin interfaces demonstrate great potential given their direct skin contact; however, conducting field studies of these devices outside of laboratories and in real settings remains a challenge. We conduct a research-through-design investigation using an extended woven practice for fabricating fully-integrated and untethered multi-sensor on-skin systems that are resilient, versatile, and capable of field deployment. We designed, implemented, and deployed a woven on-skin index-finger and thumb-based inertial measurement unit (IMU) sensing system for multi-hour use as a technology probe to understand the social, technical, and design facets towards moving integrated on-skin systems into a wearer’s daily life. Further, we integrate a woven NFC coil into the IMU on-skin system, which is wirelessly powered by a smartwatch substitute, signifying the potential of our woven approach for developing wirelessly powered on-skin systems for longer-term continuous wear. Our investigation and the lessons learned shed light on the opportunities for designing on-skin systems for everyday wear.

We propose the use of interactive vignettes as an alternative to traditional text- and video-based vignettes for conducting large-scale Human-Robot Interaction (HRI) studies. Interactive vignettes maintain the advantages of traditional vignettes while offering additional affordances for participant interaction and data collection through interactive elements. We discuss the core affordances of interactive vignettes, including explorability, responsiveness, and non-linearity, and look into how these affordances can enable HRI research with more complex scenarios. To demonstrate the strength of the approach, we present a case study of our own research project with N=87 participants and show the data we collect through interactive vignettes. We suggest that the use of interactive vignettes can benefit HRI researchers in learning how participants interact with, respond to, and perceive a robot’s behavior in pre-defined scenarios.

In Cameroon, no exam is more important than the baccalauréat, which serves as the gateway to adulthood and professional and higher-education aspirations. We build on existing literature on the benefits of practice tests to evaluate how SMS-based quizzes could help students in their final year of secondary school prepare for the baccalauréat. Students in 23 schools across Yaoundé, the capital of Cameroon, received multiple-choice questions four times per week for nine weeks leading up to the exam date. We examined baccalauréat pass rates and student feedback through 11 focus groups. We find that students used our practice tests for formative assessment and as a prompt for recall and review of study material. SMS-based quizzes were not isolated artifacts but were shared in the students' networks, built a distributed learning environment, and used as a focus of collaborative study sessions.

Novel, "space-making" robots have potential to redefine physical space and the human activities occurring in it. Categorically distinct from many robots and far removed from humanoids, space-making robots are not objects in space, not anthropomorphic, not animal-like, not mobile, but instead, integral with the physical environment, embedded in or forming walls, ceilings, floors, partitions, vehicle interiors, and building envelopes. Given their distinctiveness, space-making robots offer a novel human-machine interaction. This paper investigates whether users perceive space-making robots as agents- artificial social actors characterized by the capacity for intelligence, recognition, and intention. Results of an in-lab experiment with 11 participants and an online, between-group experiment with 120 participants show that people attribute agency metrics of intelligence, intention, recognition, cooperation, collaboration, friendliness, and welcome to our reconfigurable robotic surface embedded in a wall partition. While space-making robots may become numerous in the built environment, our results are significant, moreover, for their broader implications for conceptualizing and designing human-machine interactions.

C-Face (Contour-Face) is an ear-mounted wearable sensing technology that uses two miniature cameras to continuously reconstruct facial expressions by deep learning contours of the face. When facial muscles move, the contours of the face change from the point of view of the ear-mounted cameras. These subtle changes are fed into a deep learning model which continuously outputs 42 facial feature points representing the shapes and positions of the mouth, eyes and eyebrows. To evaluate C-Face, we embedded our technology into headphones and earphones. We conducted a user study with nine participants. In this study, we compared the output of our system to the feature points outputted by a state of the art computer vision library (Dlib) from a font facing camera. We found that the mean error of all 42 feature points was 0.77 mm for earphones and 0.74 mm for headphones. The mean error for 20 major feature points capturing the most active areas of the face was 1.43 mm for earphones and 1.39 mm for headphones. The ability to continuously reconstruct facial expressions introduces new opportunities in a variety of applications. As a demonstration, we implemented and evaluated C-Face for two applications: facial expression detection (outputting emojis) and silent speech recognition. We further discuss the opportunities and challenges of deploying C-Face in real-world applications.

3D weaving is an emerging technology for manufacturing multilayer woven textiles. In this work, we present Weavecraft: an interactive, simulation-based design tool for 3D weaving. Unlike existing textile software that uses 2D representations for design patterns, we propose a novel weave block representation that helps the user to understand 3D woven structures and to create complex multi-layered patterns. With Weavecraft, users can create blocks either from scratch or by loading traditional weaves, compose the blocks into large structures, and edit the pattern at various scales. Furthermore, users can verify the design with a physically based simulator, which predicts and visualizes the geometric structure of the woven material and reveals potential defects at an interactive rate. We demonstrate a range of results created with our tool, from simple two-layer cloth and well known 3D structures to a more sophisticated design of a 3D woven shoe, and we evaluate the effectiveness of our system via a formative user study.

Students and hobbyists build embedded systems that combine sensing, actuation and microcontrollers on solderless breadboards. To help students debug such circuits, experienced teachers apply visual inspection, targeted measurements, and circuit modifications to diagnose and localize the problem(s). However, experienced helpers may not always be available to review student projects in person. To enable remote debugging of circuit problems, we introduce Heimdall, a remote electronics workbench that allows experts to visually inspect a student's circuit; perform measurements; and to re-wire and inject test signals. These interactions are enabled by an actuated inspection camera; an augmented breadboard that enables flexible configuration of row connectivity and measurement/injection lines; and a web-based UI that teachers can use to perform measurements through interaction with the captured images. We demonstrate that common issues arising in embedded electronics classes can be successfully diagnosed remotely and report on preliminary user feedback from teaching assistants who frequently debug circuits.

We created a quiz-based intervention to help secondary school students in Cameroon with exam practice. We sent regularly-spaced, multiple-choice questions to students' own mobile devices and examined factors which influenced quiz participation. These quizzes were delivered via either SMS or WhatsApp per each student's preference. We conducted a 3-week deployment with 546 students at 3 schools during their month of independent study prior to their graduating exam. We found that participation rates were heavily impacted by trust in the intervening organization and perceptions of personal security in the socio-technical environment. Parents also played a key gate-keeping role on students' digital activities. We describe how this role - along with different perceptions of smartphones versus basic phones - may manifest in lower participation rates among WhatsApp-based users as compared to SMS. Finally, we discuss design implications for future educational interventions that target students' personal cellphones outside of the classroom.

Several systems have illustrated the concept of interactive fabrication, i.e. rather than working through a digital editor, users make edits directly on the physical workpiece. However, so far the interaction has been limited to turn-taking, i.e., users first perform a command and then the system responds with physical feedback. In this paper, we present a first step towards interactive fabrication that changes the workpiece continuously while the user is manipulating it.

We present Ondulé-an interactive design tool that allows novices to create parameterizable deformation behaviors in 3D-printable models using helical springs and embedded joints. Informed by spring theory and our empirical mechanical experiments, we introduce spring and joint-based design techniques that support a range of parameterizable deformation behaviors, including compress, extend, twist, bend, and various combinations. To enable users to design and add these deformations to their models, we introduce a custom design tool for Rhino. Here, users can convert selected geometries into springs, customize spring stiffness, and parameterize their design with mechanical constraints for desired behaviors. To demonstrate the feasibility of our approach and the breadth of new designs that it enables, we showcase a set of example 3D-printed applications from launching rocket toys to tangible storytelling props. We conclude with a discussion of key challenges and open research questions.

Emotions play a major role in how interpersonal conflicts unfold. Although several strategies and technological approaches have been proposed for emotion regulation, they often require conscious attention and effort. This often limits their efficacy in practice. In this paper, we propose a different approach inspired by self-perception theory: noticing that people are often reacting to the perception of their own behavior, we artificially change their perceptions to influence their emotions. We conducted two studies to evaluate the potential of this approach by automatically and subtly altering how people perceive their own voice. In one study, participants that received voice feedback with a calmer tone during relationship conflicts felt less anxious. In the other study, participants who listened to their own voices with a lower pitch during contentious debates felt more powerful. We discuss the implications of our findings and the opportunities for designing automatic and less perceptible emotion regulation systems.

We present the Robotic Modeling Assistant (RoMA), an interactive fabrication system providing a fast, precise, hands-on and in-situ modeling experience. As a designer creates a new model using RoMA AR CAD editor, features are constructed concurrently by a 3D printing robotic arm sharing the same design volume. The partially printed physical model then serves as a tangible reference for the designer as she adds new elements to her design. RoMA's proxemics-inspired handshake mechanism between the designer and the 3D printing robotic arm allows the designer to quickly interrupt printing to access a printed area or to indicate that the robot can take full control of the model to finish printing. RoMA lets users integrate real-world constraints into a design rapidly, allowing them to create well-proportioned tangible artifacts or to extend existing objects. We conclude by presenting the strengths and limitations of our current design.

This paper examines the opportunities and issues that arise in designing technologies to support low-income rural women in Bangladesh. Through a qualitative, empirical study with 90 participants, we reveal systemic everyday challenges that women face that form the backdrop against which technology design could potentially happen. We discuss how technology is already impacting women's lives, sometimes by reinforcing their subservient role in society and sometimes used tactically by women to gain a measure of agency. The issues raised by our participants concerning technology's place in their lives provide HCI researchers with valuable guidance about what might (or might not) be appropriate to design for them. We also show how prevalent HCI research and design strategies may fit more poorly than expected into rural women's lives, and we discuss possible alternative design directions, and the ethical and pragmatic trade-offs that they entail. Our contribution is not to "solve" the problem of designing for low-income rural women, but to expand the HCI community's understanding of technology design within deeply patriarchal societies.

Dominant approaches to programming education emphasize program construction over language comprehension. We present Reduct, an educational game embodying a new, comprehension-first approach to teaching novices core programming concepts which include functions, Booleans, equality, conditionals, and mapping functions over sets. In this novel teaching strategy, the player executes code using reduction-based operational semantics. During gameplay, code representations fade from concrete, block-based graphics to the actual syntax of JavaScript ES2015. We describe our design rationale and report on the results of a study evaluating the efficacy of our approach on young adults (18+) without prior coding experience. In a short timeframe, novices demonstrated promising learning of core concepts expressed in actual JavaScript. We also present results from an online deployment. Finally, we discuss ramifications for the design of future computational thinking games.

Speech commenting systems have been shown to facilitate asynchronous online communication from educational discussion to writing feedback. However, the production of speech comments introduces several challenges to users, including overcoming self-consciousness and time consuming editing. In this paper, we introduce TypeTalker, a speech commenting interface that presents speech as a synthesized generic voice to reduce speaker self-consciousness, while retaining the expressivity of the original speech with natural breaks and co-expressive gestures. TypeTalker streamlines speech editing through a simple textbox that respects temporal alignment across edits. A comparative evaluation shows that TypeTalker reduces speech anxiety during live-recording, and offers easier and more effective speech editing facilities than the previous state-of-the-art interface technique. A follow-up study on recipient perceptions of the produced comments suggests that while TypeTalker's generic voice may be traded-off with a loss of personal touch, it can also enhance the clarity of speech by refining the original speech's speed and accent.

The advent of high speed input sensor and display technologies and the drive for faster interactive response suggests that human-computer interaction (HCI) task processing deadlines of a few milliseconds or less may be required in future handheld devices. At the same time, users will expect the same, if not better, battery life than today's devices under these more stringent response requirements. In this paper, we present a toolbox for exploring the design space of HCI event processors. We first describe the simulation platform for interactive environments that runs mobile user interface code with inputs recorded from human users. We validate it against a hardware platform from prior work. Given system-level constraints on latency, we demonstrate how this toolbox can be used to design a custom heterogeneous event processor that maximizes battery life. We show that our toolbox can pick design points that are 1.5-2.5× more energy-efficient than general-purpose big. LITTLE architectures.

Current interactive fabrication tools offer tangible feedback by allowing users to work directly on the physical model, but they are slow because users need to participate in the physical instantiation of their designs. In contrast, CAD software offers powerful tools for 3D modeling but delays access to the physical workpiece until the end of the design process. In this paper we propose On-the-Fly Print: a 3D modeling approach that allows the user to design 3D models digitally while having a low-fidelity physical wireframe model printed in parallel. Our software starts printing features as soon as they are created and updates the physical model as needed. Users can quickly check the design in a real usage context by removing the partial physical print from the printer and replacing it afterwards to continue printing. Digital content modification can be updated with quick physical correction using a retractable cutting blade. We present the detailed description of On-the-Fly Print and showcase several examples designed and printed with our system.

New multi-modal annotation tools hold the promise of bringing the benefits of face-to-face contact to remote, asynchronous interactions. One such system, RichReview++, incorporates new techniques to improve access to the embedded multimedia commentary and allows users to annotate with new modalities, like deictic gestures. We conducted a series of field deployments of RichReview++ to characterize how these features benefit students using them for activities in the university classroom. Our first deployment investigated the use of multi-modal annotations as a way for instructors to provide feedback on student term papers. Our second deployment used annotations to support peer discussion about assigned readings in a graduate-level course. We found that presenting voice comments as interactive waveforms seems to facilitate students' consumption of the instructor's voice comments. We also found that gestural annotations clarify voice and give annotators a quick and lightweight way to alter the scope of their voice comments. Based on these results, we suggest ways to best leverage multi-modal annotation tools in education environments.

In this paper we demonstrate that it is possible to help individuals regulate their emotions with mobile interventions that leverage the way we naturally react to our bodily signals. Previous studies demonstrate that the awareness of our bodily signals, such as our heart rate, directly influences the way we feel. By leveraging these findings we designed a wearable device to regulate user's anxiety by providing a false feedback of a slow heart rate. The results of an experiment with 67 participants show that the device kept the anxiety of the individuals in low levels when compared to the control group and the other conditions. We discuss the implications of our findings and present some promising directions for designing and developing this type of intervention for emotion regulation.

We explore how to create interactive systems based on electrical muscle stimulation that offer expressive output. We present muscle-plotter, a system that provides users with input and output access to a computer system while on the go. Using pen-on-paper interaction, muscle-plotter allows users to engage in cognitively demanding activities, such as writing math. Users write formulas using a pen and the system responds by making the users' hand draw charts and widgets. While Anoto technology in the pen tracks users' input, muscle-plotter uses electrical muscle stimulation (EMS) to steer the user's wrist so as to plot charts, fit lines through data points, find data points of interest, or fill in forms. We demonstrate the system at the example of six simple applications, including a wind tunnel simulator.

We introduce a new form of low-cost 3D printer to print interactive electromechanical objects with wound in place coils. At the heart of this printer is a mechanism for depositing wire within a five degree of freedom (5DOF) fused deposition modeling (FDM) 3D printer. Copper wire can be used with this mechanism to form coils which induce magnetic fields as a current is passed through them. Soft iron wire can additionally be used to form components with high magnetic permeability which are thus able to shape and direct these magnetic fields to where they are needed. When fabricated with structural plastic elements, this allows simple but complete custom electromagnetic devices to be 3D printed. As examples, we demonstrate the fabrication of a solenoid actuator for the arm of a Lucky Cat figurine, a 6-pole motor stepper stator, a reluctance motor rotor and a Ferrofluid display. In addition, we show how printed coils which generate small currents in response to user actions can be used as input sensors in interactive devices.

Traditional 3D printers fabricate objects by depositing material to build up the model layer by layer. Instead printing only wireframes can reduce printing time and the cost of material while producing effective depictions of shape. However, wireframe printing requires the printer to undergo arbitrary 3D motions, rather than slice-wise 2D motions, which can lead to collisions with already-printed parts of the model. Previous work has either limited itself to restricted meshes that are collision free by construction, or simply dropped unreachable parts of the model, but in this paper we present a method to print arbitrary meshes on a 5DOF wireframe printer. We formalize the collision avoidance problem using a directed graph, and propose an algorithm that finds a locally minimal set of constraints on the order of edges that guarantees there will be no collisions. Then a second algorithm orders the edges so that the printing progresses smoothly. Though meshes do exist that still cannot be printed, our method prints a wide range of models that previous methods cannot, and it provides a fundamental enabling algorithm for future development of wireframe printing.

We introduce D-Coil, a new digital 3D modeling approach using wax coiling to bring tangibility to the design of digital models. After defining a shape to extrude, the users follow the lead of a hand-held actuated extruder to instantiate the actual extrusion using wax. The tangibility of the wax extrusion sets the stage to create the next components until the digital model is completed. The digital model affords all digital attributes (ease of transformation, distribution, and 3D printing) while the wax artifact can be discarded or kept as a one-of-a-kind memento. We present a proof-of-concept implementation of D-Coil and showcase how this additive approach can also be extended to a subtractive process using a digitally actuated cutter. By adding a 6DOF mouse, users can also include scaling, rotation, and bending effects to create a wide variety of shapes often difficult for novices to produce in standard CAD software.

The digitization of educational course content has proved to be problematic for math instructors due to the lack of quality feedback tools that can accommodate the commenter to efficiently express math formulae and convey descriptions about complex ideas contextualized in situ. This paper proposes that RichReview, a document annotation system which creates inking, voice and deictic gestures on top of the student's submitted work, is a possible formative math feedback solution, because it enables face-to-face like commentary within the contexts of the document at hand. A preliminary qualitative evaluation study conducted while having students receive RichReview feedback showed promise to our approach to enhance the quality of feedback, with the implication that incorporating multi-modal feedback into workflows can be an effective method to address elements of feedback submissions lacking in coursework that has moved online.

Personal fabrication is currently a one-way process: Once an object has been fabricated with a 3D printer, it cannot be changed anymore; any change requires printing a new version from scratch. The problem is that this approach ignores the nature of design iteration, i.e. that in subsequent iterations large parts of an object stay the same and only small parts change. This makes fabricating from scratch feel unnecessary and wasteful.

The orientation and repositioning of physical artefacts (such as paper documents) to afford shared viewing of content, or to steer the attention of others to specific details, is known as micro-mobility. But the role of grasp in micro-mobility has rarely been considered, much less sensed by devices. We therefore employ capacitive grip sensing and inertial motion to explore the design space of combined grasp + micro-mobility by considering three classes of technique in the context of active reading. Single user, single device techniques support grip-influenced behaviors such as bookmarking a page with a finger, but combine this with physical embodiment to allow flipping back to a previous location. Multiple user, single device techniques, such as passing a tablet to another user or working side-by-side on a single device, add fresh nuances of expression to co-located collaboration. And single user, multiple device techniques afford facile cross-referencing of content across devices. Founded on observations of grasp and micro-mobility, these techniques open up new possibilities for both individual and collaborative interaction with electronic documents.

Low power "helper" cores have been increasingly included on application processors to accomplish low intensity tasks such as music playing and motion sensing with minimum energy consumption. Recently, Guimbretière et al. [1] demonstrated that such helper cores can also be used to execute simple user interface tasks. We revisit this approach by implementing a similar system on an off-the-shelf application processor (TI OMAP4). Our study shows that in the case of high event rate interactions (pen inking and virtual keyboard), significant battery life gains (×1.7 and ×2.3 respectively) can be achieved with the helper core executing the interface. Having the helper core only dis-patch input events incurs a 18% penalty relative to the maximum savings rate, but allows for simplified deployment since it merely requires a change in toolkit infrastructure.

We explore grip and motion sensing to afford new techniques that leverage how users naturally manipulate tablet and stylus devices during pen + touch interaction. We can detect whether the user holds the pen in a writing grip or tucked between his fingers. We can distinguish bare-handed inputs, such as drag and pinch gestures produced by the nonpreferred hand, from touch gestures produced by the hand holding the pen, which necessarily impart a detectable motion signal to the stylus. We can sense which hand grips the tablet, and determine the screen's relative orientation to the pen. By selectively combining these signals and using them to complement one another, we can tailor interaction to the context, such as by ignoring unintentional touch inputs while writing, or supporting contextually-appropriate tools such as a magnifier for detailed stroke work that appears when the user pinches with the pen tucked between his fingers. These and other techniques can be used to impart new, previously unanticipated subtleties to pen + touch interaction on tablets.

Even though considered a rapid prototyping tool, 3D printing is so slow that a reasonably sized object requires printing overnight. This slows designers down to a single iteration per day. In this paper, we propose to instead print low-fidelity wireframe previews in the early stages of the design process. Wireframe previews are 3D prints in which surfaces have been replaced with a wireframe mesh. Since wireframe previews are to scale and represent the overall shape of the 3D object, they allow users to quickly verify key aspects of their 3D design, such as the ergonomic fit. To maximize the speed-up, we instruct 3D printers to extrude filament not layer-by-layer, but directly in 3D-space, allowing them to create the edges of the wireframe model directly one stroke at a time. This allows us to achieve speed-ups of up to a factor of 10 compared to traditional layer-based printing. We demonstrate how to achieve wireframe previews on standard FDM 3D printers, such as the PrintrBot or the Kossel mini. Users only need to install the WirePrint software, making our approach applicable to many 3D printers already in use today. Finally, wireframe previews use only a fraction of material required for a regular print, making it even more affordable to iterate.

This paper introduces a novel document annotation system that aims to enable the kinds of rich communication that usually only occur in face-to-face meetings. Our system, RichReview, lets users create annotations on top of digital documents using three main modalities: freeform inking, voice for narration, and deictic gestures in support of voice. RichReview uses novel visual representations and time-synchronization between modalities to simplify annotation access and navigation. Moreover, RichReview's versatile support for multi-modal annotations enables users to mix and interweave different modalities in threaded conversations. A formative evaluation demonstrates early promise for the system finding support for voice, pointing, and the combination of both to be especially valuable. In addition, initial findings point to the ways in which both content and social context affect modality choice.

As users become increasingly conscious of their energy footprint—either to improve battery life or to respect the environment—improved energy efficiency of systems has gained in importance. This is especially important in the context of information appliances such as e-book readers that are meant to replace books, since their energy efficiency impacts how long the appliance can be used on a single charge of the battery. In this article, we present a new software and hardware architecture for information appliances that provides significant advantages in terms of device lifetime. The architecture combines a low-power microcontroller with a high-performance application processor, where the low-power microcontroller is used to handle simple user interactions (e.g., turning pages, inking, entering text) without waking up the main application processor. We demonstrate how this architecture is easily adapted to the traditional way of building user interfaces using a user interface markup language. We report on our initial measurements using an E Ink-based prototype. When comparing our hybrid architecture to a simpler solution we found that we can increase the battery life by a factor of 1.72 for a reading task and by a factor of 3.23 for a writing task. We conclude by presenting design guidelines aimed at optimizing the overall energy signature of information appliances.

In laboratory studies, multi-surface slate-based reading systems have shown great promise as platforms for active reading. However, the true utility of such a system can only be ascertained through the rigors of real world use. We conducted month-long deployments of a multi-slate reading system to support the active reading activities of graduate students in the humanities. During these deployments we documented how the added display area and increased micro-mobility of multiple devices enhanced navigation and reading comfort. We also noted the essential role of writing and annotation. Finally, we observed how electronic affordances like synchronization across devices helped provide functionality that would not have been possible with paper documents. This paper contributes new information about how electronic reading solutions fit into real world reading workflows.

Having insufficient space for making annotations is a problem that afflicts both paper and digital documents. We introduce the TextTearing technique for in situ expansion of inter-line whitespace and pair it with a lightweight interaction for margin expansion as a way to address this problem. The full system leverages the dynamism of digital documents and employs a bimanual design that combines the precision of pen with the fluidity of touch. Our evaluation found that a simpler unimanual variant of TextTearing was preferred over direct annotation and margin-only expansion. Direct annotation in naturally occurring whitespace was least preferred.

We describe a mouseless, near-surface version of the Bimanual Marking Menu system. To activate the menu system, users create a pinch gesture with either their index or middle finger to initiate a left click or right click. Then they mark in the 3D space near the interactive area. We demonstrate how the system can be implemented using a commodity range camera such as the Microsoft Kinect, and report on several designs of the 3D marking system.

The availability of flexible capacitive sensors that can be fitted around mice, smartphones, and pens carries great potential in leveraging grasp as a new interaction modality. Unfortunately, most capacitive sensors only track interaction directly on the surface, making it harder to differentiate among grips and constraining user movements. We present a new optical range sensor design based on high power infrared LEDs and photo-transistors, which can be fabricat-ed on a flexible PCB and wrapped around a wide variety of graspable objects including pens, mice, smartphones, and slates. Our sensor offers a native resolution of 10 dpi with a sensing range of up to 30mm (1.2"") and sampling speed of 50Hz. Based on our prototype wrapped around the barrel of a pen, we present a summary of the characteristics of the sensor and describe the sensor output in several typical pen grips. Our design is versatile enough to apply not only to pens but to a wide variety of graspable objects including smartphones and slates.

Despite predictions of the paperless office, most knowledge workers and students still rely heavily on paper in most of their document practices. Research has shown that paper's dominance can be attributed to the fact that it supports a broad range of these users' diverse reading requirements. Our analysis of the literature suggests that a new class of reading device consisting of an interconnected environment of thin and lightweight electronic slates could potentially unify the distinct advantages of e-books, PCs, and tabletop computers to offer an electronic reading solution providing functionality comparable to, or even exceeding, that of paper. This article presents the design and construction of such a system. In it, we explain how data can be mapped to slates, detail interactions for linking the slates, and describe tools that leverage the connectivity between slates. A preliminary study of the system indicates that such a system has the potential of being an electronic alternative to paper.

To combine the affordances of paper and computers, prior research has proposed numerous interactive paper systems that link specific paper document content to digital operations such as multimedia playback and proofreading. Yet, it remains unclear to what degree these systems bridge the inherent gap between paper and computers when compared to existing paper-only and computer-only interfaces. In particular, given the special properties of paper, such as limited dynamic feedback, how well does an average new user learn to master the interactive paper system? What factors affect the user performance? And how does the paper interface work in a typical use scenario? To answer these questions, we conducted two empirical experiments on a generic pen-gesture-based command system, called PapierCraft [Liao et al. 2008], for paper-based interfaces. With it, people can select sections of printed document and issue commands such as copy and paste, linking and in-text search. The first experiment focused on the user performance of drawing pen gestures on paper. It proves that users can learn the command system in about 30 minutes and achieve a performance comparable to a table PC-based interface supporting the same gestures. The second experiment examined the application of the command system in active reading tasks. The results show promise for seamless integration of paper and computers in active reading for their combined affordances. In addition, our study identifies some key design issues, such as the pen form factor and feedback of gestures. This article contributes to better understanding on pros and cons of paper and computers, and sheds light on the design of future interfaces for document interaction.

This paper explores the interaction possibilities enabled when the barrel of a digital pen is augmented with a multi-touch sensor. We present a novel multi-touch pen (MTPen) prototype and discuss its alternate uses beyond those of a standard stylus, such as allowing new touch gestures to be performed using the index finger or thumb and detecting how users grip the device as a mechanism for mode switching. We also discuss the hardware and software implementation challenges in realizing our prototype, and showcase how one can combine different grips (tripod, relaxed tripod, sketch, wrap) and gestures (swipe and double tap) to enable new interaction techniques with the MTPen in a prototype drawing application. One specific aim is the elimination of some of the comfort problems associated with existing auxiliary controls on digital pens. Mechanical controls such as barrel buttons and barrel scroll wheels work best in only a few specific hand grips and pen rotations. Comparatively, our gestures can be successfully and comfortably performed regardless of the rotation of the pen or how the user grips it, offering greater flexibility in use. We describe a formal evaluation comparing MTPen gestures against the use of a barrel button for mode switching. This study shows that both swipe and double tap gestures are comparable in performance to commonly employed barrel buttons without its disadvantages.

We present a set of interaction techniques called TouchFace for casual videoconferencing among people in close relationships. It enables users to experience the sense of touching without the need for any special devices using interactions between cursors and live video images. After presenting the design guidelines underlying TouchFace, we describe a prototype following these guidelines. We conclude by presenting the results of a preliminary study.

MouseLight is a spatially-aware standalone mobile projector with the form factor of a mouse that can be used in combination with digital pens on paper. By interacting with the projector and the pen bimanually, users can visualize and modify the virtually augmented contents on top of the paper, and seamlessly transition between virtual and physical information. We present a high fidelity hardware prototype of the system and demonstrate a set of novel interactions specifically tailored to the unique properties of MouseLight. MouseLight differentiates itself from related systems such as PenLight in two aspects. First, MouseLight presents a rich set of bimanual interactions inspired by the ToolGlass interaction metaphor, but applied to physical paper. Secondly, our system explores novel displaced interactions, that take advantage of the independent input and output that is spatially aware of the underneath paper. These properties enable users to issue remote commands such as copy and paste or search. We also report on a preliminary evaluation of the system which produced encouraging observations and feedback.

Mesh saliency has been proposed as a computational model of perceptual importance for meshes, and it has been used in graphics for abstraction, simplification, segmentation, illumination, rendering, and illustration. Even though this technique is inspired by models of low-level human vision, it has not yet been validated with respect to human performance. Here, we present a user study that compares the previous mesh saliency approaches with human eye movements. To quantify the correlation between mesh saliency and fixation locations for 3D rendered images, we introduce the normalized chance-adjusted saliency by improving the previous chance-adjusted saliency measure. Our results show that the current computational model of mesh saliency can model human eye movements significantly better than a purely random model or a curvature-based model.

Traditional graphical user interfaces have been designed with the desktop mouse in mind, a device well characterized by Fitts' law. Yet in recent years, hand-held devices and tablet personal computers using a pen (or fingers) as the primary mean of interaction have become more and more popular. These new interaction modalities have pushed the traditional focus on pointing to its limit. In this paper we explore whether a different paradigm—goal crossing-based on pen strokes—may substitute or complement pointing as another fundamental interaction method. First we describe a study in which we establish that goal crossing is dependent on an index of difficulty analogous to Fitts' law, and that in some settings, goal crossing completion time is shorter or comparable to pointing performance under the same index of difficulty. We then demonstrate the expressiveness of the crossing-based interaction paradigm by implementing CrossY, an application which only uses crossing for selecting commands. CrossY demonstrates that crossing-based interactions can be more expressive than the standard point and click approach. We also show how crossing-based interactions encourage the fluid composition of commands. Finally after observing that users' performance could be influenced by the general direction of travel, we report on the results of a study characterizing this effect. These latter results led us to propose a general guideline for dialog box interaction. Together, these results provide the foundation for the design of effective crossing-based interactions.

The Codex is a dual-screen tablet computer, about the size of a 4"x 6 day planner, with a self-supporting binding and embedded sensors. The device can be oriented in a variety of postures to support different nuances of individual work, ambient display, or collaboration with another user. In the context of a pen-operated note taking application, we demonstrate interaction techniques that support a fluid division of labor for tasks and information across the two displays while minimizing disruption to the primary experience of authoring notes.

Digital pen systems, originally designed to digitize annotations made on physical paper, are evolving to permit a wider variety of applications. Although the type and quality of pen feedback (e.g., haptic, audio, and visual) have a huge impact on advancing the digital pen technology, dynamic visual feedback has yet to be fully investigated. In parallel, miniature projectors are an emerging technology with the potential to enhance visual feedback for small mobile computing devices. In this paper we present the PenLight system, which is a testbed to explore the interaction design space and its accompanying interaction techniques in a digital pen embedded with a spatially-aware miniature projector. Using our prototype, that simulates a miniature projection (via a standard video projector), we visually augment paper documents, giving the user immediate access to additional information and computational tools. We also show how virtual ink can be managed in single and multi-user environments to aid collaboration and data management. User evaluation with professional architects indicated promise of our proposed techniques and their potential utility in the paper-intensive domain of architecture.

CitationBuxton and Myers (1986) compared a specialized document navigation device and a scrollbar. Their device tracked finger movements along two touch-sensitive areas: one for absolute movement and one for relative movement. They found that their system was faster when navigating a 5-page document. We identified finger-tracking issues with the Buxton and Myers device for larger documents (100 pages) and developed an alternative device employing a range sensor and rotary encoder to track finger movement. We ran a new experiment comparing the traditional scrollbar, the Buxton and Myers design, and our new design. Both the Buxton and Myers design and our new design were poorly received by users compared to the scrollbar. Our results indicate that in a large document, factors beyond finger tracking accuracy influence the performance of a device providing absolute movement. From these results, we identify possible improvements necessary to implement effective and practical absolute navigation devices.

Recent advancements in rapid prototyping techniques such as 3D printing and laser cutting are changing the perception of physical 3D models in architecture and industrial design. Physical models are frequently created not only to finalize a project but also to demonstrate an idea in early design stages. For such tasks, models can easily be annotated to capture comments, edits, and other forms of feedback. Unfortunately, these annotations remain in the physical world and cannot easily be transferred back to the digital world. Our system, ModelCraft, addresses this problem by augmenting the surface of a model with a traceable pattern. Any sketch drawn on the surface of the model using a digital pen is recovered as part of a digital representation. Sketches can also be interpreted as edit marks that trigger the corresponding operations on the CAD model. ModelCraft supports a wide range of operations on complex models, from editing a model to assembling multiple models, and offers physical tools to capture free-space input. Several interviews and a formal study with the potential users of our system proved the ModelCraft system useful. Our system is inexpensive, requires no tracking infrastructure or per object calibration, and we show how it could be extended seamlessly to use current 3D printing technology.

Existing e-book readers do not do a good job supporting many reading tasks that people perform, as ethnographers report that when reading, people frequently read from multiple display surfaces. In this paper we present our design of a dual display e-book reader and explore how it can be used to interact with electronic documents. Our design supports embodied interactions like folding, flipping, and fanning for local/lightweight navigation. We also show how mechanisms like Space Filling Thumbnails can use the increased display space to aid global navigation. Lastly, the detachable faces in our design can facilitate inter-document operations and flexible layout of documents in the workspace. Semi-directed interviews with seven users found that dual-displays have the potential to improve the reading experience by supporting several local navigation tasks better than a single display device. Users also identified many reading tasks for which the device would be valuable. Users did not find the embodied interface particularly useful when reading in our controlled lab setting, however.

We present an empirical analysis of crossing-based dialog boxes. First, we study the spatial constraints required for efficient crossing-based interactions in the case of a simple multi-parameter dialog box. Through a series of 3 tasks, we establish the minimal value of the landing margin, the takeoff margin, and the column width. We also offer an estimation of the role of stroke shape on user performance. After studying the reasons for errors during our experiment, we propose a relaxed crossing semantic that combines aspects of pointing and crossing-based interfaces.

Paper persists as an integral component of active reading and other knowledge-worker tasks because it provides ease of use unmatched by digital alternatives. Paper documents are light to carry, easy to annotate, rapid to navigate, flexible to manipulate, and robust to use in varied environments. Interactions with paper documents create rich webs of annotation, cross reference, and spatial organization. Unfortunately, the resulting webs are confined to the physical world of paper and, as they accumulate, become increasingly difficult to store, search, and access. XLibris [Schilit et al. 1998] and similar systems address these difficulties by simulating paper with tablet PCs. While this approach is promising, it suffers not only from limitations of current tablet computers (e.g., limited screen space) but also from loss of invaluable paper affordances. In this article, we describe PapierCraft, a gesture-based command system that allows users to manipulate digital documents using paper printouts as proxies. Using an Anoto [Anoto 2002] digital pen, users can draw command gestures on paper to tag a paragraph, e-mail a selected area, copy selections to a notepad, or create links to related documents. Upon pen synchronization, PapierCraft executes the commands and presents the results in a digital document viewer. Users can then search the tagged information and navigate the web of annotated digital documents resulting from interactions with the paper proxies. PapierCraft also supports real time interactions across mix-media, for example, letting users copy information from paper to a Tablet PC screen. This article presents the design and implementation of the PapierCraft system and describes user feedback from initial use.

We present ExperiScope, an analytical tool to help designers and experimenters explore the results of quantitative evaluations of interaction techniques. ExperiScope combines a new visualization incorporating aspects of the KLM and the three-state model with an interface helping users to rapidly cluster similar patterns of interactions. The tool makes it easy to identify and compare key patterns of use encountered during data collection. This promotes a deeper understanding of the results of a given evaluation.We illustrate the advantages of this tool by revisiting the data collected for an experiment conducted by Hinckley et al. [19] which compared different mode switching techniques. Our results show that our tool complements the previously reported results by offering insights about error behavior and the impact of mode switching on user performance.By providing a more fine-grained analysis of the data gathered during empirical evaluations, we hope that our tool will improve researchers' understanding of existing and newly developed interaction techniques.

Modes allow a few inputs to invoke many operations, yet if a user misclassifies or forgets the state of a system, modes can result in errors. Spring-loaded modes (quasimodes) maintain a mode while the user holds a control such as a button or key. The Springboard is an interaction technique for tablet computers that extends quasimodes to encompass multiple tool modes in a single spring-loaded control. The Springboard allows the user to continue holding down a nonpreferred-hand command button after selecting a tool from a menu as a way to repeatedly apply the same tool. We find the Springboard improves performance for both a local marking menu and for a non-local marking menu ("lagoon") at the lower left corner of the screen. Despite the round-trip costs incurred to move the pen to a tool lagoon, a keystroke-level analysis of the true cost of each technique reveals the local marking menu is not significantly faster.

We present a study that evaluates conventional Pan and Zoom Navigation and Rubber Sheet Navigation, a rectilinear Focus+Context technique. Each of the two navigation techniques was evaluated both with and without an overview. All interfaces guaranteed that regions of interest would remain visible, at least as a compressed landmark, independent of navigation actions. Interfaces implementing these techniques were used by 40 subjects to perform a task that involved navigating a large hierarchical tree dataset and making topological comparisons between nodes in the tree. Our results show that Pan and Zoom Navigation was significantly faster and required less mental effort than Rubber Sheet Navigation, independent of the presence or absence of an overview. Also, overviews did not appear to improve performance, but were still perceived as beneficial by users. We discuss the implications of our task and guaranteed visibility on the results and the limitations of our study, and we propose preliminary design guidelines and recommendations for future work.

Through a study of field biology practices, we observed that biology fieldwork generates a wealth of heterogeneous information, requiring substantial labor to coordinate and distill. To manage this data, biologists leverage a diverse set of tools, organizing their effort in paper notebooks. These observations motivated ButterflyNet, a mobile capture and access system that integrates paper notes with digital photographs captured during field research. Through ButterflyNet, the activity of leafing through a notebook expands to browsing all associated digital photos. ButterflyNet also facilitates the transfer of captured content to spreadsheets, enabling biologists to share their work. A first-use study with 14 biologists found this system to offer rich data capture and transformation, in a manner felicitous with current practice.

Current paper-based interfaces such as PapierCraft, provide very little feedback and this limits the scope of possible interactions. So far, there has been little systematic exploration of the structure, constraints, and contingencies of feedback-mechanisms in paper-based interaction systems for paper-only environments. We identify three levels of feedback: discovery feedback (e.g., to aid with menu learning), status-indication feedback (e.g., for error detection), and task feedback (e.g., to aid in a search task). Using three modalities (visual, tactile, and auditory) which can be easily implemented on a pen-sized computer, we introduce a conceptual matrix to guide systematic research on pen-top feedback for paper-based interfaces. Using this matrix, we implemented a multimodal pen prototype demonstrating the potential of our approach. We conducted an experiment that confirmed the efficacy of our design in helping users discover a new interface and identify and correct their errors.

With the availability of affordable new desktop fabrication techniques such as 3D printing and laser cutting, physical models are used increasingly often during the architectural and industrial design cycle. Models can easily be annotated to capture comments, edits and other forms of feedback. Unfortunately, these annotations remain in the physical world and cannot be easily transferred back to the digital world. Here we present a simple solution to this problem based on a tracking pattern printed on the surface of each model. Our solution is inexpensive, requires no tracking infrastructure or per object calibration, and can be used in the field without a computer nearby. It lets users not only capture annotations, but also edit the model using a simple yet versatile command system. Once captured, annotations and edits are merged into the original CAD models. There they can be easily edited or further refined. We present the design of a SolidWorks plug-in implementing this concept, and report initial feedback from potential users using our prototype. We also present how this prototype could be extended seamlessly to a fully functional system using current 3D printing technology.

We present a quantitative analysis of delimiters for pen gestures. A delimiter is "something different" in the input stream that a computer can use to determine the structure of input phrases. We study four techniques for delimiting a selection-action gesture phrase consisting of lasso selection plus marking-menu-based command activation. Pigtail is a new technique that uses a small loop to delimit lasso selection from marking (Fig. 1). Handle adds a box to the end of the lasso, from which the user makes a second stroke for marking. Timeout uses dwelling with the pen to delimit the lasso from the mark. Button uses a button press to signal when to delimit the gesture. We describe the role of delimiters in our Scriboli pen interaction testbed, and show how Pigtail supports scope selection, command activation, and direct manipulation all in a single fluid pen gesture.

Knowledge workers use paper extensively for document reviewing and note-taking due to its versatility and simplicity of use. As users annotate printed documents and gather notes, they create a rich web of annotations and cross references. Unfortunately, as paper is a static media, this web often gets trapped in the physical world. While several digital solutions such as XLibris [15] and Digital Desk [18] have been proposed, they suffer from a small display size or onerous hardware requirements.To address these limitations, we propose PapierCraft, a gesture-based interface that allows users to manipulate digital documents directly using their printouts as proxies. Using a digital pen, users can annotate a printout or draw command gestures to indicate operations such as copying a document area, pasting an area previously copied, or creating a link. Upon pen synchronization, our infrastructure executes these commands and presents the result in a customized viewer. In this paper we describe the design and implementation of the PapierCraft command system, and report on early user feedback.

Toolglass [Bier et al. 1993] demonstrated a two-handed command selection technique that combined command selection and direct manipulation. While empirical evaluations showed a speed advantage for ToolGlass, they did not examine the relative importance of two possible factors in its improved performance: (1) the use of two hands and (2) the merging of command selection and direct manipulation.We conducted a study comparing the relative benefits of three command selection techniques that merge command selection and direct manipulation: one two-handed technique, Toolglass, and two one-handed techniques, namely, control menus [Pook et al. 2000] and FlowMenu [Guimbretière and Winograd 2000]. Participants performed sequences of operations that required both selecting a color and designating the endpoints of a line. Our results show that control menus and FlowMenu are significantly faster than Toolglass. Further analysis suggests that the merging of command selection and direct manipulation is the most important factor in the performance of all three techniques.

We introduce CrossY, a simple drawing application developed as a benchmark to demonstrate the feasibility of goal-crossing as the basis for a graphical user interface. While crossing was previously identified as a potential substitute for the classic point-and-click interaction, this work is the first to report on the practical aspects of implementing an interface solely based on goal-crossing.

Stitching is a new interaction technique that allows users to combine pen-operated mobile devices with wireless networking by using pen gestures that span multiple displays. To stitch, a user starts moving the pen on one screen, crosses over the bezel, and finishes the stroke on the screen of a nearby device. Properties of each portion of the pen stroke are observed by the participating devices, synchronized via wireless network communication, and recognized as a unitary act performed by one user, thus binding together the devices. We identify the general requirements of stitching and describe a prototype photo sharing application that uses stitching to allow users to copy images from one tablet to another that is nearby, expand an image across multiple screens, establish a persistent shared workspace, or use one tablet to present images that a user selects from another tablet. We also discuss design issues that arise from proxemics, that is, the sociological implications of users collaborating in close quarters.

We introduce CrossY, a simple drawing application developed as a benchmark to demonstrate the feasibility of goal crossing as the basis for a graphical user interface. We show that crossing is not only as expressive as the current point-and-click interface, but also offers more flexibility in interaction design. In particular, crossing encourages the fluid composition of commands which supports the development of more fluid interfaces. While crossing was previously identified as a potential substitute for the classic point-and-click interaction, this work is the first to report on the practical aspects of implementing an interface based on goal crossing as the fundamental building block.

Several experiments by psychologists and human factors researchers have shown that when young children execute pointing tasks, they perform at levels below older children and adults. However, these experiments have not provided user interface designers with an understanding of the severity or the nature of the difficulties young children have when using input devices. To address this need, we conducted a study to gain a better understanding of 4 and 5 year-old children's use of mice. We compared the performance of thirteen 4 year-olds, thirteen 5 year-olds and thirteen young adults in point-and-click tasks. Plots of the paths taken by the participants show severe differences between adults' and preschool children's ability to control the mouse. We were not surprised then to find age had a significant effect on accuracy, target reentry, and efficiency. We also found that target size had a significant effect on accuracy and target reentry. Measuring movement time at four different times (first entering target, last entering target, pressing button, releasing button) yielded the result that Fitts' law models children well only up to the time they first enter the target. Overall, we found that the difference between the performance of children and adults was large enough to warrant user interface interactions designed specifically for preschool children. The results additionally suggest that children need the most help once they get close to targets.

Paper Augmented Digital Documents (PADD), are digital documents that can be manipulated either on a computer screen or on paper. PADD, and the infrastructure supporting them, can be seen as a bridge between the digital and the paper worlds. As digital documents, PADD are easy to edit, distribute and archive; as paper documents, PADD are easy to navigate, annotate and well accepted in social settings. The chimeric nature of PADD makes them well suited for many tasks such as proofreading, editing, and annotation of large format document like blueprints. We are presenting an architecture which supports the seamless manipulation of PADs using today's technologies and reports on the lessons we learned while implementing the first PADD system. Keywords: Paper Augmented Digital Document, Paper based user interface, digital pen UMIACS-TR-2003-40

Structural comparison of large trees is a difficult task that is only partially supported by current visualization techniques, which are mainly designed for browsing. We present TreeJuxtaposer, a system designed to support the comparison task for large trees of several hundred thousand nodes. We introduce the idea of "guaranteed visibility", where highlighted areas are treated as landmarks that must remain visually apparent at all times. We propose a new methodology for detailed structural comparison between two trees and provide a new nearly-linear algorithm for computing the best corresponding node from one tree to another. In addition, we present a new rectilinear Focus+Context technique for navigation that is well suited to the dynamic linking of side-by-side views while guaranteeing landmark visibility and constant frame rates. These three contributions result in a system delivering a fluid exploration experience that scales both in the size of the dataset and the number of pixels in the display. We have based the design decisions for our system on the needs of a target audience of biologists who must understand the structural details of many phylogenetic, or evolutionary, trees. Our tool is also useful in many other application domains where tree comparison is needed, ranging from network management to call graph optimization to genealogy.

This paper describes new interaction techniques for direct pen-based interaction on the Interactive Mural, a large (6′x3.5′) high resolution (64 dpi) display. They have been tested in a digital brainstorming tool that has been used by groups of professional product designers. Our "interactive wall" metaphor for interaction has been guided by several goals: to support both free-hand sketching and high-resolution materials, such as images, 3D models and GUI application windows; to present a visual appearance that does not clutter the content with control devices; and to support fluid interaction, which minimizes the amount of attention demanded and interruption due to the mechanics of the interface. We have adapted and extended techniques that were developed for electronic whiteboards and generalized the use of the FlowMenu to execute a wide variety of actions in a single pen stroke, While these techniques were designed for a brainstorming tool, they are very general and can be used in a wide variety of application domains using interactive surfaces.

We present a new kind of marking menu that was developed for use with a pen device on display surfaces such as large, high resolution, wall-mounted displays. It integrates capabilities of previously separate mechanisms such as marking menus and Quikwriting, and facilitates the entry of multiple commands. While using this menu, the pen never has to leave the active surface so that consecutive menu selections, data entry (text and parameters) and direct manipulation tasks can be integrated fluidly.