We present Patch-O, a novel deformable interface devised as a woven patch that enables diverse movement-based interactions adaptive to garments or on-skin wearing. Patch-O interfaces are uniquely detachable and relocatable soft actuation units that can be sewn or attached to clothing or skin at various locations. To optimize the morphing effect while preserving a slim form factor, we introduce a construction approach that integrates actuators at a structural level and varies the texture and stiffness of the woven substrate locally. We implement three basic actuation primitives, including bending, expanding, and shrinking, and experiment with aggregation parameters to exhaustively extend the design space. A final workshop study inviting textile practitioners to create personalized designs of Patch-O provides insights into the expressiveness of the approach for wearable interactions. We conclude with three applications inspired by users’ designs and showcase the aesthetic and functional usages enabled by the deformable woven patches.

Haptic feedback significantly enhances virtual experiences. However, supporting haptics currently requires modifying the codebase, making it impractical to add haptics to popular, high-quality experiences such as best selling games, which are typically closed-source. We present HapticSeer, a multi-channel, black-box, platform-agnostic approach to detecting game events for real-time haptic feedback. The approach is based on two key insights: 1) all games have 3 types of data streams: video, audio, and controller I/O, that can be analyzed in real-time to detect game events, and 2) a small number of user interface design patterns are reused across most games, so that event detectors can be reused effectively. We developed an open-source HapticSeer framework and implemented several real-time event detectors for commercial PC and VR games. We validated system correctness and real-time performance, and discuss feedback from several haptics developers that used the HapticSeer framework to integrate research and commercial haptic devices.

JetController is a novel haptic technology capable of supporting high-speed and persistent 3-DoF ungrounded force feedback. It uses high-speed pneumatic solenoid valves to modulate compressed air to achieve 20-50Hz of full impulses at 4.0-1.0N, and combines multiple air propulsion jets to generate 3-DoF force feedback. Compared to propeller-based approaches, JetController supports 10-30 times faster impulse frequency, and its handheld device is significantly lighter and more compact. JetController supports a wide range of haptic events in games and VR experiences, from firing automatic weapons in games like Halo (15Hz) to slicing fruits in Fruit Ninja (up to 45Hz). To evaluate JetController, we integrated our prototype with two popular VR games, Half-life: Alyx and Beat Saber, to support a variety of 3D interactions. Study results showed that JetController significantly improved realism, enjoyment, and overall experience compared to commercial vibrating controllers, and was preferred by most participants.

Zippers are common in a wide variety of objects that we use daily. This work investigates how we can take advantage of such common daily activities to support seamless interaction with technology. We look beyond simple zipper-sliding interactions explored previously to determine how to weave foreground and background interactions into a vocabulary of natural usage patterns. We begin by conducting two user studies to understand how people typically interact with zippers. The findings identify several opportunities for zipper input and sensing, which inform the design of Zippro, a self-contained prototype zipper slider, which we evaluate with a standard jacket zipper. We conclude by demonstrating several applications that make use of the identified foreground and background input methods.

We propose a new sensing technique for one-dimensional touch input workable on an interactive thread of less than 0.4 mm thick. Our technique locates up to two touches using impedance sensing with a spacing resolution unachievable by the existing methods. Our approach is also unique in that it locates a touch based on a mathematical model describing the change in thread impedance in relation to the touch locations. This allows the system to be easily calibrated by the user touching a known location(s) on the thread. The system can thus quickly adapt to various environmental settings and users. A system evaluation showed that our system could track the slide motion of a finger with an average error distance of 6.13 mm and 4.16 mm using one and five touches for calibration, respectively. The system could also distinguish between single touch and two concurrent touches with an accuracy of 99% and could track two concurrent touches with an average error distance of 8.55 mm. We demonstrate new interactions enabled by our sensing approach in several unique applications.

Modern touchscreen devices have recently introduced customizable touchscreen settings to improve accessibility for users with motor impairments. For example, iOS 10 introduced the following four Touch Accommodation settings: 1) Hold Duration, 2) Ignore Repeat, 3) Tap Assistance, and 4) Tap Assistance Gesture Delay. These four independent settings lead to a total of more than 1 million possible configurations, making it impractical to manually determine the optimal settings. We present PersonalTouch, which collects and analyzes touchscreen gestures performed by individual users, and recommends personalized, optimal touchscreen accessibility settings. Results from our user study show that PersonalTouch significantly improves touch input success rate for users with motor impairments (20.2%, N=12, p=.00054) and for users without motor impairments (1.28%, N=12, p=.032).

Augmented Reality (AR) provides real-time information by superimposing virtual information onto users' view of the real world. Our work is the first to explore how peripheral vision, instead of central vision, can be used to read text on AR and smart glasses. We present Peritext, a multiword reading interface using rapid serial visual presentation (RSVP). This enables users to observe the real world using central vision, while using peripheral vision to read virtual information. We first conducted a lab-based study to determine the effect of different text transformation by comparing reading efficiency among 3 capitalization schemes, 2 font faces, 2 text animation methods, and 3 different numbers of words for RSVP paradigm. We found that title case capitalization, sans-serif font and word-wise typewriter animation with multiword RSVP display resulted in better reading efficiency, which together formed our Peritext design. Another lab-based study followed, investigating the performance of the Peritext against control text, and the results showed significant better performance. Finally, we conducted a field study to collect user feedback while using Peritext in real-world walking scenarios, and all users reported a preference of 5° eccentricity over 8°.

Proper ergonomics improves productivity and reduces risks for injuries such as tendinosis, tension neck syndrome, and back injuries. Despite having ergonomics standards and guidelines for computer usage since the 1980s, injuries due to poor ergonomics remain widespread. We present ActiveErgo, the first active approach to improving ergonomics by combining sensing and actuation of motorized furniture. It provides automatic and personalized ergonomics of computer workspaces in accordance to the recommended ergonomics guidelines. Our prototype system uses a Microsoft Kinect sensor for skeletal sensing and monitoring to determine the ideal furniture positions for each user, then uses a combination of automatic adjustment and real-time feedback to adjust the computer monitor, desk, and chair positions. Results from our 12-person user study demonstrated that ActiveErgo significantly improves ergonomics compared to manual configuration in both speed and accuracy, and helps significantly more users to fully meet ergonomics guidelines.

Electric current and voltage are fundamental to learning, understanding, and debugging circuits. Although both can be measured using tools such as multimeters and oscilloscopes, electric current is much more difficult to measure because users have to unplug parts of a circuit and then insert the measuring tools in serial. Furthermore, users need to restore the circuits back to its original state after measurements have been taken. In practice, this cumbersome process poses a formidable barrier to knowing how current flows throughout a circuit. We present CurrentViz, a system that can sense and visualize the electric current flowing through a circuit, which helps users quickly understand otherwise invisible circuit behavior. It supports fully automatic, ubiquitous, and real-time collection of amperage information of breadboarded circuits. It also supports visualization of the amperage data on a circuit schematic to provide an intuitive view into the current state of a circuit.

The rise of Maker communities and open-source electronic prototyping platforms have made electronic circuit projects increasingly popular around the world. Although there are software tools that support the debugging and sharing of circuits, they require users to manually create the virtual circuits in software, which can be time-consuming and error-prone. We present CircuitSense, a system that automatically recognizes the wires and electronic components placed on breadboards. It uses a combination of passive sensing and active probing to detect and generate the corresponding circuit representation in software in real-time. CircuitSense bridges the gap between the physical and virtual representations of circuits. It enables users to interactively construct and experiment with physical circuits while gaining the benefits of using software tools. It also dramatically simplifies the sharing of circuit designs with online communities.

Current virtual reality (VR) and augmented reality (AR) devices rely on handheld devices, hand gestures, head tracking, and voice input. This paper presents an initial exploration of using eye expressions as hands-free input modality for head-mounted AR/VR devices (HMDs). We consulted interaction designers and ophthalmologists and enumerated 12 eye expressions, and conducted a 12-person user study to better understand users' ability to perform them as well as preferences.

Augmented Reality (AR) provides real-time information by super-imposing virtual information onto users' view of the real world. Our work is the first to explore how peripheral vision, instead of central vision, can be used to read text on AR and smart glasses. We present PeriText+, a multiword reading interface using rapid serial visual presentation (RSVP). This enables users to observe the real world using central vision, while using peripheral vision to read text. We conducted a lab study to compare reading efficiency among 40 different conditions of text transformation. We also conducted a field study to evaluate the information transfer while using PeriText+ in a real-world walking scenario.