Sketch recognition has the potential to be an important input method for computers in the coming years, particularly for STEM (science, technology, engineering, and math) education. However, designing and building an accurate and sophisticated sketch recognition system is a time-consuming and daunting task. Since sketch recognition mistakes are still common, it is important to understand how users perceive and tolerate recognition errors and other user interface elements with these imperfect systems. In order to solve this problem, we developed a Wizard of Oz sketch recognition tool, the WOZ Recognizer, that supports controlled recognition accuracy, multiple recognition modes, and multiple sketching domains for performing controlled experiments. We present the design of the WOZ Recognizer and our process for representing recognition domains using graphs and symbol alphabets. In addition, we discuss how sketches are altered, how to control the WOZ Recognizer, and how users interact with it. Finally, we present an expert user case study that examines the WOZ Recognizer’s usability.

ABSTRACT Code Bubbles is an attempt to redefine the user interface for an integrated programming environment. As it represents a whole new user interface, implementing it as a plug‐in is inherently difficult. We get around this difficulty by combining two different plug‐in architectures, a standard one based on registrations and callbacks and a message‐based one that puts the plug‐in at arm's length and defines a narrower two‐way interface. This paper describes both how we have implemented Code Bubbles as a plug‐in to Eclipse and how Code Bubbles itself is implemented as a set of plug‐ins representing the different aspects of the environment, using both traditional and message‐based plug‐in architectures as appropriate. It also shows how the resultant architecture is flexible enough to support collaboration, different back ends, and a cloud‐based environment. Copyright © 2013 John Wiley & Sons, Ltd.

Our original work on the Code Bubbles environment demonstrated that a working-set based framework for software development showed promise. We have spent the past several years extending the underlying concepts into a fully-functional system. In our demonstration, we will show the current Code Bubbles environment for Java, how it works, how it can be used, and why we prefer it over more traditional programming environments. We will also show how we have extended the framework to enhance software development tasks such as complex debugging, testing, and collaboration. This paper describes the features we will demonstrate.

In recent years the popularity of music and rhythm-based games has experienced tremendous growth. However almost all of these games require custom hardware to be used as input devices, and these devices control only one or two similar instruments. In this paper we describe One Man Band, a prototype video game for musical expression that uses novel 3D spatial interaction techniques using accelerometer-based motion controllers. One Man Band provides users with 3D gestural interfaces to control both the timing and sound of the music played, with both single and collaborative player modes. We further investigate the ability to detect different musical gestures without explicit selection of mode, giving the user the ability to seamlessly transition between instrument types with a single input device. A formal user study is then presented comparing the musical interface of One Man Band to that of Nintendo's Wii Music. Our results indicate that users generally preferred the interface of One Man Band over that of Wii Music. We also found that users desire to express their own ideas and have explicit control of the melodies created in music-based video games.

In recent years the popularity of music and rhythm-based games has experienced tremendous growth. However, almost all of these games require custom hardware to be used as input devices, and these devices control only one or two similar instruments. In this paper we describe One Man Band, a prototype video game for musical expression that uses novel 3D spatial interaction techniques using accelerometer-based motion controllers. One Man Band provides users with 3D gestural interfaces to control both the timing and sound of the music played, with both single and collaborative player modes. We further investigate the ability to detect different musical gestures without explicit selection of mode, giving the user the ability to seamlessly transition between instrument types with a single input device.