We present Machete, a straightforward segmenter one can use to isolate custom gestures in continuous input. Machete uses traditional continuous dynamic programming with a novel dissimilarity measure to align incoming data with gesture class templates in real time. Advantages of Machete over alternative techniques is that our segmenter is computationally efficient, accurate, device-agnostic, and works with a single training sample. We demonstrate Machete’s effectiveness through an extensive evaluation using four new high-activity datasets that combine puppeteering, direct manipulation, and gestures. We find that Machete outperforms three alternative techniques in segmentation accuracy and latency, making Machete the most performant segmenter. We further show that when combined with a custom gesture recognizer, Machete is the only option that achieves both high recognition accuracy and low latency in a video game application.

Those who design gesture recognizers and user interfaces often use data collection applications that enable users to comfortably produce gesture training samples. In contrast, games present unique contexts that impact cognitive load and have the potential to elicit rapid gesticulations as players react to dynamic conditions, which can result in high gesture form variability. However, the extent to which these gestures differ is presently unknown. To this end, we developed two games with unique mechanics, Follow the Leader (FTL) and Sleepy Town, as well as a standard data collection application. We collected gesture samples from 18 participants across all conditions for gestures of varying complexity, and through an analysis using relative, global, and distribution coverage measures, we confirm significant differences between conditions. We discuss the implications of our findings, and show that our FTL design is closer to being an ecologically valid data collection protocol with low implementation complexity.

While we are in the midst of a renaissance of interest in augmented reality (AR), there remain a small number of application domains which have seen significant development. Education is a domain that often drives innovation with emerging technologies. One particular subject which benefits from additional visualization capabilities is physics. In this paper, we present the results of a series of interviews with secondary school teachers about their experience with AR and the features which would be most beneficial to them from a pedagogical perspective. To gather meaningful information, a prototype application was developed and presented to the teachers. Based on the feedback collected from the teachers, we present a set of design recommendations for AR physics education tools, as well as other useful collects comments.

Despite decades of research, there is yet no general rapid prototyping recognizer for dynamic gestures that can be trained with few samples, work with continuous data, and achieve high accuracy that is also modality-agnostic. To begin to solve this problem, we describe a small suite of accessible techniques that we collectively refer to as the Jackknife gesture recognizer. Our dynamic time warping based approach for both segmented and continuous data is designed to be a robust, go-to method for gesture recognition across a variety of modalities using only limited training samples. We evaluate pen and touch, Wii Remote, Kinect, Leap Motion, and sound-sensed gesture datasets as well as conduct tests with continuous data. Across all scenarios we show that our approach is able to achieve high accuracy, suggesting that Jackknife is a capable recognizer and good first choice for many endeavors.

We explore the benefits of intelligent prototype selection for $-family recognizers. Currently, the state of the art is to randomly select a subset of prototypes from a dataset without any processing. This results in reduced computation time for the recognizer, but also increases error rates. We propose applying optimization algorithms, specifically random mutation hill climb and a genetic algorithm, to search for reduced sets of prototypes that minimize recognition error. After an evaluation, we found that error rates could be reduced compared to random selection and rapidly approached the baseline accuracies for a number of different $-family recognizers.

Training gesture recognizers with synthetic data generated from real gestures is a well known and powerful technique that can significantly improve recognition accuracy. In this paper we introduce a novel technique called gesture path stochastic resampling (GPSR) that is computationally efficient, has minimal coding overhead, and yet despite its simplicity is able to achieve higher accuracy than competitive, state-of-the-art approaches. GPSR generates synthetic samples by lengthening and shortening gesture subpaths within a given sample to produce realistic variations of the input via a process of nonuniform resampling. As such, GPSR is an appropriate rapid prototyping technique where ease of use, understandability, and efficiency are key. Further, through an extensive evaluation, we show that accuracy significantly improves when gesture recognizers are trained with GPSR synthetic samples. In some cases, mean recognition errors are reduced by more than 70%, and in most cases, GPSR outperforms two other evaluated state-of-the-art methods.

We explore the capabilities of head tracking combined with head mounted displays (HMD) as an input modality for robot navigation. We use a Parrot AR Drone to test five techniques which include metaphors for plane-like banking control, car-like turning control and virtual reality-inspired translation and rotation schemes which we compare with a more traditional game controller interface. We conducted a user study to observe the effectiveness of each of the interfaces we developed in navigating through a number of archways in an indoor course. We examine a number of qualitative and quantitative metrics to determine performance and preference among each metaphor. Our results show an appreciation for head rotation based controls over other head gesture techniques, with the classic controller being preferred overall. We discuss possible shortcomings with head tracked HMDs as a primary input method as well as propose improved metaphors that alleviate some of these drawbacks.