Training a real-time gesture recognition model heavily relies on annotated data. However, manual data annotation is costly and demands substantial human effort. In order to address this challenge, we propose a framework that can automatically annotate gesture classes and identify their temporal ranges. Our framework consists of two key components: (1) a novel annotation model that leverages the Connectionist Temporal Classification (CTC) loss, and (2) a semi-supervised learning pipeline that enables the model to improve its performance by training on its own predictions, known as pseudo labels. These high-quality pseudo labels can also be used to enhance the accuracy of other downstream gesture recognition models. To evaluate our framework, we conducted experiments using two publicly available gesture datasets. Our ablation study demonstrates that our annotation model design surpasses the baseline in terms of both gesture classification accuracy (3–4 % improvement) and localization accuracy (71-75% improvement). Additionally, we illustrate that the pseudo-labeled dataset produced from the proposed framework significantly boosts the accuracy of a pre-trained downstream gesture recognition model by 11-18%. We believe that this annotation framework has immense potential to improve the training of downstream gesture recognition models using unlabeled datasets.

Providing users with accurate gestural interfaces, such as gesture recognition based on wrist-worn devices, is a key challenge in mixed reality. However, static machine learning processes in gesture recognition assume that training and test data come from the same underlying distribution. Unfortunately, in real-world applications involving gesture recognition, such as gesture recognition based on wrist-worn devices, the data distribution may change over time. We formulate this problem of adapting recognition models to new tasks, where new data patterns emerge, as open-world gesture recognition (OWGR). We propose the use of continual learning to enable machine learning models to be adaptive to new tasks without degrading performance on previously learned tasks. However, the process of exploring parameters for questions around when, and how, to train and deploy recognition models requires resource-intensive user studies may be impractical. To address this challenge, we propose a design engineering approach that enables offline analysis on a collected large-scale dataset by systematically examining various parameters and comparing different continual learning methods. Finally, we provide design guidelines to enhance the development of an open-world wrist-worn gesture recognition process.

While research continues to investigate and improve the accuracy, fairness, and normative appropriateness of content moderation processes on large social media platforms, even the best process cannot be effective if users reject its authority as illegitimate. We present a survey experiment comparing the perceived institutional legitimacy of four popular content moderation processes. We conducted a within-subjects experiment in which we showed US Facebook users moderation decisions and randomized the description of whether those decisions were made by paid contractors, algorithms, expert panels, or juries of users. Prior work suggests that juries will have the highest perceived legitimacy due to the benefits of judicial independence and democratic representation. However, expert panels had greater perceived legitimacy than algorithms or juries. Moreover, outcome alignment -agreement with the decision - played a larger role than process in determining perceived legitimacy. These results suggest benefits to incorporating expert oversight in content moderation and underscore that any process will face legitimacy challenges derived from disagreement about outcomes.

Simulation offers many advantages when designing analog circuits. Designers can explore alternatives quickly, without added cost or risk of hardware faults. However, it is challenging to use simulation as an aid during interactive debugging of physical circuits, due to difficulties in comparing simulated analyses with hardware measurements. Designers must continually configure simulations to match the state of the physical circuit (e.g. capturing sensor inputs), and must manually rework the hardware to replicate changes or analyses performed in simulation. We propose techniques leveraging instrumentation and programmable test hardware to create a tight coupling between a physical circuit and its simulated model. Bridging these representations helps designers to compare simulated and measured behaviors, and to quickly perform analytical techniques on hardware (e.g. parameter-response analysis) that are typically cumbersome outside of simulation. We implement these techniques in a prototype and show how it aids in efficiently debugging a variety of analog circuits.

Difficulties in accessing, isolating, and iterating on the components and connections of a printed circuit board (PCB) create unique challenges in PCB debugging. Manual probing methods are slow and error prone, and even dedicated PCB testing equipment remains limited by its inability to modify the circuit during testing. We present Pinpoint, a tool that facilitates in-circuit PCB debugging through techniques such as programmatically probing signals, dynamically disconnecting components and subcircuits to test in isolation, and splicing in new elements to explore potential modifications. Pinpoint automatically instruments a PCB design and generates designs for a physical jig board that interfaces the user's PCB to our custom testing hardware and to software tools. We evaluate Pinpoint's ability to facilitate the debugging of various PCB issues by instrumenting and testing different classes of boards, as well as by characterizing its technical limitations and by soliciting feedback through a guided exploration with PCB designers.

We present Haptic Links, electro-mechanically actuated physical connections capable of rendering variable stiffness between two commodity handheld virtual reality (VR) controllers. When attached, Haptic Links can dynamically alter the forces perceived between the user's hands to support the haptic rendering of a variety of two-handed objects and interactions. They can rigidly lock controllers in an arbitrary configuration, constrain specific degrees of freedom or directions of motion, and dynamically set stiffness along a continuous range. We demonstrate and compare three prototype Haptic Links: Chain, Layer-Hinge, and Ratchet-Hinge. We then describe interaction techniques and scenarios leveraging the capabilities of each. Our user evaluation results confirm that users can perceive many two-handed objects or interactions as more realistic with Haptic Links than with typical unlinked VR controllers.

We present three new novel haptic controllers that render shape force feedback during interaction. 1) CLAW is a multi-purpose controller that renders tactile forces for common hand interactions, such as grasping, touching, and triggering grasped objects. 2) Haptic Revolver is a general-purpose handheld VR controller that renders touch contact with virtual surfaces, motion shear along a surface, textures, and shapes using interchangeable wheels. 3) Haptic Links haptic render shape feedback between two controllers using variable-stiffness locking mechanisms to provide force feedback for grasping and interacting with two-handed objects such as wind instruments, steering wheels, handle bars, or bow and arrow.

Haptic interfaces are ideal in situations where visual/auditory attention is impossible, unsafe, or socially unacceptable. However, conventional (vibrotactile) wearable interfaces often possess a limited bandwidth for expressing information. We explore a novel form of tactile stimulation through brushing, and demonstrate BrushTouch, a wearable prototype for brushing haptics. We also present schemes for conveying information such as time and direction through multi-tactor wrist-worn haptic interfaces. To evaluate BrushTouch, two user studies were run, comparing it to a conventional vibrotactile wristband across a number of tasks in both lab and mobile conditions. We show that for certain cues brushing can be more accurately recognized than vibration, enabling more effective spatial schemes for presenting information through haptic means. We then show that BrushTouch is capable of greater information transfer using such cues. We believe that brushing, as with other non-vibrotactile haptic techniques, merits further investigation as potential vehicles for richer haptic feedback.

Currently, virtual (i.e. touchscreen) controls are dynamic, but lack the advantageous tactile feedback of physical controls. Similarly, devices may also have dedicated physical controls, but they lack the flexibility to adapt for different contexts and applications. On mobile and wearable devices in particular, space constraints further limit our input and output capabilities. We propose utilizing reconfigurable tactile elements around the edge of a mobile device to enable dynamic physical controls and feedback. These tactile elements can be used for physical touch input and output, and can reposition according to the application both around the edge of and hidden within the device. We present shiftIO, two implementations of such a system which actuate physical controls around the edge of a mobile device using magnetic locomotion. One version utilizes PCB-manufactured electromagnetic coils, and the other uses switchable permanent magnets. We perform a technical evaluation of these prototypes and compare their advantages in various applications. Finally, we demonstrate several mobile applications which leverage shiftIO to create novel mobile interactions.

Analog circuit design is a complex, error-prone task in which the processes of gathering observations, formulating reasonable hypotheses, and manually adjusting the circuit raise significant barriers to an iterative workflow. We present Scanalog, a tool built on programmable analog hardware that enables users to rapidly explore different circuit designs using direct manipulation, and receive immediate feedback on the resulting behaviors without manual assembly, calculation, or probing. Users can interactively tune modular signal transformations on hardware with real inputs, while observing real-time changes at all points in the circuit. They can create custom unit tests and assertions to detect potential issues. We describe three interactive applications demonstrating the expressive potential of Scanalog. In an informal evaluation, users successfully conditioned analog sensors and described Scanalog as both enjoyable and easy to use.