Pointing transfer functions define the mapping between input devices and onscreen cursor movement. Despite being used by millions daily, only marginal improvements in pointing performance have been achieved by tuning transfer functions since the introduction of acceleration-based gains. We present TFTune, a reinforcement learning-based approach for improving pointing by automatically tuning personalized transfer functions. We show that TFTune-generated functions outperform operating system defaults, improving movement times by 7% on macOS when using a trackpad (7 minutes of tuning) and 8% on participants’ personal Windows computers with hardware (i.e. mice and monitors) of varying characteristics (after just 1 minute of tuning). Further, we show that TFTune generalizes beyond traditional pointing devices, providing 16% improvement for a muscle-computer interface (2 minutes of tuning). TFTune demonstrates an initial approach for scalable and meaningful performance improvements in input–output mappings, opening a new direction for exploring the use of machine learning for improving fundamental computer inputs.

Electromyography (EMG) has been explored as an HCI input modality following a long history of success for prosthesis control. While EMG has the potential to address a range of hands-free interaction needs, it has yet to be widely accepted outside of prosthetics due to a perceived lack of robustness and intuitiveness. To understand how EMG input systems can be better designed, we sampled the ACM digital library to identify limitations in the approaches taken. Leveraging these works in combination with our research group’s extensive interdisciplinary experience in this field, four themes emerged (1) interaction design, (2) model design, (3) system evaluation, and (4) reproducibility. Using these themes, we provide a step-by-step framework for designing EMG-based input systems to strengthen the foundation on which EMG-based interactions are built. Additionally, we provide a call-to-action for researchers to unlock the hidden potential of EMG as a widely applicable and highly usable input modality.

While there has been renewed interest in the use of myoelectric control for general-purpose applications, the burden of training and maintaining robust models still limits its real-world viability. Online unsupervised adaptation has been proposed to solve this issue by updating the model using predicted pseudo-labels in real time during regular device use. Until now, however, these unsupervised strategies have been limited as they rely on the very classifier outputs they are adapting, making them ill-suited when there is a drastic shift in the input space (e.g., after donning and doffing a device) or there is insufficient training data. In such situations, leveraging context (i.e., task-specific information that can help understand or assess a circumstance) could provide additional guidance for adaptation and improve its robustness. Although difficult to extract in traditional prosthesis control use cases without additional sensors, context may be more readily available in other general-purpose applications, such as in human-computer interaction. In this study, we explore leveraging context, both positive (i.e., reinforcing correct actions) and negative (i.e., correcting poor actions), for conditioning pseudo-label predictions within an adaptive gamified target acquisition setting. The results show that leveraging this additional con-text significantly outperforms the current state-of-the-art high-confidence unsupervised adaptation (p<0.05) using both offline and online performance metrics. This pilot work contributes novel findings and contextual approaches that do not rely on additional sensors, and thus outlines a promising direction of study for myoelectric control as a reliable and effective interaction technique.