The wide range of algorithms for recognizing two-dimensional stroke gestures raises a crucial question: which recognizer should be selected depending on the gesture set and its context of use? Since the context of use expresses a category of users performing interactive tasks on a device in a physical environment, many contextual conditions are feasible. Recognizers have been tested in only a few of them, making them difficult to compare and select. A recognizer suitable for one context of use does not necessarily translate to another. Recognizer variables, such as the number of resampling points, the number of templates, and the number of users, vary from one recognizer to another, making them challenging to specify for practitioners and heterogeneous to compare for researchers. To address these issues, this article performs comparative tests of 16 recognizers (15 state-of-the-art recognizers and one for rotation invariance) in 11 gesture sets (9 reference sets and 2 for multi-device rotation invariance) against their accuracy, speed, and algorithmic complexity, in user-(in)dependent scenarios based on the variables mentioned above. In particular, the impact of two contextual dimensions, i.e., the visual impairments of end users and input devices, on accuracy is further studied. The comparative testing results in an empirically validated decision table for the researcher to select a recognizer depending on contextual conditions and a flow chart for the practitioner to be guided. We break down our design implications into two categories: gesture acquisition, recognition, and gesture set design. We also publicly release Gester , a JavaScript software for conducting comparative testing of recognizers under various contextual conditions, allowing the practitioner to test a recognizer in a project and the researcher to benchmark recognizers.

The performance of state-of-the-art stroke gesture recognizers has been extensively tested against accuracy and speed in the user-dependent or user-independent procedure on a wide variety of gesture sets. These tests were carried out under heterogeneous experimental conditions, making the results non-comparable from one source to another, and calling for a consistent assessment that ensures reproducibility of the results. When a new gesture set should be incorporated in a gesture-based user interface and/or when a new recognizer is made available, this gesture set needs to be tested with existing and new recognizers to determine which recognizer offers the best performance in a given context of use. To this end, this technical note presents Gester , a software that automates the performance testing of stroke gesture recognizers in terms of accuracy (through three recognition rates) and speed (through four execution times). It exports data for visualizing confusion matrices and confusion wheels. We elaborate on typical use cases supported by Gester , which include the two traditional testing procedures, i.e., user-dependent and user-independent scenarios, and provide two new ones, i.e., dataset-dependent and dataset-independent, to test the performance of recognizers under new conditions. Then, we illustrate a series of activities.

Finger-based gesture input becomes a major interaction modality for surface computing. Due to the low precision of the finger and the variation in gesture production, multistroke gestures are still challenging to recognize in various setups. In this paper, we present µV, a multistroke gesture recognizer that addresses the properties of articulation, rotation, scaling, and translation invariance by combining $P+'s cloud-matching for articulation invariance with !FTL's local shape distance for RST-invariance. We evaluate µV against five competitive recognizers on MMG, an existing gesture set, and on two new versions for smartphones and tablets, MMG+ and RMMG+, a randomly rotated version on both platforms. µV is significantly more accurate than its predecessors when rotation invariance is required and not significantly inferior when it is not. µV is also significantly faster than others with many samples and not significantly slower with few samples.

Arm-and-hand tracking by technological means allows gathering data that can be elaborated for determining gesture meaning. To this aim, machine learning (ML) algorithms have been mostly investigated looking for a balance between the highest recognition rate and the lowest recognition time. However, this balance comes mainly from statistical models, which are challenging to interpret. In contrast, we present$\mu C^{1}$and$\mu C^{2}$, two geometric model-based approaches to gesture recognition which support the visualization and geometrical interpretation of the recognition process. We compare$\mu C^{1}$and$\mu C^{2}$with respect to two classical ML algorithms, k-nearest neighbor (k-NN) and support vector machine (SVM), and two state-of-the-art (SotA) deep learning (DL) models, bidirectional long short-term memory (BiLSTM) and gated recurrent unit (GRU), on an experimental dataset of ten gesture classes from the Italian Sign Language (LIS), each repeated 100 times by five inexperienced non-native signers, and gathered with wearable technology (a sensory glove and inertial measurement units). As a result, we achieve a compromise between high recognition rates ($>90\%$) and low recognition times ($ < 0.1 {\mathrm{ s}}$) that is adequate for human–computer interaction. Moreover, we elaborate on the algorithms’ geometric interpretation based on geometric algebra, which supports some understanding of the recognition process.

While end users can acquire full 3D gestures with many input devices, they often capture only 3D trajectories, which are 3D uni-path, uni-stroke single-point gestures performed in thin air. Such trajectories with their $(x,y,z)$ coordinates could be interpreted as three 2D stroke gestures projected on three planes,\ie, $XY$, $YZ$, and $ZX$, thus making them admissible for established 2D stroke gesture recognizers. To investigate whether 3D trajectories could be effectively and efficiently recognized, four 2D stroke gesture recognizers, \ie, \$P, \$P+, \$Q, and Rubine, are extended to the third dimension: $\$P^3$, $\$P+^3$, $\$Q^3$, and Rubine-Sheng, an extension of Rubine for 3D with more features. Two new variations are also introduced: $\F for flexible cloud matching and FreeHandUni for uni-path recognition. Rubine3D, another extension of Rubine for 3D which projects the 3D gesture on three orthogonal planes, is also included. These seven recognizers are compared against three challenging datasets containing 3D trajectories, \ie, SHREC2019 and 3DTCGS, in a user-independent scenario, and 3DMadLabSD with its four domains, in both user-dependent and user-independent scenarios, with varying number of templates and sampling. Individual recognition rates and execution times per dataset and aggregated ones on all datasets show a highly significant difference of $\$P+^3$ over its competitors. The potential effects of the dataset, the number of templates, and the sampling are also studied.

A gesture elicitation study, as originally defined, consists of gathering a sample of participants in a room, instructing them to produce gestures they would use for a particular set of tasks, materialized through a representation called referent, and asking them to fill in a series of tests, questionnaires, and feedback forms. Until now, this procedure is conducted manually in a single, physical, and synchronous setup. To relax the constraints imposed by this manual procedure and to support stakeholders in defining and conducting such studies in multiple contexts of use, this paper presents Gelicit, a cloud computing platform that supports gesture elicitation studies distributed in time and space structured into six stages: (1) define a study: a designer defines a set of tasks with their referents for eliciting gestures and specifies an experimental protocol by parameterizing its settings; (2) conduct a study: any participant receiving the invitation to join the study conducts the experiment anywhere, anytime, anyhow, by eliciting gestures and filling forms; (3) classify gestures: an experimenter classifies elicited gestures according to selected criteria and a vocabulary; (4) measure gestures: an experimenter computes gesture measures, like agreement, frequency, to understand their configuration; (5) discuss gestures: a designer discusses resulting gestures with the participants to reach a consensus; (6) export gestures: the consensus set of gestures resulting from the discussion is exported to be used with a gesture recognizer. The paper discusses Gelicit advantages and limitations with respect to three main contributions: as a conceptual model for gesture management, as a method for distributed gesture elicitation based on this model, and as a cloud computing platform supporting this distributed elicitation. We illustrate Gelicit through a study for eliciting 2D gestures executing Internet of Things tasks on a smartphone.