Summary Precisely detecting bare hands and recognizing the characters are two major stages in gesticulated character recognition systems. It is very challenging to implement them in an uncontrolled environment. Additional variations, particularly (i) background feature domination (BFD) effect and motion blur in detection, (ii) gesturing style, pattern, and case sensitivity in recognition, make the system more complex. To address these challenges, a gesticulated character recognition (GCR‐Net) model is designed. To detect the bare hand precisely, a pixel‐wise segmentation approach, HandSNet, is presented, which is able to overcome the BFD effect. To handle the motion blur in the frames, a tracking module comprised of a point‐tracker and Kalman filter is applied. To reduce the computational time, a mini‐SqueezeNet network is designed, which is used in HandSNet and recognition models as the backend network. It has 0.39 million parameters only. Four separate deep convolutional neural networks (DCNNs) are connected with the network section module at the recognition end. This network selection module activates one DCNN at a time to recognize the gesticulated characters accurately. The proposed GCR‐Net reduces the complexity between similar characters and provides a high precision rate compared to the existing approaches.

Abstract Detection and tracking are the vital stages to form the gesture trajectory in gesture recognition. It becomes more challenging when the variations in illumination, pose, position, occlusion, scale, speed, blurring effect and complex environment are introduced. Additionally, the background feature domination effect affects the existing deep learning models. A semantic segmentation model is implemented in this work to detect the bare hand to overcome these challenges. A pre‐trained network VGG‐16 is utilized by training with the proposed NITS S‐Net database. Evaluation of the SegNet model is done on EgoHands, Oxford and OUHands databases. To track the bare hand, a SegNet‐based detection and tracking approach is proposed using Kalman filter and point‐tracker. This model achieves 97.01% accuracy (a relative improvement of ~8% from the baseline models) at 0.068 s per frame computational time on NITS hand gesture database VIIIB. The gesticulated characters, that is, alphabets, numbers, operators, special characters, are gesticulated without any constraints on the pattern/strokes. To recognize these 95 multi‐stroke gestures, a deep convolutional neural network (DCNN) is presented using AlexNet. The DCNN model achieves 97.60% (a relative improvement of ~14% from the baseline models) accuracy on the NITS hand gesture database VIIIB merged. Evaluation of the handwritten EMNIST merged (balanced) database resulted in average recognition accuracy of 91.60%.

Recognition of the characters has gained much attention due to its potential applications like document analysis, license plate detection, house number detection, virtual text entry system, etc., in pattern recognition. However, it is very challenging to recognize the characters under the variations in pattern, style, translation, scale, rotation. This work develops a computationally efficient deep learning model to recognize handwritten, printable, and gesticulated characters. For gesture, the NITS gesticulated database having 60 characters (10 digits, 26 English uppercase alphabets, 4 operations, 18 special symbols) is proposed with the variation in pattern, style, scale in this work. To evaluate the ability and robustness of the proposed model, the handwritten characters (MNIST, EMNIST), printable characters (SVHN, Chars74) databases are considered. This network achieves 94.55%, 89.54%, 87.33, and 93.90% recognition accuracy on NITS gesticulated, EMNIST merge (balanced), SVHN, and Chars74 databases.

Abstract Gesture formation, a pre‐processing step, has its importance when variations in patterns, scale, and speed come into play. Self co‐articulations are intentional movements performed by an individual to complete a gesture, whose presence in the trajectory alters its original meaning. For recognition, most researchers have directly used the trajectory formed along with these self co‐articulated strokes, with a few removing it using visible trait‐like velocity. Usage of velocity has shortcomings as gesturing in air differs from gesturing over a solid surface; hence, we propose a gesture formation model, which incorporates global and local measures to remove these self co‐articulations. The global measure uses Euclidean distance, instantaneous velocity, and polarity calculated from the complete gesture, while the local measure segments the gesture into stroke‐level segments by using the minimum–maximum‐polarity algorithm and applies the selective bypass rules. The proposed model, when experimented on gestures patterns with premeditated speed variation, has a mean error rate of 0.0069 and 7.40% self co‐articulations;individuals’ natural gesticulation has a mean error rate of 0.0371 and 12.07% self co‐articulations. Experimentation on each gesture of NITS hand gesture databases showed a relative improvement of 40% (accuracy 97%) over the existing baseline models.