For the hard-of-hearing population, communicating medical issues to doctors who do not understand sign language can be challenging. To address this problem, researchers have focused extensively on sign language gesture recognition. However, existing methods often struggle with conversion time and the accuracy of recognizing similar gestures. In this paper, we propose a sign language gesture recognition system utilizing the Yolov9 model. The system's performance was evaluated using a publicly available Kaggle dataset, achieving an impressive Mean Average Precision ([email protected]) of 99.5%. Experimental results show that the proposed method surpasses state-of-the-art techniques in both efficiency and accuracy.

Abstract The researchers have paid significant attention to dynamic images for hand gesture recognition. Dynamic images are gesture representation patterns that simultaneously capture spatial, temporal, and structural information from the video. Existing techniques to generate dynamic images provide low discriminability for the gestures, which follow the same trajectory, but in opposite directions, such as “swiping hand right” versus “swiping hand left.” Also, limited to similar gestures such as “Snap fingers” versus “Dual fingers heart.” To address these issues, we have proposed an algorithm to convert a depth video into a single dynamic image known as a linked motion image (LMI). We give the LMI to a classifier consisting of an ensemble of three modified pretrained convolutional neural networks. We conduct the experiments using a multimodal large‐scale EgoGesture dataset and The MSR Gesture 3D dataset. For the EgoGesture dataset, the proposed method achieved an accuracy of 92.91%, which is better than the state‐of‐the‐art methods. For the MSR Gesture 3D dataset, the proposed method accuracy is 100%, which outperforms the state‐of‐the‐art methods. This work also highlights the recognition accuracy and precision of each gesture. The experiments demonstrate the work's economic efficiency using a web‐based data science environment called Kaggle rather than high‐end systems like GPU.

Summary Hand gestures can be categorized based on their applications as conversational, manipulative, controlling, and communicative gestures. Hand gesture recognition is an aspect of human action recognition, which plays a notable role in communication among the deaf and dumb community. Accurate recognition and classification of hand gestures with similar postures is still a complex problem. The main objective of this work is to employ deep learning based convolutional neural networks for human action recognition. This article introduces a convolutional neural network based on VGG16 architecture with an attention approach to mitigate the issue. Incorporating an attention‐based module with the VGG16 architecture is the prime reason for potentially learning distinguishing image features by the network. Besides, the proposed robust system collectively recognizes and classifies all the available 36 hand gesture classes of the Massey database, where most previous work selected only a few distinguishable gestures. Experimental results show that the average recognition accuracy for characters with similar hand postures like “m” and “n” is better than the state‐of‐the‐art networks. Additionally, the proposed method attains about 3% higher recognition accuracy than the state‐of‐the‐art networks for all gesture classes. The efficacy of the proposed architecture has also been validated via precision, recall, and F‐score.

Summary Gesture recognition is the foremost need in building intelligent human‐computer interaction systems to solve many day‐to‐day problems and simplify human life in this digital world. The traditional machine learning (ML) algorithm tried to capture specific handcrafted features, failed miserably in some real‐world environments. Deep learning (DL) techniques have become a sensation among researchers in recent years, making the traditional ML approaches quite obsolete. However, existing reviews consider only a few datasets on which DL algorithm has been applied, and the categorization of the DL algorithms is vague in their review. This study provides the precise categorization of DL algorithms and considers around 15 gesture datasets on which these techniques have been applied. This study also provides a brief overview of the numerous challenging dataset available among the research community and insight into various challenges and limitations of a DL algorithm in vision‐based dynamic gesture recognition.