Correct exercise form is essential to prevent injuries and optimize performance. This paper introduces LiftMate, a computer vision based system for gym exercise posture classification in biceps curl, plank, and squat using machine learning and deep learning models. Our dataset consists of self-recorded and public videos, with MediaPipe employed for real-time keypoint detection. We extract spatial features (joint angles and distances) and evaluate multiple classification models. Experimental results demonstrate that deep models achieve near-perfect accuracy. In particular, our 3-layer neural network attained almost 100% classification accuracy on all three exercises, significantly outperforming traditional classifiers (e.g. logistic regression). These findings confirm the potential for fully automated real-time posture assessment in fitness. While our models achieved very high accuracy, limitations such as dataset diversity and size suggest avenues for future work.

In this study, we introduce UJ-AQA-CricketVision, a dataset comprising 8,540 video clips of cricket strokes, each annotated with detailed phase breakdowns. We develop a novel multi-variate approach for Action Quality Assessment (AQA) at a body level that leverages an Autoencoder for extracting sophisticated feature representations from video frames and pose estimated keypoints. These features are subsequently utilised by a multilayer perceptron regression-based model to accurately predict the quality of cricket actions in terms of their head, shoulder, hands, hips, and feet. Our approach is benchmarked against contemporary state-of-the-art AQA methods and achieves a Spearman Rank Correlation score of 0.84. The performance highlights the significance of integrating pose keypoint and frame data for the nuanced analysis of short and complex action sequences in sports such as cricket. This work aims to foster the development of accurate Action Quality Assessment methods on Cricket Video data. The dataset can be found here: https://github.com/dvanderhaar/uj-aqa-cricketvision.

Systems dedicated to the Action Quality Assessment (AQA) have seen a notable surge in interest from both scholars and industry experts. Such systems seek to provide an objective measure of the quality of athletes’ physical movements, offering perspectives that were once exclusive to skilled human evaluators. Our research builds upon the observation that previous studies have primarily considered spatio-temporal features and pose estimation keypoints as distinct elements within action analysis. We introduce an innovative two-stream methodology that merges these representations of an action. Our approach leverages the combined capabilities of various techniques: utilising Inflated 3D ConvNet (I3D) for the extraction of spatial–temporal characteristics from video content, employing OpenPose for detailed pose estimation keypoints that furnish an Autoencoder (AE) with intricate information to concentrate on key aspects of action, and employing Long Short Term Memory (LSTM) networks. The integration of the proposed two-stream methodology allows for a more comprehensive analysis of athletic movements. By capturing both spatial and temporal dynamics together, we can better understand the nuances of an action. Traditional methods often treat these aspects separately, leading to a fragmented understanding. In addition, incorporating detailed pose estimation through OpenPose allows the Autoencoder to focus on key aspects of the action. This targeted focus ensures that the evaluation is not only about how movements look in a general sense but also how they align with optimal performance criteria. By combining the representations, we enhance the model’s ability to recognise and evaluate complex movements more accurately. Furthermore, we propose a new multi-variate scoring system designed to assess action quality based on the scores from individual judges. Multi-variate scoring introduces a richer dataset for assessing action quality, enabling more sophisticated analyses and decision-making. Multi-variate scoring can better highlight discrepancies or consensus among judges, leading to more reliable assessments. The method has an average Spearman Rank Correlation of 97.35%, which outperforms current state-of-the-art methods and underscores the effectiveness of merging spatio-temporal and pose estimation keypoints into a unified action representation. Finally, training the model to predict the scores assigned by each judge reveals additional advantages. In sports involving multiple scoring criteria, the proposed approach enables the extraction of more detailed insights, marking a significant advancement that allows for more precise and detailed predictions of scores. • Introducing a 2-Stream Inflated 3D ConvNet-Autoencoder-Long Short Term:I3D-AE-LSTM . • Reviewing MTL-AQA related works and showcasing the I3D-AE-LSTM model performance. • Introducing a multi-variate scoring mechanism to score actions for individual scores.