Post-stroke rehabilitation of upper extremity (UE) motor function is essential. Despite the widespread use of UE rehabilitation in clinical settings, assessing the success of these treatments is challenging. Methods for evaluating UE motor function include task performance under clinical supervision, patient self-reports, and data analysis from wearable devices equipped with accelerometers and gyroscopes. In prior research, we demonstrated that machine learning and deep learning models using data from a single wrist-worn accelerometer sensor could accurately differentiate between functional and non-functional UE movements in stroke patients. To overcome the limitations of single wrist-worn accelerometer sensors – challenges in capturing the full context of functional movements – this study presents a new deep learning (DL) based framework designed to classify functional and non-functional arm movements in videos captured from a conventional camera. The system is entirely automated and comprises two DL networks. The first network performs human pose estimation, extracting 2D pose key points of the paretic arm(s) and torso on 2-second sequences of the frames. The second network then uses these 2D pose landmarks to classify the movement sequence as functional or non-functional. This system offers two key benefits for rehabilitation. First, it can automatically generate initial annotations for video frames, significantly reducing the time needed for manual labeling. Second, analyzing the effectiveness of UE rehabilitation through third-person videos allows for objective outcome measurement of UE treatments in stroke survivors' home environments.

The rehabilitation of the upper extremity (UE) plays a pivotal role in the recovery process for stroke survivors. Despite the routine practice of UE rehabilitation, assessing its effectiveness poses a significant challenge. Evaluation methods vary, ranging from clinical assessments of UE motor skills and patient feedback to the utilization of wearable technology equipped with accelerometer or gyro sensors. In this study, we employed deep learning approaches such as Multi-Layer Perceptron, Long Short-term Memory, and Gated Recurrent Units, in conjunction with data from accelerometry sensors embedded in wrist-wearable Inertial Measurement Units. Our findings demonstrate the successful differentiation between functional and non-functional UE movements in individuals recovering from strokes with 90% and 74% of classification accuracy for intra-subject and inter-subject models, respectively. The methodologies employed in this study exhibit consistency and reliability, providing precise outcomes for individual and acceptable outcomes for cross-participant evaluations. Notably, our advanced models directly utilized raw accelerometry data instead of manual extracted features that was often used in traditional machine learning algorithms. These innovative techniques offer cost-effective and adaptable tools for monitoring UE functionality in real-world settings. The insights derived from this study hold the potential to be transformative in tailoring rehabilitation strategies for adults affected by strokes.

Upper Extremity (UE) rehabilitation is often needed post-stroke. The main goal of UE treatment in stroke survivors is to increase the use of the affected UE in the home and community. However, the effectiveness of UE treatments are difficult to quantify because no objective evaluation of UE use exists. In practice, a clinician rates the patient's ability to perform specific motor tasks associated with functional use in a clinic or the patient self-reports the amount or quality of arm movement for a standard set of activities. Both methods do not objectively measure the performance of the affected UE in the home or community environment, and there is growing evidence that motor performance in the laboratory is a poor proxy for the actual amount of UE use. Using a single wrist-worn sensor (i.e., accelerometry data) and machine learning, we have reported that it is possible to separate UE functional use from nonfunctional movement after stroke. Specifically, we reported that we correctly classified sensor data with an average of 94.80% in controls and 88.38% in stroke subjects in intra-subject test trials, and 91.53% for controls and 70.18% in stroke subjects in inter-subject test trials. In this paper, we employed feature selection techniques and explored different machine learning methods to improve the classification accuracy. Our enhanced methods are robust and reliable, and work in both intra-subject and inter-subject training and testing. Our result showed better accuracy in stroke patients than previously reported with the same dataset. The enhanced models reached an average of 96% accuracy in control subjects and 94% in stroke subjects for intra-subject trials, and an average of 90% accuracy in control subjects and 83% in stroke subjects for the inter-subject trials. The proposed methods provide an inexpensive and feasible way to quantify the UE functional use in home and community. This information can provide guidance for clinical practice in the rehabilitative care of adults recovering from stroke.