Mannequin-based simulations are widely used to train novice nurses. However, current mannequins have no dynamic facial expressions, which decreases the mannequins’ fidelity and impacts students’ learning outcomes and experience. This study proposes a projection-based AR system for overlaying dynamic facial expressions on a mannequin and implements the system in a stroke simulation. Thirty-six undergraduate nursing students participated in the study and were equally divided into the control (without the system) and experimental group (with the system). The participants’ gaze behavior, simulation performance, and subjective evaluation were measured. Results illustrated that the participants focused more on the face-animated mannequin than the traditional mannequin during the simulation. Nursing experts believed that the face-animated mannequin increased the participants’ performance in recognizing deviations but decreased their performance in seeking additional information. Moreover, the participants reported that the face-animated mannequin was more interactive and helpful for performing appropriate assessments than the traditional mannequin.

Safety practitioners widely use the lifting index (LI) to determine workers’ lifting risk but are hampered by the difficulties of estimating the lifting load without intervention or intrusive sensors. This study proposes a computer vision method for estimating the LI across varying lifting loads. The proposed method can also predict the Brog rating of perceived exertion (RPE), a measure associated with the lifting load. A controlled lifting experiment was conducted to demonstrate the approach. Thirty participants performed 2176 lifting tasks at three LI levels. These levels were controlled by varying the lifting load and fixing other task variables (e.g., the lifting distance). The proposed method combined the pose estimation (OpenPose) and the optical flow estimation (SelFlow) techniques for extracting the participants’ body motion and posture features; a facial expression recognition algorithm (OpenFace) built upon the facial action unit coding system (FACS) was used to extract the participants’ facial features. The extracted features were combined and used to develop prediction models. The best-performing model was an integration of the 1-D convolutional neural network and the long short-term memory network. It achieved an area under curve of 0.890 in classifying the LI and a root mean square of 2.264 in predicting the participants’ RPE. Critical indicators were identified by investigating the contribution of the features through interpretable machine learning techniques. In summary, this study demonstrates a nonintrusive method for lifting risk assessment and discovers behavioral indicators that predict changes in the LI and RPE due to varying loads.

Physical injuries induced by lifting are commonly reported in the workplace. Early risk detection is essential for reducing lifting injuries but requires trained observers to perform assessments manually. Machine learning and computer vision techniques have been proposed to aid ergonomists in lifting risk assessments. However, these methods may not bring the practitioners into the decision-making process and frequently not interpretable to practitioners. We conducted a user study with a proposed risk assessment system that consists of a prediction module, explanation module, and prototype user interface. The prediction module consists of a logistics regression model capable of distinguishing the injury risk levels induced by different levels of force exertion in common lifting tasks. The logistics regression model makes predictions based on explainable body motion, posture, and facial features extracted through computer vision techniques. The explanation module makes up of explainable AI techniques. Specifically, a surrogate model provides local explanations for presenting how the system makes each prediction to users. The prototype interface presents the system’s predictions and explanations. A usability study shows that the proposed system increases crowd-workers’ and domain scholars’ performance in assessing workers’ injury risks in lifting. Furthermore, the usability study also shows that the proposed system increases their confidence in the assessment tasks when the system’s evaluations agree with their subjective evaluations.