Human Action Recognition (HAR) plays a vital role in manufacturing assembly tasks, addressing key areas such as worker safety, operational support, production optimization, employee training, and facilitating human-robot collaboration. This paper introduces a skeleton-based action recognition approach based on a CNN deep neural network architecture. Joint-to-joint distances are used to represent human movements during assembly tasks, enabling the model to capture intricate motion patterns. The primary focus of this work is on structuring the input data in various ways to analyze how these variations influence the network performance. Studying the spatial configurations of input data for human action recognition in an assembly task is an insightful and challenging research topic. In assembly tasks, the high similarity between actions and the operator-specific execution variations make distinguishing actions more complex. This work investigates how the arrangement of input data impacts model accuracy. In particular, two input data configurations are analyzed: onechannel and multi-channel types. The assembly actions are classified using a CNN-based architecture. So, the different data configurations directly influence the type of CNN applied, which can be 2D or 3D. The proposed approach is evaluated on the publicly available HA4M dataset. The obtained results showed that the proposed data structure greatly influences the model performance measurement.

With Industry 5.0, operators’ physical and cognitive behavior is crucial in any field, particularly manufacturing and production lines. To this end, the need to monitor humans performing specific tasks when working alongside robotic systems has grown further. Guaranteeing the well-being of operators sharing a workspace with industrial robots can drastically reduce risky and harmful situations for the operators while leading the robot to adapt to humans fully. For this purpose, monitoring systems can be very helpful in studying the most suitable deep learning methodologies to obtain information from the movements of the individual operator performing a specific task. Therefore, action segmentation can be fundamental to establishing a new and safe way of communication among operators and robots. In this work, a system for segmenting actions performed by operators assembling an industrial object is developed. The public HA4M dataset has been used to train and test temporal action segmentation models by using MS-TCN++ architecture. Highly discriminant features have been extracted from the dataset, and different training approaches based on multimodal data, including RGB and skeletal joints in Depth and RGB resolutions, have been considered. Results show the effectiveness of the proposed system, laying the foundation for further studies for detecting the operators’ actions in the challenging context of Human-Robot Interaction and Collaboration.

This paper introduces a dataset on Human-cobot collaboration for Action Recognition in Manufacturing Assembly (HARMA3). It is a collection of RGB frames, Depth maps, RGB-to-depth-Aligned (RGB-A) frames and Skeleton data relative to actions performed by different subjects in collaboration with a cobot for building an Epicyclic Gear Train (EGT). In particular, 27 subjects executed several trials of the assembly task, which consisted of 7 actions. Data were collected in a laboratory scenario using two Microsoft®Azure Kinect cameras positioned in frontal and lateral positions. The dataset represents a good foundation for developing and testing advanced action recognition as well as action segmentation systems with far-reaching implications beyond human-cobot collaboration. Further potential applications include Computer Vision, Machine Learning, and Smart Manufacturing. Preliminary experiments for action segmentation by applying a state-of-the-art method on features extracted from RGB and skeletal data are presented in this paper, showing high-performance rates.

The widespread availability of network resources and the development of new generation devices allowed the industry to pass into the so-called Industry 4.0 era. In this fourth industrial revolution, Internet of Things (IoT) and robotic systems closely cooperate, reshaping their relations. This way of integration of robotic agents and IoT leads to the concept of Internet of Robotic Things (IoRT). Such disruptive technology opens new possibilities also in research fields other than manufacturing, such as agriculture, health, surveillance, and education. This paper reviews the key technologies of Industry 4.0 and the way they are implemented in IoRT architectures. In addition, it sheds light on the impact of the IoRT on other research fields, focusing on the main open challenges of the integration of robotic technologies into smart spaces.