Human activity recognition (HAR) in industrial domains is important for workflow optimization, throughput estimation, and bottleneck detection. Skeleton-based models achieve high HAR accuracy by exploiting rich spatial and temporal cues, but they are difficult to deploy in industrial sites due to occlusions, camera placement, and privacy concerns. IMU sensors, especially smartwatches, are practical for deployment but lack spatial awareness, resulting in weaker performance. This work aims to enable robust HAR using only a wrist-worn IMU by distilling knowledge from richer modalities. Knowledge distillation allows transferring information from a skeleton teacher to a single-IMU student, but the large modality gap has limited the success of prior teacher–student approaches. To address this issue, we propose a teacher-assistant-student (TAS) learning framework, in which a multi-IMU assistant model bridges the skeleton-based teacher and the single-IMU student. To support TAS, we develop the following techniques: (i) Dense temporal Contrastive Learning, aligning structural representations of skeleton and IMU segments; (ii) Spatial Relationship Learning, guiding models to capture spatial priors from skeleton data; and (iii) Temporal Attention Transfer, distilling attention patterns for key atomic actions. We further boost the robustness to behavioral variation with motion-guided IMU data diversification using physics-based simulation. Experiments on industrial HAR sensor data show that our framework consistently improves single-IMU recognition across diverse operational scenarios, highlighting its potential for practical deployment.

Complex work activity recognition based on wearable sensors is crucial for streamlining work processes in industrial domains. Unlike basic activities such as walking or running, which involve simple repetitive motions, a complex work activity consists of discrete atomic actions such as an action of spreading a shipping label or cutting tape in a packaging task. In addition, the atomic actions sometimes involve characteristic short-term sensor data patterns. In addition, these actions can be performed in different orders by different workers to achieve similar outcomes, resulting in different long-term sensor data trends for different workers. Because multilayer networks for activity recognition may learn short-term features from shallow-level layers and long-term trends from deeper layers, we propose a new transfer learning method called multilevel knowledge transfer (MLKT), which performs level-wise source selection according to trend similarity across workers in different levels. For example, for training shallow layers, highly similar workers are selected for specific short motions (e.g., pasting a shipping label), and to train the deeper layers, workers with similar cadence are selected. This method also enables the adaptive thresholding of source data selection for each layer level during network training using the proposed adaptive level-wise discerning module.

Owing to the cost of collecting labeled sensor data, self-supervised learning (SSL) methods for human activity recognition (HAR) that effectively use unlabeled data for pretraining have attracted attention. However, applying prior SSL to COMPLEX activities in real industrial settings poses challenges. Despite the consistency of work procedures, varying circumstances, such as different sizes of packages and contents in a packing process, introduce significant variability within the same activity class. In this study, we focus on sensor data corresponding to characteristic and necessary actions (sensor data motifs) in a specific activity such as a stretching packing tape action in an assembling a box activity, and propose to train a neural network in self-supervised learning so that it identifies occurrences of the characteristic actions, i.e., Motif Identification Learning (MoIL). The feature extractor in the network is subsequently employed in the downstream activity recognition task, enabling accurate recognition of activities containing these characteristic actions, even with limited labeled training data. The MoIL approach was evaluated on real-world industrial activity data, encompassing the state-of-the-art SSL tasks with an improvement of up to 23.85% under limited training labels.

This seminar introduces recent trends in sensor-based activity recognition technology. Technology to recognize human activities using sensors has been a hot topic in the field of mobile and ubiquitous computing for many years. Recent developments in deep learning and sensor technology have expanded the application of activity recognition to various domains such as industrial and natural science fields. However, because activity recognition in the new domains suffers from various real problems such as the lack of sufficient training data and complexity of target activities, new solutions have been proposed for the practical problems in applying activity recognition to real-world applications in the new domains. In this seminar, we introduce recent topics in activity recognition from the viewpoints of (1) recent trends in state-of-the-art machine learning methods for practical activity recognition, (2) recently focused domains for human activity recognition such as industrial and medical domains and their public datasets, and (3) applications of activity recognition to the natural science field, especially in animal behavior understanding.

This study presents a new method for recognizing complex human activities within the logistics domain, such as packaging operations, using acceleration data from a body-worn sensor. The recognition of packaging tasks using standard supervised machine learning is complex because the observed data vary considerably depending on the number of items to be packed, the size of the items, and other parameters. In this study, we focused on the characteristics and necessary key actions (motions) that occur during a specific operation. For instance, when the packaging tape is stretched while assembling the shipping boxes. To focus on these characteristic actions when recognizing data, we propose the use of an attention-based neural network. With our method, the attention-based neural network’s training is guided such that its focus is on the motifs. In addition, this method was designed to accurately recognize short operations by leveraging data augmentation techniques. We tested our method on two logistics datasets and achieved a 3.9% improvement over the previous MGA-Net approach.

This study presents a new method for recognizing complex human activities in a logistical domain, such as packaging, using acceleration data from a body-worn sensor. Recognition of packaging tasks using standard supervised machine learning is difficult because the observed data vary considerably depending on the number of items to pack, the size of the items, and other parameters. In this study, we focus on characteristic and necessary actions (motions) that occur in a specific operation such as an action of stretching packing tape when assembling shipping boxes. We propose the use of an attention-based neural network to focus on these characteristic actions when recognizing the data. However, training of a such deep network model is a data-intensive process, and obtaining a huge amount of labeled training data in actual industrial settings is difficult. To address this problem, we employ motif-detection algorithms to detect sensor data motifs (segments corresponding to characteristic actions) that can be useful for recognizing operations in advance. Moreover, we propose that the training of the attention-based network should be guided such that it pays attention to the detected motifs, i.e., motif-guided training.