Complex work activity recognition based on wearable sensors is crucial for streamlining work processes in industrial domains. Unlike basic activities such as walking or running, which involve simple repetitive motions, a complex work activity consists of discrete atomic actions such as an action of spreading a shipping label or cutting tape in a packaging task. In addition, the atomic actions sometimes involve characteristic short-term sensor data patterns. In addition, these actions can be performed in different orders by different workers to achieve similar outcomes, resulting in different long-term sensor data trends for different workers. Because multilayer networks for activity recognition may learn short-term features from shallow-level layers and long-term trends from deeper layers, we propose a new transfer learning method called multilevel knowledge transfer (MLKT), which performs level-wise source selection according to trend similarity across workers in different levels. For example, for training shallow layers, highly similar workers are selected for specific short motions (e.g., pasting a shipping label), and to train the deeper layers, workers with similar cadence are selected. This method also enables the adaptive thresholding of source data selection for each layer level during network training using the proposed adaptive level-wise discerning module.

Owing to the cost of collecting labeled sensor data, self-supervised learning (SSL) methods for human activity recognition (HAR) that effectively use unlabeled data for pretraining have attracted attention. However, applying prior SSL to COMPLEX activities in real industrial settings poses challenges. Despite the consistency of work procedures, varying circumstances, such as different sizes of packages and contents in a packing process, introduce significant variability within the same activity class. In this study, we focus on sensor data corresponding to characteristic and necessary actions (sensor data motifs) in a specific activity such as a stretching packing tape action in an assembling a box activity, and propose to train a neural network in self-supervised learning so that it identifies occurrences of the characteristic actions, i.e., Motif Identification Learning (MoIL). The feature extractor in the network is subsequently employed in the downstream activity recognition task, enabling accurate recognition of activities containing these characteristic actions, even with limited labeled training data. The MoIL approach was evaluated on real-world industrial activity data, encompassing the state-of-the-art SSL tasks with an improvement of up to 23.85% under limited training labels.

This study presents a new method for recognizing complex human activities within the logistics domain, such as packaging operations, using acceleration data from a body-worn sensor. The recognition of packaging tasks using standard supervised machine learning is complex because the observed data vary considerably depending on the number of items to be packed, the size of the items, and other parameters. In this study, we focused on the characteristics and necessary key actions (motions) that occur during a specific operation. For instance, when the packaging tape is stretched while assembling the shipping boxes. To focus on these characteristic actions when recognizing data, we propose the use of an attention-based neural network. With our method, the attention-based neural network’s training is guided such that its focus is on the motifs. In addition, this method was designed to accurately recognize short operations by leveraging data augmentation techniques. We tested our method on two logistics datasets and achieved a 3.9% improvement over the previous MGA-Net approach.

This study presents a new method for recognizing complex human activities in a logistical domain, such as packaging, using acceleration data from a body-worn sensor. Recognition of packaging tasks using standard supervised machine learning is difficult because the observed data vary considerably depending on the number of items to pack, the size of the items, and other parameters. In this study, we focus on characteristic and necessary actions (motions) that occur in a specific operation such as an action of stretching packing tape when assembling shipping boxes. We propose the use of an attention-based neural network to focus on these characteristic actions when recognizing the data. However, training of a such deep network model is a data-intensive process, and obtaining a huge amount of labeled training data in actual industrial settings is difficult. To address this problem, we employ motif-detection algorithms to detect sensor data motifs (segments corresponding to characteristic actions) that can be useful for recognizing operations in advance. Moreover, we propose that the training of the attention-based network should be guided such that it pays attention to the detected motifs, i.e., motif-guided training.

In a line production system of a factory, a worker repetitively performs predefined operation processes. This paper tries to recognize work by factory workers in an unsupervised manner. Specifically, we propose an unsupervised measurement method for estimating lead time (duration) of each period of an operation process using a wrist-worn accelerometer because the lead time greatly affects productivity of the line production system. Our proposed method automatically finds a frequent sensor data segment as a "motif" that occurs once in each operation period using only prior knowledge about predefined standard lead time of the operation process, and uses the occurrence intervals of the motif to estimate the lead time. We evaluated our method using real factory data and the estimation error was only about 3.5%.

This paper presents novel unconstrained person identification for tabletop systems using a ceiling-mounted depth camera that overlooks a table. Recent state-of-the-art ubicomp, computer-vision, and CSCW studies have tried to recognize a user's activities and actions on a table using a ceiling-mounted device that overlooks the table. However, conventional unconstrained person identification methods such as face identification cannot be used for providing personalized services in such settings. In this study, we focus on a user's soft biometrics that can be captured from the ceiling such as the shoulder length, shape of the head, and posture of the back to achieve unconstrained person identification by using a ceiling-mounted depth camera. We achieve robust person identification by combining the soft biometrics within a framework of multiview learning. Multiview learning allows us to deal effectively with data consisting of features from multiple sources with different data distributions, i.e., multiple soft biometrics in our case. To the best of our knowledge, this is the first study that investigates the feasibility of person identification for tabletop users by a ceiling-mounted depth camera.

In distributed control systems (DCS), designed distributed controllers, which interact with each other operate plants. Therefore, in the development of the system, verification of whether the controllers satisfy given specification is important. However, the existing description languages, for example, temporal interval, are too complicated to formulize the specification. In this paper, first, Sequence Oriented Language (SOL), which is intelligible language in order to describe specification for distributed sequential control systems, is proposed. Secondly, we developed a tool for it. In this method, the tool automatically verifies that behavior of models satisfies specifications which is described using SOL.