This paper introduces a vision-based framework and dataset for capturing and understanding human behavior in industrial assembly lines, focusing on car door manufacturing. The framework leverages advanced computer vision techniques to estimate workers’ locations and 3D poses and analyze work postures, actions, and task progress. A key contribution is the introduction of the CarDA dataset, which contains domain-relevant assembly actions captured in a realistic setting to support the analysis of the framework for human pose and action analysis. The dataset comprises time-synchronized multi-camera RGB-D videos, motion capture data recorded in a real car manufacturing environment, and annotations for EAWS-based ergonomic risk scores and assembly activities. Experimental results demonstrate the effectiveness of the proposed approach in classifying worker postures and robust performance in monitoring assembly task progress.

Currently deployed NAT devices are designed primarily around the client/server paradigm, in which relatively anonymous client machines inside a private network initiate connections to public servers with stable IP addresses and DNS names. Thus, the asymmetric addressing and connectivity regimes established by NAT devices have created unique problems for Peer-to-Peer (P2P) applications and protocols. Multiple NAT-traversal techniques have been developed to overcome these shortcomings, each offering a different set of pros and cons. In the context of a P2P collaborative virtual reality (CVR) system, the difficulty of selecting a convenient and effective NAT-traversal technique increases exponentially because of the added constraints related to CVR. In this view, this article discusses the trade-offs of different NAT-traversal techniques and the CVR challenges that need to be taken into account when choosing a NAT-traversal technique. Finally, it presents a relay-based approach that leverages container migration to mitigate the drawbacks that come with this solution and accentuate its advantages.

The ACM Multimedia 2022 Computational Paralinguistics Challenge addresses four different problems for the first time in a research competition under well-defined conditions: In the Vocalisations and Stuttering Sub-Challenges, a classification on human non-verbal vocalisations and speech has to be made; the Activity Sub-Challenge aims at beyond-audio human activity recognition from smartwatch sensor data; and in the Mosquitoes Sub-Challenge, mosquitoes need to be detected. We describe the Sub-Challenges, baseline feature extraction, and classifiers based on the 'usual' ComParE and BoAW features, the auDeep toolkit, and deep feature extraction from pre-trained CNNs using the DeepSpectrum toolkit; in addition, we add end-to-end sequential modelling, and a log-mel-128-BNN.

This work introduces the harAGEdataset: a novel multimodal smartwatch-based dataset for Human Activity Recognition (HAR) with more than 17 hours of data collected from 19 participants using a Garmin Vivoactive 3 device. The dataset contains samples from resting, lying, sitting, standing, washing hands, walking, running, stairs climbing, strength workout, flexibility workout, and cycling activities. The resting activity, excluded from the set of activities to recognise, was explicitly conducted while avoiding stressors and external stimuli, so the data collected can be used to compute the personal, baseline heart rate at rest. We also present the HAR-based models trained using the accelerometer data to recognise different sets of activities. Specifically, we focus on different strategies to combine, fuse, and enrich the accelerometer measurements, so they can be used end-to-end. Model performances are assessed following a Leave-One-Subject-Out Cross-Validation (LOSO-CV) approach, and we use the Unweighted Average Recall (UAR) as the evaluation metric to compare the ground truth and the inferred information. The best UAR score of 98.1 % is obtained when recognising the static and the dynamic activities, excluding the samples corresponding to the washing hands, strength workout, and flexibility workout activities. When recognising the specific activities included in these two sets, the model with the best performance scores a UAR of 70.1 %. Finally, when recognising all the activities considered in the harAGEdataset, the highest UAR achieved is 64.3 %.

Abstract\n\nThe paper outlines the sustAGE system, a smart solution that builds upon strategic technology trends, such as Internet-of-Things, machine learning and recommender systems, to support sustainable work environments and increase wellness at work and well-being with a focus on the ageing workforce. Acknowledging the interrelation of the work and private arrays for healthy ageing, the developed solution utilizes a recommendation-based approach providing personalized warnings and preventive recommendations regarding occupational risks, as well as personalized cognitive and physical training activities for the off-work context with the overall goal of maintaining Work Ability and enabling sustainable work. The piloting of the proposed solution in two critical industrial domains provides promising results towards the use of personalized recommendation-based interventions for the working context and beyond for improving workers’ occupational safety and health, performance and general well-bei ng.

Recent introduction of low-cost depth cameras triggered a number of interesting works, pushing forward the state-of-the-art in human body pose extraction and tracking. However, despite the remarkable progress, many of the contemporary methods cope inadequately with complex scenarios, involving multiple interacting users, under the presence of severe inter- and intra-occlusions. In this work, we present a model-based approach for markerless articulated full body pose extraction and tracking in RGB-D sequences. A cylinder-based model is employed to represent the human body. For each body part a set of hypotheses is generated and tracked over time by a Particle Filter. To evaluate each hypothesis, we employ a novel metric that considers the reprojected Top View of the corresponding body part. The latter, in conjunction with depth information, effectively copes with difficult and ambiguous cases, such as severe occlusions. For evaluation purposes, we conducted several series of experiments using data from a public human action database, as well as own-collected data involving varying number of interacting users. The performance of the proposed method has been further compared against that of the Microsoft's Kinect SDK and NiTE (TM) using ground truth information. The results obtained attest for the effectiveness of our approach.

Articulated human body tracking is one of the most thoroughly examined, yet still challenging, tasks in Human Robot Interaction. The emergence of low-cost real-time depth cameras has greatly pushed forward the state of the art in the field. Nevertheless, the overall performance in complex, real life scenarios is an open-ended problem, mainly due to the high-dimensionality of the problem, the common presence of severe occlusions in the observed scene data, and errors in the segmentation and pose initialization processes. In this paper we propose a novel model-based approach for markerless pose detection and tracking of the articulated upper body of multiple users in RGB-D sequences. The main contribution of our work lies in the introduction and further development of a virtual User Top View, a hypothesized view aligned to the main torso axis of each user, to robustly estimate the 3D torso pose even under severe intra- and inter-personal occlusions, exempting at the same time the requirement of arbitrary initialization. The extracted 3D torso pose, along with a human arm kinematic model, gives rise to the generation of arms hypotheses, tracked via Particle Filters, and for which ordered rendering is used to detect possible occlusions and collisions. Experimental results in realistic scenarios, as well as comparative tests against the NiTETM user generator middleware using ground truth data, validate the effectiveness of the proposed method.

We address the question of whether service robots that interact with humans in public spaces must express socially appropriate behaviour. To do so, we implemented a robot bartender which is able to take drink orders from humans and serve drinks to them. By using a high-level automated planner, we explore two different robot interaction styles: in the task only setting, the robot simply fulfils its goal of asking customers for drink orders and serving them drinks; in the socially intelligent setting, the robot additionally acts in a manner socially appropriate to the bartender scenario, based on the behaviour of humans observed in natural bar interactions. The results of a user study show that the interactions with the socially intelligent robot were somewhat more efficient, but the two implemented behaviour settings had only a small influence on the subjective ratings. However, there were objective factors that influenced participant ratings: the overall duration of the interaction had a positive influence on the ratings, while the number of system order requests had a negative influence. We also found a cultural difference: German participants gave the system higher pre-test ratings than participants who interacted in English, although the post-test scores were similar.

We introduce a humanoid robot bartender that is capable of dealing with multiple customers in a dynamic, multi-party social setting. The robot system incorporates state-of-the-art components for computer vision, linguistic processing, state management, high-level reasoning, and robot control. In a user evaluation, 31 participants interacted with the bartender in a range of social situations. Most customers successfully obtained a drink from the bartender in all scenarios, and the factors that had the greatest impact on subjective satisfaction were task success and dialogue efficiency.

In this paper we address an important issue in human-robot interaction, that of accurately deriving pointing information from a corresponding gesture. Based on the fact that in most applications it is the pointed object rather than the actual pointing direction which is important, we formulate a novel approach which takes into account prior information about the location of possible pointed targets. To decide about the pointed object, the proposed approach uses the Dempster-Shafer theory of evidence to fuse information from two different input streams: head pose, estimated by visually tracking the off-plane rotations of the face, and hand pointing orientation. Detailed experimental results are presented that validate the effectiveness of the method in realistic application setups.

In this paper we present a novel methodology for detection and tracking of facial features like eyes, nose and mouth in image sequences. The proposed methodology is intended to support natural interaction with autonomously navigating robots that guide visitors in museums and exhibition centers and, more specifically, to provide input for the analysis of facial expressions that humans utilize while engaged in various conversational states. For face and facial feature region detection and tracking, we propose a methodology that combines appearance-based and feature-based methods for recognition and tracking, respectively. For the stage of face tracking the introduced method is based on least squares matching (LSM), a matching technique able to model effectively radiometric and geometric differences between image patches in different images. Thus, compared with previous research, the LSM approach can overcome the problems of variable scene illumination and head in-plane rotation. Another significant characteristic of the proposed approach is that tracking is performed on the image plane only wherever laser range information suggests so. The increased computational efficiency meets the real time demands of human-robot interaction applications and hence facilitates the development of relevant systems.