Robustness against specific kinds of noise is of high importance for safety-critical components in industrial robot applications, as legal and normative regulations demand the identification and handling of all unacceptable risks. This includes risks from environmental conditions, like noisy data. One such component is human pose estimation, which is needed and crucial for human-robot collaboration tasks and applications. However, little research on human pose estimation under specific noise types has been performed. In our work, we focus on extensively evaluating human pose estimation under specific noise and propose potential countermeasures. We leverage Gaussian noise as specific noise type and the hourglass model as human pose estimator. We show that human pose estimation is already vulnerable to small amounts of Gaussian noise. As countermeasures we propose either denoising images upfront or training the hourglass model to be robust against Gaussian noise. All methods achieve a significantly higher robustness against Gaussian noise, typically at the cost of slightly worse performance on clean data. Three of our methods also achieved slight improvements on clean data.

Obtaining reliable detections of a human is crucial for many safety-related robotic tasks. This can be done by human pose estimation methods, which predict the position of several different keypoints of the human body. In most cases, recent approaches based on neural networks produce ‘good’ results, i.e. predictions with small localization errors, however, large errors do also occur. For an individual keypoint prediction, the magnitude of the error is unknown, posing a risk to safety. In this work, we extend a neural network architecture for single-person 2D human pose estimation, so that it predicts not only the keypoints of the human body, but also corresponding upper bounds for their localization errors. These upper bounds correspond to the neural network's confidence in its output, and are obtained by one of two general strategies based on (i) a direct estimation of the localization error or (ii) the predicted standard deviations of a 2D Gaussian. We propose several approaches employing these strategies and evaluate them on the MPII Human Pose dataset. In addition, we consider two quality criteria for the results: closeness of the predicted keypoint position to the actual one, and closeness of the predicted upper bound to the localization error. The best results are achieved by a Gaussian-based approach, which predicted correct upper bounds in 94.7% of the cases, while also sufficiently fulfilling the quality criteria.

In the industrial domain, the application of any system as a safety function has to follow strict rules and requirements, defined in safety standards such as ISO 13849 [1] and ISO 13855 [2]. Two core requirements are an extremely low rate of dangerous errors and an upper limit for the response time of the system (hard real-time requirement). Current approaches in the field of human pose estimation achieve neither of both.In our work we approach the second requirement by introducing a general procedure to achieve hard real-time capability for interchangeable 3D human pose estimation systems. We use the detections of the pose estimation to model the human as 3D volume consisting of spheres and spherical cones. To bridge the time between arriving detections, the volume is adjusted in a way that ensures coherence, continuity and, most important, conformance with necessary surcharges from safety standard ISO 13855 [2]. Low and fixed computational cost for the adaption makes the procedure real-time capable. Our modelling approach using spheres and spherical cones also allows distance calculations at low and fixed computational costs, e.g. between human and robot. We show the benefit of our approach via human-robot distance calculation experiments, outperforming a safety-certified laser scanner for most of the time.

Modern mobile phones are equipped with many sensors, which can increasingly be used to sense various environmental phenomena. In particular, mobile sensing has enabled crowdsourced data collection at an unprecedented scale. However, as laypersons are involved in this, concerns regarding the data quality arise. This work explores the gamification of smartphone-based measurement processes in practice by embedding a sensing task into a mobile minigame. The underlying idea is – rather than to educate the user on how to correctly perform a measurement task – to opportunistically execute the measurement in the background once the smartphone is in a suitable context. To this end, this paper presents the design and evaluation of SpaceMaze, a smartphone game with the goal of minimising user error by introducing appropriate game mechanics to influence the phone context, using the example of mobile noise level monitoring. A large user study that compares SpaceMaze to two non-gamified apps for noise level monitoring (N = 360 in total) shows that SpaceMaze can successfully reduce user errors when compared to simple non-gamified ambient noise level monitoring applications and that the minigame is generally perceived as being enjoyable. Solutions for remaining problems, such as noise generated by the players, are discussed.

In recent years, human pose estimation has seen great improvements by the use of neural networks. However, these approaches are unsuitable for safety-critical applications such as human-robot interaction (HRI), as no guarantees are given whether a produced detection is correct or not and false detections with high confidence scores are produced on a regular basis. In this work, we propose a method to identify and eliminate false detections by comparing keypoint detections from different neural networks and assigning a 'Don't know' label in the case of a mismatch. Our approach is driven by the principle of software diversity, a technique recommended by the safety standard IEC 61508-7 [1] for dealing with software implementation faults. We evaluate our general concept on the MPII human pose dataset [2] using available ground truth data to calculate a suitable threshold for our keypoint comparison, reducing the number of false detections by approx. 61%. For the application at runtime, where no ground truth data is available, we introduce a method to calculate the needed threshold directly from keypoint detections. In further experiments, it was possible to reduce the number of false detections by approx. 75%. Eliminating keypoints by comparison also lowers the correct detection rate, which we maintained above 75% in all experiments. As this effect is limited and non-critical regarding safety we believe that the proposed approach can lead the way to a safe use of neural networks for human pose estimation in the future.