Inertial human motion capture (IHMC) has become a robust tool to estimate human kinematics in the wild such as industrial facilities. In contrast to optical motion capture, where occlusions might take place, the kinematics of a worker can be continuously provided. This is for instance a prerequisite for an ergonomic assessments of the workers. State-of-the-art IHMC solutions require inertial sensors to be tightly attached to body segments. This requires an additional setup time and lowers the practicability and ease of use when it comes to an industrial application. In contrast, sensors integrated into loose clothing such as a working jacket, may yield corrupted kinematics estimates due to the additional motion of loose clothing. In this work we present a study of orientations deviations obtained from kinematics estimates using tightly attached inertial sensors and into a working jacket integrated ones. We performed a quantitative analysis using data from the two hardware setups worn by 19 subjects performing different industry related tasks and measures of their body shapes. Using this data we approximated probability distributions of the deviation angles for each person and body segment. Applying different statistical measures we could gain insights to questions like, how severe orientation deviations are, if there is an influence of body shapes on the distribution and how probability distributions of the deviation angles can indicate physical motion limitations of a sensor attached to a segment.

Inertial motion capture has become an attractive alternative to optical motion capture for human joint angle estimation outside the laboratory. Usually inertial sensors are assumed to be tightly fixed to the body segments, which can be cumbersome regarding setup-time and ease-of-use. However, integrating the sensors directly into loose clothing, usually, results in additional clothing motion relative to the motion of the underlying bones that should be captured. In this work we propose the Difference Mapping distributions approach that corrects the segment orientations of a given inertial motion capture system that assumes tightly coupled sensors. The approach allows to reduce the joint angle errors due to clothing artefacts by at least 77.2% for people with similar morphology performing a similar task as seen in the training data, including an ergonomic assessments scenario at work places with ten participants. Moreover, we show that the uncertainty of the distribution can be used to measure the reliability of the predicted map if e.g. the motion is further away from the training data to allow for an artefact aware inertial motion tracking approach. The experimental data for this study is available online under [1].

Inertial measurement units (IMUs) enable the capture of human motion in-field. This can be used in various analysis applications ranging from the medical domain over sports to daily activities. Manual data labelling for classification or regression tasks is often time-consuming and cumbersome, in particular when it comes to larger datasets. Active learning algorithms try to reduce the labelling cost, for instance via suggesting samples with high prediction uncertainty that should be explicitly labelled. In this work, we apply two probabilistic deep learning approaches on different state-of-the-art deep neural network structures for timeseries data, with uncertainty based measures to actively query sample labels. This is applied to gait phase classification using IMU data as inputs. We demonstrate the performance on a newly captured dataset, where we obtained high accuracy (up to 96%) with up to 43% fewer samples as compared to random sampling in an online setting. In an offline setting, we could extract heel strike and toe-off foot events with an accuracy of 99.9% using active learning strategies with up to 58% fewer samples.

The ability to accurately capture locomotion is relevant in various use cases, in particular in the sports and health area. With the major goal of providing a measurement system that can deliver different types of relevant information (3D body segment kinematics, spatiotemporal locomotion parameters, and locomotion patterns) in-field and in real-time, we propose a novel probabilistic (single-plane) ground contact estimation method, using four contact points defined through a biomechanical foot model, and integrate this into an existing inertial motion capturing method. The resulting method is quantitatively evaluated on simulated and real IMU data in comparison to an optical motion capture system on walking, running, and jumping sequences. The results show its ability to maintain a good average 3D kinematics estimation error on low- and high-acceleration locomotion, whereas many previous accuracy studies restrict themselves to movements with low to moderate global accelerations, such as upper body activities or slow locomotion. Moreover, a qualitative evaluation of the estimated ground contact probabilities demonstrates the method's ability to also provide consistent information also for deriving spatiotemporal locomotion parameters as well as locomotion patterns (e.g., over-pronation/-supination) simultaneously with the 3D kinematics.

This paper addresses the problem of video registration for dense non-rigid structure from motion under suboptimal conditions, such as noise, self-occlusions, considerable external occlusions or specularities, i.e. the computation of optical flow between the reference image and each of the subsequent images in a video sequence when the camera observes a highly deformable object. We tackle this challenging task by improving previously proposed variational optimization techniques for multi-frame optical flow (MFOF) through detection, tracking and handling of uncertain flow field estimates. This is based on a novel Bayesian inference approach incorporated into the MFOF. At the same time, computational costs are significantly reduced through iterative pre-computation of the flow fields. As shown through experiments, the resulting method performs superior to other state-of-the-art (MF)OF methods on video sequences showing a highly non-rigidly deforming object with considerable occlusions.

In the context of a smart user assistance system for industrial manipulation tasks it is necessary to capture motions of the upper body and limbs of the worker in order to derive his or her interactions with the task space. While such capturing technology already exists, the novelty of the proposed work results from the strong requirements of the application context: The method should be flexible and use only on-body sensors, work accurately in industrial environments that suffer from severe magnetic disturbances, and enable consistent registration between the user body frame and the task space. Currently available systems cannot provide this. This paper suggests a novel egocentric solution for visual-inertial upper-body motion tracking based on recursive filtering and model-based sensor fusion. Visual detections of the wrists in the images of a chest-mounted camera are used as substitute for the commonly used magnetometer measurements. The on-body sensor network, the motion capturing system, and the required calibration procedure are described and successful operation is shown in a real industrial environment.

This paper addresses two fundamental requirements of full body motion monitoring: (a) the ability to sense the input of the user and (b) the means to interpret the captured input. Appropriate technology in both areas is required for an interactive virtual reality system to provide feedback in a useful and natural way. This paper combines technologies for both areas: It develops a sensor fusion approach for capturing user input based on miniature on-body inertial and magnetic motion sensors. Furthermore, it presents work in progress to automatically generate models for motion patterns from the captured input. The technology is then used and evaluated in the context of a personalized virtual rehabilitation trainer application.

Visual simultaneous localisation and mapping (SLAM) is since the last decades an often addressed problem. Online mapping enables tracking in unknown environments. However, it also suffers from high computational complexity and potential drift. Moreover, in augmented reality applications the map itself is often not needed and the target environment is partially known, e.g. in a few 3D anchor or marker points. In this paper, rather than using SLAM, measurements based on optical flow are introduced. With these measurements, a modified visual-inertial tracking method is derived, which in Monte Carlo simulations reduces the need for 3D points and allows tracking for extended periods of time without any 3D point registrations.

The use of a particle filter (PF) for camera pose estimation is an ongoing topic in the robotics and computer vision community, especially since the FastSLAM algorithm has been utilised for simultaneous localisation and mapping (SLAM) applications with a single camera. The major problem in this context consists in the poor proposal distribution of the camera pose particles obtained from the weak motion model of a camera moved freely in 3D space. While the FastSLAM 2.0 extension is one possibility to improve the proposal distribution, this paper addresses the question of how to use measurements from low-cost inertial sensors (gyroscopes and accelerometers) to compensate for the missing control information. However, the integration of inertial data requires the additional estimation of sensor biases, velocities and potentially accelerations, resulting in a state dimension, which is not manageable by a standard PF. Therefore, the contribution of this paper consists in developing a real-time capable sensor fusion strategy based upon the marginalised particle filter (MPF) framework. The performance of the proposed strategy is evaluated in combination with a marker-based tracking system and results from a comparison with previous visual-inertial fusion strategies based upon the extended Kalman filter (EKF), the standard PF and the MPF are presented.

We present a new visual-inertial tracking device for augmented and virtual reality applications. The paper addresses two fundamental issues of such systems. The first one concerns the definition and modelling of the sensor fusion. Much work has been done in this area and several models for exploiting the data of the gyroscopes and linear accelerometers have been proposed. However, the respective advantages of each model and in particular the benefits of the integration of the accelerometer data in the filter are still unclear. The paper therefore provides an evaluation of different models with special investigation of the effects of using accelerometers on the tracking performance. The second contribution is about the development of an image processing approach that does not require special landmarks but uses natural features. Our solution relies on a 3D model of the scene that enables to predict the appearances of the features by rendering the model using the prediction data of the sensor fusion filter. The feature localisation is robust and accurate mainly because local lighting is also estimated. The final system is evaluated with help of ground-truth and real data. High stability and accuracy is demonstrated also for large environments.

One of the key requirements of augmented reality systems is a robust real-time camera pose estimation. In this paper we present a robust approach, which does neither depend on offline pre-processing steps nor on pre-knowledge of the entire target scene. The connection between the real and the virtual world is made by a given CAD model of one object in the scene. However, the model is only needed for initialization. A line model is created out of the object rendered from a given camera pose and registrated onto the image gradient for finding the initial pose. In the tracking phase, the camera is not restricted to the modeled part of the scene anymore. The scene structure is recovered automatically during tracking. Point features are detected in the images and tracked from frame to frame using a brightness invariant template matching algorithm. Several template patches are extracted from different levels of an image pyramid and are used to make the 2D feature tracking capable for large changes in scale. Occlusion is detected already on the 2D feature tracking level. The features' 3D locations are roughly initialized by linear triangulation and then refined recursively over time using techniques of the Extended Kalman Filter framework. A quality manager handles the influence of a feature on the estimation of the camera pose. As structure and pose recovery are always performed under uncertainty, statistical methods for estimating and propagating uncertainty have been incorporated consequently into both processes. Finally, validation results on synthetic as well as on real video sequences are presented.