Abstract We present a method for automatically generating reduced marker layouts for marker‐based optical motion capture of human hands. The employed motion reconstruction method is based on subspace‐constrained inverse kinematics, which allows for the recovery of realistic hand movements even from sparse input data. We additionally present a user‐specific hand model calibration procedure that fits an articulated hand model to point cloud data of the user's hand. Our marker layout optimization is sensitive to the kinematic structure and the subspace representations of hand articulations utilized in the reconstruction method, in order to generate sparse marker configurations that are optimal for solving the constrained inverse kinematics problem. We propose specific quality criteria for reduced marker sets that combine numerical stability with geometric feasibility of the resulting layout. These criteria are combined in an objective function that is minimized using a specialized surface‐constrained particle swarm optimization scheme, which generates marker layouts bound to the surface of an animated hand model. Our method provides a principled way for determining reduced marker layouts based on subspace representations of hand articulations. We demonstrate the effectiveness of our motion reconstruction and model calibration methods in a thorough evaluation.

A necessary skill when using liquids in the preparation of food is to be able to estimate viscosity, e.g. in order to control the pouring velocity or to determine the thickness of a sauce. We introduce a method to allow a robotic kitchen assistant discriminate between different but visually similar liquids. Using a Kinect depth camera, surface changes, induced by a simple pushing motion, are recorded and used as input to nearest neighbour and polynomial regression classification models. Results reveal that even when the classifier is trained on a relatively small dataset it generalises well to unknown containers and liquid fill rates. Furthermore, the regression model allows us to determine the approximate viscosity of unknown liquids.

We present a method for automatically generating reduced marker layouts for marker-based optical motion capture of human hand motions. Reducing the number of markers on the hand is important to ensure the generated motions are performed in a natural way and indeed a reduced marker set might be a technical requirement should simultaneous body motion capture also have to be carried out. The employed motion reconstruction method is based on subspace-constrained inverse kinematics, which allows for the recovery of realistic hand movements even from sparse input data. Our marker layout optimization is sensitive to the kinematic structure and the subspace representations of hand articulations utilized in the reconstruction method in order to generate sparse marker configurations that are optimal for solving the constrained inverse kinematics problem. We propose specific quality criteria for reduced marker sets that combine numerical stability with geometric feasibility of the resulting layout. These criteria are combined in an objective function that is minimized using a specialized surface-constrained particle swarm optimization scheme. Our method provides a principled way for determining reduced marker layouts based on subspace representations of hand articulations.

We present a method for real-time bare hand tracking that utilizes natural hand synergies to reduce the complexity and improve the plausibility of the hand posture estimation. The hand pose and posture are estimated by fitting a virtual hand model to the 3D point cloud obtained from a Kinect camera using an inverse kinematics approach. We use real human hand movements captured with a Vicon motion tracking system as the ground truth for deriving natural hand synergies based on principal component analysis. These synergies are integrated in the tracking scheme by optimizing the posture in a reduced parameter space. Tracking in this reduced space combined with joint limit avoidance constrains the posture estimation to natural hand articulations. The information loss associated with dimension reduction can be dealt with by employing a hierarchical optimization scheme. We show that our synergistic hand tracking approach improves runtime performance and increases the quality of the posture estimation.

We implemented a system, called the VICON-EyeTracking Visualizer, that combines mobile eye tracking data with motion capture data to calculate and visualize the 3D gaze vector within the motion capture co-ordinate system. To ensure that both devices were temporally synchronized we used previously developed software by us. By placing reflective markers on objects in the scene, their positions are known and by spatially synchronizing both the eye tracker and the motion capture system allows us to automatically compute how many times and where fixations occur, thus overcoming the time consuming and error-prone disadvantages of the traditional manual annotation process. We evaluated our approach by comparing its outcome for a simple looking task and a more complex grasping task against the average results produced by the manual annotation process. Preliminary data reveals that the program only differed from the average manual annotation results by approximately 3 percent in the looking task with regard to the number of fixations and cumulative fixation duration on each point in the scene. In case of the more complex grasping task the results depend on the object size: for larger objects there was good agreement (less than 16 percent (or 950ms)), but this degraded for smaller objects, where there are more saccades towards object boundaries. The advantages of our approach are easy user calibration, the ability to have unrestricted body movements (due to the mobile eye-tracking system), and that it can be used with any wearable eye tracker and marker based motion tracking system. Extending existing approaches, our system is also able to monitor fixations on moving objects. The automatic analysis of gaze and movement data in complex 3D scenes can be applied to a variety of research domains, i. e., Human Computer Interaction, Virtual Reality or grasping and gesture research.

To enable the creation of manual interaction databases, aiding the replication of dexterous capabilities with anthropomorphic robot hands by utilizing information about how humans perform complex manipulation tasks, requires the capability to record and analyze large amounts of manual interaction sequences. For this goal we have studied and compared three mappings from captured human hand motion data to a simulated model, which allow for robust and accurate real-time hand posture tracking. We evaluate the effectiveness of these mappings and discuss their pros and cons in various real-world scenarios. The first method is based on data glove data and aims for direct gaging of hand joints. The other two methods utilize a VICON motion tracking system which monitors markers placed on all finger segments. Here we compare two approaches: a direct computation of hand postures from angles between adjacent markers and an iterative inverse kinematics approach to optimally reproduce fingertip positions. For a quantitative evaluation, we employ a “calibration objects” technique to obtain a reliable ground truth of task-relevant hand posture data.