Image inpainting is the task of removing undesired objects or flaws in images. This work advances an exemplar-based global optimization image inpainting algorithm. For that purpose, the inpainting area is iteratively refined through the minimization of a cost function. The minimization outcome depends on the initial values of the inpainting area. We compare three initialization methods with a new sparsity-driven approach. Lastly, we propose the new wavelet contrast costs which increase the inpainting quality. Wavelet contrasts reduce computational complexity in comparison to wavelet histograms while preserving their ability of measuring the density of image texture.

This work adapts common HMD interfaces for the use on a handheld device. The proposed interfaces focus on: Easy interaction on the mobile device and independence of the provided content to the user's view. We compare two AR interface techniques for object identification and three AR interfaces for troubleshooting. The results show that exocentric-based AR interfaces outperform egocentric ones in respect to completion time, walking distance and number of interactions.

In this work, we propose a device-based manipulation technique named RayOnPlane, which maintains the ease of use also in case of increasing work space size. We compare this technique to the state-of-the-art device-based manipulation technique HOMER-S. An experiment incorporating different work space sizes indicates comparable performance in completion time, while minimizing gesture size as well as user frustration and physical strain.

Despite the growing research interest in 2.5D and 3D video face processing, 3D facial videos are actually scarcely available. This work introduces a new open source tool, named FaceGrabber, for capturing human faces using Microsoft's Kinect 2.0. FaceGrabber permits the concurrent recording of various formats, including the raw 2D and 2.5D video streams, 3D point clouds and the 3D registered face model provided by the Kinect. The software is also able to convert different data formats and playback recorded results directly in 3D. In order to encourage research with Kinect 2.0 face data, we publish a new public video face database which was captured using FaceGrabber. The database comprises 40 individuals, performing the six universal emotions (disgust, sadness, happiness, fear, anger, surprise) and two additional sequences.

Diminished reality (DR) aims at removing objects auto-matically in live video streams, in a way which is unrecognizable to human observers. Approaches working on mobile devices utilize image inpainting algorithms to fill the holes (target region) of the removed objects in a plausible way. We propose a new DR algorithm, which extracts a scene model from a sparse point cloud of a mono SLAM algorithm. The scene model allows to inpaint complex scenes consisting of multiple planes intersecting at the target region. A plane-wise key frame estimation preserves scene structure as well as improving inpainting results. Furthermore, we introduce a viewing angle-based key frame update policy, ensuring high quality of the key frames. The multi-threaded design of the algorithm achieves nearly real-time performance on a mobile device.

Visual attributes are high-level semantic descriptions of visual data that are close to the human language. They have been used intensively in various applications such as image classification, active learning, and interactive search. However, the usage of attributes in subspace learning (or dimensionality reduction) has not been considered yet. In this work, we propose to utilize relative attributes as semantic cues in subspace learning. To this end, we employ Non-negative Matrix Factorization (NMF) constrained by embedded relative attributes to learn a subspace representation of image content. Experiments conducted on two datasets show the efficiency of attributes in discriminative subspace learning.

Image inpainting algorithms aim to cut out parts of the image without leaving holes. Various algorithms exist, but no wider comparison has been made, yet. This work fills the gap by comparing state-of-the-art algorithms in a user study. We create and publish a database consisting of multiple base images and inpaint them using different inpainting concepts. Afterwards, 21 participants are asked to rate the quality of these inpainted images. The subjective feedback indicates that different image inpainting algorithms are favorable depending on the characteristics of the base image and target region. Furthermore, the results show that general image quality measures such as the peak signal-to-noise ratio (PSNR) or the structural similarity (SSIM) index are not suited for judging inpainting quality.

In the area of foreground and background segmentation the dynamics of a scene often influence the model of the background. There exist different methods trying to overcome the difficulty in estimating and handling these dynamics. In this paper, we propose an adaption of proportional-integral-derivative (PID) controllers to this regulation problem. In our approach, PID controllers regulate the decision threshold of the background dynamics and the update rate of the background model. The regulators are integrated in the Pixel-based Adaptive Segmenter (PBAS), which is publicly available and offers good results in the Change Detection challenge. We show that the performance of the PBAS can be further improved through an empirical tuning of the single PID constants.

Mobile touch PCs allow interactions with virtual objects in augmented reality scenes. Manipulations of 3D objects are a common way of such interactions, which can be performed in three different coordinate systems: the camera-, object- and world coordinate systems. The camera coordinate system changes continuously in augmented reality as it depends on the mobile device's pose. The axis orientations of the world coordinate system are steady, whereas the axes of the object coordinates base on previous manipulations. The selection of a coordinate system therefore influences the 3D transformation's orientation independent from the used manipulation type.

This work presents three techniques for 3D manipulation on mobile touch devices, taking the specifics of mobile AR scenes into account. We compare the common direct manipulation technique with two indirect techniques, which utilize only the thumbs to perform the transformations. The evaluation of the manipulation variants is conducted in a mixed reality (MR) environment which takes advantage of the controlled conditions of a full virtual reality (VR) system. A study with 18 participants shows that the two-thumb method tops the other techniques. It performs better with respect to the total manipulation time and total number of gestures.