Abstract Non-rigid structure from motion (nrsfm), is a long standing and central problem in computer vision and its solution is necessary for obtaining 3D information from multiple images when the scene is dynamic. A main issue regarding the further development of this important computer vision topic, is the lack of high quality data sets. We here address this issue by presenting a data set created for this purpose, which is made publicly available, and considerably larger than the previous state of the art. To validate the applicability of this data set, and provide an investigation into the state of the art of nrsfm, including potential directions forward, we here present a benchmark and a scrupulous evaluation using this data set. This benchmark evaluates 18 different methods with available code that reasonably spans the state of the art in sparse nrsfm. This new public data set and evaluation protocol will provide benchmark tools for further development in this challenging field.

This paper presents a new large scale dataset targeting evaluation of local shape descriptors and 3d object recognition algorithms. The dataset consists of point clouds and triangulated meshes from 292 physical scenes taken from 11 different views, a total of approximately 3204 views. Each of the physical scenes contain 10 occluded objects resulting in a dataset with 32040 unique object poses and 45 different object models. The 45 object models are full 360 degree models which are scanned with a high precision structured light scanner and a turntable. All the included objects belong to different geometric groups, concave, convex, cylindrical and flat 3D object models. The object models have varying amount of local geometric features to challenge existing local shape feature descriptors in terms of descriptiveness and robustness. The dataset is validated in a benchmark which evaluates the matching performance of 7 different state-of-the-art local shape descriptors. Further, we validate the dataset in a 3D object recognition pipeline. Our benchmark shows as expected that local shape feature descriptors without any global point relation across the surface have a poor matching performance with flat and cylindrical objects. It is our objective that this dataset contributes to the future development of next generation of 3D object recognition algorithms. The dataset is public available at http://roboimagedata.compute.dtu.dk/.

A current trend in industrial automation implies a need for doing automatic scene understanding, from optical 3D sensors, which in turn imposes a need for a lightweight and reliable 3D optical sensor to be mounted on a collaborative robot e.g., Universal Robot UR5 or Kuka LWR. Here, we empirically evaluate the feasibility of structured light scanners for this purpose, by presenting a system optimized for this task. The system incorporates several recent advances in structured light scanning, such as Large-Gap Gray encoding for dealing with defocusing, automatic creation of illumination masks for noise removal, as well as employing a multi exposure approach dealing with different surface reflectance properties. In addition to this, we investigate expanding the traditional structured light setup to using three cameras, instead of one or two. Also, a novel method for fusing multiple exposures and camera pairs is given. We present an in-depth evaluation, that lead us to conclude, that this setup performs well on tasks relevant for an industrial environment, where many metallic and other surfaces with difficult reflectance properties are in abundance. We demonstrate, that the added components contribute to the robustness of the system. Hereby, we demonstrate that structured light scanning is a technology well suited for hyper flexible industrial automation, by proposing an appropriate system.

The seminal multiple view stereo benchmark evaluations from Middlebury and by Strecha et al. have played a major role in propelling the development of multi-view stereopsis methodology. Although seminal, these benchmark datasets are limited in scope with few reference scenes. Here, we try to take these works a step further by proposing a new multi-view stereo dataset, which is an order of magnitude larger in number of scenes and with a significant increase in diversity. Specifically, we propose a dataset containing 80 scenes of large variability. Each scene consists of 49 or 64 accurate camera positions and reference structured light scans, all acquired by a 6-axis industrial robot. To apply this dataset we propose an extension of the evaluation protocol from the Middlebury evaluation, reflecting the more complex geometry of some of our scenes. The proposed dataset is used to evaluate the state of the art multiview stereo algorithms of Tola et al., Campbell et al. and Furukawa et al. Hereby we demonstrate the usability of the dataset as well as gain insight into the workings and challenges of multi-view stereopsis. Through these experiments we empirically validate some of the central hypotheses of multi-view stereopsis, as well as determining and reaffirming some of the central challenges.

Classification of high resolution urban remote sensing imagery is addressed. The classification is done by both considering the panchromatic imagery and the multi-spectral image obtained using the spectrally consistent fusion method introduced in [1]. The data are classified using support vector machines (SVM). To further enhance the classification accuracy, mathematical morphology is used to derive local spatial information from the panchromatic data. In particular we use the Morphological Profile (MP) in classification of satellite imagery as was proposed in [2, 3]. We also use the derivative of the MP (DMP). In the majority of the image fusion (pansharpening) techniques proposed today, there is a compromise between the spatial enhancement and the spectral consistency. By comparing classification results obtained by using our model based scheme [1] to results obtained using the IHS and Brovey fusion methods, we find that spectrally consistent data give better results when it comes to classification.

This video presents a practical navigation scheme for indoor mobile robots using hybrid maps. The method makes use of metric maps for local navigation and a topological map for global path planning. Metric maps are generated as occupancy grids by a laser range finder to represent local information about partial areas. The global topological map is used to indicate the connectivity of the `places-of-interests' in the environment and the interconnectivity of the local maps. Visual tags on the ceiling to be detected by the robot provide valuable information and contribute to reliable localization. The navigation scheme based on the hybrid metric-topological maps saves memory space and is also scalable and adaptable since new local maps can be easily added to the global topology, and the method can be deployed with minimum amount of modification if new areas are to be explored. The video demonstrated that the method is implemented successfully on physical robot in a hospital environment, which provides a practical solution for indoor navigation.

Many change detection algorithms work by calculating the probability of change on a pixel-wise basis. This is a disadvantage since one is usually looking for regions of change, and such information is not used in pixel-wise classification - per definition. This issue becomes apparent in the face of noise, implying that the pixel-wise classifier is also noisy. There is thus a need for incorporating local homogeneity constraints into such a change detection framework. For this modelling task Markov Random Fields are suitable. Markov Random Fields have, however, previously been plagued by lack of efficient optimization methods or numerical solvers. We here address the issue of efficient incorporation of local homogeneity constraints into change detection algorithms. We do this by exploiting recent advances in graph based algorithms for Markov Random Fields. This is combined with an IR-MAD change detector, and demonstrated on real data with good results.

Abstract Scalar functions defined on manifold triangle meshes is a starting point for many geometry processing algorithms such as mesh parametrization, skeletonization, and segmentation. In this paper, we propose the Auto Diffusion Function (ADF) which is a linear combination of the eigenfunctions of the Laplace‐Beltrami operator in a way that has a simple physical interpretation. The ADF of a given 3D object has a number of further desirable properties: Its extrema are generally at the tips of features of a given object, its gradients and level sets follow or encircle features, respectively, it is controlled by a single parameter which can be interpreted as feature scale, and, finally, the ADF is invariant to rigid and isometric deformations. We describe the ADF and its properties in detail and compare it to other choices of scalar functions on manifolds. As an example of an application, we present a pose invariant, hierarchical skeletonization and segmentation algorithm which makes direct use of the ADF.

A method is proposed for pixel-level satellite image fusion derived directly from a model of the imaging sensor. By design, the proposed method is spectrally consistent. It is argued that the proposed method needs regularization, as is the case for any method for this problem. A framework for pixel neighborhood regularization is presented. This framework enables the formulation of the regularization in a way that corresponds well with our prior assumptions of the image data. The proposed method is validated and compared with other approaches on several data sets. Lastly, the intensity-hue-saturation method is revisited in order to gain additional insight of what implications the spectral consistency has for an image fusion method.

Different spatial priors for satellite image fusion are evaluated through experiments on three different data sets. The results are judged visually as well as quantified via different image quality metrics on a down-sampled data-set. It is done within our previously proposed spectrally consistent pan- sharpening framework (SCP). This is a per pixel based fusion framework constructed by considering the imaging physics.

Several widely used methods have been proposed for fusing high resolution panchromatic data and lower resolution multi-channel data. However, many of these methods fail to maintain the spectral consistency of the fused high resolution image, which is of high importance to many of the applications based on satellite data. Additionally, most conventional methods are loosely connected to the image forming physics of the satellite image, giving these methods an ad hoc feel. Vesteinsson et al. [1] proposed a method of fusion of satellite images that is based on the properties of imaging physics in a statistically meaningful way and was called spectral consistent panshapening (SCP). In this paper we improve this framework for satellite image fusion by introducing a better image prior, via data-dependent image smoothing. The dependency is obtained via total variation edge detection method.

Many visual cues for surface reconstruction from known views are sparse in nature, e.g., specularities, surface silhouettes, and salient features in an otherwise textureless region. Often, these cues are the only information available to an observer. To allow these constraints to be used either in conjunction with dense constraints such as pixel-wise similarity, or alone, we formulate such constraints in a variational framework. We propose a sparse variational constraint in the level set framework, enforcing a surface to pass through a specific point, and a sparse variational constraint on the surface normal along the observed viewing direction, as is the nature of, e.g., specularities. These constraints are capable of reconstructing surfaces from extremely sparse data. The approach has been applied and validated on the shape from specularities problem.

The normals of closed, smooth surfaces have long been used to determine whether a point is inside or outside such a surface. It is tempting to also use this method for polyhedra represented as triangle meshes. Unfortunately, this is not possible since, at the vertices and edges of a triangle mesh, the surface is not C1 continuous, hence, the normal is undefined at these loci. In this paper, we undertake to show that the angle weighted pseudonormal (originally proposed by Thürmer and Wüthrich and independently by Séquin) has the important property that it allows us to discriminate between points that are inside and points that are outside a mesh, regardless of whether a mesh vertex, edge, or face is the closest feature. This inside-outside information is usually represented as the sign in the signed distance to the mesh. In effect, our result shows that this sign can be computed as an integral part of the distance computation. Moreover, it provides an additional argument in favor of the angle weighted pseudonormals being the natural extension of the face normals. Apart from the theoretical results, we also propose a simple and efficient algorithm for computing the signed distance to a closed C0 mesh. Experiments indicate that the sign computation overhead when running this algorithm is almost negligible.

Factorization algorithms for recovering structure and motion from an image stream have many advantages, but they usually require a set of well-tracked features. Such a set is in generally not available in practical applications. There is thus a need for making factorization algorithms deal effectively with errors in the tracked features. We propose a new and computationally efficient algorithm for applying an arbitrary error function in the factorization scheme. This algorithm enables the use of robust statistical techniques and arbitrary noise models for the individual features. These techniques and models enable the factorization scheme to deal effectively with mismatched features, missing features, and noise on the individual features. The proposed approach further includes a new method for Euclidean reconstruction that significantly improves convergence of the factorization algorithms. The proposed algorithm has been implemented as a modification of the Christy-Horaud factorization scheme, which yields a perspective reconstruction. Based on this implementation, a considerable increase in error tolerance is demonstrated on real and synthetic data. The proposed scheme can, however, be applied to most other factorization algorithms.