We introduce a simple and efficient procedure for the segmentation of rigidly moving objects, imaged under an affine camera model. For this purpose we revisit the theory of "linear combination of views" (LCV), proposed by Ullman and Basri [20], which states that the set of 2d views of an object undergoing 3d rigid transformations, is embedded in a low-dimensional linear subspace that is spanned by a small number of basis views. Our work shows, that one may use this theory for motion segmentation, and cluster the trajectories of 3d objects using only two 2d basis views. We therefore propose a practical motion segmentation method, built around LCV, that is very simple to implement and use, and in addition is very fast, meaning it is well suited for real-time SfM and tracking applications. We have experimented on real image sequences, where we show good segmentation results, comparable to the state-of-the-art in literature. If we also consider computational complexity, our proposed method is one of the best performers in combined speed and accuracy. © 2011. The copyright of this document resides with its authors.

We propose a method for segmenting an arbitrary number of moving objects using the geometry of 6 points in 2D images to infer motion consistency. This geometry allows us to determine whether or not observations of 6 points over several frames are consistent with a rigid 3D motion. The matching between observations of the 6 points and an estimated model of their configuration in 3D space is quantified in terms of a geometric error derived from distances between the points and 6 corresponding lines in the image. This leads to a simple motion inconsistency score that is derived from the geometric errors of 6 points, that in the ideal case should be zero when the motion of the points can be explained by a rigid 3D motion. Initial clusters are determined in the spatial domain and merged in motion trajectory domain based on the score. Each point is then assigned to a cluster by assigning the point to the segment of the lowest score. Our algorithm has been tested with real image sequences from the Hopkins155 database with very good results, competing with the state of the art methods, particularly for degenerate motion sequences. In contrast the motion segmentation methods based on multi-body factorization, that assumes an affine camera model, the proposed method allows the mapping from the 3D space to the 2D image to be fully projective.

The paper describes a minimal set of 18 parameters that can represent any trifocal tensor consistent with the internal constraints. 9 parameters describe three orthogonal matrices and 9 parameters describe 10 elements of a sparse tensor T̃ with 17 elements in well-defined positions equal to zero. Any valid trifocal tensor is then given as some specific T̃ transformed by the orthogonal matrices in the respective image domain. The paper also describes a simple approach for estimating the three orthogonal matrices in the case of a general 3 × 3 × 3 tensor, i.e., when the internal constraints are not satisfied. This can be used to accomplish a least squares approximation of a general tensor to a tensor that satisfies the internal constraints. This type of constraint enforcement, in turn, can be used to obtain an improved estimate of the trifocal tensor based on the normalized linear algorithm, with the constraint enforcement as a final step. This makes the algorithm more similar to the corresponding algorithm for estimation of the fundamental matrix. An experiment on synthetic data shows that the constraint enforcement of the trifocal tensor produces a significantly better result than without enforcement, expressed by the positions of the epipoles, given that the constraint enforcement is made in normalized image coordinates.

Triangulation of a 3D point from two or more views can be solved in several ways depending on how perturbations in the image coordinates are dealt with. A common approach is optimal triangulation which minimizes the total L2 reprojection error in the images, corresponding to finding a maximum likelihood estimate of the 3D point assuming independent Gaussian noise in the image spaces. Computational approaches for optimal triangulation have been published for the stereo case and, recently, also for the three-view case. In short, they solve an independent optimization problem for each 3D point, using relatively complex computations such as finding roots of high order polynomials or matrix decompositions. This paper discuss three-view triangulation and reports the following results: (1) the 3D point can be computed as multi-linear mapping (tensor) applied on the homogeneous image coordinates, (2) the set of triangulation tensors forms a 7-dimensional space determined by the camera matrices, (3) given a set of corresponding 3D/2D calibration points, the 3D residual L1 errors can be optimized over the elements in the 7-dimensional space, (4) using the resulting tensor as initial value, the error can be further reduced by tuning the tensor in a two-step iterative process, (5) the 3D residual L1 error for a set of evaluation points which lie close to the calibration set is comparable to the three-view optimal method. In summary, three-view triangulation can be done by first performing an optimization of the triangulation tensor and once this is done, triangulation can be made with 3D residual error at the same level as the optimal method, but at a much lower computational cost. This makes the proposed method attractive for real-time three-view triangulation of large data sets provided that the necessary calibration process can be performed. 1

This paper presents a method for triangulation of 3D points given their projections in two images. Recent results show that the triangulation mapping can be represented as a linear operator K applied to the outer product of corresponding homogeneous image coordinates, leading to a triangulation of very low computational complexity. K can be determined from the camera matrices, together with a so-called blind plane, but we show here that it can be further refined by a process similar to gold standard methods for camera matrix estimation. In particular, it is demonstrated that K can be adjusted to minimize the Euclidean L1residual 3D error, bringing it down to the same level as the optimal triangulation by Hartley and Sturm. The resulting K optimally fits a set of 2D+2D+3D data where the error is measured in the 3D space. Assuming that this calibration set is representative for a particular application, where later only the 2D points are known, this K can be used for triangulation of 3D points in an optimal way, which in addition is very efficient since the optimization need only be made once for the point set. The refinement of K is made by iteratively reducing errors in the 3D and 2D domains, respectively. Experiments on real data suggests that very few iterations are needed to accomplish useful results.

Point-of-interest detection is a way of reducing the amount of data that needs to be processed in a certain application and is widely used in 2D image analysis. In 2D image analysis, point-of-interest detection is usually related to extraction of local descriptors for object recognition, classification, registration or pose estimation. In analysis of range data however, some local descriptors have been published in the last decade or so, but most of them do not mention any kind of point-of-interest detection. We here show how to use an extended Harris detector on range data and discuss variants of the Harris measure. All described variants of the Harris detector for 3D should also be usable in medical image analysis, but we focus on the range data case. We do present a performance evaluation of the described variants of the Harris detector on range data.

We have developed a system which integrates the information output from several pose estimation algorithms and from several views of the scene. It is tested in a real setup with a robotic manipulator. It is shown that integrating pose estimates from several algorithms increases the overall performance of the pose estimation accuracy as well as the robustness as compared to using only a single algorithm. It is shown that increased robustness can be achieved by using pose estimation algorithms based on complementary features, so called algorithmic multi-cue integration (AMC). Furthermore it is also shown that increased accuracy can be achieved by integrating pose estimation results from different views of the scene, so-called temporal multi-cue integration (TMC). Temporal multi-cue integration is the most interesting aspect of this paper

A runtime system for implementation of image processing operations is presented. It is designed for working in a flexible and distributed environment related to the software architecture of a newly developed UAV system. The software architecture can be characterized at a coarse scale as a layered system, with a deliberative layer at the top, a reactive layer in the middle, and a processing layer at the bottom. At a finer scale each of the three levels is decomposed into sets of modules which communicate using CORBA, allowing system development and deployment on the UAV to be made in a highly flexible way. Image processing takes place in a dedicated module located in the process layer, and is the main focus of the paper. This module has been designed as a runtime system for data flow graphs, allowing various processing operations to be created online and on demand by the higher levels of the system. The runtime system is implemented in Java, which allows development and deployment to be made on a wide range of hardware/software configurations. Optimizations for particular hardware platforms have been made using Java's native interface.

This paper describes a method for extracting point features from an image, corresponding to corners and crossings of lines. The method is based on a local estimation of a 6/spl times/6 tensor which describes the parameters of a pair of line segments. By considering the rank of the tensor, it is possible to find points of interest. These points can then be further analyzed to provide detailed information about the configuration of the segments. The proposed method is intended for features which can be used for estimation of position and pose of 3D objects, e.g., for the purpose of grasping.

The paper makes a short presentation of three existing methods for estimation of orientation tensors, the so-called structure tensor, quadrature filter based techniques, and techniques based on approximating a local polynomial model. All three methods can be used for estimating an orientation tensor, which in the 3D case can be used for motion estimation. The methods are based on rather different approaches in terms of the underlying signal models. However, they produce more or less similar results, which indicates that there should be a common framework for estimation of the tensors. Such a framework is proposed, in terms of a second order mapping from signal to tensor with additional conditions on the mapping. It is also shown that the three methods in principle fall into this framework.

Recently substantial research has been devoted to Unmanned Aerial Vehicles (UAVs). One of a UAV's most demanding subsystem is vision. The vision subsystem must dynamically combine different algorithms as the UAVs goal and surrounding change. To fully utilize the available hardware, a run time system must be able to vary the quality and the size of regions the algorithms are applied to, as the number of image processing tasks changes. To allow this the run time system and the underlying computational model must be integrated. In this paper we present a computational model suitable for integration with a run time system. The computational model is called Image Processing Data Flow Graph (IP-DFG). IP-DFG has been developed for modeling of complex image processing algorithms. IP-DFG is based on data flow graphs, but has been extended with hierarchy and new rules for token consumption, which makes the computational model more flexible and more suitable for human interaction. In this paper we also show that IP-DFGs are suitable for modelling expressions, including data dependent decisions and iterations, which are common in complex image processing algorithms.

The paper discusses the concepts of invariance and equivariance in the context of continuous operator groups, or Lie groups, acting on both the image and a descriptor of some image feature. The theory is illustrated by an example of how a 3D image neighbourhood that contains a linear structure can be mapped to a tensor that represents the orientation of the structure. A second order tensor is defined which represents both local orientation and local phase in a continuous way, the phase is given by the relation between the symmetric and anti-symmetric part of this tensor. By computing a tensor inner product of the tensor, a phase invariant orientations representation is obtained.

This paper establishes an algebraic relation between two methods recently reported; normalized convolution and normalized differential convolution. These are general methods for filtering incomplete or uncertain data and are based on the separation of both data and operator into a signal part and a certainty part. General filtering can be performed without preprocessing input data with an interpolation step. The methods allow both data and operators to be scalars, vectors or tensors of higher order. Normalized differential convolution has been used in a wide range of applications. Examples are estimation of gradient estimation in irregularly sampled data, estimation of differential invariants in sparse image flow fields and image edge effect reduction. It was previously shown that normalized convolution produces a description of the neighbourhood which is optimal in a least square sense. The algebraic relation to normalized differential convolution presented in this paper proves that the latter method is also optimal in the same sense as well.>

Invariance is a very important property of features that are useful for vision. A great deal of research on this subject is going on at different labs. While invariance mechanisms can be prescribed for certain descriptors, it is our firm belief that this is not feasible for descriptors of higher level properties in general. As a consequence, these invariance mechanisms have to be learned by the vision system. In this paper, such a learning structure is proposed. A major contribution in this paper, as well as a crucial component for a successful operation, is the use of a coordinate-free information representation: the channel representation. Furthermore, each processing unit is a linear perceptron which operates on outer products of input data, implying a complex space of invariance. Two examples of how the representation can be employed are included. The examples shows an excellent separation of invariance modes, good accuracy, as well as a fast convergence.>