Detecting building change in bitemporal remote sensing (RS) imagery requires a model to highlight the changes in buildings and ignore the irrelevant changes of other objects and sensing conditions. Buildings have comparatively less diverse textures than other objects and appear as repetitive visual patterns on RS images. In this paper, we propose Gabor Feature Network (GFN) to extract the distinctive repetitive texture features of buildings. Furthermore, we also design Feature Fusion Module (FFM) to fuse the extracted multiscale features from GFN with the features from a Transformer-based encoder to pass on the texture features to different parts of the model. Using GFN and FFM, we design a Transformer-based model, called GabFormer for building change detection. Experimental results on the LEVIR-CD and WHU-CD datasets indicate that GabFormer outperforms other SOTA models and in particular show significant improvement in the generalization capability. Our code is available on https://github.com/Ayana-Inria/GabFormer.

A novel learnable line segment detector and descriptor is proposed which allows efficient extraction and matching of 2D lines via the angular distance of 128 dimensional unit descriptor vectors. While many handcrafted and deep features have been proposed for keypoints, only a few methods exist for line segments. It is well known, however, that line segments are commonly found in man-made environments, in particular urban scenes, thus they are important for applications like pose estimation, visual odometry, or 3D reconstruction. Our method relies on a 2-stage deep convolutional neural network architecture: In stage 1, candidate 2D line segments are detected, and in stage 2, a descriptor is generated for the extracted lines. The network is trained in a self-supervised way using an automatically collected dataset of matching and non-matching line segments across (substantially) different views of 3D lines. Experimental results confirm the state-of-the-art performance of the proposed L2D2 network on two well-known datasets for autonomous driving both in terms of detected line matches as well as when used for line-based camera pose estimation and tracking.

A novel method is proposed for the absolute pose estimation of a central 2D camera with respect to 3D depth data without the use of any dedicated calibration pattern or explicit point correspondences. The proposed method has no specific assumption about the data source: plain depth information is expected from the 3D sensing device and a central camera is used to capture the 2D images. Both the perspective and omnidirectional central cameras are handled within a single generic camera model. Pose estimation is formulated as a 2D-3D nonlinear shape registration task which is solved without point correspondences or complex similarity metrics. It relies on a set of corresponding planar regions, and the pose parameters are obtained by solving an overdetermined system of nonlinear equations. The efficiency and robustness of the proposed method were confirmed on both large scale synthetic data and on real data acquired from various types of sensors.

Low-rank regions capture geometrically meaningful structures in an image which encompass typical local features such as edges, corners and all kinds of regular, symmetric, often repetitive patterns, that are commonly found in man-made environment. While such patterns are challenging current state-of-the-art feature correspondence methods, the recovered homography of a low-rank texture readily provides 3D structure with respect to a 3D plane, without any prior knowledge of the visual information on that plane. However, the automatic and efficient detection of the broad class of low-rank regions is unsolved. Herein, we propose a novel self-supervised low-rank region detection deep network that predicts a low-rank likelihood map from an image. The evaluation of our method on real-world datasets shows not only that it reliably predicts low-rank regions in the image similarly to our baseline method, but thanks to the data augmentations used in the training phase it generalizes well to difficult cases (e.g. day/night lighting, low contrast, underexposure) where the baseline prediction fails.

We propose a novel method to compute the absolute pose of a generalized camera based on straight lines, which are common in urban environment. The only assumption about the imaging model is that 3D straight lines are projected via projection planes determined by the line and camera projection directions, i.e. correspondences are given as a 3D world line and its projection plane. Since modern cameras are frequently equipped with various location and orientation sensors, we assume that the vertical direction (e.g. a gravity vector) is available. Therefore we formulate the problem in terms of 4 unknowns using 3D line - projection plane correspondences which yields a closed form solution. The solution can be used as a minimal solver as well as a least squares solver without reformulation. The proposed algorithm have been evaluated on various synthetic datasets as well as on real data. Experimental results confirm state of the art performance both in terms of quality and computing time.

A workflow is proposed for Cultural Heritage applications in which the fusion of 3D and 2D visual data is required. Using data acquired by cheap, standard devices, like a 3D scanner having a low quality 2D camera in it, and a high resolution DSLR camera, one can produce high quality color calibrated 3D model for documenting purpose. The proposed processing workflow combines a novel region based calibration method with an ICP alignment used for refining the results. It works on 3D data, that do not necessarily contain intensity information in them, and 2D images of a calibrated camera. These can be acquired with commercial 3D scanners and color cameras without any special constraint. In contrast with the typical solutions, the proposed method is not using any calibration patterns or markers. The efficiency and robustness of the proposed calibration method has been confirmed on both synthetic and real data.

This paper presents a novel approach to deformable image registration using a parametric transformation model. The aligning transformation is simply found by solving a system of non-linear equations. Each equation is generated by integrating the product of a set of polynomial functions and the image intensity function over corresponding image domains. The capabilities of the algorithm have been studied on synthetic and real datasets on which it compares favorably to state of the art methods in terms of alignment accuracy.

This paper addresses the problem of simultaneous estimation of different linear deformations, resulting in a global non-linear transformation, between an original object and its broken fragments. A general framework is proposed without using correspondences, where the solution of a polynomial system of equations directly provides the parameters of the alignment. We quantitatively evaluate the proposed algorithm on a large synthetic dataset containing 2D and 3D images, where linear (rigid-body and affine) transformations are considered. We also conduct an exhaustive analysis of the robustness against segmentation errors and the numerical stability of the proposed method. Moreover, we present experiments on 2D real images as well as on volumetric medical images.

A novel affine registration technique is proposed which efficiently handles the partial occlusion problem. The proposed method is working without point wise correspondences or radiometric information's. The aligning transformation is found by solving a system of non-linear equations and minimizing the area of occlusions. Each equation is generated by integrating a non-linear function over the foreground domains of the shapes. In order to deal with occlusions, the shapes are approximated by polygons and the occluded areas are determined with simple boolean operators. Based on the tests on a large synthetic dataset, the proposed approach outperforms state of the art algorithms.

In this paper, we propose a Multilayer Markovian model for change detection in registered aerial image pairs with large time differences. A Three Layer Markov Random Field takes into account information from two different sets of features namely the Modified HOG (Histogram of Oriented Gradients) difference and the Gray-Level (GL) Difference. The third layer is the resultant combination of the two layers. Thus we integrate both the texture level as well as the pixel level information to generate the final result. The proposed model uses pair wise interaction retaining the sub-modularity condition for energy. Hence a global energy optimization can be achieved using a standard min-cut/ max flow algorithm ensuring homogeneity in the connected regions.

A novel correspondence-less approach is proposed to find a thin plate spline map between a pair of deformable 3D objects represented by triangular surface meshes. The proposed method works without landmark extraction and feature correspondences. The aligning transformation is found simply by solving a system of nonlinear equations. Each equation is generated by integrating a nonlinear function over the object's domains. We derive recursive formulas for the efficient computation of these integrals. Based on a series of comparative tests on a large synthetic dataset, our triangular mesh-based algorithm outperforms state of the art methods both in terms of computing time and accuracy. The applicability of the proposed approach has been demonstrated on the registration of 3D lung CT volumes.

In this paper, we propose a novel framework to estimate the parameters of a diffeomorphism that aligns a known shape and its distorted observation. Classical registration methods first establish correspondences between the shapes and then compute the transformation parameters from these landmarks. Herein, we trace back the problem to the solution of a system of nonlinear equations which directly gives the parameters of the aligning transformation. The proposed method provides a generic framework to recover any diffeomorphic deformation without established correspondences. It is easy to implement, not sensitive to the strength of the deformation, and robust against segmentation errors. The method has been applied to several commonly used transformation models. The performance of the proposed framework has been demonstrated on large synthetic data sets as well as in the context of various applications.

We propose a registration method to find affine transformations between 3D objects by constructing and solving an overdetermined system of polynomial equations. We utilize voxel coverage information for more precise object boundary description. An iterative solution enables us to easily adjust the method to recover e.g. rigid-body and similarity transformations. Synthetic tests show the advantage of the voxel coverage representation, and reveal the robustness properties of our method against different types of segmentation errors. The method is tested on a real medical CT volume.

We propose a novel Markovian segmentation model which takes into account edge information. By construction, the model uses only pairwise interactions and its energy is sub modular. Thus the exact energy minima is obtained via a max-flow/min-cut algorithm. The method has been quantitatively evaluated on synthetic images as well as on fluorescence microscopic images of live cells.

Images taken from different views of a planar object are related by planar homography. Recovering the parameters of such transformations is a fundamental problem in computer vision with various applications. This paper proposes a novel method to estimate the parameters of a homography that aligns two binary images. It is obtained by solving a system of nonlinear equations generated by integrating linearly independent functions over the domains determined by the shapes. The advantage of the proposed solution is that it is easy to implement, less sensitive to the strength of the deformation, works without established correspondences and robust against segmentation errors. The method has been tested on synthetic as well as on real images and its efficiency has been demonstrated in the context of two different applications: alignment of hip prosthesis X-ray images and matching of traffic signs.

We propose a binary Markov random field (MRF) model that assigns high probability to regions in the image domain consisting of an unknown number of circles of a given radius. We construct the model by discretizing the `gas of circles' phase field model in a principled way, thereby creating an `equivalent'MRF. The behaviour of the resulting MRF model is analyzed, and the performance of the new model is demonstrated on various synthetic images as well as on the problem of tree crown detection in aerial images.

A direct approach for parametric estimation of 2D affine deformations between compound shapes is proposed. It provides the result as a least-square solution of a linear system of equations. The basic idea is to fit Gaussian densities over the objects yielding covariant functions, which preserves the effect of the unknown transformation. Based on these functions, linear equations are constructed by integrating nonlinear functions over appropriate domains. The main advantages are: linear complexity, easy implementation, works without any time consuming optimization or established correspondences. Comparative tests show that it outperforms state-of-the-art methods both in terms of precision, robustness and complexity.

A novel approach is proposed to estimate the parameters of a diffeomorphism that aligns two binary images. Classical approaches usually define a cost function based on a similarity metric and then find the solution via optimization. Herein, we trace back the problem to the solution of a system of nonlinear equations which directly provides the parameters of the aligning transformation. The proposed method works without any time consuming optimization step or established correspondences. The advantage of our algorithm is that it is easy to implement, less sensitive to the strength of the deformation, and robust against segmentation errors. The efficiency of the proposed approach has been demonstrated on a large synthetic dataset as well as in the context of an industrial application.

We propose a new Bayesian method for detecting the regions of object displacements in aerial image pairs. We use a robust but coarse 2-D image registration algorithm. Our main challenge is to eliminate the registration errors from the extracted change map. We introduce a three-layer Markov random field (L(3)MRF) model which integrates information from two different features, and ensures connected homogenous regions in the segmented images. Validation is given on real aerial photos.

We consider the problem of planar object registration on binary images where the aligning transformation is restricted to the group of affine transformations. Previous approaches usually require established correspondences or the solution of nonlinear optimization problems. Herein we show that it is possible to formulate the problem as the solution of a system of up to third order polynomial equations. These equations are constructed in a simple way using some basic geometric information of binary images. It does not need established correspondences nor the solution of complex optimization problems. The resulting algorithm is fast and provides a direct solution regardless of the magnitude of transformation.

In this paper, we give a probabilistic model for automatic change detection on airborne images taken with moving cameras. To ensure robustness, we adopt an unsupervised coarse matching instead of a precise image registration. The challenge of the proposed model is to eliminate the registration errors, noise and the parallax artifacts caused by the static objects having considerable height (buildings, trees, walls etc.) from the difference image. We describe the background membership of a given image point through two different features, and introduce a novel three-layer Markov random field (MRF) model to ensure connected homogenous regions in the segmented image.

Reversible jump Markov chain Monte Carlo (RJMCMC) is a recent method which makes it possible to construct reversible Markov chain samplers that jump between parameter subspaces of different dimensionality. In this paper, we propose a new RJMCMC sampler for multivariate Gaussian mixture identification and we apply it to color image segmentation. For this purpose, we consider a first order Markov random field (MRF) model where the singleton energies derive from a multivariate Gaussian distribution and second order potentials favor similar classes in neighboring pixels. The proposed algorithm finds the most likely number of classes, their associated model parameters and generates a segmentation of the image by classifying the pixels into these classes. The estimation is done according to the Maximum A Posteriori (MAP) criterion. Experimental results are promising, we have obtained accurate results on a variety of real color images. 1

Herein, we propose a novel multilayer Markov random field (MRF) image segmentation model which aims at combining color and texture features: each feature is associated to a so called feature layer, where an MRF model is defined using only the corresponding feature. A special layer is assigned to the combined MRF model. This layer interacts with each feature layer and provides the segmentation based on the combination of different features. The model is quite generic and isn't restricted to a particular texture feature. Herein we will test the algorithm using Gabor and MRSAR texture features. Furthermore, the algorithm automatically estimates the number of classes at each layer (there can be different classes at different layers) and the associated model parameters.

In this paper, we will show how non-photorealistic rendering (NPR) can take a new role in content-based image retrieval (CBIR). The proposed CBIR method applies a novel image similarity measure: unlike traditional features like color, texture, or shape, our measure is based on a painted representation of the original image. This is produced by a stochastic paintbrush algorithm which simulates a painting process. We use the stroke parameters (color, size, orientation, and location) as features and similarity is measured by matching strokes of a pair of images. The advantage of our approach is that it provides information not only about the color content but also about the structural properties of an image without the segmentation of the image. Experimental results show that the CBIR method using paintbrush features has higher retrieval rate than traditional methods using color or texture features only.

We propose a new content based image retrieval method. The novelty of our approach lies in the applied image similarity measure: unlike traditional features, such as color, texture or shape, our measure is based on a painted representation of the original image. We use paintbrush stroke parameters as features. These strokes are produced by a stochastic paintbrush algorithm which simulates a painting process. Stroke parameters include color, orientation and location. Therefore, it provides information not only about the color content but also about the structural properties of an image. Experimental results on a database of more than 500 images show that the CBIR method using paintbrush features has a higher retrieval rate than methods using color features only.

This paper deals with the problem of unsupervised Bayesian segmentation of images modeled by Markov random fields (MRF). If the model parameters are known then we have various methods to solve the segmentation problem (simulated annealing, ICM, etc...). However, when they are not known, the problem becomes more difficult. One has to estimate the hidden label field parameters from the available image only. Our approach consists of a recent iterative method of estimation, called iterative conditional estimation (ICE), applied to a monogrid Markovian image segmentation model. The method has been tested on synthetic and real satellite images.

The paper deals with the problem of unsupervised classification of images modeled by Markov random fields (MRF). If the model parameters are known then we have various methods to solve the segmentation problem (simulated annealing, ICM, etc...). However, when they are not known, the problem becomes more difficult. One has to estimate the hidden label field parameters from the only observable image. Our approach consists of extending a recent iterative method of estimation, called iterative conditional estimation (ICE) to a hierarchical Markovian model. The idea resembles the estimation-maximization (EM) algorithm as we recursively look at the maximum a posteriori (MAP) estimate of the label field given the estimated parameters then we look at the maximum likelihood (ML) estimate of the parameters given a tentative labeling obtained at the previous step. We propose unsupervised image classification algorithms using a hierarchical model. The only parameter supposed to be known is the number of regions, all the other parameters are estimated. The presented algorithms have been implemented on a Connection Machine CM200. Comparative tests have been done on noisy synthetic and real images (remote sensing).>

Deterministic pseudo-annealing (DPA) is a new deterministic optimization method for finding the maximum a posteriori (MAP) labeling in a Markov random field, in which the probability of a tentative labeling is extended to a merit function on continuous labelings. This function is made convex by changing its definition domain. This unambiguous maximization problem is solved, and the solution is followed down to the original domain, yielding a good, if suboptimal, solution to the original labeling assignment problem. The performance of DPA is analyzed on randomly weighted graphs.

As it is well known, optimization of the energy function of Markov random fields is very expensive. Hierarchical models have usually much more communication per pixel than monogrid ones. This is why classical annealing schemes are too slow, even on a parallel machine, to minimize the energy associated with such a model. However, taking benefit of the pyramidal structure of the model, we can define a new annealing scheme: the multitemperature annealing (MTA), which consists of associating higher temperatures to coarser levels, in order to be less sensitive to local minima at coarser grids. The convergence to the global optimum is proved by a generalisation of the annealing theorem of Geman and Geman (1984). We have applied the algorithm to image classification and tested it on synthetic and real images.

The authors consider multiscale Markov random field (MRF) models. It is well known that multigrid methods can improve significantly the convergence rate and the quality of the final results of iterative relaxation techniques. A hierarchical model is proposed, which consists of a label pyramid and a whole observation field. The parameters of the coarse grid can be derived by simple computation from the finest grid. In the label pyramid, a new local interaction is introduced between two neighbor grids. This model gives a relaxation algorithm which can be run in parallel on the entire pyramid. The model allows propagation of local interactions more efficiently, giving estimates closer to the global optimum for deterministic as well as for stochastic relaxation schemes. It can also be seen as a way to incorporate cliques with far apart sites for a reasonable price.>

A pseudo-stochastic variation of the Metropolis dynamics for combinatorial optimization in image classification using Markov random fields is presented. At high temperature, the behavior of the algorithm is similar to the stochastic ones. However, if the temperature is less than a certain threshold, it becomes deterministic. The length of the pseudo-stochastic phase is controlled by a constant threshold used in the modified dynamics. The algorithm yields an approximate but usually good solution to the optimization problem. The algorithm runs on a connection machine. It is applied to the standard pixel classification problem; objective and subjective comparisons with other algorithms have been made.>