This paper presents a new image sonification system that strives to help visually impaired users access visual information via an audio (easily decodable) signal that is generated in real time when the users explore the image on a touch screen or with a pointer. The sonified signal, which is generated for each position within the image, tries to capture the most useful and discriminant local information about the image content at different levels of abstraction, ranging from low-level (at the pixel level) to high-level (segmentation) and combining low-level (color edges and texture), mid-level and high-level (gradient or color distribution for each region of the image) features. The proposed system mainly uses musical notes at several octaves, the notion of timbre, and loudness but also uses pitch, rhythm and the distortion effect in an intuitive way to sonify the image content both locally and globally. To this end, we use perceptually meaningful mappings, in which the properties of an image are directly reflected in the audio domain, in a very predictable way. The listener can then draw simple and reliable conclusions about the image by quickly decoding the sonified result.

This paper presents a new approach to classify environmental sounds using a texture feature local binary pattern (LBP) and audio features collaboration. To our knowledge, this is the first time that the LBP (or its variants), which has a proven track record in the field of image recognition and classification, has been generalized for 1D and combined with audio features for an environmental sound classification task. To this end, we have generalized and defined LBP-1D and local phase quantization (LPQ)-1D on the 1-dimensional (1D) audio signal and have applied the original LBP, the variance LBP (VARLBP) and the extended LBP (ELBP) thus generated to the spectrogram of the audio signal in order to model the sound texture. We have also extensively compared these new LBP-based features to the classical audio descriptors commonly used in environmental sound classification, such as MFCC, GFCC, CQT, chromagram, STE and ZCR. We have evaluated our algorithm on ESC-10 and ESC-50 datasets using classical machine learning algorithms, such as support vector machines (SVM), random forest and k-nearest neighbor (kNN). The results showed that the LBP features outperform the classical audio features. We mix the LBP features with the audio descriptors, and our best mixed model achieves state-of-the-art results for environmental sound classification: 88.5$\%$on ESC-10 and 64.6$\%$on ESC-50. Those results outperform the results of methods that used handcrafted features with classical machine learning algorithms and are similar to some convolutional neural network-based methods. Although our method is not the cutting edge of the state-of-the-art methods, it is faster than any convolutional neural network methods and represents a better choice when there is data scarcity or minimal computing power.

Change detection in heterogeneous bitemporal satellite images has become an emerging, important, and challenging research topic in remote sensing for rapid damage assessment. In this article, we explore a new parametric mapping strategy based on a modified geometric fractal decomposition and a contractive mapping approach allowing us to project the before image on any after imaging modality type. This projection exploits the fact that any satellite image data can be approximatively encoded in terms of spatial self-similarities at different scales and this property remains quite invariant to a given imaging modality type. Once the projection is performed and that a pixelwise difference map between the two images (presented in the same imaging modality) is then binarized in the unsupervised Bayesian framework. At this stage, we will test several parameter estimation procedures combined with several segmentation strategies based on different Bayesian cost functions. The experiments for change detection, with real images showing different multimodalities and changed events, indicate that this new fractal-based projection method, which is entirely based on a series of structural and spatial information, is an interesting alternative to classical regression-based projection methods (based only on luminance transformation). Besides, the experiments also show that the difference map, resulting in this novel projection strategy, is also particularly amenable for an unsupervised Markovian binarization approach.

This work presents a Bayesian statistical approach to the multimodal change detection (CD) problem in remote sensing imagery. More precisely, we formulate the multimodal CD problem in the unsupervised Markovian framework. The main novelty of the proposed Markovian model lies in the use of an observation field built up from a pixel pairwise modeling and on the bitemporal heterogeneous satellite image pair. Such modeling allows us to rely instead on a robust visual cue, with the appealing property of being quasi-invariant to the imaging (multi-) modality. To use this observation cue as part of a stochastic likelihood model, we first rely on a preliminary iterative estimation technique that takes into account the variety of the laws in the distribution mixture and estimates the parameters of the Markovian mixture model. Once this estimation step is completed, the Maximum a posteriori (MAP) solution of the change detection map, based on the previously estimated parameters, is then computed with a stochastic optimization process. Experimental results and comparisons involving a mixture of different types of imaging modalities confirm the robustness of the proposed approach.

In the last few years, there has been considerable interest in scene parsing. This task consists of assigning a predefined class label to each pixel (or pre-segmented region) in an image. To best address the complexity challenge of this task, first, we propose a new geometric retrieval strategy to select nearest neighbors from a database containing fully segmented and annotated images. Then, we introduce a novel and simple energy-minimization model. The proposed cost function of this model combines efficiently different global nonparametric semantic likelihood energy terms. These terms are computed from the (pre-)segmented regions of the (query) image and their structural properties (location, texture, color, context, and shape). Different from the traditional approaches, we use a simple and local optimization procedure derived from the iterative conditional modes algorithm to optimize our energy-based model. Experimental results on two challenging datasets: 1) microsoft research Cambridge dataset and 2) Stanford background dataset demonstrate the feasibility and the success of the proposed approach. Compared to existing annotation methods that require training classifiers for each object and learning many parameters, our method is easy to implement, has a few parameters, and combines different criteria.

Dynamic texture (DT) segmentation is the problem of clustering into groups various characteristics and phenomena that reproduce in both time and space, assigning a unique label to each group or region. Though this problem is highly complex, it has recently become the focus of considerable interest. This paper presents a simple and effective fusion framework for dynamic texture segmentation, whose objective is to combine multiple and weak region-based segmentation maps to get a final better segmentation result. The different label fields to be fused, are given by a simple clustering technique applied to an input video (based on three orthogonal planes xy, xt and yt). This is using as features a set of values of the requantized local binary patterns (LBP) histogram around the pixel to be classified. Promising preliminary experimental results have been achieved by our method on the challenging SynthDB dataset. Compared to existing dynamic texture segmentation approaches that require estimation of parameters or training classifiers, our method is easy to implement, simple and has few parameters.

Image change detection (CD) is a challenging problem, particularly when images come from different sensors. In this paper, we present a novel and reliable CD model, which is first based on the estimation of a robust similarity-feature map generated from a pair of bitemporal heterogeneous remote sensing images. This similarity-feature map, which is supposed to represent the difference between the multitemporal multisensor images, is herein defined, by specifying a set of linear equality constraints, expressed for each pair of pixels existing in the before-and-after satellite images acquired through different modalities. An estimation of this overconstrained problem, also formulated as a nonlocal pairwise energy-based model, is then carried out, in the least square sense, by a fast linear-complexity algorithm based on a multidimensional scaling mapping technique. Finally, the fusion of different binary segmentation results, obtained from this similarity-feature map by different automatic thresholding algorithms, allows us to precisely and automatically classify the changed and unchanged regions. The proposed method is tested on satellite data sets acquired by real heterogeneous sensor, and the results obtained demonstrate the robustness of the proposed model compared with the best existing state-of-the-art multimodal CD methods recently proposed in the literature.

Semantic image segmentation has recently become the focus of considerable interest. This task consists in assigning a predefined class label to each pixel (or pre-segmented region) in an image. To address the complexity challenge of this task, we develop, in this work, a novel and simple energy-minimization model. The proposed cost function of this model combines efficiently different global non-parametric semantic likelihood energy terms computed from the (pre-)segmented regions of the (query) image and their structural properties (location, texture, color, context and shape). To optimize our energy-based model, we use a local optimization procedure derived from the iterative conditional modes (ICM) algorithm. Experimental results on the challenging Microsoft Research Cambridge dataset (MSRC-21) clearly shows the feasibility and the merits of the proposed approach.

Image segmentation fusion is defined as the set of methods which aim at merging several image segmentations, in a manner that takes full advantage of the complementarity of each one. Previous relevant researches in this field have been impeded by the difficulty in identifying an appropriate single segmentation fusion criterion, providing the best possible, i.e., the more informative, result of fusion. In this paper, we propose a new model of image segmentation fusion based on multi-objective optimization which can mitigate this problem, to obtain a final improved result of segmentation. Our fusion framework incorporates the dominance concept in order to efficiently combine and optimize two complementary segmentation criteria, namely, the global consistency error and the F-measure (precision-recall) criterion. To this end, we present a hierarchical and efficient way to optimize the multi-objective consensus energy function related to this fusion model, which exploits a simple and deterministic iterative relaxation strategy combining the different image segments. This step is followed by a decision making task based on the so-called "technique for order performance by similarity to ideal solution". Results obtained on two publicly available databases with manual ground truth segmentations clearly show that our multi-objective energy-based model gives better results than the classical mono-objective one.

In this paper, we introduce a new fusion model whose objective is to fuse multiple region-based segmentation maps to get a final better segmentation result. The suggested new fusion model is based on an energy function originated from the global consistency error (GCE), a perceptual measure which takes into account the inherent multiscale nature of an image segmentation by measuring the level of refinement existing between two spatial partitions. Combined with a region merging/splitting prior, this new energy-based fusion model of label fields allows to define an interesting penalized likelihood estimation procedure based on the GCE criterion with which the fusion of basic, rapidly-computed segmentation results appears as a relevant alternative compared with other (possibly complex) segmentation techniques proposed in the image segmentation field. The performance of our fusion model was evaluated on the Berkeley dataset including various segmentations given by humans (manual ground truth segmentations). The obtained results clearly demonstrate the efficiency of this fusion model.

This paper presents a simple and efficient method for action recognition based on the learning of an explicit representation for an intrinsic dynamic shape manifold of human action. The proposed model relies on a short temporal set of FastMap dimensionality reduction-based technique for embedding a sequence of raw moving silhouettes, associated to an action video into a low-dimensional space, in order to characterize the spatio-temporal property of the action, as well as to preserve much of the geometric structure. The objective is to provide a recognition method that is both simple, fast and applicable in many scenarios. Moreover, we demonstrate the robustness of our method to partial occlusion, deformation of shapes, significant changes in scale and viewpoint, irregularities in the performance of an action, and low-quality video.

In this work, we introduce a new fusion model whose objective is to fuse multiple region-based segmentation maps to get a final better segmentation result. This new fusion model is based on an energy function originated from the global consistency error (GCE), a perceptual measure which takes into account the inherent multiscale nature of an image segmentation by measuring the level of refinement existing between two spatial partitions. Combined with a region merging/splitting prior, this new energy-based fusion model of label fields allows to define an interesting penalized likelihood estimation procedure based on the global consistency error criterion with which the fusion of basic, rapidly-computed segmentation results appears as a relevant alternative compared with other segmentation techniques proposed in the image segmentation field. The performance of our fusion model was evaluated on the Berkeley dataset including various segmentations given by humans.

Fusion of image segmentations using consensus clustering and based on the optimization of a single criterion (commonly called the median partition based approach) may bias and limit the performance of an image segmentation model. To address this issue, we propose, in this paper, a new fusion model of image segmentation based on multi-objective optimization which aims to avoid the bias caused by a single criterion and to achieve a final improved segmentation. The proposed fusion model combines two conflicting and complementary segmentation criteria, namely; the region-based variation of information (VoI) criterion and the contour-based F-Measure (precision-recall) criterion with an entropy-based confidence weighting factor. To optimize our energy-based model we use an optimization procedure derived from the iterative conditional modes (ICM) algorithm. The experimental results on the Berkeley database with manual ground truth segmentations clearly show the effectiveness and the robustness of our multi-objective median partition based approach.

Recently, there has been renewed interest in the fusion of image segmentation. However, previous relevant research has been impeded by the lack of an appropriate single segmentation criterion, which yields an improved final segmentation result. This paper proposes a new framework to tackle this problem. It is based on multi-objective optimization strategy, followed by a decision making technique called: technique for order performance by similarity to ideal solution (TOPSIS). This new fusion framework aims to overcome the limits caused by using a single criterion by combining and optimizing, simultaneously, two different and complementary segmentation criteria; namely, the global consistency error (GCE) (region-based criterion) and the F-measure (edge-based criterion). This new multi-criterion fusion framework is validated on the Berkeley image dataset and compared to different segmentation algorithms (with or without fusion strategy). Experiments show that the results of our new multi-objective approach improve the state of the art in terms of popular indices.

In this paper, we propose a new unsupervised change detection method designed to analyze multispectral remotely sensed image pairs. It is formulated as a segmentation problem to discriminate the changed class from the unchanged class in the difference images. The proposed method is in the category of the committee machine learning model that utilizes an ensemble of classifiers (i.e., the set of segmentation results obtained by several thresholding methods) with a dynamic structure type. More specifically, in order to obtain the final “change/no-change” output, the responses of several classifiers are combined by means of a mechanism that involves the input data (the difference image) under an iterative Bayesian-Markovian framework. The proposed method is evaluated and compared to previously published results using satellite imagery.

We investigate the contour detection task in complex natural images. We propose a novel contour detection algorithm which jointly tracks at two scales small pieces of edges called edgelets. This multiscale edgelet structure naturally embeds semi-local information and is the basic element of the proposed recursive Bayesian modeling. Prior and transition distributions are learned offline using a shape database. Likelihood functions are learned online, thus are adaptive to an image, and integrate color and gradient information via local, textural, oriented, and profile gradient-based features. The underlying model is estimated using a sequential Monte Carlo approach, and the final soft contour detection map is retrieved from the approximated trajectory distribution. We also propose to extend the model to the interactive cut-out task. Experiments conducted on the Berkeley Segmentation data sets show that the proposed MultiScale Particle Filter Contour Detector method performs well compared to competing state-of-the-art methods.

In this paper, we propose an algorithm to detect smooth local symmetries and contours of ribbon-like objects in natural images. The detection is formulated as a spatial tracking task using a particle filtering approach, extracting one part of a structure at a time. Using an adaptive local geometric model, the method can detect straight reflection symmetries in perfectly symmetrical objects as well as smooth local symmetries in curved elongated objects. In addition, the proposed approach jointly estimates spine and contours, making it possible to generate back ribbon objects. Experiments for local symmetry detection have been conducted on a recent extension of the Berkeley segmentation data sets. We also show that it is possible to retrieve specific geometrical objects using intuitive prior structural information.

This paper proposes a new nonlinear dimensionality-reduction model based on a bicriteria global optimization approach for the color display of hyperspectral images. The proposed fusion model is derived from two well-known and contradictory criteria of good visualization, which are useful in any multidimensional imagery color display, namely, accuracy, with the preservation of spectral distance criterion, and contrast, guaranteeing that colors are well distinguished or concretely allowing the good separability of each observed existing material in the final visualized color image. An internal parameter allows our algorithm to express the contribution or the importance of these two criteria for a specific application. In this framework, which also can be viewed as a classical Bayesian optimization strategy involving a tradeoff between fidelity to the unreduced (raw) spectral data and the expected highly contrasted resulting mapping, we will show that a hybrid optimization strategy, combining a global and deterministic optimization procedure and a local stochastic search using the Metropolis criterion, can be exploited to efficiently minimize the complex nonlinear objective cost function related to our model. The experiments reported in this paper demonstrate that the proposed model, taking into account these two criteria of good visualization, makes easier and more reliable the interpretation and quick overview of such multidimensional hyperspectral images.

In this correspondence, we present an original energy-based model that achieves the edge-histogram specification of a real input image and thus extends the exact specification method of the image luminance (or gray level) distribution recently proposed by Coltuc et al. Our edge-histogram specification approach is stated as an optimization problem in which each edge of a real input image will tend iteratively toward some specified gradient magnitude values given by a target edge distribution (or a normalized edge histogram possibly estimated from a target image). To this end, a hybrid optimization scheme combining a global and deterministic conjugate-gradient-based procedure and a local stochastic search using the Metropolis criterion is proposed herein to find a reliable solution to our energy-based model. Experimental results are presented, and several applications follow from this procedure.

In this paper, we present an efficient coarse-to-fine multiresolution framework for multidimensional scaling and demonstrate its performance on a large-scale nonlinear dimensionality reduction and embedding problem in a texture feature extraction step for the unsupervised image segmentation problem. We demonstrate both the efficiency of our multiresolution algorithm and its real interest to learn a nonlinear low-dimensional representation of the texture feature set of an image which can then subsequently be exploited in a simple clustering-based segmentation algorithm. The resulting segmentation procedure has been successfully applied on the Berkeley image database, demonstrating its efficiency compared to the best existing state-of-the-art segmentation methods recently proposed in the literature.

In this paper, we present a nonstationary Markov random field (MRF) fusion model for the color display of hyperspectral images. The proposed fusion or dimensionality reduction model is derived from the preservation of spectral distance criterion. This quantitative metric of good dimensionality reduction and meaningful visualization allows us to derive an appealing fusion model of high-dimensional spectral data, expressed as a Gibbs distribution or a nonstationary MRF model defined on a complete graph. In this framework, we propose a computationally efficient coarse-to-fine conjugate-gradient optimization method to minimize the cost function related to this energy-based fusion model. The experiments reported in this paper demonstrate that the proposed visualization method is efficient (in terms of preservation of spectral distances and discriminality of pixels with different spectral signatures) and performs well compared to the best existing state-of-the-art multidimensional imagery color display methods recently proposed in the literature.

This paper presents a novel segmentation approach based on a Markov random field (MRF) fusion model which aims at combining several segmentation results associated with simpler clustering models in order to achieve a more reliable and accurate segmentation result. The proposed fusion model is derived from the recently introduced probabilistic Rand measure for comparing one segmentation result to one or more manual segmentations of the same image. This non-parametric measure allows us to easily derive an appealing fusion model of label fields, easily expressed as a Gibbs distribution, or as a nonstationary MRF model defined on a complete graph. Concretely, this Gibbs energy model encodes the set of binary constraints, in terms of pairs of pixel labels, provided by each segmentation results to be fused. Combined with a prior distribution, this energy-based Gibbs model also allows for definition of an interesting penalized maximum probabilistic rand estimator with which the fusion of simple, quickly estimated, segmentation results appears as an interesting alternative to complex segmentation models existing in the literature. This fusion framework has been successfully applied on the Berkeley image database. The experiments reported in this paper demonstrate that the proposed method is efficient in terms of visual evaluation and quantitative performance measures and performs well compared to the best existing state-of-the-art segmentation methods recently proposed in the literature.

In this paper, a new unsupervised hierarchical approach to textured color images segmentation is proposed. To this end, we have designed a two-step procedure based on a grey-scale Markovian over-segmentation step, followed by a Markovian graph-based clustering algorithm, using a decreasing merging threshold schedule, which aims at progressively merging neighboring regions with similar textural features. This hierarchical segmentation method, using two levels of representation, has been successfully applied on the Berkeley Segmentation Dataset and Benchmark (BSDB, Martin et al., 2001). The experiments reported in this paper demonstrate that the proposed method is efficient in terms of visual evaluation and quantitative performance measures and performs well compared to the best existing state-of-the-art segmentation methods recently proposed in the literature.

3D Brain SPECT imagery is a well established functional imaging method which has become a great help to physicians in the diagnosis of several neurological and cerebrovascular diseases. However, mainly due to the effects of attenuation and the scattering of emitted photons, inherent to this imaging process, 3D SPECT images are generally blurred and exhibit poor spatial resolution. This leads to substantial errors in measurements of regional brain blood flow, and therefore in the estimations of brain activity. In order to improve the resolution of these images and then to facilitate their interpretation, we herein propose an original extension of the NAS-RIF (Recursive Inverse Filtering) deconvolution technique proposed by Kundur and Hatzinakos [1]. The proposed extension allows to efficiently integrate, in the deconvolution process, a set of soft constraints given by a probabilistic MRI atlas containing experts's prior knowledge about the spatial localization of the different brain structures (or tissue classes). This extension has three interesting properties ; first it allows to exploit (or fuse) reliable anatomical and (high resolution) geometrical information extracted horn a probabilistic 3D MRI atlas. Second, it allows to incorporate, into the NAS-RIF method, a regularization term which efficiently stabilizes the inverse solution. Third and contrary to multi-modal restoration techniques, it does not require a MRI scan of the patient. This method has been successfully tested on numerous real brain SPECT images (of different patients suffering from epilepsy), yielding promising restoration results.

This paper presents a new, simple, and efficient segmentation approach, based on a fusion procedure which aims at combining several segmentation maps associated to simpler partition models in order to finally get a more reliable and accurate segmentation result. The different label fields to be fused in our application are given by the same and simple (K-means based) clustering technique on an input image expressed in different color spaces. Our fusion strategy aims at combining these segmentation maps with a final clustering procedure using as input features, the local histogram of the class labels, previously estimated and associated to each site and for all these initial partitions. This fusion framework remains simple to implement, fast, general enough to be applied to various computer vision applications (e.g., motion detection and segmentation), and has been successfully applied on the Berkeley image database. The experiments herein reported in this paper illustrate the potential of this approach compared to the state-of-the-art segmentation methods recently proposed in the literature.

In this paper, we present a new model for deformations of shapes. A pseudo-likelihood is based on the statistical distribution of the gradient vector field of the gray level. The prior distribution is based on the Probabilistic Principal Component Analysis (PPCA). We also propose a new model based on mixtures of PPCA that is useful in the case of greater variability in the shape. A criterion of global or local object specificity based on a preliminary color segmentation of the image, is included into the model. The localization of a shape in an image is then viewed as minimizing the corresponding Gibbs field. We use the Exploration/Selection (E/S) stochastic algorithm in order to find the optimal deformation. This yields a new unsupervised statistical method for localization of shapes. In order to estimate the statistical parameters for the gradient vector field of the gray level, we use an Iterative Conditional Estimation (ICE) procedure. The color segmentation of the image can be computed with an Exploration/Selection/Estimation (ESE) procedure.

In this letter, we show that the performance of image denoising algorithms using wavelet transforms can be improved by a post-processing deconvolution step that takes into account the inherent blur function created by the considered wavelet based denoising system. The interest of the proposed deblurring procedure is illustrated on denoised images reconstructed by shrinkage of curvelet and undecimated wavelet coefficients. Experimental results reported here show that the proposed post-processing technique yields improvements in term of image quality and lower mean square error, especially when the image is corrupted by strong additive white Gaussian noise.

Most statistical background subtraction techniques are based on the analysis of temporal color/intensity distribution. However, learning statistics on a series of time frames can be problematic, especially when no frame absent of moving objects is available or when the available memory is not sufficient to store the series of frames needed for learning. In this letter, we propose a spatial variation to the traditional temporal framework. The proposed framework allows statistical motion detection with methods trained on one background frame instead of a series of frames as is usually the case. Our framework includes two spatial background subtraction approaches suitable for different applications. The first approach is meant for scenes having a nonstatic background due to noise, camera jitter or animation in the scene (e.g.,waving trees, fluttering leaves). This approach models each pixel with two PDFs: one unimodal PDF and one multimodal PDF, both trained on one background frame. In this way, the method can handle backgrounds with static and nonstatic areas. The second spatial approach is designed to use as little processing time and memory as possible. Based on the assumption that neighboring pixels often share similar temporal distribution, this second approach models the background with one global mixture of Gaussians.

In this paper, we put forward a novel fusion framework that mixes together label fields instead of observation data as is usually the case. Our framework takes as input two label fields: a quickly estimated and to-be-refined segmentation map and a spatial region map that exhibits the shape of the main objects of the scene. These two label fields are fused together with a global energy function that is minimized with a deterministic iterative conditional mode algorithm. As explained in the paper, the energy function may implement a pure fusion strategy or a fusion-reaction function. In the latter case, a data-related term is used to make the optimization problem well posed. We believe that the conceptual simplicity, the small number of parameters, the use of a simple and fast deterministic optimizer that admits a natural implementation on a parallel architecture are among the main advantages of our approach. Our fusion framework is adapted to various computer vision applications among which are motion segmentation, motion estimation and occlusion detection.

This paper investigates the problem of image denoising when the image is corrupted by additive white Gaussian noise. We herein propose a spatial adaptive denoising method which is based on an averaging process performed on a set of Markov Chain Monte-Carlo simulations of region partition maps constrained to be spatially piecewise uniform (i.e., constant in grey level value sense) for each estimated constant-value regions. For the estimation of these region partition maps, we have adopted the unsupervised Markovian framework in which parameters are automatically estimated in the least square sense. This sequential averaging allows to obtain, under our image model, an approximation of the image to be recovered in the minimal mean square sense error. The experiments reported in this paper demonstrate that the discussed method performs competitively and sometimes better than the best existing state-of-the-art wavelet-based denoising methods in benchmark tests.

This paper presents a novel region-based approach for detecting occlusion between two consecutive frames. Based on a generalization of Marr and Poggio’s uniqueness assumption, the explicit goal of our method is to reduce the number of false positives while optimizing the hit rate. To do so, our method relies on a fusion procedure that blends together two segmentation maps: one pre-estimated occlusion binary map and one color segmentation map. While the occlusion map is obtained after a simple thresholding procedure, the color segmentation map is obtained with an unsupervised Markovian approach. Assuming that the color segmentation regions exhibit more precise edges, the occlusion areas are iteratively modified to fit the colorregion shapes. Since our method has been entirely implemented on a parallel architecture (a Graphics Processor Unit), its processing times are remarkably low. Our method is compared with other occlusion approaches both quantitatively and qualitatively on scenes that represent different challenges. 1

Nowadays, brain 3D SPECT is a well established functional imaging method that is widely used in clinical settings for the assessment of neurological and cerebrovascular diseases. However, due to the scattering of the emitted photons, inherent to this imaging process, brain 3D SPECT images exhibit poor spatial and inter-slice resolution. More precisely, SPECT images are blurred, leading to substantial errors in measurement of regional brain activity and making difficult and subjective, a reliable and accurate diagnosis by the nuclear physician. In order to improve the resolution of these images and then to facilitate their interpretation, we herein propose an original extension of the NAS-RIF deconvolution technique of Kundur and Hatzinakos. The proposed extension has two interesting properties; it allows to exploit or fuse anatomical and geometrical information extracted from a high resolution anatomical magnetic resonance (MR) image and also to efficiently incorporate, into the NAS-RIF method, a regularization term to stabilize the inverse solution. In our application, this anatomical-based regularization term exploits the result of an unsupervised Markovian segmentation obtained after a preliminary registration step between the MR and SPECT data volume coming from a same patient. This method has been successfully tested on numerous pairs of brain MR and SPECT images of different patients, yielding very promising restoration results.

This paper presents a differential optical flow method which accounts for two typical motion-estimation problems : (1) flow regularization within regions of uniform motion while (2) preserving sharp edges near motion discontinuities i.e., where motion is mul-timodal by nature. The method proposed is a modified version of the well known Lucas Kanade (LK) algorithm. Based on documented assumptions, our method computes motion with a classical least-square fit on a local neighborhood shifted away from where motion is likely to be multimodal. This edge-avoidance procedure is based on the non-parametric mean-shift algorithm which shifts the LK integration window away from local sharp edges. Our method also locally regularizes motion by performing a fusion of local motion estimates. Our method is compared with other edge-preserving methods on image sequences representing different challenges.

In this paper, we propose a light and fast pixel-based statistical motion detection method based on a background subtraction procedure. The statistical representation of the background relies on its spatial color distributions herein modeled by a mixture of Gaussians. The Gaussian parameters are obtained after segmenting one reference frame with an unsupervised Bayesian approach whose parameter estimation step is ensured by the K-means and the iterated conditional estimation (ICE) algorithms. Since the motion detection function only depends on a global mixture of M Gaussians, only a few bits per pixel need to be stored in memory. Our method achieves real-time performances, especially when look up tables are used to store pre-calculated data. Results have been obtained on synthetic and real video sequences and compared with other statistical methods.

In this paper, we present a Hidden Markov Random Field (HMRF) data-fusion model. The proposed model is applied to the segmentation of natural images based on the fusion of colors and textons into Julesz ensembles. The corresponding Exploration/ Selection/Estimation (ESE) procedure for the estimation of the parameters is presented. This method achieves the estimation of the parameters of the Gaussian kernels, the mixture proportions, the region labels, the number of regions, and the Markov hyper-parameter. Meanwhile, we present a new proof of the asymptotic convergence of the ESE procedure, based on original finite time bounds for the rate of convergence.

This paper proposes a new and original inhomogeneous restoration (deconvolution) model under the Bayesian framework for observed images degraded by space-invariant blur and additive Gaussian noise. In this model, regularization is achieved during the iterative restoration process with a segmentation-based a priori term. This adaptive edge-preserving regularization term applies a local smoothness constraint to pre-estimated constant-valued regions of the target image. These constant-valued regions (the segmentation map) of the target image are obtained from a preliminary Wiener deconvolution estimate. In order to estimate reliable segmentation maps, we have also adopted a Bayesian Markovian framework in which the regularized segmentations are estimated in the maximum a posteriori (MAP) sense with the joint use of local Potts prior and appropriate Gaussian conditional luminance distributions. In order to make these segmentations unsupervised, these likelihood distributions are estimated in the maximum likelihood sense. To compute the MAP estimate associated to the restoration, we use a simple steepest descent procedure resulting in an efficient iterative process converging to a globally optimal restoration. The experiments reported in this paper demonstrate that the discussed method performs competitively and sometimes better than the best existing state-of-the-art methods in benchmark tests.

This paper proposes an original inhomogeneous restoration (deconvolution) model under the Bayesian framework. In this model, regularization is achieved, during the iterative restoration process, with an adaptive segmentation-based regularization term whose goal is to apply local smoothness constraints on estimated constant areas of the image to be recovered. To this end, the parameters of this restoration a priori model relies on an unsupervised Markovian over-segmentation. To compute the MAP estimate associated to the restoration, we use a simple steepest descent procedure resulting in an efficient iterative process converging to a globally optimal restoration. The experiments reported in this paper demonstrate that the discussed method performs competitively and sometimes better than the best existing state-of-the-art methods in benchmark tests.

We propose a new stochastic algorithm for computing useful Bayesian estimators of hidden Markov random field (HMRF) models that we call exploration/selection/estimation (ESE) procedure. The algorithm is based on an optimization algorithm of O. François, called the exploration/selection (E/S) algorithm. The novelty consists of using the a posteriori distribution of the HMRF, as exploration distribution in the E/S algorithm. The ESE procedure computes the estimation of the likelihood parameters and the optimal number of region classes, according to global constraints, as well as the segmentation of the image. In our formulation, the total number of region classes is fixed, but classes are allowed or disallowed dynamically. This framework replaces the mechanism of the split-and-merge of regions that can be used in the context of image segmentation. The procedure is applied to the estimation of a HMRF color model for images, whose likelihood is based on multivariate distributions, with each component following a Beta distribution. Meanwhile, a method for computing the maximum likelihood estimators of Beta distributions is presented. Experimental results performed on 100 natural images are reported. We also include a proof of convergence of the E/S algorithm in the case of nonsymmetric exploration graphs.

In this paper, we present a robust method for estimating the model parameters in a mixture of probabilistic principal component analyzers. This method is based on the stochastic version of the expectation maximization (SEM) algorithm. Parameters of this mixture model are herein used to constrain the 3D reconstruction problem of scoliotic rib cage from a pair of planar and conventional calibrated radiographic images (postero-anterior with normal incidence (I/sub PA/) and lateral (I/sub LAT/)). More precisely, the proposed PPCA mixture model is herein robustly exploited for dimensionality reduction and to get a set of probabilistic prior models associated to each detected class of pathological deformations observed on a representative training scoliotic rib cage population. By using an appropriate likelihood and for each considered class-conditional prior model, the proposed 3D reconstruction is stated as an energy function minimization problem, which is solved with a stochastic optimization algorithm. The optimal 3D reconstruction then corresponds to the class of deformation and parameters leading to the minimal energy. This 3D method of reconstruction has been successfully tested on several biplanar radiographic images, yielding very promising results.

In this work, we propose an unsupervised Bayesian model for the detection of moving objects from dynamic scenes. This unsupervised solution is a three-step approach that uses a statistical model of an interframe gradient norm field (as likelihood model) with a local regularization term (as prior model) combined with strong intraframe spatial constraints. In the first step, the spatial constraints are estimated by making an unsupervised Markovian spatial over-segmentation of two input frames. In the second step, the interframe gradient (derived from the input frames) is restored to minimize undesired noise. In the last step, an unsupervised Markovian temporal segmentation (with global spatial constraints) is performed to generate the desired motion label field. The maximum a posteriori (MAP) estimation of the label field associated with the spatial segmentations (in the first step) and the motion label field (in the third step) is performed by a classical Iterative Conditional Mode (ICM) algorithm. An Iterative Conditional Estimation (ICE) procedure is exploited for estimating the parameters of the spatial model and the region-constrained temporal model. This new statistical method of motion detection has been successfully applied to real dynamic scenes and seems to be well suited for the temporal detection of noisy image sequences.

This paper investigates the use of a region-based approach for the stereo matching problem. We have stated this problem in a commonly adopted global energy-based framework. Our energy-based model mixes a local and robust regularization term with global spatial constraints. These constraints are related to a (precomputed) partition into homogeneous regions with identical disparity. In practice, our approach assigns a single disparity to regions instead of individual pixels. These regions, used to globally constrain the ill-posed nature of our minimization problem, are estimated by combining an unsupervised Markovian segmentation and a roughly estimated disparity map. This disparity map is computed with a basic winner-take-all (WTA) procedure. The proposed global energy function seems to be well suited to find good disparity discontinuities at object boundaries, especially when the number of disparities is large. An iterated conditional modes (ICM) algorithm is used to optimize this global energy function. We provide experimental results on real stereo image pairs. A quality measure, based on ground truth data, is used to evaluate the performance of our algorithm. Results indicate that our approach is fast and performs well compared to other existing methods.

In this paper, we describe a statistical model for the gradient vector field of the gray level in images validated by different experiments. Moreover, we present a global constrained Markov model for contours in images that uses this statistical model for the likelihood. Our model is amenable to an Iterative Conditional Estimation (ICE) procedure for the estimation of the parameters; our model also allows segmentation by means of the Simulated Annealing (SA) algorithm, the Iterated Conditional Modes (ICM) algorithm, or the Modes of Posterior Marginals (MPM) Monte Carlo (MC) algorithm. This yields an original unsupervised statistical method for edge-detection, with three variants. The estimation and the segmentation procedures have been tested on a total of 160 images. Those tests indicate that the model and its estimation are valid for applications that require an energy term based on the log-likelihood ratio. Besides edge-detection, our model can be used for semiautomatic extraction of contours, localization of shapes, non-photo-realistic rendering; more generally, it might be useful in various problems that require a statistical likelihood for contours.

This paper investigates the use of a nonparametric regularization energy term for devising a example-based rendering and segmentation technique. We have stated this problem in the multiresolution energy minimization framework and exploited the multiscale structure proposed by Wei and Levoy for the texture synthesis problem. In this nonparametric energy minimization framework, we also propose a computationally efficient coarse-to-fine recursive optimization method to minimize the cost function related to this hierarchical model. In this context, the formulation of our example-based regularization term also allows to directly infer an intuitive dissimilarity measure between two contour shapes. This measure is herein exploited to define an efficient shape descriptor for the contour-based shape recognition and indexing problem.

In this paper, we propose a new and accurate 3D reconstruction technique for the scoliotic spine from a pair planar and conventional radiographic images (postero-anterior and lateral). The proposed model uses a priori hierarchical global knowledge, both on the geometric structure of the whole spine and of each vertebra. More precisely, it relies on the specification of two 3D templates. The first, a rough geometric template on which rigid admissible deformations are defined, is used to ensure a crude registration of the whole spine. 3D reconstruction is then refined for each vertebra, by a template on which nonlinear admissible global deformations are modeled, with statistical modal analysis of the pathological deformations observed on a representative scoliotic vertebra population. This unsupervised coarse-to-fine 3D reconstruction procedure is stated as a double energy function minimization problems efficiently solved with a stochastic optimization algorithm. The proposed method, tested on several pairs of biplanar radiographic images with scoliotic deformities, is comparable in terms of accuracy with the classical CT-scan technique while being unsupervised and requiring a lower amount of radiation for the patient.

In this paper, we describe a new hidden Markov random field model, which we call hierarchical multidata model, and which is based on a triplet of random fields (two hidden random fields and one observed field) in order to capture interscale and within-scale dependencies between various scales of resolution of wavelet-based texture features. We present a variation of the iterated conditional modes (ICM) algorithm for the segmentation, and an adaptation of the iterative conditional estimation (ICE) procedure for the estimation of the statistical parameters of the model. Results of tests performed on 75 mosaics of Brodatz textures are reported.

This paper investigates the use of the Bayesian inference for devising an unsupervised sketch rendering procedure. As likelihood model of this inference, we exploit the recent statistical model of the gradient vector field distribution proposed by Destrempes et al. for contour detection. A global prior deformation model for each pencil stroke is also considered. In this Bayesian framework, the placement of each stroke is viewed as the search of the maximum a posteriori estimation of the posterior distribution of its deformations. We use a stochastic optimization algorithm in order to find these optimal deformations. This yields an unsupervised method to create realistic hand-sketched pencil drawings. Combined with an example-based local rendering model, used to transfer the textural tone value of a given depiction style, the proposed scheme allows to simulate automatic synthesis of various artistic illustration styles.

In this paper, we describe an unsupervised segmentation method for contours which proves quite adapted for the images obtained by electronic acquisition. We present two statistical models for the norm of the gradient of the gray level at the pixels of an Image, one for contour points and one for points outside contours. We also describe a Markov model with constraint which incorporates those two statistical distributions as likelihood together with a simple a priori model. Our model is suitable for an iterative conditional estimation (ICE) procedure for the estimation of the parameters and an iterated conditional modes (ICM) algorithm, or simulated annealing, for the segmentation. A preliminary step proceeds to the segmentation of the image into sub-regions and uses a Markov model without constraint based on the gray level distribution on the image.

In this paper we present a new and robust shape descriptor which can be efficiently used to quickly prune a search for similar shapes in a large Image database. The proposed shape descriptor is based on a multiscale representation of the discrete set of points, sampled from the internal and external contour points of the query and the candidate shapes. In this approach, dissimilarity between two shapes is defined as the reconstruction error of the candidate shape, made by using multiscale elements of contours extracted from the query shape. This dissimilarity measure allows one to quickly produce an accurate short-list of candidate matches, ranked from the most similar to the least similar one, suitable for a more careful and more time consuming matching algorithm. Experiments on the Snodgrass & Vanderwart database allows one to attest the discriminating power of this measure and its robustness to possible distortions, warping and occlusion artifacts.

This paper presents a new 3D reconstruction method of the scoliotic vertebrae of a spine, using two conventional radiographic views (postero-anterior and lateral), and global prior knowledge on the geometrical structure of each vertebra. This geometrical knowledge is efficiently captured by a statistical deformable template integrating a set of admissible deformations, expressed by the first modes of variation in the Karhunen-Loeve expansion of the pathological deformations observed on a representative scoliotic vertebra population. The proposed reconstruction method consists in fitting the projections of this deformable template with the segmented contours of the corresponding vertebra on the two radiographic views. The 3D reconstruction problem is stated as the minimization of a cost function for each vertebra and solved with a gradient descent technique. The reconstruction of the spine is then made vertebra by vertebra. This 3D reconstruction method has been successfully tested on several biplanar radiographic images, yielding very promising results.

We present an original statistical classification method using a deformable template model to separate natural objects from man-made objects in an image provided by a high resolution sonar. A prior knowledge of the manufactured object shadow shape is captured by a prototype template, along with a set of admissible linear transformations, to take into account the shape variability. Then, the classification problem is defined as a two-step process: 1) the detection problem of a region of interest in the input image is stated as the minimization of a cost function; and 2) the value of this function at convergence allows one to determine whether the desired object is present or not in the sonar image. The energy minimization problem is tackled using relaxation techniques. In this context, we compare the results obtained with a deterministic relaxation technique and two stochastic relaxation methods: simulated annealing and a hybrid genetic algorithm. This latter method has been successfully tested on real and synthetic sonar images, yielding very promising results.

This paper is concerned with hierarchical Markov random field (MRP) models and their application to sonar image segmentation. We present an original hierarchical segmentation procedure devoted to images given by a high-resolution sonar. The sonar image is segmented into two kinds of regions: shadow (corresponding to a lack of acoustic reverberation behind each object lying on the sea-bed) and sea-bottom reverberation. The proposed unsupervised scheme takes into account the variety of the laws in the distribution mixture of a sonar image, and it estimates both the parameters of noise distributions and the parameters of the Markovian prior. For the estimation step, we use an iterative technique which combines a maximum likelihood approach (for noise model parameters) with a least-squares method (for MRF-based prior). In order to model more precisely the local and global characteristics of image content at different scales, we introduce a hierarchical model involving a pyramidal label field. It combines coarse-to-fine causal interactions with a spatial neighborhood structure. This new method of segmentation, called the scale causal multigrid (SCM) algorithm, has been successfully applied to real sonar images and seems to be well suited to the segmentation of very noisy images. The experiments reported in this paper demonstrate that the discussed method performs better than other hierarchical schemes for sonar image segmentation.

We present a new method to shape-based segmentation of deformable anatomical structures in medical images and validate this approach by detecting and tracking the endocardial border in an echographic image sequence. To this end, a global prior knowledge of the endocardial contour is captured by a prototype template with a set of admissible deformations to take into account its inherent natural variability over time. In this approach, the data likelihood model rely on an accurate statistical modeling of the grey level distribution of each class present in the image. The parameters of this distribution mixture are given by a preliminary estimation step which takes into account the distribution shape of each class. Then the tracking problem is stated in a Bayesian framework where it ends up as an optimization problem. This one is then efficiently solved by a genetic algorithm combined with a steepest ascent procedure. This technique has been successfully applied on synthetic images and on a real echocardiographic image sequence.

We present a new classification method using a deformable template model to separate natural objects from man made objects in an image given by a high resolution sonar. A prior knowledge of the manufactured object shadow shape is described by a prototype template and a set of admissible linear transformations to take into account the shape variability. Then, the classification problem is defined as a two step process; firstly the detection problem of a region of interest in the input image is stated in a Bayesian framework and is posed as an equivalent energy minimization problem of an objective function: in this paper, this energy minimization problem is solved by using a hybrid genetic algorithm (GA). Secondly, the value of this function at convergence allows one to determine the presence of the desired object in the sonar image. This method has been successfully tested on real and synthetic sonar images, yielding very promising results.

This work deals with unsupervised sonar image segmentation. We present a new estimation segmentation procedure using the an iterative method called iterative conditional estimation (ICE). This method takes into account the variety of the laws in the distribution mixture of a sonar image and the estimation of the parameters of the label field (modeled by a Markov random field (MRF)). For the estimation step we use a maximum likelihood estimation for the noise model parameters and the least square method proposed by Derin et al. (1987) to estimate the MRF prior model. Then, in order to obtain a good segmentation and to speed up the convergence rate, we use a multigrid strategy with the previously estimated parameters. This technique has been successfully applied to real sonar images and is compatible with an automatic treatment of massive amounts of data.