Segmentation and detection of biological objects in fluorescence microscopy is of paramount importance in cell imaging. Deep learning approaches have recently shown promise to advance, automatize and accelerate analysis. However, most of the interest has been given to the segmentation of static objects of 2D/3D images whereas the segmentation of dynamic processes obtained from time-lapse acquisitions has been less explored. Here we adapted DeepFinder, a U-Net originally designed for 3D noisy cryo-electron tomography (cryo-ET) data, for the detection of rare dynamic exocytosis events (termed ExoDeepFinder) observed in temporal series of 2D Total Internal Reflection Fluorescence Microscopy (TIRFM) images. ExoDeepFinder achieved good absolute performances with a relatively small training dataset of 12000 events in 60 cells. We rigorously compared deep learning performances with unsupervised conventional methods from the literature. ExoDeepFinder outcompeted the tested methods, but also exhibited a greater plasticity to the experimental conditions when tested under drug treatments and after changes in cell line or imaged reporter. This robustness to unseen experimental conditions did not require re-training demonstrating generalization capability of our deep learning model. ExoDeepFinder, as well as the annotated training datasets, were made transparent and available through an open-source software as well as a Napari plugin and can directly be applied to custom user data. The apparent plasticity and performances of ExoDeepFinder to detect dynamic events open new opportunities for future deep learning guided analysis of dynamic processes in live-cell imaging.

Abstract. We propose a unified view of nonlocal methods for single-image denoising, for which BM3D is the most popular representative, that operate by gathering noisy patches together according to their similarities in order to process them collaboratively. Our general estimation framework is based on the minimization of the quadratic risk, which is approximated in two steps, and adapts to photon and electronic noises. Relying on an unbiased risk estimate (URE) for the first step and on “internal adaptation,” a concept borrowed from deep learning theory, for the second, we show that our approach enables one to reinterpret and reconcile previous state-of-the-art nonlocal methods. Within this framework, we propose a novel denoiser called NL-Ridge that exploits linear combinations of patches. While conceptually simpler, we show that NL-Ridge can outperform well-established state-of-the-art single-image denoisers.

In the past decade, deep neural networks have revolutionized image denoising in achieving significant accuracy improvements by learning on datasets composed of noisy/clean image pairs. However, this strategy is extremely dependent on training data quality, which is a well-established weakness. To alleviate the requirement to learn image priors externally, single-image (a.k.a., self-supervised or zero-shot) methods perform denoising solely based on the analysis of the input noisy image without external dictionary or training dataset. This work investigates the effectiveness of linear combinations of patches for denoising under this constraint. Although conceptually very simple, we show that linear combinations of patches are enough to achieve state-of-the-art performance. The proposed parametric approach relies on quadratic risk approximation via multiple pilot images to guide the estimation of the combination weights. Experiments on images corrupted artificially with Gaussian noise as well as on real-world noisy images demonstrate that our method is on par with the very best single-image denoisers, outperforming the recent neural network-based techniques, while being much faster and fully interpretable.

In many information processing systems, it may be desirable to ensure that any change of the input, whether by shifting or scaling, results in a corresponding change in the system response. While deep neural networks are gradually replacing all traditional automatic processing methods, they surprisingly do not guarantee such normalization-equivariance (scale + shift) property, which can be detrimental in many applications. To address this issue, we propose a methodology for adapting existing neural networks so that normalization-equivariance holds by design. Our main claim is that not only ordinary convolutional layers, but also all activation functions, including the ReLU (rectified linear unit), which are applied element-wise to the pre-activated neurons, should be completely removed from neural networks and replaced by better conditioned alternatives. To this end, we introduce affine-constrained convolutions and channel-wise sort pooling layers as surrogates and show that these two architectural modifications do preserve normalization-equivariance without loss of performance. Experimental results in image denoising show that normalization-equivariant neural networks, in addition to their better conditioning, also provide much better generalization across noise levels.

We propose a unified view of unsupervised non-local methods for image denoising that linearly combine noisy image patches. The best methods, established in different modeling and estimation frameworks, are two-step algorithms. Leveraging Stein’s unbiased risk estimate (SURE) for the first step and the “internal adaptation”, a concept borrowed from deep learning theory, for the second one, we show that our NL-Ridge approach enables to reconcile several patch aggregation methods for image denoising. In the second step, our closed-form aggregation weights are computed through multivariate Ridge regressions. Experiments on artificially noisy images demonstrate that NL-Ridge may outperform well established state-of-the-art unsupervised denoisers such as BM3D and NL-Bayes, as well as recent unsupervised deep learning methods, while being simpler conceptually.

Timelines are common visualizations to represent and manipulate temporal data. However, timeline visualizations rarely consider spatio-temporal 3D data (e.g. mesh or volumetric models) directly. In this paper, leveraging the increased workspace and 3D interaction capabilities of virtual reality (VR), we first propose a timeline design space for 3D temporal data extending the timeline design space proposed by Brehmer et al. [7]. The proposed design space adapts the scale, layout and representation dimensions to account for the depth dimension and how the 3D temporal data can be partitioned and structured. Moreover, an additional dimension is introduced, the support, which further characterizes the 3D dimension of the visualization. The design space is complemented by discussing the interaction methods required for the efficient visualization of 3D timelines in VR. Secondly, we evaluate the benefits of 3D timelines through a formal evaluation (n=21). Taken together, our results showed that time-related tasks can be achieved more comfortably using timelines, and more efficiently for specific tasks requiring the analysis of the surrounding temporal context. Finally, we illustrate the use of 3D timelines with a use-case on morphogenetic analysis in which domain experts in cell imaging were involved in the design and evaluation process.

SUMMARY: Analysis of intra- and extracellular dynamic like vesicles transport involves particle tracking algorithms. The design of a particle tracking pipeline is a routine but tedious task. Therefore, particle dynamics analysis is often performed by combining several pieces of software (filtering, detection, tracking, etc.) requiring many manual operations, and thus leading to poorly reproducible results. Given the new segmentation tools based on deep learning, modularity and interoperability between software have become essential in particle tracking algorithms. A good synergy between a particle detector and a tracker is of paramount importance. In addition, a user-friendly interface to control the quality of estimated trajectories is necessary. To address these issues, we developed STracking, a Python library that allows combining algorithms into standardized particle tracking pipelines. AVAILABILITY AND IMPLEMENTATION: STracking is available as a Python library using 'pip install' and the source code is publicly available on GitHub (https://github.com/sylvainprigent/stracking). A graphical interface is available using two napari plugins: napari-stracking and napari-tracks-reader. These napari plugins can be installed via the napari plugins menu or using 'pip install'. The napari plugin source codes are available on GitHub (https://github.com/sylvainprigent/napari-tracks-reader, https://github.com/sylvainprigent/napari-stracking). SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

This work tackles the issue of noise removal from images, focusing on the well-known DCT image denoising algorithm. The latter, stemming from signal processing, has been well studied over the years. Though very simple, it is still used in crucial parts of state-of-the-art "traditional" denoising algorithms such as BM3D. For a few years however, deep convolutional neural networks (CNN), especially DnCNN, have outperformed their traditional counterparts, making signal processing methods less attractive. In this paper, we demonstrate that a DCT denoiser can be seen as a shallow CNN and thereby its original linear transform can be tuned through gradient descent in a supervised manner, improving considerably its performance. This gives birth to a fully interpretable CNN called DCT2net. To deal with remaining artifacts induced by DCT2net, an original hybrid solution between DCT and DCT2net is proposed combining the best that these two methods can offer; DCT2net is selected to process non-stationary image patches while DCT is optimal for piecewise smooth patches. Experiments on artificially noisy images demonstrate that two-layer DCT2net provides comparable results to BM3D and is as fast as DnCNN algorithm.

The new generation of confocal microscopes are equipped with an array detector that generates an array of images corresponding to a multiview of the same sample. Several computational methods have been proposed to reconstruct a single super-resolution image from a stack of images associated to detectors. Each method has its pros and cons depending on the targeted application. In this paper, we review the most commonly used reconstruction methods and propose a SURE approach to automatically estimate parameters and improve reconstruction. Methods described in this paper are available in an open source software.

It is of primary interest for biologists to be able to visualize the dynamics of proteins within the cell. In this paper, we propose a new mapping method to robustly estimate dynamics in the entire cell from particle tracks. To obtain satisfying diffusion and drift maps, we use a spatiotemporal kernel estimator. Trajectory classification data is used as input and allows to automatically label particle movements into three classes: confined motion (or subdiffusion), Brownian motion, and directed motion (or superdiffusion). We then use this information to calculate diffusion coefficient and drift maps separately on each class of motion.

We present an overview of diffusion models commonly used for quantifying the dynamics of intracellular particles (e.g. biomolecules) inside eukaryotic living cells. It is established that inference on the modes of mobility of molecules is central in cell biology since it reflects interactions between structures and determines functions of biomolecules in the cell. In that context, Brownian motion is a key component in short distance transportation (e.g. connectivity for signal transduction). Another dynamical process that has been heavily studied in the past decade is the motor-mediated transport (e.g. dynein, kinesin and myosin) of molecules. Primarily supported by actin filament and microtubule network, it ensures spatial organization and temporal synchronization in the intracellular mechanisms and structures. Nevertheless, the complexity of internal structures and molecular processes in the living cell influence the molecular dynamics and prevent the systematic application of pure Brownian or directed motion modeling. On the one hand, cytoskeleton density will hinder the free displacement of the particle, a phenomenon called subdiffusion. On the other hand, the cytoskeleton elasticity combined with thermal bending can contribute a phenomenon called superdiffusion. This paper discusses the basics of diffusion modes observed in eukariotic cells, by introducing the essential properties of these processes. Applications of diffusion models include protein trafficking and transport and membrane diffusion.

MOTIVATION: Recent advances in molecular biology and fluorescence microscopy imaging have made possible the inference of the dynamics of single molecules in living cells. Changes of dynamics can occur along a trajectory. Then, an issue is to estimate the temporal change-points that is the times at which a change of dynamics occurs. The number of points in the trajectory required to detect such changes will depend on both the magnitude and type of the motion changes. Here, the number of points per trajectory is of the order of 102, even if in practice dramatic motion changes can be detected with less points. RESULTS: We propose a non-parametric procedure based on test statistics computed on local windows along the trajectory to detect the change-points. This algorithm controls the number of false change-point detections in the case where the trajectory is fully Brownian. We also develop a strategy for aggregating the detections obtained with different window sizes so that the window size is no longer a parameter to optimize. A Monte Carlo study is proposed to demonstrate the performances of the method and also to compare the procedure to two competitive algorithms. At the end, we illustrate the efficacy of the method on real data in 2D and 3D, depicting the motion of mRNA complexes-called mRNA-binding proteins-in neuronal dendrites, Galectin-3 endocytosis and trafficking within the cell. AVAILABILITY AND IMPLEMENTATION: A user-friendly Matlab package containing examples and the code of the simulations used in the paper is available at http://serpico.rennes.inria.fr/doku.php? id=software:cpanalysis:index. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

MOTIVATION: The revolution in light sheet microscopy enables the concurrent observation of thousands of dynamic processes, from single molecules to cellular organelles, with high spatiotemporal resolution. However, challenges in the interpretation of multidimensional data requires the fully automatic measurement of those motions to link local processes to cellular functions. This includes the design and the implementation of image processing pipelines able to deal with diverse motion types, and 3D visualization tools adapted to the human visual system. RESULTS: Here, we describe a new method for 3D motion estimation that addresses the aforementioned issues. We integrate 3D matching and variational approach to handle a diverse range of motion without any prior on the shape of moving objects. We compare different similarity measures to cope with intensity ambiguities and demonstrate the effectiveness of the Census signature for both stages. Additionally, we present two intuitive visualization approaches to adapt complex 3D measures into an interpretable 2D view, and a novel way to assess the quality of flow estimates in absence of ground truth. AVAILABILITY AND IMPLEMENTATION: https://team.inria.fr/serpico/data/3d-optical-flow-data/. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

BACKGROUND: Over the last two decades, an innovative technology called Tissue Microarray (TMA), which combines multi-tissue and DNA microarray concepts, has been widely used in the field of histology. It consists of a collection of several (up to 1000 or more) tissue samples that are assembled onto a single support - typically a glass slide - according to a design grid (array) layout, in order to allow multiplex analysis by treating numerous samples under identical and standardized conditions. However, during the TMA manufacturing process, the sample positions can be highly distorted from the design grid due to the imprecision when assembling tissue samples and the deformation of the embedding waxes. Consequently, these distortions may lead to severe errors of (histological) assay results when the sample identities are mismatched between the design and its manufactured output. The development of a robust method for de-arraying TMA, which localizes and matches TMA samples with their design grid, is therefore crucial to overcome the bottleneck of this prominent technology. RESULTS: In this paper, we propose an Automatic, fast and robust TMA De-arraying (ATMAD) approach dedicated to images acquired with brightfield and fluorescence microscopes (or scanners). First, tissue samples are localized in the large image by applying a locally adaptive thresholding on the isotropic wavelet transform of the input TMA image. To reduce false detections, a parametric shape model is considered for segmenting ellipse-shaped objects at each detected position. Segmented objects that do not meet the size and the roundness criteria are discarded from the list of tissue samples before being matched with the design grid. Sample matching is performed by estimating the TMA grid deformation under the thin-plate model. Finally, thanks to the estimated deformation, the true tissue samples that were preliminary rejected in the early image processing step are recognized by running a second segmentation step. CONCLUSIONS: We developed a novel de-arraying approach for TMA analysis. By combining wavelet-based detection, active contour segmentation, and thin-plate spline interpolation, our approach is able to handle TMA images with high dynamic, poor signal-to-noise ratio, complex background and non-linear deformation of TMA grid. In addition, the deformation estimation produces quantitative information to asset the manufacturing quality of TMAs.

Detecting spot-like objects of different sizes in images is needed in many applications. Multiple image scales must then be handled for reliable spot segmentation. We define an original criterion based on the a contrario approach and the LoG scale-space framework to automatically select the meaningful scales. We then design a coarse-to-fine multi-scale spot segmentation scheme involving a locally adaptive thresholding across scales, to come up with the final map of segmented spots. We report experimental results on simulated and real images of different types, and we demonstrate that our method outperforms other existing methods.

BACKGROUND: Characterizing membrane dynamics is a key issue to understand cell exchanges with the extra-cellular medium. Total internal reflection fluorescence microscopy (TIRFM) is well suited to focus on the late steps of exocytosis at the plasma membrane. However, it is still a challenging task to quantify (lateral) diffusion and estimate local dynamics of proteins. RESULTS: A new model was introduced to represent the behavior of cargo transmembrane proteins during the vesicle fusion to the plasma membrane at the end of the exocytosis process. Two biophysical parameters, the diffusion coefficient and the release rate parameter, are automatically estimated from TIRFM image sequences, to account for both the lateral diffusion of molecules at the membrane and the continuous release of the proteins from the vesicle to the plasma membrane. Quantitative evaluation on 300 realistic computer-generated image sequences demonstrated the efficiency and accuracy of the method. The application of our method on 16 real TIRFM image sequences additionally revealed differences in the dynamic behavior of Transferrin Receptor (TfR) and Langerin proteins. CONCLUSION: An automated method has been designed to simultaneously estimate the diffusion coefficient and the release rate for each individual vesicle fusion event at the plasma membrane in TIRFM image sequences. It can be exploited for further deciphering cell membrane dynamics.

In this paper, a new denoising algorithm to deal with the additive white Gaussian noise model is described. Following the nonlocal (NL) means approach, we propose an adaptive estimator based on the weighted average of observations taken in a neighborhood with weights depending on the similarity of local patches. The idea is to compute adaptive weights that best minimize an upper bound of the pointwise $L_2$ risk. In the framework of adaptive estimation, we show that the “oracle” weights are optimal if we consider triangular kernels instead of the commonly used Gaussian kernel. Furthermore, we propose a way to automatically choose the spatially varying smoothing parameter for adaptive denoising. Under conventional minimal regularity conditions, the obtained estimator converges at the usual optimal rate. The implementation of the proposed algorithm is also straightforward and the simulations show that our algorithm significantly improves the classical NL means and is competitive when compared to the more sophisticated NL means filters, both in terms of peak signal-to-noise ratio values and visual quality.

One of the major challenges in multiple particle tracking is the capture of extremely heterogeneous movements of objects in crowded scenes. The presence of numerous assignment candidates in the expected range of particle motion makes the tracking ambiguous and induces false positives. Lowering the ambiguity by reducing the search range, on the other hand, is not an option, as this would increase the rate of false negatives. We propose here a piecewise-stationary motion model (PMM) for the particle transport along an iterative smoother that exploits recursive tracking in multiple rounds in forward and backward temporal directions. By fusing past and future information, our method, termed PMMS, can recover fast transitions from freely or confined diffusive to directed motions with linear time complexity. To avoid false positives, we complemented recursive tracking with a robust inline estimator of the search radius for assignment (a.k.a. gating), where past and future information are exploited using only two frames at each optimization step. We demonstrate the improvement of our technique on simulated data especially the impact of density, variation in frame to frame displacements, and motion switching probability. We evaluated our technique on the 2D particle tracking challenge dataset published by Chenouard et al. in 2014. Using high SNR to focus on motion modeling challenges, we show superior performance at high particle density. On biological applications, our algorithm allows us to quantify the extremely small percentage of motor-driven movements of fluorescent particles along microtubules in a dense field of unbound, diffusing particles. We also show with virus imaging that our algorithm can cope with a strong reduction in recording frame rate while keeping the same performance relative to methods relying on fast sampling.

Accurately detecting subcellular particles in fluorescence microscopy is of primary interest for further quantitative analysis such as counting, tracking, or classification. Our primary goal is to segment vesicles likely to share nearly the same size in fluorescence microscopy images. Our method termed adaptive thresholding of Laplacian of Gaussian (LoG) images with autoselected scale (ATLAS) automatically selects the optimal scale corresponding to the most frequent spot size in the image. Four criteria are proposed and compared to determine the optimal scale in a scale-space framework. Then, the segmentation stage amounts to thresholding the LoG of the intensity image. In contrast to other methods, the threshold is locally adapted given a probability of false alarm (PFA) specified by the user for the whole set of images to be processed. The local threshold is automatically derived from the PFA value and local image statistics estimated in a window whose size is not a critical parameter. We also propose a new data set for benchmarking, consisting of six collections of one hundred images each, which exploits backgrounds extracted from real microscopy images. We have carried out an extensive comparative evaluation on several data sets with ground-truth, which demonstrates that ATLAS outperforms existing methods. ATLAS does not need any fine parameter tuning and requires very low computation time. Convincing results are also reported on real total internal reflection fluorescence microscopy images.

Image analysis applied to fluorescence live cell microscopy has become a key tool in molecular biology since it enables to characterize biological processes in space and time at the subcellular level. In fluorescence microscopy imaging, the moving tagged structures of interest, such as vesicles, appear as bright spots over a static or nonstatic background. In this paper, we consider the problem of vesicle segmentation and time-varying background estimation at the cellular scale. The main idea is to formulate the joint segmentation-estimation problem in the general conditional random field framework. Furthermore, segmentation of vesicles and background estimation are alternatively performed by energy minimization using a min cut-max flow algorithm. The proposed approach relies on a detection measure computed from intensity contrasts between neighboring blocks in fluorescence microscopy images. This approach permits analysis of either 2D + time or 3D + time data. We demonstrate the performance of the so-called C-CRAFT through an experimental comparison with the state-of-the-art methods in fluorescence video-microscopy. We also use this method to characterize the spatial and temporal distribution of Rab6 transport carriers at the cell periphery for two different specific adhesion geometries.

Assessing crowd behaviors from videos is a difficult task while of interest in many applications. We have defined a novel approach which identifies from two successive frames only, crowd behaviors expressed by simple image motion patterns. It relies on the estimation of a collection of sub-affine motion models in the image, a local motion classification based on a penalized likelihood criterion, and a regularization stage involving inhibition and reinforcement factors. We have also developed an original and simple method for recovering the dominant paths followed by people in the observed scene. It involves the introduction of local paths determined from the space-time average of the parametric motion subfields selected in each image block. Experiments on synthetic and real scenes have demonstrated the performance of our method.

Cryo-electron tomography allows 3D observation of biological specimens in their native and hydrated state at high spatial resolution (4-5 nanometers). Traditionally cryo-tomograms have very low signal-to-noise ratios and conventional image segmentation methods are limited yet. In this paper, we formulate the segmentation problem of both small tubulin aggregates and microtubules against the background as a two class labeling problem in the Conditional Random Field framework. In our approach, we exploit image patches to take into account spatial contexts and to improve robustness to noise. Because of the contrast anisotropy in the specimen thickness direction, each 2D section of the 3D tomogram is segmented separately with an optional update of reference patches. This method is evaluated on synthetic data and on cryo-electron tomograms of in vitro microtubules.

In this paper, we present a stochastic NL-means-based de-noising algorithm for generalized non-parametric noise models. First, we provide a statistical interpretation to current patch-based neighborhood filters and justify the Bayesian inference that needs to explicitly accounts for discrepancies between the model and the data. Furthermore, we investigate the Approximate Bayesian Computation (ABC) rejection method combined with density learning techniques for handling situations where the posterior is intractable or too prohibitive to calculate. We demonstrate our stochastic Gamma NL-means (SGNL) on real images corrupted by non-Gaussian noise.

Recognizing dynamic behaviors of dense crowds in videos is of great interest in many surveillance applications. In contrast to most existing methods which are based on trajectories or tracklets, our approach for crowd motion analysis provides a crowd motion classification on a frame-by-frame and pixel-wise basis. Indeed, we only compute affine motion models from pairs of two consecutive video images. The classification itself relies on simple rules on the coefficients of the computed affine motion models, and therefore does not imply any prior learning stage. The overall method proceeds in four steps: (i) detection of moving points, (ii) computation of a set of motion model candidates over a collection of windows, (iii) selection of the best motion model at each point owing to a maximum likelihood criterion, (iv) determination of the crowd motion class at each pixel with a hierarchical classification tree regularized by majority votes. The algorithm is almost parameter-free, and is efficient in terms of memory and computation load. Experiments on computer-generated sequences and real video sequences demonstrate that our method is accurate, and can successfully handle complex situations.

Global variational methods for optical flow estimation usually suffer from an over-smoothing effect. We propose a semi-local estimation framework designed to integrate and improve any variational method. The idea is to implicitly segment the minimization domain into coherently moving windows. In a first time, local variational estimations are performed in overlapping candidate square regions. Then, a global discrete optimization, non subject to the over-smoothing introduced by variational approaches, selects the optimal window for each pixel. Experimental results show an increasing of the sharpness of discontinuities and a significant improvement of global registration errors compared to the results of the baseline global variational method.

We propose a robust statistical framework for reconstructing lifetime map corrupted by vesicle motion in frequency domain FLIM imaging. Instrumental noise is taken into account to improve lifetime estimation. Robust M-estimators and ML-estimators allow to jointly estimate motion and lifetime. Performances are demonstrated on simulated and real samples.

We present a nonparametric regression method for denoising 3-D image sequences acquired via fluorescence microscopy. The proposed method exploits the redundancy of the 3-D+time information to improve the signal-to-noise ratio of images corrupted by Poisson-Gaussian noise. A variance stabilization transform is first applied to the image-data to remove the dependence between the mean and variance of intensity values. This preprocessing requires the knowledge of parameters related to the acquisition system, also estimated in our approach. In a second step, we propose an original statistical patch-based framework for noise reduction and preservation of space-time discontinuities. In our study, discontinuities are related to small moving spots with high velocity observed in fluorescence video-microscopy. The idea is to minimize an objective nonlocal energy functional involving spatio-temporal image patches. The minimizer has a simple form and is defined as the weighted average of input data taken in spatially-varying neighborhoods. The size of each neighborhood is optimized to improve the performance of the pointwise estimator. The performance of the algorithm (which requires no motion estimation) is then evaluated on both synthetic and real image sequences using qualitative and quantitative criteria.

In image processing, restoration is expected to improve the qualitative inspection of the image and the performance of quantitative image analysis techniques. In this paper, an adaptation of the nonlocal (NL)-means filter is proposed for speckle reduction in ultrasound (US) images. Originally developed for additive white Gaussian noise, we propose to use a Bayesian framework to derive a NL-means filter adapted to a relevant ultrasound noise model. Quantitative results on synthetic data show the performances of the proposed method compared to well-established and state-of-the-art methods. Results on real images demonstrate that the proposed method is able to preserve accurately edges and structural details of the image.

A critical issue in image restoration is the problem of noise removal while keeping the integrity of relevant image information. Denoising is a crucial step to increase image quality and to improve the performance of all the tasks needed for quantitative imaging analysis. The method proposed in this paper is based on a 3-D optimized blockwise version of the nonlocal (NL)-means filter (Buades, et al., 2005). The NL-means filter uses the redundancy of information in the image under study to remove the noise. The performance of the NL-means filter has been already demonstrated for 2-D images, but reducing the computational burden is a critical aspect to extend the method to 3-D images. To overcome this problem, we propose improvements to reduce the computational complexity. These different improvements allow to drastically divide the computational time while preserving the performances of the NL-means filter. A fully automated and optimized version of the NL-means filter is then presented. Our contributions to the NL-means filter are: 1) an automatic tuning of the smoothing parameter; 2) a selection of the most relevant voxels; 3) a blockwise implementation; and 4) a parallelized computation. Quantitative validation was carried out on synthetic datasets generated with BrainWeb (Collins, et al., 1998). The results show that our optimized NL-means filter outperforms the classical implementation of the NL-means filter, as well as two other classical denoising methods [anisotropic diffusion (Perona and Malik, 1990)] and total variation minimization process (Rudin, et al., 1992) in terms of accuracy (measured by the peak signal-to-noise ratio) with low computation time. Finally, qualitative results on real data are presented .

We present a novel space-time patch-based method for image sequence restoration. We propose an adaptive statistical estimation framework based on the local analysis of the bias-variance trade-off. At each pixel, the space-time neighborhood is adapted to improve the performance of the proposed patch-based estimator. The proposed method is unsupervised and requires no motion estimation. Nevertheless, it can also be combined with motion estimation to cope with very large displacements due to camera motion. Experiments show that this method is able to drastically improve the quality of highly corrupted image sequences. Quantitative evaluations on standard artificially noise-corrupted image sequences demonstrate that our method outperforms other recent competitive methods. We also report convincing results on real noisy image sequences.

This paper describes a method for separating moving objects from temporally varying background in time-lapse confocal microscopy image sequences representing fluorescently tagged moving vesicles. A temporal linear model is considered for background modeling whose parameters are robustly estimated using asymmetric M-estimators combined with a bias-variance trade-off criterion. Furthermore, we propose an original approach for automatically detecting moving objects in the image sequence. Experimental results demonstrate the interest of this proposed method which can be relevant for biological studies from image sequences.

Psychovision have shown that many grouping laws come into play to structure human vision. They use informations of different kinds, not only gray (or color)-level values. Here we will show how an orientation interpolation operator working in S1(angle in [0, 2π]) can be used to recover geometrical information in images. The operator is presented and is used to produce fields that drive a fast marching contour extraction algorithm and a LIC-based smoothing method. Experiment on real images are reported to validate the proposed approach.

A novel adaptive and patch-based approach is proposed for image denoising and representation. The method is based on a pointwise selection of small image patches of fixed size in the variable neighborhood of each pixel. Our contribution is to associate with each pixel the weighted sum of data points within an adaptive neighborhood, in a manner that it balances the accuracy of approximation and the stochastic error, at each spatial position. This method is general and can be applied under the assumption that there exists repetitive patterns in a local neighborhood of a point. By introducing spatial adaptivity, we extend the work earlier described by Buades et al. which can be considered as an extension of bilateral filtering to image patches. Finally, we propose a nearly parameter-free algorithm for image denoising. The method is applied to both artificially corrupted (white Gaussian noise) and real images and the performance is very close to, and in some cases even surpasses, that of the already published denoising methods.

We present a spatio-temporal filtering method for significantly increasing the signal-to-noise ratio in noisy fluorescence microscopic image sequences where small particles have to be tracked from frame to frame. Image sequences restoration is achieved using a spatio-temporal adaptive window approach with an appropriate on-line window geometry specification. We have applied this method to noisy synthetic and real 3D image sequences where a large number of small fluorescently labelled vesicles are moving in regions close to the Golgi apparatus. The SNR is shown to be drastically improved and the enhanced vesicles can be segmented. This novel approach can be further used for biological studies where the dynamic of small objects of interest has to be analyzed in molecular and sub-cellular bio-imaging.

An energy model based approach for estimating object boundaries is presented. We study a particular energy whose minimizer can be determined. The method estimates the unknown number of objects and draws object boundaries by selecting the "best" level lines computed from level sets of the original image. Unlike previous standard methods, the proposed method does not require iteration for minimizing the energy. In addition, our segmentation algorithm combines anisotropic diffusion based regularization with level line selection to extract smooth object boundaries. Experimental results on 2D biomedical and meteorological images are reported.

We present a statistical method for the motion-based segmentation of deformable structures undergoing nonrigid movements. The proposed approach relies on two models describing the shape of interest, its variability, and its movement. The first model corresponds to a statistical deformable template that constrains the shape and its deformations. The second model is introduced to represent the optical flow field inside the deformable template. These two models are combined within a single probability distribution, which enables to derive shape and motion estimates using a maximum likelihood approach. The method requires no manual initialization and is demonstrated on synthetic data and on a medical X-ray image sequence.

We present a statistical method for the motion-based segmentation of deformable structures undergoing non-rigid movements. The proposed approach relies on two models describing the shape of interest, its variability and its movement. The first model corresponds to a statistical deformable template that constrains the shape and its deformations. The second model is introduced to represent the optical flow field inside the deformable template. These two models are combined within a single probability distribution which enables to derive optimal shape and motion estimates using a maximum likelihood approach. The method requires no manual initialization and is demonstrated on medical X-ray image sequences.

We describe a method for the temporal tracking of stochastic deformable models in image sequences. The object representation relies on a hierarchical statistical description of the deformations applied to a template. The optimal Bayesian estimate of deformations is obtained by maximizing nonlinear probability distributions using optimization techniques. The method may be sensitive to local maxima of the distributions and require an initial configuration close to the optimal solution. In our approach, the initialization is provided by a robust estimate of the rigid and statistically constrained nonrigid motions from the normal optical flow computed along the deformable contour. The approach is demonstrated on real-world sequences showing mouth movements and cardiac motions with missing data.

Many studies have proven that statistical model-based texture segmentation algorithms yield good results provided that the model parameters and the number of regions be known a priori. In this correspondence, we present an unsupervised texture segmentation method that does not require knowledge about the different texture regions, their parameters, or the number of available texture classes. The proposed algorithm relies on the analysis of local and global second and higher order spatial statistics of the original images. The segmentation map is modeled using an augmented-state Markov random field, including an outlier class that enables dynamic creation of new regions during the optimization process. A Bayesian estimate of this map is computed using a deterministic relaxation algorithm. Results on real-world textured images are presented.

In this paper, we propose a new statistical framework for modeling and extracting 2D moving deformable objects from image sequences. The object representation relies on a hierarchical description of the deformations applied to a template. Global deformations are modeled using a Karhunen Loeve expansion of the distortions observed on a representative population. Local deformations are modeled by a (first-order) MarKov process. The optimal bayesian estimate of the global and local deformations is obtained by maximizing a non-linear joint probability distribution using stochastic and deterministic optimization techniques. The use of global optimization techniques yields robust and reliable segmentations in adverse situations such as low signal-to-noise ratio, non-gaussian noise or occlusions. Moreover, no human interaction is required to initialize the model. The approach is demonstrated on synthetic as well as on real-world image sequences showing moving hands with partial occlusions.>

We describe a method for the temporal tracking of stochastic deformable models in long image sequences. The object representation relies on a hierarchical statistical description of the deformations applied to a template. A Bayesian estimate of the deformations is obtained by maximizing a highly non-linear joint probability distribution. Time consuming global (stochastic) optimization techniques are necessary to obtain optimal solutions unless a good initial guess is available. A good initialization is provided by a recursive temporal filtering of the parameters of the deformable template, combined with a detection of abrupt changes. This procedure yields robust segmentations and enables to track reliably complex deformable structures as is demonstrated here on real-world image sequences showing hand and mouth movements.>