Interaction while performing physical tasks is inherently challenging, as both hands are fully engaged. In Minimally-Invasive Surgery (MIS), for instance, navigating images requires either delegation, causing frustration and delays, or hand de-sterilization, increasing risk. We introduce Interaction Through Instruments, an interaction paradigm in which task instruments become interaction devices. To design this technique in MIS, we first conduct a survey (N=23) identifying intraoperative needs, interaction strategies and workarounds, and persistent challenges. Then, through five participatory design workshops (N=10), we identify challenges in blending a user interface into views of a physical space, informing the design of InteractOR, a system that combines surgical instrument segmentation with pinch-gesture recognition to enable interaction within the surgical view. Finally, in a Comparative Structured Observation study (N=12) we compare two visualization strategies (side-by-side and overlay) against delegation, showing that interaction through instruments can reduce focus shifts, increase efficiency, and foster autonomy.

This article advances a method to analyze a large corpus of historical photographs using artificial intelligence tools and data modeling. This research was conducted within the framework of the EyCon (Early Conflict Photography 1890-1918 and Visual AI) and HighVision projects, which aim at leveraging the power of digital tools, exploiting both visual and textual information, to investigate the development of war photography at the turn of the 20th century. To do so, one of the objectives of the project was to develop a method to extract robust features and to overcome the challenges posed by the halftone printing techniques, the most common way to reproduce photographs in daily newspapers, periodicals and books at the time. By combining visual and textual similarity measures, the proposed approach enables the identification of significant subsets of similarity within the dataset. The findings from this research hold important implications for the broader field of image analysis and provide insights into the unique characteristics and complexities of historical visual data. This work contributes to the advancement of computer vision techniques in the analysis of historical photographic collections, opening up new avenues for research in visual AI and archival studies.

Bayesian Neural Networks (BNNs) have long been considered an ideal, yet unscalable solution for improving the robustness and the predictive uncertainty of deep neural networks. While they could capture more accurately the posterior distribution of the network parameters, most BNN approaches are either limited to small networks or rely on constraining assumptions, e.g., parameter independence. These drawbacks have enabled prominence of simple, but computationally heavy approaches such as Deep Ensembles, whose training and testing costs increase linearly with the number of networks. In this work we aim for efficient deep BNNs amenable to complex computer vision architectures, e.g., ResNet-50 DeepLabv3+, and tasks, e.g., semantic segmentation and image classification, with fewer assumptions on the parameters. We achieve this by leveraging variational autoencoders (VAEs) to learn the interaction and the latent distribution of the parameters at each network layer. Our approach, called Latent-Posterior BNN (LP-BNN), is compatible with the recent BatchEnsemble method, leading to highly efficient (in terms of computation and memory during both training and testing) ensembles. LP-BNNs attain competitive results across multiple metrics in several challenging benchmarks for image classification, semantic segmentation, and out-of-distribution detection.

The spatial resolution and temporal frame-rate of dynamic magnetic resonance imaging (MRI) can be improved by reconstructing images from sparsely sampled k -space data with mathematical modeling of the underlying spatiotemporal signals. These models include sparsity models, linear subspace models, and non-linear manifold models. This work presents a novel linear tangent space alignment (LTSA) model-based framework that exploits the intrinsic low-dimensional manifold structure of dynamic images for accelerated dynamic MRI. The performance of the proposed method was evaluated and compared to state-of-the-art methods using numerical simulation studies as well as 2D and 3D in vivo cardiac imaging experiments. The proposed method achieved the best performance in image reconstruction among all the compared methods. The proposed method could prove useful for accelerating many MRI applications, including dynamic MRI, multi-parametric MRI, and MR spectroscopic imaging.

Explaining a decision requires some properties that have been studied and established in cognitive sciences. An important one is the contrastive nature of explanations: an explanation should answer questions such as "why make decision P rather than Q?". This principle has been formalized recently by T. Miller in a logical framework exploiting knowledge represented as structural causal graphs, with variables taking crisp values. However this framework does not allow us to cope easily with imprecise knowledge or data, nor with imprecise formulations of explanations, that could be preferred in some situations. This paper discusses the principles of such fuzzy extensions of this model, exploring the various levels for integrating fuzzy semantics: it discusses successively (i) the input level, for imprecisely described data instances for which an explanation is required, (ii) the level of the structural causal graph itself, to model imprecise knowledge about the functional relations between the variables involved in the model, and (iii) the output level, to express the explanation, e.g. using fuzzy modalities. The combination of these levels is considered as well. Finally, the paper proposes a discussion about the definition of minimality in the fuzzy framework.

CNNs are often assumed to be capable of using contextual information about distinct objects (such as their directional relations) inside their receptive field. However, the nature and limits of this capacity has never been explored in full. We explore a specific type of relationship – directional – using a standard U-Net trained to optimize a cross-entropy loss function for segmentation. We train this network on a pretext segmentation task requiring directional relation reasoning for success and state that, with enough data and a sufficiently large receptive field, it succeeds to learn the proposed task. We further explore what the network has learned by analysing scenarios where the directional relationships are perturbed, and show that the network has learned to reason using these relationships.

Abstract Augmented reality applications have rapidly spread across online retail platforms and social media, allowing consumers to virtually try‐on a large variety of products, such as makeup, hair dying, or shoes. However, parametrizing a renderer to synthesize realistic images of a given product remains a challenging task that requires expert knowledge. While recent work has introduced neural rendering methods for virtual try‐on from example images, current approaches are based on large generative models that cannot be used in real‐time on mobile devices. This calls for a hybrid method that combines the advantages of computer graphics and neural rendering approaches. In this paper, we propose a novel framework based on deep learning to build a real‐time inverse graphics encoder that learns to map a single example image into the parameter space of a given augmented reality rendering engine. Our method leverages self‐supervised learning and does not require labeled training data, which makes it extendable to many virtual try‐on applications. Furthermore, most augmented reality renderers are not differentiable in practice due to algorithmic choices or implementation constraints to reach real‐time on portable devices. To relax the need for a graphics‐based differentiable renderer in inverse graphics problems, we introduce a trainable imitator module. Our imitator is a generative network that learns to accurately reproduce the behavior of a given non‐differentiable renderer. We propose a novel rendering sensitivity loss to train the imitator, which ensures that the network learns an accurate and continuous representation for each rendering parameter. Automatically learning a differentiable renderer, as proposed here, could be beneficial for various inverse graphics tasks. Our framework enables novel applications where consumers can virtually try‐on a novel unknown product from an inspirational reference image on social media. It can also be used by computer graphics artists to automatically create realistic rendering from a reference product image.

Tracking an entire high-density crowd composed of more than five hundred individuals is a difficult task that has not yet been accomplished. In this article, we propose to track pedestrians using a model composed of a Particle Filter (PF) and three Deep Convolutional Neural Networks (DCNN). The first network is a detector that learns to localize the persons. The second one is a pretrained network that estimates the optical flow, and the last one corrects the flow. Our contribution resides in the way we train this last network by PF supervision, and in Markov Random Field linking the different tracks.

In this work, we perform an in-depth analysis of the specific difficulties a crowded scene dataset raises for tracking algorithms. Starting from the standard characteristics depicting the crowd and their limitations, we introduce six entropy measures related to the motion patterns and to the appearance variability of the individuals forming the crowd, and one appearance measure based on Principal Component Analysis. The proposed measures are discussed on synthetic configurations and on multiple real datasets. These criteria are able to characterize the crowd behavior at a more detailed level and may be helpful for evaluating the tracking difficulty of different datasets. The results are in agreement with the perceived difficulty of the scenes.

The study of vascular morphometry requires segmenting vessels with high precision. Of particular clinical interest is the morpho-metric analysis of arterial bifurcations in Adaptive Optics Ophthalmoscopy (AOO) images of eye fundus. In this paper, we extend our previous approach for segmenting retinal vessel branches to the segmentation of bifurcations. This enables us to recover the microvascular tree and extract biomarkers that charactarize the blood flow. Segmentation results are shown to be within the range of intra-and inter-user variability, allowing a preliminary study on biomarkers derived from vessel diameter estimates at arterial bifurcations.

We extend recent work on mathematical morphology for signal processing on weighted graphs, based on discrete tropical algebra. The framework is general and can be applied to any scalar function defined on a graph. We show applications in structure tensors analysis and the regularisation of greyscale images.

Brain magnetic resonance imaging (MRI) is widely used to assess brain development in neonates and to diagnose a wide range of neurological diseases in adults. Such studies are usually based on quantitative analysis of different brain tissues, so it is essential to be able to classify them accurately. In this paper, we propose a fast automatic method that segments 3D brain MR images into different tissues using fully convolutional network (FCN) and transfer learning. As compared to existing deep learning-based approaches that rely either on 2D patches or on fully 3D FCN, our method is way much faster: it only takes a few seconds, and only a single modality (T1 or T2) is required. In order to take the 3D information into account, all 3 successive 2D slices are stacked to form a set of 2D “color” images, which serve as input for the FCN pre-trained on ImageNet for natural image classification. To the best of our knowledge, this is the first method that applies transfer learning to segment both neonatal and adult brain 3D MR images. Our experiments on two public datasets show that our method achieves state-of-the-art results.

in Frontiers in Artificial Intelligence and Applications, vol. 285

In this paper we extend some previously established links between the derivation operators used in formal concept analysis and some mathematical morphology operators to fuzzy concept analysis. We also propose to use mathematical morphology to navigate in a fuzzy concept lattice and perform operations on it. Links with other lattice-based for malisms such as rough sets and F-transforms are also established. This paper proposes a discussion and new results on such links and their potential interest.

In order to segment elongated structures, we propose a new approach for integrating an approximate parallelism constraint in deformable models . The proposed Parallel Double Snakes evolve simultaneously two contours, in order to minimize an energy functional which attracts these contours towards high image gradients and enforces the approximate parallelism between them by controlling their distance to a centerline under regularity constraints of this line. The proposed approach is applied on retina images, for segmenting retinal layers in optical coherence tomography images of pathological subjects (and it applies to healthy subjects as well). Results are evaluated by comparing with manual segmentations for three retinal layers, and provide a similarity index above 0.87, sensitivity between 0.85 and 0.93, and specificity between 0.84 and 0.94. These results are within the range of intra and inter-expert variability. Moreover, quantitative studies demonstrate that, in our application, our Parallel Double Snake (PDS) model outperforms other parametric active contour algorithms integrating parallelism information.

A number of similarity measures for comparing description logic concepts have been proposed. Criteria have been developed to evaluate a measure's fitness for an application. These criteria include on the one hand those that ensure compatibility with the semantics, such as equivalence soundness, and on the other hand the properties of a metric, such as the triangle inequality. In this work we present two classes of dissimilarity measures that are at the same time equivalence sound and satisfy the triangle inequality: a simple dissimilarity measure, based on description trees for the lightweight description logic EL; and an instantiation of a general framework, presented in our previous work, using dilation operators from mathematical morphology, and which exploits the link between Hausdorff distance and dilations using balls of the ground distance as structuring elements.

We consider the Chan and Vese multiphase segmentation based on the partition of an image minimizing an energy involving a region-based data fidelity term and a regularization term. The common implementation of the continuous optimization of this segmentation framework, with multiple level set functions, raises some numerical issues which lead to poor performance of the method when handling more than two phases. We propose a general formulation of the multiphase model, and a permutation method, incorporated in the level-set based implementation of the multi-phase approach to handle in an original way the so-called hidden phase problem.

In this paper, an extension of the framework proposed by Deledalle et al. [1] for Non Local Means (NLM) method is proposed. This extension is a general adaptive method to denoise images containing multiple noises. It takes into account a segmentation stage that indicates the noise type of a given pixel in order to select the similarity measure and suitable parameters to perform the denoising task, considering a certain patch on the image. For instance, it has been experimentally observed that fetal 3D ultrasound images are corrupted by different types of noise, depending on the tissue. Finally, the proposed method is applied to denoise these images, showing very good results.

In this paper, we present a method for automatically segmenting the walls of retinal arteries in adaptive optics images. To the best of our knowledge, this is the first method addressing this problem in such images. To achieve this goal, we propose to model these walls as four curves approximately parallel to a common reference line located near the center of vessels. Once this line is detected, the curves are simultaneously positioned as close as possible to the borders of walls using an original tracking procedure to cope with deformations along vessels. Then, their positioning is refined using a deformable model embedding a parallelism constraint. Such an approach enables us to control the distance of the curves to their reference line and improve the robustness to image noise. This model was evaluated on healthy subjects by comparing the results against segmentations from physicians. Noticeably, the error introduced by this model is smaller or very near the inter-physicians error.

Sensorimotor assessment aims at evaluating sensorial and motor capabilities of children who are likely to present a pervasive developmental disorder, such as autism. It relies on playful activities which are proposed by a psychomotrician expert to the child, with the intent of observing how the latter responds to various physical and cognitive stimuli. Each session is recorded so that the psychomotrician can use the video as a support for reviewing in-session impressions and drawing final conclusions. These recordings carry a wealth of information that could be exploited for research purposes and contribute to a better understanding of autism spectrum disorders. However, the systematic inspection of these data by clinical professionals would be time-consuming and impracticable. In order to make these analyses feasible, we discuss a computer vision approach to prospect precise behavior information from the available visual data acquired throughout assessment sessions.

International audience

In this paper, we propose an original way of enriching description logics with abduction reasoning services. Under the aegis of set and lattice theories, we put together ingredients from mathematical morphology, description logics, and formal concept analysis. We propose computing the best explanations of an observation through algebraic erosion over the concept lattice of a background theory that is efficiently constructed using tools from formal concept analysis. We show that the defined operators are sound and complete and satisfy important rationality postulates of abductive reasoning. As a typical illustration, we consider a scene understanding problem. In fact, scene understanding can benefit from prior structural knowledge represented as an ontology and the reasoning tools of description logics. We formulate model based scene understanding as an abductive reasoning process. A scene is viewed as an observation and the interpretation is defined as the best explanation, considering the terminological knowledge part of a description logic about the scene context. This explanation is obtained from morphological operators applied on the corresponding concept lattice.

Brain-computer interfaces (BCIs) are systems that record brain signals and then classify them to generate computer commands. Keeping a minimal number of channels (electrodes) is essential for developing portable BCIs. Unlike existing methods choosing channels without optimization of time segment for classification, this work proposes a novel subject-specific channel selection method based on a criterion derived from Fisher's discriminant analysis to realize the parametrization of both time segment and channel positions. The experimental results show that the method can efficiently reduce the number of channels (from 118 channels to no more than 11), and shorten the training time, without a significant decrease of classification accuracy on a standard dataset.

In this paper, we present a method for simultaneously tracking thousands of targets in biological image sequences, which is of major importance in modern biology. The complexity and inherent randomness of the problem lead us to propose a unified probabilistic framework for tracking biological particles in microscope images. The framework includes realistic models of particle motion and existence and of fluorescence image features. For the track extraction process per se, the very cluttered conditions motivate the adoption of a multiframe approach that enforces tracking decision robustness to poor imaging conditions and to random target movements. We tackle the large-scale nature of the problem by adapting the multiple hypothesis tracking algorithm to the proposed framework, resulting in a method with a favorable tradeoff between the model complexity and the computational cost of the tracking procedure. When compared to the state-of-the-art tracking techniques for bioimaging, the proposed algorithm is shown to be the only method providing high-quality results despite the critically poor imaging conditions and the dense target presence. We thus demonstrate the benefits of advanced Bayesian tracking techniques for the accurate computational modeling of dynamical biological processes, which is promising for further developments in this domain.

Alignment and parallelism are frequently found between objects in high-resolution remote sensing images and can be used to interpret and describe the observed scenes. In this paper, we propose new representations of parallelism and alignment as fuzzy spatial relations, which capture the imprecision in the semantics of both relations. We propose two novel definitions of alignment between objects: local and global. In local alignment, each object of the group is aligned with its neighbors, while in global alignment, every object of the group is aligned to all other members. Both definitions consider each object as a whole and are based on relative position measures. They are robust with respect to segmentation errors. Furthermore, we propose an efficient graph-based method to determine which are the locally and the globally aligned groups of objects from a set of segmented objects. In addition, we propose a fuzzy definition for the parallel relation, which is also based on relative position measures and is adequate to represent the parallelism between a globally aligned group of objects and another object or group of objects. Illustrative examples on optical satellite images show the description power of these two relations and their combination for image interpretation.

The aim of the approach proposed in this paper is to determine a potential crop extent prior to the crop season, by determining regions that might change in time vs. those that surely do not change. We use multi-annual PALSAR-1 data since in dry conditions, L-band HH/HV data have a potential of distinguishing between bare soil and other classes. In addition, a more accurate map can be reached with multi-temporal data than using a single date. We work on HH and HV data sets separately and analyze the two outputs using ground-truth information. In a final phase, we combine these two outputs and compare the result with the ground-truth too, to test the usefulness of fusing the HH/HV information. This approach is the first step in our three-step procedure for estimation of cultivated area in small plot agriculture in Malawi. Validation results show that the proposed approach is promising.

Parametric deformable models are an important technique for image segmentation. In order to improve the robustness of the model, it may be interesting to incorporate a priori information about the shape of the objects to be segmented. In this paper, we propose to add a parallelism constraint. Such a model is relevant in many applications where elongated structures have to be detected. One main advantage of our formulation is that it only needs few parameters to be adjusted in addition to those of traditional snakes. The pro- posed model has been applied for the segmentation of OCT images of tile retina and for the segmentation of retinal vessels. Experimental results, obtained on 25 OCT images and 30 eye fundus images, demonstrated the robustness, flexibility and large potential applicability of this new formulation. The accuracy of the method has been assessed by comparing manual segmentations, made by experts, with the automatic ones.

In this paper, we propose a novel method to introduce spatial information in particle filters. This information may be expressed as spatial relations (orientation, distance, etc.), velocity, scaling, or shape information. Spatial information is modeled in a generic fuzzy-set framework. The fuzzy models are then introduced in the particle filter and automatically define transition and prior spatial distributions. We also propose an efficient importance distribution to produce relevant particles, which is dedicated to the proposed fuzzy framework. The fuzzy modeling provides flexibility both in the semantics of information and in the transitions from one instant to another one. This allows one to take into account situations where a tracked object changes its direction in a quite abrupt way and where poor prior information on dynamics is available, as demonstrated on synthetic data. As an illustration, two tests on real video sequences are performed in this paper. The first one concerns a classical tracking problem and shows that our approach efficiently tracks objects with complex and unknown dynamics, outperforming classical filtering techniques while using only a small number of particles. In the second experiment, we show the flexibility of our approach for modeling: Fuzzy shapes are modeled in a generic way and allow the tracking of objects with changing shape.

Bipolarity is an important feature of spatial information, involved in the expression of preferences and constraints about spatial positioning or in pairs of opposite spatial relations such as left and right. Another important feature is imprecision which has to be taken into account to model vagueness, inherent to many spatial relations (as for instance vague expressions such as close to, to the right of), and to gain in robustness in the representations. In previous works, we have shown that fuzzy sets and fuzzy mathematical morphology are appropriate frameworks, on the one hand, to represent bipolarity and imprecision of spatial relations and, on the other hand, to combine qualitative and quantitative reasoning in description logics extended with fuzzy concrete domains. The purpose of this paper is to integrate the bipolarity feature in the latter logical framework based on bipolar and fuzzy mathematical morphology and description logics with fuzzy concrete domains. Two important issues are addressed in this paper: the modeling of the bipolarity of spatial relations at the terminological level and the integration of bipolar notions in fuzzy description logics. At last, we illustrate the potential of the proposed formalism for spatial reasoning on a simple example in brain imaging.

In this paper we present a new procedure for tracking bi-directional objects in kymographs. The proposed technique is based on a novel adaptive and directional band-pass filtering method which allows us to separate particles which move in opposite directions. The filtering method exploits the curvelet analysis of the kymograph image to automatically adapt to the objects trails characteristics and select oriented features. The separation of bi-directional objects in separated images allows us to reliably detect and track fluorescent particles in fluorescence image sequences, despite numerous crossroad points in the kymograph space. The new abilities provided by the proposed technique are highlighted by the analysis of biological images which were previously impossible to analyze reliably.

This paper proposes a new morphological filter for SAR interferograms. It is based on a modified version of alternate sequential filters with reconstruction (MASF), in which the structuring elements are adaptively defined according to the fringe directions. This provides a good fidelity to the fringe information while efficiently removing noise. Another feature of the proposed approach is to apply the filter on the original interferogram and on shifted version, to overcome the wrapping of the phase, and to combine the two results. The proposed filtering technique is then tested on both simulated and real data with different levels of noise. It is also compared to previous techniques according to simplicity and noise reduction.

In this article we present a method for extracting groups of aligned objects from a labeled image. Our method is based on fuzzy measures of relative direction between the objects, leading to a fuzzy approach for defining alignment as a spatial relation. The method is able to capture the ambiguities presented when defining alignment between objects of different sizes. Two definitions of alignment are presented; local and global. The local alignments are first extracted and are used as candidates for the global alignments. Applications of the alignment relation on real images illustrates its interest for high level image interpretation.

Fuzzy connected filters were recently introduced as an extension of connected filters within the fuzzy set framework. They rely on the representation of the image gray levels by fuzzy quantities, which are suitable to represent imprecision usually contained in images. No robust construction method of these fuzzy images has been introduced so far. In this paper we propose a generic method to fuzzify a crisp image in order to explicitly take imprecision on grey levels into account. This method is based on the conversion of statistical noise present in an image, which cannot be directly represented by fuzzy sets, into a denoising imprecision. The detectability of constant gray level structures in these fuzzy images is also discussed.

Particle detection and tracking methods generally assume a simplistic image model that is rarely valid when imaging biological processes in fluorescence microscopy. The tracking task may become nearly impossible when complex biological structures are visible and interfere with the signal of interest. To address this limitation we have adapted a source separation technique based on sparsity principles to the characteristics of fluorescent biological images. Since it allows the discrimination of objects with different morphologies, we present an approach to detect and track particles that exploits its results. The tracking algorithm resolves particles that temporarily aggregate by exploiting the proposed model of image. We prove in a real case the ability of the method to track numerous particles in a complex and dynamic background, something which was not feasible until now, hence offering new tools to document interactions between cellular compartments.

Multiple hypothesis tracking (MHT) is a preferred technique for solving the data association problem in modern multiple targets tracking systems. However its computational cost is generally considered prohibitive for tracking numerous objects in cluttered environments due to numerous targets and spurious measurements. We present in this paper a new MHT formulation in which target perceivability is modeled whereby automatic early track termination and false measurements exclusion reduce the problem complexity and improve the method robustness to clutter. Moreover we propose a MHT implementation exploiting the tree structure of the potential tracks to take full advantages of parallel computing technologies. We provide experimental results showing that both the track model and algorithmic design make the algorithm fast and robust even in highly complex situations such as tracking numerous particles in fluorescent microscopy images.

High resolution remote sensing (HR RS) images allow discriminating between different objects in a scene. Spatial reasoning techniques can be used to interpret and describe the scene. One component of spatial reasoning deals with the modeling and assessment of spatial relations among objects. In this work we propose three models that seize the semantics of the spatial relations "to go across" and "to go through" between a linear object and a region. To develop these three models we considered the usual perception of these natural language expressions which leads to the development of the three fuzzy models. They have been implemented and tested in scenes of HR RS images. Results are in good agreement with intuition.

Spatial relations play a crucial role in model-based image recognition and interpretation due to their stability compared to many other image appearance characteristics, and graphs are well adapted to represent such information. Sequential methods for knowledgebased recognition of structures require to define in which order the structures have to be recognized, which can be expressed as the optimization of a path in the representation graph. We propose to integrate pre-attention mechanisms in the optimization criterion, in the form of a saliency map, by reasoning on the saliency of spatial area defined by spatial relations. Such mechanisms extract knowledge from an image without object recognition in advance and do not require any a priori knowledge on the image. Therefore, pre-attentional mechanisms provide useful knowledge for object segmentation and recognition. The derived algorithms are applied on brain image understanding.

In some application domains, such as medical imaging, the objects that compose the scene are known as well as some of their properties and their spatial arrangement. We can take advantage of this knowledge to perform the segmentation and recognition of structures in medical images. We propose here to formalize this problem as a constraint network and we perform the segmentation and recognition by iterative domain reductions, the domains being sets of regions. For computational purposes we represent the domains by their upper and lower bounds and we iteratively reduce the domains by updating their bounds. We show some preliminary results on normal and pathological brain images.

We present a new feature-aided tracking algorithm dedicated to the task of tracking multiple and closely-spaced biological particles. We propose a new function to score associations, based on kinetic models, and enriched with an additional feature. This feature is based on adaptive profiles and the physical properties of the acquisition system. A key property is that this feature definition allows to resolve the challenging task of tracking particles that appear fused. Results on simulations show improved performances over existing methods both on tracking and on the resolution of fused particles.

Tracking nano-metric particles in a biological environment is a very difficult task because of the low signal intensity and the high mobility of these small objects. The task becomes nearly impossible for classical tracking procedures when the targets are labeled with a marker that is not strictly specific, because in this case dynamic structures in the cell are also visible. To address this limitation, we propose to use a source separation technique based on sparsity principles which allows the discrimination of objects with different morphologies. We prove in a real case that tracking in the source separated images allows to track particles that interact with other sources, something which was not feasible until now. This capability opens up new perspectives for the analysis documenting intricate interactions between cellular compartments.

Two approaches for combining humanitarian mine detection sensors are presented—one based on belief functions and the other one based on possibility theory. The approaches are described in parallel. First, different measures are extracted from the sensor data. Mass functions and possibility distributions are then derived from the measures based on prior information. After that, the combination of masses and the combination of possibility degrees are performed in two steps, on a separate sensor level and between the sensors. Combination operators are chosen to account for different characteristics of the sensors. The selection of the decision rules is discussed for both approaches. The proposed approaches are illustrated on a set of real mines and nondangerous objects, and promising results have been obtained.

An image segmentation method based on structural pattern recognition is presented. Two graphs are generated from the image to be segmented. A model graph is generated from an oversegmentation of the image and from traces provided by the user. An input graph is generated from the oversegmented image. Image segmentation is then obtained by matching the input graph to the model graph. An objective function is defined and optimized using a new approach to find the most suitable clique of the corresponding association graph. The structural information encoded in the graphs leads to a robust segmentation performance even in the case of non-homogeneous textured regions. Successful experimental results obtained from real images are provided.

We propose a method for combining humanitarian mine detection sensors based on possibility theory. Firstly, different features are extracted from the sensor data. Possibility distributions are then derived from the features based on prior information. After that, the combination of possibility degrees is performed in two steps, on separate sensor level and between the sensors. Combination operators are chosen to account for the different characteristics of the sensors. The final decision is obtained by thresholding the fusion result. Promising results have been obtained on a set of real mines and non-dangerous objects. In particular a 100% mine recognition rate was achieved, with a limited number of false alarms.

The aim of this paper is to develop a registration methodology in order to combine anatomical and functional information provided by thoracic/abdominal computed tomography (CT) and whole-body positron emission tomography (PET) images. The proposed procedure is based on the incorporation of prior anatomical information in an intensity-based nonrigid registration algorithm. This incorporation is achieved in an explicit way, initializing the intensity-based registration stage with the solution obtained by a nonrigid registration of corresponding anatomical structures. A segmentation algorithm based on a hierarchically ordered set of anatomy-specific rules is used to obtain anatomical structures in CT and emission PET scans. Nonrigid deformations are modeled in both registration stages by means of free-form deformations, the optimization of the control points being achieved by means of an original vector field-based approach instead of the classical gradient-based techniques, considerably reducing the computational time of the structure registration stage. We have applied the proposed methodology to 38 sets of images (33 provided by standalone machines and five by hybrid systems) and an assessment protocol has been developed to furnish a qualitative evaluation of the algorithm performance.

This paper proposes a method based on the Dempster-Shafer evidence theory for the detection of blotches in digitized archive film sequences. The detection scheme relies on the fusion of two uncorrelated fast, no motion compensated, spatio-temporal blotch detectors. The imprecision and uncertainty of both detectors are modeled using Dempster-Shafer evidence theory, which improves the decision, by taking into account the ignorance and the conflict between detectors. We found that this combination scheme improves the global performance, and compares favorably to the motion compensated, complex and time consuming blotch detection methods, for real archive film sequences

The spatial relation "between" is a notion which is intrinsically both fuzzy and contextual, and depends, in particular, on the shape of the objects. The literature is quite poor on this and the few existing definitions do not take into account these aspects. In particular, an object B that is in a concavity of an object A1 not visible from an object A2 is considered between A1 and A2 for most definitions, which is counter intuitive. Also, none of the definitions deal with cases where one object is much more elongated than the other. Here, we propose definitions which are based on convexity, morphological operators, and separation tools, and a fuzzy notion of visibility. They correspond to the main intuitive exceptions of the relation. We distinguish between cases where objects have similar spatial extensions and cases where one object is much more extended than the other. Extensions to cases where objects, themselves, are fuzzy and to three-dimensional space are proposed as well. The original work proposed in this paper covers the main classes of situations and overcomes the limits of existing approaches, particularly concerning nonvisible concavities and extended objects. Moreover, the definitions capture the intrinsic imprecision attached to this relation. The main proposed definitions are illustrated on real data from medical images.

We propose an OMR method based on fuzzy modeling of the information extracted from the scanned score and of musical rules. The aim is to disambiguate the recognition hypotheses output by the individual symbol analysis process. Fuzzy modeling allows to account for imprecision in symbol detection, for typewriting variations, and for flexibility of rules. Tests conducted on a hundred of music sheets result in a global recognition rate of 98.55%, and show good performances compared to SmartScore.

This paper describes a probabilistic method of inferring the position of a point with respect to a reference point knowing their relative spatial position to a third point. We address this problem in the case of incomplete information where only the angular spatial relationships are known. The use of probabilistic representations allows us to model prior knowledge. We derive exact formulae expressing the conditional probability of the position given the two known angles, in typical cases: uniform or Gaussian random prior distributions within rectangular or circular regions. This result is illustrated with respect to two different simulations: The first is devoted to the localization of a mobile phone using only angular relationships, the second, to geopositioning within a city. This last example uses angular relationships and some additional knowledge about the position.

The problem of classification is so fundamental that it has been intensively investigated by many researchers from different domains. In this paper, we present a novel unsupervised clustering algorithm derived from the techniques of probabilistic modeling which is implemented by a stochastic gradient algorithm. Then its application to challenging landcover classification based on Daedalus data of the SMART project is explored by combining both spectral feature and spatial contextual information. Our first experiments show its potential usefulness in remote sensing.

In this paper, we propose a new representation of the cortical surface that may be used to study the cortex folding process and to recover some putative stable anatomical landmarks called sulcal roots usually buried in the depth of adult brains. This representation is a primal sketch derived from a scale space computed for the mean curvature of the cortical surface. This scale-space stems from a diffusion equation geodesic to the cortical surface. The primal sketch is made up of objects defined from mean curvature minima and saddle points. The resulting sketch aims first at highlighting significant elementary cortical folds, second at representing the fold merging process during brain growth. The relevance of the framework is illustrated by the study of central sulcus sulcal roots from antenatal to adult age. Some results are proposed for ten different brains. Some preliminary results are also provided for superior temporal sulcus.

A two-level approach for modeling and fusion of antipersonnel mine detection sensors in terms of belief functions within the Dempster-Shafer framework is presented. Three promising and complementary sensors are considered: a metal detector, an infrared camera, and a ground-penetrating radar. Since the metal detector, the most often used mine detection sensor, provides measures that have different behaviors depending on the metal content of the observed object, the first level aims at identifying this content and at providing a classification into three classes. Depending on the metal content, the object is further analyzed at the second level toward deciding the final object identity. This process can be applied to any problem where one piece of information induces different reasoning schemes depending on its value. A way to include influence of various factors on sensors in the model is also presented, as well as a possibility that not all sensors refer to the same object. An original decision rule adapted to this type of application is proposed, as well as a way for estimating confidence degrees. More generally, this decision rule can be used in any situation where the different types of errors do not have the same importance. Some examples of obtained results are shown on synthetic data mimicking reality and with increasing complexity. Finally, applications on real data show promising results.

This paper develops an evaluation of the position probability of a point C which is known to be in a direction /spl beta/ with respect to a point B, itself in the direction /spl alpha/ with respect to another point A. The obtained results can be used in the problem of inference of directional relationships in the case of spatial reasoning.

We propose a morphological approach to segment several structures of 3D head magnetic resonance images dedicated to the construction of individual models of the head for applications where topology is one of the main constraints. The originality of the approach lies in the satisfaction of such constraints and in an effort towards robustness.

The problem of combining pieces of information issued from several sources can be encountered in various fields of application. This paper aims at presenting the different aspects of information fusion in different domains, such as databases, regulations, preferences, sensor fusion, etc., at a quite general level. We first present different types of information encountered in fusion problems, and different aims of the fusion process. Then we focus on representation issues which are relevant when discussing fusion problems. An important issue is then addressed, the handling of conflicting information. We briefly review different domains where fusion is involved, and describe how the fusion problems are stated in each domain. Since the term fusion can have different, more or less broad, meanings, we specify later some terminology with respect to related problems, that might be included in a broad meaning of fusion. Finally we briefly discuss the difficult aspects of validation and evaluation. © 2001 John Wiley & Sons, Inc.

A new generic definition of graph fuzzy morphism is introduced that includes classical graph related problem definitions as sub-cases. Two practical interpretations as well as some properties are discussed. This definition is a first attempt towards a unified theoretic framework for graph morphism.

We present an original method to reconstruct 3D magnetic resonance images of high resolution in the 3 directions of space from two anisotropic volumes. The resolution of each volume is degraded in a different direction. The reconstruction method is based on an optimization technique, the constraints being fidelity to the acquired data on the one hand, smoothness and edge preservation on the other hand. The interest of such a method is to significantly decrease the acquisition time of MR images, without degrading the spatial resolution.

In order to cope with the ambiguity of spatial relative position concepts, we propose a new definition of the relative position between two objects in a fuzzy set framework. This definition is based on a morphological and fuzzy pattern matching approach, and consists in comparing an object to a fuzzy landscape representing the degree of satisfaction of a directional relationship to a reference object. We detail its formal properties, and show that it is flexible and fits the intuition. Moreover, it applies also in 3D, and for fuzzy objects issued from images. It can be used for structural pattern recognition in images under imprecision.

In order to cope with the ambiguity of spatial relative position concepts, we propose a new definition of the relative position between two objects in a fuzzy set framework. This definition is based on a morphological and fuzzy pattern-matching approach, and consists of comparing an object to a fuzzy landscape representing the degree of satisfaction of a directional relationship to a reference object. It has good formal properties, it is flexible, it fits the intuition, and it can be used for structural pattern recognition under imprecision. Moreover, it also applies in 3D and for fuzzy objects issued from images.

The authors propose a method aiming to characterize the spatial organization of the main cartographic elements of a synthetic aperture radar (SAR) image and thus giving an almost automatic interpretation of the scene. Their approach is divided into three main steps which build the whole image interpretation gradually. The first step consists of applying low-level detectors taking the speckle statistics into account and extracting some raw information from the scene. The detector responses are then fused in a second step using Dempster-Shafer theory, thus allowing the modeling of the knowledge that there is about operators, including possible ignorance and their limits. A third step gives the final image interpretation using contextual knowledge between the different classes. Results of the whole method applied to different SAR images and to various landscapes are presented.

In this paper, we show how fuzzy set theory can be used to represent structural information in images, in particular, relationships between imprecise objects, defined as spatial fuzzy sets. We distinguish two types of relationships: on the one hand, relationships that are well defined in the case of crisp objects (like adjacency or distance), and on the other hand, relationships that do not find any consensual definition even in the binary case (typically relative position between objects). We propose several ways to generalize relationships of the first class in order to incorporate imprecision attached to the objects. For the second class, we argue that fuzzy definitions are appropriate even when dealing with crisp objects, and we propose original definitions.

Continuous label segmentation approaches have recently attracted much interest as they provide a formalism for handling image artifacts due to the partial volume effect which is common in for instance medical images. Here, the authors propose a new approach to this type of segmentation. Their work represents an extension of the now classic Markovian Bayesian discrete label segmentation approaches and provides good results on synthetic images simulating the presence of partial volumes as well as on real patient MR images.

In this paper we propose a new unsupervised method for estimating class membership functions from statistical data. It combines in an original way information derived from the histogram as well as prior knowledge of the requirements that the functions must satisfy and that cannot be derived from the histogram. The method has been tested successfully on MR brain images, and applications to image fusion are illustrated.

The notion of adjacency has a strong interest for image processing and pattern recognition, since it denotes an important relationship between objects or regions in an image, widely used as a feature in model-based pattern recognition. A crisp definition of adjacency often leads to low robustness in the presence of noise, imprecision, or segmentation errors. We propose two approaches to cope with spatial imprecision in image processing applications, both based on the framework of fuzzy sets. These approaches lead to two completely different classes of definitions of a degree of adjacency. In the first approach, we introduce imprecision as a property of the adjacency relation, and consider adjacency between two (crisp) objects to be a matter of degree. We represent adjacency by a fuzzy relation whose value depends on the distance between the objects. In the second approach, we introduce imprecision (in particular spatial imprecision) as a property of the objects, and consider objects to be fuzzy subsets of the image space. We then represent adjacency by a relation between fuzzy sets. This approach is, in our opinion, more powerful and general. We propose several ways for extending adjacency to fuzzy sets, either by using α-cuts, or by using a formal translation of binary equations into fuzzy ones. Since set equations are more easily translated into fuzzy terms, we shall privilege set representations of adjacency, particularly in the framework of fuzzy mathematical morphology. Finally, we give some hints on how to compare degrees of adjacency, typically for applications in model-based pattern recognition.

The aim of this paper is to show that Dempster-Shafer evidence theory may be successfully applied to unsupervised classification in multisource remote sensing. Dempster-Shafer formulation allows for consideration of unions of classes, and to represent both imprecision and uncertainty, through the definition of belief and plausibility functions. These two functions, derived from mass function, are generally chosen in a supervised way. In this paper, the authors describe an unsupervised method, based on the comparison of monosource classification results, to select the classes necessary for Dempster-Shafer evidence combination and to define their mass functions. Data fusion is then performed, discarding invalid clusters (e.g. corresponding to conflicting information) thank to an iterative process. Unsupervised multisource classification algorithm is applied to MAC-Europe'91 multisensor airborne campaign data collected over the Orgeval French site. Classification results using different combinations of sensors (TMS and AirSAR) or wavelengths (L- and C-bands) are compared. Performance of data fusion is evaluated in terms of identification of land cover types. The best results are obtained when all three data sets are used. Furthermore, some other combinations of data are tried, and their ability to discriminate between the different land cover types is quantified.

In order to cope with the ambiguity of spatial relative position concepts, we propose new definitions of the relative position between two objects in the fuzzy set framework, which are based on fuzzy pattern matching approaches. They have good properties, are flexible, fit the intuition, and they can be used for structural pattern recognition under imprecision. Moreover, they apply also in 3D, and for fuzzy image objects.

The authors' work contributes to the accurate segmentation of blood vessels from 3D medical images. The blood vessel axis and surface are optimized in an alternating way. Starting from an initial blood vessel axis estimate, slices are resampled in the 3D data volume perpendicular to this axis. In these slices, blood vessel contour candidate points are extracted at maximum gradient positions on a star pattern. The selection of a closed contour among these candidates is optimized with respect to a cost function by dynamic programming. The blood vessel axis is re-estimated at the center of the extracted contours and the process is repeated until convergence. Results are shown on both synthetic and real spiral CT angiographic images.

In most data fusion systems, the information extracted from each sensor (either numerical or symbolic) is represented as a degree of belief in an event with real values, taking in this way into account the imprecise, uncertain, and incomplete nature of the information. The combination of such degrees of belief is performed through numerical fusion operators. A very large variety of such operators has been proposed in the literature. We propose in this paper a classification of these operators issued from the different data fusion theories with respect to their behavior. Three classes are thus defined. This classification provides a guide for choosing an operator in a given problem. This choice can then be refined from the desired properties of the operators, from their decisiveness, and by examining how they deal with conflictive situations.

The authors present a fast stochastic method aiming at segmenting cerebral internal structures in 3D magnetic resonance images. An original method introducing context permits the authors to obtain reliable radiometric characteristics even for hardly discriminable brain structures. Segmentation is formulated as the labeling of a region adjacency graph. The graph is constructed by an extension to 3D of the watershed algorithm and the labeling is performed using a Markovian relaxation process. This leads to consistent results with a very low computational burden.

In the framework of fuzzy set theory, we propose: (i) a classification scheme for multi-modality image fusion, where membership degrees to a class issued from several images are combined before taking a decision, (ii) a classification of fusion operators, depending on their behaviour.>

A new kind of entropy is proposed, which associates spatial and radiometric properties of images. The possible use of this entropy is shown firstly to measure the effect of picture processing algorithms, then to control the evolution of iterative contextual classification algorithms like Markov random fields.>

We propose here a pre-processing of 3D shapes which allows to give greater importance to the match between some parts of the surface of one object and some parts of the surface of the other object during the matching of the two objects. The method is working with objects defined on a digital grid and consists in a segmentation step to separate the main components of the shapes, and in a distance computation step to determine matching weights to be assigned to the surface points. Several new algorithms are developed: a 3D segmentation algorithm based on mathematical morphology, a fast method to compute geodesic distances on a 3D surface, a simple sphere creation algorithm on a digital grid. The method has been applied to chemical molecules.