Capturing diversity is crucial in conditional and prompt-based image generation, particularly when conditions contain uncertainty that can lead to multiple plausible outputs. To generate diverse images reflecting this diversity, traditional methods often modify random seeds, making it difficult to discern meaningful differences between samples, or diversify the input prompt, which is limited in verbally interpretable diversity. We propose \modelnamenospace, a novel conditional image generation framework, applicable to any pretrained conditional generative model, that addresses inherent condition/prompt uncertainty and generates diverse plausible images. \modelname is based on a simple yet effective idea: decomposing the input condition into diverse latent representations, each capturing an aspect of the uncertainty and generating a distinct image. First, we integrate a latent graph, parameterized by Generative Flow Networks (GFlowNets), into the prompt representation computation. Second, leveraging GFlowNets' advanced graph sampling capabilities to capture uncertainty and output diverse trajectories over the graph, we produce multiple trajectories that collectively represent the input condition, leading to diverse condition representations and corresponding output images. Evaluations on natural image and medical image datasets demonstrate \modelnamenospace’s improvement in both diversity and fidelity across image synthesis, image generation, and counterfactual generation tasks.

The integration of diverse clinical modalities such as medical imaging and the tabular data extracted from patients’ Electronic Health Records (EHRs) is a crucial aspect of modern healthcare. Integrative analysis of multiple sources can provide a comprehensive understanding of the clinical condition of a patient, improving diagnosis and treatment decision. Deep Neural Networks (DNNs) consistently demonstrate outstanding performance in a wide range of multimodal tasks in the medical domain. However, the complex endeavor of effectively merging medical imaging with clinical, demographic and genetic information represented as numerical tabular data remains a highly active and ongoing research pursuit. We present a novel framework based on hypernetworks to fuse clinical imaging and tabular data by conditioning the image processing on the EHR’s values and measurements. This approach aims to leverage the complementary information present in these modalities to enhance the accuracy of various medical applications. We demonstrate the strength and generality of our method on two different brain Magnetic Resonance Imaging (MRI) analysis tasks, namely, brain age prediction conditioned by subject’s sex and multi-class Alzheimer’s Disease (AD) classification conditioned by tabular data. We show that our framework outperforms both single-modality models and state-of-the-art MRI tabular data fusion methods. A link to our code can be found at https://github.com/daniel4725/HyperFusion . • We present a HyperFusion network - a novel hypernetwork for medical imaging and tabular data fusion. • A hypernetwork controls a primary network by producing parameters to predefined layers. • This mechanism is exploited to condition image processing predictions by tabular data. • The HyperFusion outperforms existing imaging-tabular fusion methods for Alzheimer’s disease classification. • The HyperFusion versatility is demonstrated for brain age prediction conditioned by sex.

Once deployed, medical image analysis methods are often faced with unexpected image corruptions and noise perturbations. These unknown covariate shifts present significant challenges to deep learning based methods trained on "clean" images. This often results in unreliable predictions and poorly calibrated confidence, hence hindering clinical applicability. While recent methods have been developed to address specific issues such as confidence calibration or adversarial robustness, no single framework effectively tackles all these challenges simultaneously. To bridge this gap, we propose LaDiNE, a novel ensemble learning method combining the robustness of Vision Transformers with diffusion-based generative models for improved reliability in medical image classification. Specifically, transformer encoder blocks are used as hierarchical feature extractors that learn invariant features from images for each ensemble member, resulting in features that are robust to input perturbations. In addition, diffusion models are used as flexible density estimators to estimate member densities conditioned on the invariant features, leading to improved modeling of complex data distributions while retaining properly calibrated confidence. Extensive experiments on tuberculosis chest X-rays and melanoma skin cancer datasets demonstrate that LaDiNE achieves superior performance compared to a wide range of state-of-the-art methods by simultaneously improving prediction accuracy and confidence calibration under unseen noise, adversarial perturbations, and resolution degradation.

We propose an analysis in fair learning that preserves the utility of the data while reducing prediction disparities under the criteria of group sufficiency. We focus on the scenario where the data contains multiple or even many subgroups, each with limited number of samples. As a result, we present a principled method for learning a fair predictor for all subgroups via formulating it as a bilevel objective. Specifically, the subgroup specific predictors are learned in the lower-level through a small amount of data and the fair predictor. In the upper-level, the fair predictor is updated to be close to all subgroup specific predictors. We further prove that such a bilevel objective can effectively control the group sufficiency and generalization error. We evaluate the proposed framework on real-world datasets. Empirical evidence suggests the consistently improved fair predictions, as well as the comparable accuracy to the baselines.

Although deep networks have been shown to perform very well on a variety of medical imaging tasks, inference in the presence of pathology presents several challenges to common models. These challenges impede the integration of deep learning models into real clinical workflows, where the customary process of cascading deterministic outputs from a sequence of image-based inference steps (e.g. registration, segmentation) generally leads to an accumulation of errors that impacts the accuracy of downstream inference tasks. In this paper, we propose that by embedding uncertainty estimates across cascaded inference tasks, performance on the downstream inference tasks should be improved. We demonstrate the effectiveness of the proposed approach in three different clinical contexts: (i) We demonstrate that by propagating T2 weighted lesion segmentation results and their associated uncertainties, subsequent T2 lesion detection performance is improved when evaluated on a proprietary large-scale, multi-site, clinical trial dataset acquired from patients with Multiple Sclerosis. (ii) We show an improvement in brain tumour segmentation performance when the uncertainty map associated with a synthesised missing MR volume is provided as an additional input to a follow-up brain tumour segmentation network, when evaluated on the publicly available BraTS-2018 dataset. (iii) We show that by propagating uncertainties from a voxel-level hippocampus segmentation task, the subsequent regression of the Alzheimer's disease clinical score is improved.

The number of biomedical image analysis challenges organized per year is steadily increasing. These international competitions have the purpose of benchmarking algorithms on common data sets, typically to identify the best method for a given problem. Recent research, however, revealed that common practice related to challenge reporting does not allow for adequate interpretation and reproducibility of results. To address the discrepancy between the impact of challenges and the quality (control), the Biomedical Image Analysis ChallengeS (BIAS) initiative developed a set of recommendations for the reporting of challenges. The BIAS statement aims to improve the transparency of the reporting of a biomedical image analysis challenge regardless of field of application, image modality or task category assessed. This article describes how the BIAS statement was developed and presents a checklist which authors of biomedical image analysis challenges are encouraged to include in their submission when giving a paper on a challenge into review. The purpose of the checklist is to standardize and facilitate the review process and raise interpretability and reproducibility of challenge results by making relevant information explicit.

This paper investigates the effects of sampling on action recognition performance. Currently, dense (regular grid) sampling and uniform random sampling are popular strategies that achieve state-of-the-art performance. However, they are data-blind and pay equal attention to locations of different informativeness. In this paper, a Shannon information based adaptive sampling approach is proposed for action recognition. Results of different sampling approaches are compared on three benchmark datasets: the basic KTH and the challenging HMDB51 and UCF101 datasets. The method is shown to improve recognition accuracy as well as computational efficiency over the current state-of-the-art using less than one percent of the total pixels.

Studies have demonstrated the feasibility of late Gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) imaging for guiding the management of patients with sequelae to myocardial infarction, such as ventricular tachycardia and heart failure. Clinical implementation of these developments necessitates a reproducible and reliable segmentation of the infarcted regions. It is challenging to compare new algorithms for infarct segmentation in the left ventricle (LV) with existing algorithms. Benchmarking datasets with evaluation strategies are much needed to facilitate comparison. This manuscript presents a benchmarking evaluation framework for future algorithms that segment infarct from LGE CMR of the LV. The image database consists of 30 LGE CMR images of both humans and pigs that were acquired from two separate imaging centres. A consensus ground truth was obtained for all data using maximum likelihood estimation. Six widely-used fixed-thresholding methods and five recently developed algorithms are tested on the benchmarking framework. Results demonstrate that the algorithms have better overlap with the consensus ground truth than most of the n-SD fixed-thresholding methods, with the exception of the Full-Width-at-Half-Maximum (FWHM) fixed-thresholding method. Some of the pitfalls of fixed thresholding methods are demonstrated in this work. The benchmarking evaluation framework, which is a contribution of this work, can be used to test and benchmark future algorithms that detect and quantify infarct in LGE CMR images of the LV. The datasets, ground truth and evaluation code have been made publicly available through the website: https://www.cardiacatlas.org/web/guest/challenges.

Recent literature shows that facial attributes, i.e., contextual facial information, can be beneficial for improving the performance of real-world applications, such as face verification, face recognition, and image search. Examples of face attributes include gender, skin color, facial hair, etc. How to robustly obtain these facial attributes (traits) is still an open problem, especially in the presence of the challenges of real-world environments: non-uniform illumination conditions, arbitrary occlusions, motion blur and background clutter. What makes this problem even more difficult is the enormous variability presented by the same subject, due to arbitrary face scales, head poses, and facial expressions. In this paper, we focus on the problem of facial trait classification in real-world face videos. We have developed a fully automatic hierarchical and probabilistic framework that models the collective set of frame class distributions and feature spatial information over a video sequence. The experiments are conducted on a large real-world face video database that we have collected, labelled and made publicly available. The proposed method is flexible enough to be applied to any facial classification problem. Experiments on a large, real-world video database McGillFaces [1] of 18,000 video frames reveal that the proposed framework outperforms alternative approaches, by up to 16.96 and 10.13%, for the facial attributes of gender and facial hair, respectively.

We propose a conditional random field (CRF) based classifier for segmentation of small enhanced pathologies. Specifically, we develop a temporal hierarchical adaptive texture CRF (THAT-CRF) and apply it to the challenging problem of gad enhancing lesion segmentation in brain MRI of patients with multiple sclerosis. In this context, the presence of many nonlesion enhancements (such as blood vessels) renders the problem more difficult. In addition to voxel-wise features, the framework exploits multiple higher order textures to discriminate the true lesional enhancements from the pool of other enhancements. Since lesional enhancements show more variation over time as compared to the nonlesional ones, we incorporate temporal texture analysis in order to study the textures of enhanced candidates over time. The parameters of the THAT-CRF model are learned based on 2380 scans from a multi-center clinical trial. The effect of different components of the model is extensively evaluated on 120 scans from a separate multi-center clinical trial. The incorporation of the temporal textures results in a general decrease of the false discovery rate. Specifically, THAT-CRF achieves overall sensitivity of 95% along with false discovery rate of 20% and average false positive count of 0.5 lesions per scan. The sensitivity of the temporal method to the trained time interval is further investigated on five different intervals of 69 patients. Moreover, superior performance is achieved by the reviewed labelings of our model compared to the fully manual labeling when applied to the context of separating different treatment arms in a real clinical trial.

In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients-manually annotated by up to four raters-and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.

In this paper, we introduce a fully automated multistage graphical probabilistic framework to segment brain tumours from multimodal Magnetic Resonance Images (MRIs) acquired from real patients. An initial Bayesian tumour classification based on Gabor texture features permits subsequent computations to be focused on areas where the probability of tumour is deemed high. An iterative, multistage Markov Random Field (MRF) framework is then devised to classify the various tumour subclasses (e.g. edema, solid tumour, enhancing tumour and necrotic core). Specifically, an adapted, voxel-based MRF provides tumour candidates to a higher level, regional MRF, which then leverages both contextual texture information and relative spatial consistency of the tumour subclass positions to provide updated regional information down to the voxel-based MRF for further local refinement. The two stages iterate until convergence. Experiments are performed on publicly available, patient brain tumour images from the MICCAI 2012 [11] and 2013 [12] Brain Tumour Segmentation Challenges. The results demonstrate that the proposed method achieves the top performance in the segmentation of tumour cores and enhancing tumours, and performs comparably to the winners in other tumour categories.

Head pose estimation has been receiving a lot of attention due to its wide range of possible applications. However, most approaches in the literature have focused on head pose estimation in controlled environments. Head pose estimation has recently begun to be applied to real-world environments. However, the focus has been on estimation from single images or video frames. Furthermore, most approaches frame the problem as classification into a set of coarse pose bins, rather than performing continuous pose estimation. The proposed multi-layer probabilistic temporal graphical model robustly estimates continuous head pose angle while leveraging the strengths of multiple features into account. Experiments performed on a large, real-world video database show that our approach not only significantly outperforms alternative head pose approaches, but also provides a pose probability assigned at each video frame, which permits robust temporal, probabilistic fusion of pose information over the entire video sequence.

Detection of new Multiple Sclerosis (MS) lesions on magnetic resonance imaging (MRI) is important as a marker of disease activity and as a potential surrogate for relapses. We propose an approach where sequential scans are jointly segmented, to provide a temporally consistent tissue segmentation while remaining sensitive to newly appearing lesions. The method uses a two-stage classification process: 1) a Bayesian classifier provides a probabilistic brain tissue classification at each voxel of reference and follow-up scans, and 2) a random-forest based lesion-level classification provides a final identification of new lesions. Generative models are learned based on 364 scans from 95 subjects from a multi-center clinical trial. The method is evaluated on sequential brain MRI of 160 subjects from a separate multi-center clinical trial, and is compared to 1) semi-automatically generated ground truth segmentations and 2) fully manual identification of new lesions generated independently by nine expert raters on a subset of 60 subjects. For new lesions greater than 0.15 cc in size, the classifier has near perfect performance (99% sensitivity, 2% false detection rate), as compared to ground truth. The proposed method was also shown to exceed the performance of any one of the nine expert manual identifications.

This paper presents a novel probabilistic voxel selection strategy for medical image registration in time-sensitive contexts, where the goal is aggressive voxel sampling (e.g., using less than 1% of the total number) while maintaining registration accuracy and low failure rate. We develop a Bayesian framework whereby, first, a voxel sampling probability field (VSPF) is built based on the uncertainty on the transformation parameters. We then describe a practical, multi-scale registration algorithm, where, at each optimization iteration, different voxel subsets are sampled based on the VSPF. The approach maximizes accuracy without committing to a particular fixed subset of voxels. The probabilistic sampling scheme developed is shown to manage the tradeoff between the robustness of traditional random voxel selection (by permitting more exploration) and the accuracy of fixed voxel selection (by permitting a greater proportion of informative voxels).

Gadolinium-enhancing lesions in brain magnetic resonance imaging of multiple sclerosis (MS) patients are of great interest since they are markers of disease activity. Identification of gadolinium-enhancing lesions is particularly challenging because the vast majority of enhancing voxels are associated with normal structures, particularly blood vessels. Furthermore, these lesions are typically small and in close proximity to vessels. In this paper, we present an automatic, probabilistic framework for segmentation of gadolinium-enhancing lesions in MS using conditional random fields. Our approach, through the integration of different components, encodes different information such as correspondence between the intensities and tissue labels, patterns in the labels, or patterns in the intensities. The proposed algorithm is evaluated on 80 multimodal clinical datasets acquired from relapsing-remitting MS patients in the context of multicenter clinical trials. The experimental results exhibit a sensitivity of 98% with a low false positive lesion count. The performance of the proposed algorithm is also compared to a logistic regression classifier, a support vector machine and a Markov random field approach. The results demonstrate superior performance of the proposed algorithm at successfully detecting all of the gadolinium-enhancing lesions while maintaining a low false positive lesion count.

In this paper, we propose a new multi-scale technique for multi-modal image registration based on the alignment of selected gradient orientations of reduced uncertainty. We show how the registration robustness and accuracy can be improved by restricting the evaluation of gradient orientation alignment to locations where the uncertainty of fixed image gradient orientations is minimal, which we formally demonstrate correspond to locations of high gradient magnitude. We also embed a computationally efficient technique for estimating the gradient orientations of the transformed moving image (rather than resampling pixel intensities and recomputing image gradients). We have applied our method to different rigid multi-modal registration contexts. Our approach outperforms mutual information and other competing metrics in the context of rigid multi-modal brain registration, where we show sub-millimeter accuracy with cases obtained from the retrospective image registration evaluation project. Furthermore, our approach shows significant improvements over standard methods in the highly challenging clinical context of image guided neurosurgery, where we demonstrate misregistration of less than 2 mm with relation to expert selected landmarks for the registration of pre-operative brain magnetic resonance images to intra-operative ultrasound images.

In unconstrained environments, head pose detection can be very challenging due to the joint and arbitrary occurrence of facial expressions, background clutter, partial occlusions and illumination conditions. Despite the wide range of head pose literature, most current methods can address this problem only up to a certain degree, and mostly for restricted scenarios. In this paper, we address the problem of head pose classification from real world images with large appearance variation. We represent each pose with a probabilistic and spatial template learned from facial codewords. The inference of the best template representing a test image is achieved probabilistically and spatially at the codebook. The experimental results are obtained from 5500 video frames collected under different illumination and background conditions. Our probabilistic framework is shown to outperform the current state-of-the-art in head pose classification.

EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.

This paper presents feature-based morphometry (FBM), a new, fully data-driven technique for identifying group-related differences in volumetric imagery. In contrast to most morphometry methods which assume one-to-one correspondence between all subjects, FBM models images as a collage of distinct, localized image features which may not be present in all subjects. FBM thus explicitly accounts for the case where the same anatomical tissue cannot be reliably identified in all subjects due to disease or anatomical variability. A probabilistic model describes features in terms of their appearance, geometry, and relationship to subgroups of a population, and is automatically learned from a set of subject images and group labels. Features identified indicate group-related anatomical structure that can potentially be used as disease biomarkers or as a basis for computer-aided diagnosis. Scale-invariant image features are used, which reflect generic, salient patterns in the image. Experiments validate FBM clinically in the analysis of normal (NC) and Alzheimer's (AD) brain images using the freely available OASIS database. FBM automatically identifies known structural differences between NC and AD subjects in a fully data-driven fashion, and obtains an equal error classification rate of 0.78 on new subjects.

This paper presents a novel framework for detecting, localizing, and classifying faces in terms of visual traits, e.g., sex or age, from arbitrary viewpoints and in the presence of occlusion. All three tasks are embedded in a general viewpoint-invariant model of object class appearance derived from local scale-invariant features, where features are probabilistically quantified in terms of their occurrence, appearance, geometry, and association with visual traits of interest. An appearance model is first learned for the object class, after which a Bayesian classifier is trained to identify the model features indicative of visual traits. The framework can be applied in realistic scenarios in the presence of viewpoint changes and partial occlusion, unlike other techniques assuming data that are single viewpoint, upright, prealigned, and cropped from background distraction. Experimentation establishes the first result for sex classification from arbitrary viewpoints, an equal error rate of 16.3 percent, based on the color FERET database. The method is also shown to work robustly on faces in cluttered imagery from the CMU profile database. A comparison with the geometry-free bag-of-words model shows that geometrical information provided by our framework improves classification. A comparison with support vector machines demonstrates that Bayesian classification results in superior performance.

In freehand 3-D ultrasound (US), the relative positions of US images are usually measured using a position tracking device despite its cumbersome nature. The probe trajectory can instead be estimated from image data, using registration techniques to recover in-plane motion and speckle decorrelation to recover out-of-plane transformations. The relationship between speckle decorrelation and elevational separation is typically represented by a single curve, estimated from calibration data. Distances read off such a curve are corrupted by bias and uncertainty, and only provide an absolute estimate of elevational displacement. This paper presents a probabilistic model of the relationship between correlation measurements and elevational separation. This representation captures the skewed distribution of distance estimates based on high correlations and the uncertainties attached to each measurement. Multiple redundant correlation measurements can then be integrated within a maximum likelihood estimation framework. This paper also introduces a new method based on the traveling salesman problem for resolving sign ambiguities in data sets resulting from nonmonotonic probe motion and frame intersections. Experiments with real and synthetic US data show that by combining these new methods, out-of-plane US probe motion is recovered with improved accuracy over baseline methods using a deterministic model and fewer measurements.

In this paper, we investigate detection and localization of general 3D object classes by relating local scale-invariant features to a viewpoint invariant reference frame. This can generally be achieved by either a multi-view representation, where features and reference frame are modeled as a collection of distinct views, or by a viewpoint invariant representation, where features and reference frame are modeled independently of viewpoint. We compare multi-view and viewpoint invariant representations trained and tested on the same data, where the viewpoint invariant approach results in fewer false positive detections and higher average precision. We present a new, iterative learning algorithm to determine an optimal viewpoint invariant reference frame from training images in a data-driven manner. The learned optimal reference frame is centrally located with respect to the 3D object class and to image features in a given view, thereby minimizing reference frame localization error as predicted by theory and maintaining a consistent geometrical interpretation with respect to the underlying object class. Modeling and detection based on the optimal reference frame improves detection performance for both multiview and viewpoint invariant representations. Experimentation is performed on the class of 3D faces, using the public color FERET database for training, the CMU profile database for testing and SIFT image features.

In this paper, we present a statistical parts-based model (PBM) of appearance, applied to the problem of modeling intersubject anatomical variability in magnetic resonance (MR) brain images. In contrast to global image models such as the active appearance model (AAM), the PBM consists of a collection of localized image regions, referred to as parts, whose appearance, geometry and occurrence frequency are quantified statistically. The parts-based approach explicitly addresses the case where one-to-one correspondence does not exist between all subjects in a population due to anatomical differences, as model parts are not required to appear in all subjects. The model is constructed through a fully automatic machine learning algorithm, identifying image patterns that appear with statistical regularity in a large collection of subject images. Parts are represented by generic scale-invariant features, and the model can, therefore, be applied to a wide variety of image domains. Experimentation based on 2-D MR slices shows that a PBM learned from a set of 102 subjects can be robustly fit to 50 new subjects with accuracy comparable to 3 human raters. Additionally, it is shown that unlike global models such as the AAM, PBM fitting is stable in the presence of unexpected, local perturbation.

Feature points for image correspondence are often selected according to subjective criteria (e.g. edge density, nostrils). In this paper, we present a general, nonsubjective criterion for selecting informative feature points, based on the correspondence model itself. We describe the approach within the framework of the Bayesian Markov random field (MRF) model, where the degree of feature point information is encoded by the entropy of the likelihood term. We propose that feature selection according to minimum entropy-of-likelihood (EOL) is less likely to lead to correspondence ambiguity, thus improving the optimization process in terms of speed and quality of solution. Experimental results demonstrate the criterion's ability to select optimal features points in a wide variety of image contexts (e.g. objects, faces). Comparison with the automatic Kanade-Lucas-Tomasi feature selection criterion shows correspondence to be significantly faster with feature points selected according to minimum EOL in difficult correspondence problems.

In this paper we propose a paradigm called the Interactive Visual Dialog (IVD) as a means of facilitating a system’s ability to recognize objects p-resented to it by a human. The presentation centers around a supermarket checkout scenario in which an operator presents an item to be tallied to a sta-tionary television camera. An active vision approach is used to provide feed-back to the operator in the form of an image (or images) depicting what the system thinks the operator is most likely holding, shown in a viewpoint that suggests how the object should next be presented to improve the certainty of interpretation. Interaction proceeds iteratively until the system converges on the correct interpretation. We show how the IVD can be implemented using an entropy-based gaze planning strategy and a sequential Bayes recognition system using optical flow as input. Experimental results show that the system does, in practice, improve recognition accuracy, leading to convergence to a correct solution in a minimal number of iterations. 1

This paper introduces an object recognition strategy based on the following premises: i) an object can be identified on the basis of the optical flow it induces on a stationary observer, and ii) a basis for recognition can be built on the appearance of flow corresponding to local curvilinear motion. Unlike other approaches that seek to recognize particular motions, ours focuses on the problem of recognizing objects by training on an expected set of motions. A sequential estimation framework is used to solve the implicit factorization problem, which is itself simplified in that the task is to discriminate between different objects as opposed to recovering motion or structure. Training is accomplished automatically using a robot mounted camera system to induce a set of canonical motions at different locations on a viewsphere. Experimen-tal results are presented to support our contention that the resulting motion basis can generalize to a fairly wide range of motions, leading to a practical method for recognizing moving objects. 1

In this paper, we show how entropy maps can be used to guide an active observer along an optimal trajectory, by which the identity and pose of objects in the world can be inferred with confidence, while minimizing the amount of data that must be gathered. Specifically we consider the case of active object recognition where entropy maps are used to encode prior knowledge about the discriminability of objects as a function of viewing position. The paper describes how these maps are computed using optical flow signatures as a case study, and how a gaze-planning strategy can be formulated by using entropy minimization as a basis for choosing a next best view. Experimental results are presented which show the strategy's effectiveness for active object recognition using a single monochrome television camera.

This paper describes a new framework for parametric shape recognition. The key result is a method for generating classifiers in the form of conditional probability densities for recognizing an unknown from a set of reference models. The authors' procedure is automatic. Off-line, it invokes an autonomous process to estimate reference model parameters and their statistics. On-line, during measurement, it combines these with a priori context-dependent information, as well as the parameters and statistics estimated for an unknown object, into a conditional probability density function, which represents the belief that the unknown is a particular reference model. The paper also describes the implementation of this procedure in a system for automatically generating and recognizing 3-D part-oriented models. The authors show that recognition performance is near perfect for cases in which complete surface information is accessible to the algorithm, and that it falls off gracefully when only partial information is available. This leads to the possibility of an active recognition strategy in which the belief measures associated with each classification can be used as feedback for the acquisition of further evidence as required.