Automatic cell tracking in dense environments is plagued by inaccurate correspondences and misidentification of parent-offspring relationships. In this paper, we introduce a novel cell tracking algorithm named DenseTrack, which integrates deep learning with mathematical model-based strategies to effectively establish correspondences between consecutive frames and detect cell division events in crowded scenarios. We formulate the cell tracking problem as a deep learning-based temporal sequence classification task followed by solving a constrained one-to-one matching optimization problem exploiting the classifier's confidence scores. Additionally, we present an eigendecomposition-based cell division detection strategy that leverages knowledge of cellular geometry. The performance of the proposed approach has been evaluated by tracking densely packed cells in 3D time-lapse image sequences of bacterial biofilm development. The experimental results on simulated as well as experimental fluorescence image sequences suggest that the proposed tracking method achieves superior performance in terms of both qualitative and quantitative evaluation measures compared to recent state-of-the-art cell tracking approaches.

Ultrasound images are corrupted by signal-dependent speckle, degrading the image quality and presenting challenges for downstream tasks such as segmentation and classification. The ultrasound transducer, as modeled by the point spread function (PSF), further distorts the speckle and the signal. The PSF has different lateral and axial distortions which should be considered in the design of efficient speckle removal methods. To this end, we propose a novel lateral and axial distortion-aware diffusion network that encodes the spectrum of lateral and axial distortions, thus enabling adaptive denoising of images corrupted with speckle. The distortions have been modeled in the forward and reverse processes of a multiplicative noise-based diffusion model. Extensive experiments on two datasets establish the efficiency of the proposed model over state-of-the-art methods. The code and data are available at https://github.com/soumeeguha/PSF-SRDN.

We present a causal state space model (CSSM) for video understanding that couples a learned causal DAG with latent state dynamics. Latent factors form DAG nodes, enabling explicit cause–effect modeling over time; the state-space form provides efficient sequence inference, while the graph adds interpretability and robustness to distribution shifts. We learn the latent graph and inject its adjacency into the transition operator. On HMDB-51, UCF-101, and HAR, CSSM improves accuracy over strong baselines and supports counterfactual reasoning about video events.

This paper presents a novel spatiotemporal transformer network that introduces several original components to detect actions in untrimmed videos. First, the multi-feature selective semantic attention model calculates the correlations between spatial and motion features to model spatiotemporal interactions between different action semantics properly. Second, the motion-aware network encodes the locations of action semantics in video frames utilizing the motion-aware 2D positional encoding algorithm. Such a motion-aware mechanism memorizes the dynamic spatiotemporal variations in action frames that current methods cannot exploit. Third, the sequence-based temporal attention model captures the heterogeneous temporal dependencies in action frames. In contrast to standard temporal attention used in natural language processing, primarily aimed at finding similarities between linguistic words, the proposed sequence-based temporal attention is designed to determine both the differences and similarities between video frames that jointly define the meaning of actions. The proposed approach outperforms the state-of-the-art solutions on four spatiotemporal action datasets: AVA 2.2, AVA 2.1, UCF101-24, and EPIC-Kitchens.

Single-molecule tracking is a powerful tool to measure the dynamics of proteins in living cells. Analysis of molecule diffusion aids our understanding of molecular mechanisms and helps us distinguish between freely diffusing cytosolic protein populations and the more slowly moving membrane-associated population. There is no existing technique that distinguishes the diffusion coefficients of single-molecule images without analyzing their trajectories, which extend over multiple sequential camera frames. To this end, we propose a spatial and channel attention-based convolutional neural network (CNN) architecture with latent entropy minimization that efficiently classifies individual single-molecule images by the imaged molecules’ diffusion coefficients. We also propose a loss function which minimizes the entropy of the attention maps. Experiments demonstrate that the diffusion coefficients can efficiently identify different types of molecules in simulated and experimental datasets and our method outperforms state-of-the-art models for image classification (e.g., Resnet-18 and Densenet).

The papers in this special section focus on large-scale medical imaging and video analytics for clinical decision support systems. Biomedical images and videos are ubiquitous and overwhelming in volume, amounting to a database that can be measured in zettabytes.With increased access to open image and video datasets and the recent development of effective image and video analysis systems, there is a unique opportunity for the development of artificial intelligence (AI) systems that can be trained and tested on large-scale biomedical image and video databases. The special issue summarizes emerging methods associated with the development of computer-aided diagnostic systems. More specifically, the special issue discusses the development of methods for dealing with small or large or creating new datasets, biomedical image segmentation, and image classification. The development of new dataset methods allows us to develop methods for specific diseases, employ meta-learning for training on small datasets, or develop methods for reducing larger video datasets. Biomedical image segmentation is a primary focus of the special issue.

Transform-domain least mean squares (TDLMS) adaptive filters encompass the class of learning algorithms where the input data are subjected to a data-independent unitary transform followed by a power normalization stage as preprocessing steps. Because conventional transformations are not data-dependent, this preconditioning procedure was shown theoretically to improve the convergence of the least mean squares (LMS) filter only for certain classes of input data. So, one can tailor the transformation to the class of data. However, in reality, if the class of input data is not known beforehand, it is difficult to decide which transformation to use. Thus, there is a need to devise a learning framework to obtain such a preconditioning transformation using input data prior to applying on the input data. It is hypothesized that the underlying topology of the data affects the selection of the transformation. With the input modeled as a weighted finite graph, our method, called preconditioning using graph (PrecoG), adaptively learns the desired transform by recursive estimation of the graph Laplacian matrix. We show the efficacy of the transform as a generalized split preconditioner on a linear system of equations and in Hebbian-LMS learning models. In terms of the improvement of the condition number after applying the transformation, PrecoG performs significantly better than the existing state-of-the-art techniques that involve unitary and nonunitary transforms.

MOTIVATION: Data-driven deep learning techniques usually require a large quantity of labeled training data to achieve reliable solutions in bioimage analysis. However, noisy image conditions and high cell density in bacterial biofilm images make 3D cell annotations difficult to obtain. Alternatively, data augmentation via synthetic data generation is attempted, but current methods fail to produce realistic images. RESULTS: This article presents a bioimage synthesis and assessment workflow with application to augment bacterial biofilm images. 3D cyclic generative adversarial networks (GAN) with unbalanced cycle consistency loss functions are exploited in order to synthesize 3D biofilm images from binary cell labels. Then, a stochastic synthetic dataset quality assessment (SSQA) measure that compares statistical appearance similarity between random patches from random images in two datasets is proposed. Both SSQA scores and other existing image quality measures indicate that the proposed 3D Cyclic GAN, along with the unbalanced loss function, provides a reliably realistic (as measured by mean opinion score) 3D synthetic biofilm image. In 3D cell segmentation experiments, a GAN-augmented training model also presents more realistic signal-to-background intensity ratio and improved cell counting accuracy. AVAILABILITY AND IMPLEMENTATION: https://github.com/jwang-c/DeepBiofilm. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Recent deep learning methods have provided successful initial segmentation results for generalized cell segmentation in microscopy. However, for dense arrangements of small cells with limited ground truth for training, the deep learning methods produce both over-segmentation and under-segmentation errors. Post-processing attempts to balance the trade-off between the global goal of cell counting for instance segmentation, and local fidelity to the morphology of identified cells. The need for post-processing is especially evident for segmenting 3D bacterial cells in densely-packed communities called biofilms. A graph-based recursive clustering approach, m-LCuts, is proposed to automatically detect collinearly structured clusters and applied to post-process unsolved cells in 3D bacterial biofilm segmentation. Construction of outlier-removed graphs to extract the collinearity feature in the data adds additional novelty to m-LCuts. The superiority of m-LCuts is observed by the evaluation in cell counting with over 90% of cells correctly identified, while a lower bound of 0.8 in terms of average single-cell segmentation accuracy is maintained. This proposed method does not need manual specification of the number of cells to be segmented. Furthermore, the broad adaptation for working on various applications, with the presence of data collinearity, also makes m-LCuts stand out from the other approaches.

In visual object tracking, robust and accurate scale estimation of a target is a challenging task. Despite the associated computational expense, existing tracking methods cannot accommodate large scale variations. Here, we propose a scale searching scheme that obtains robust and accurate scale estimation by incorporating a novel and robust criterion, the average peak-to-correlation energy, into a multi-resolution translation filter framework. To address the problem of computational expense, we introduce an expeditious search strategy. The resulting system is named FAST: Fast and Accurate Scale estimation for Tracking. Comprehensive evaluation using the publicly available tracking benchmark datasets demonstrates that the proposed scale searching framework can accommodate large scale variation while also yielding computational efficiency.

Robust and accurate scale estimation of a target object is a challenging task in visual object tracking. Most existing tracking methods cannot accommodate large scale variation in complex image sequences and thus result in inferior performance. In this paper, we propose to incorporate a novel criterion called the average peak-to-correlation energy into the multi-resolution translation filter framework to obtain robust and accurate scale estimation. The resulting system is named SITUP: Scale Invariant Tracking using Average Peak-to-Correlation Energy. SITUP effectively tackles the problem of fixed template size in standard discriminative correlation filter based trackers. Extensive empirical evaluation on the publicly available tracking benchmark datasets demonstrates that the proposed scale searching framework meets the demands of scale variation challenges effectively while providing superior performance over other scale adaptive variants of standard discriminative correlation filter based trackers. Also, SITUP obtains favorable performance compared to state-of-the-art trackers for various scenarios while operating in real-time on a single CPU.

We propose a shape-based solution for speckle removal from ultrasound images. The method is operable in both low contrast and high contrast imaging scenarios. The approach introduces shape information alongside structural information in the speckle removing filter. By iteratively minimizing a shape fidelity penalty to reduce speckle, the proposed filter facilitates superior segmentation of blood vessels. The effectiveness of the proposed method is established through experimentation on the ultrasound images of human blood vessels. The results show at more than 7% improvement in PSNR values compared to other state-of-the-art approaches.

Gait recognition is a leading remote-based identification method, suitable for real-world surveillance and medical applications. Model-based gait recognition methods have been particularly recognized due to their scale and view-invariant properties. We present the first model-based gait recognition methodology, Glidar3DJ using a skeleton model extracted from sequences generated by a single flash lidar camera. Existing successful model-based approaches take advantage of high quality skeleton data collected by Kinect and Mocap, for example, are not practicable for applications outside the laboratory. The low resolution and noisy imaging process of lidar negatively affects the performance of state-of-the-art skeleton-based systems, generating a significant number of outlier skeletons. We propose a rule-based filtering mechanism that adopts robust statistics to correct for skeleton joint measurements. Quantitative measurements validate the efficacy of the proposed method in improving gait recognition.

In this paper, we present a novel region-based segmentation method for cortical spreading depressions in 2-photon microscopy images. Fluorescent microscopy has become an important tool in neuroscience, but segmentation approaches are challenged by the opaque properties and structures of brain tissue. These challenges are made more extreme when segmenting events such as cortical spreading depressions, where low signal-to-noise ratios and intensity inhomogeneity dominate images. The method we propose uses a local intensity similarity measure that takes advantage of normalized Euclidean and geodesic distance maps of the image. This method provides a smooth segmentation boundary which is robust to the noise and inhomogeneity within cortical spreading depression images. Experimental results yielded a DICE index of 0.9859, an increase of 6% over the current state-of-the-art, and a reduction of root mean square error by 79.9%.

Bacterial biofilm segmentation poses significant challenges due to lack of apparent structure, poor imaging resolution, limited contrast between conterminous cells and high density of cells that overlap. Although there exist bacterial segmentation algorithms in the existing art, they fail to delineate cells in dense biofilms, especially in 3D imaging scenarios in which the cells are growing and subdividing in a complex manner. A graph-based data clustering method, £Cuts, is presented with the application on bacterial cell segmentation. By constructing a weighted graph with node features in locations and principal orientations, the proposed method can automatically classify and detect differently oriented aggregations of linear structures (represent by bacteria in the application). The method assists in the assessment of several facets, such as bacterium tracking, cluster growth, and mapping of migration patterns of bacterial biofilms. Quantitative and qualitative measures for 2D data demonstrate the superiority of proposed method over the state of the art. Preliminary 3D results exhibit reliable classification of the cells with 97% accuracy.

This work proposes a novel computational modeling to estimate 3D dense skeleton and corresponding joint locations from Lidar (light detection and ranging) full motion video (FMV). Unlike motion capture (MoCap) video, where body mounted reflectors are used to capture 3D skeleton in a controlled research environment, the proposed model obtains full 3D dense skeleton in Lidar FMV. Our proposed method extracts 3D pose for subjects with walking motion from the 3D dense joints. The second contribution involves extraction of silhouette-based features and augmentation of the pose features with silhouette-based features generated over small windows of the video for human subject identification. We evaluate our model with a 10-person in-house Lidar FMV dataset and the proposed method offers 91.69% of cross-validated accuracy using a support vector machine (SVM) classification. For comparison, we implement transfer learning for another well-known deep learning-based human identification method, OpenPose, using the same Lidar FMV. The fully tuned OpenPose offers 85.00% cross-validated identification rate using the same dataset. The comparison suggests that the proposed computational modeling offers better human identification performance when compared to OpenPose transfer learning method using the 10-person Lidar FMV.

In this paper, we describe a novel enhancement method for images containing filamentous structures. Our method combines a gradient sparsity constraint with a filamentous structure constraint for the effective removal of clutter and noise from the background. The method is applied and evaluated on three types of data: 1) confocal microscopy images of neurons; 2) calcium imaging data; and 3) images of road pavement. We found that the images enhanced by our method preserve both the structure and the intensity details of the original object. In the case of neuron microscopy, we find that the neurons enhanced by our method are better correlated with the original structure intensities than the neurons enhanced by well-known vessel enhancement methods. Experiments on simulated calcium imaging data indicate that both the number of detected neurons and the accuracy of the derived calcium activity are improved. Applying our method to real calcium data, more regions exhibiting calcium activity in the full field of view were found. In road pavement crack detection, smaller or milder cracks were detected after using our enhancement method.

We propose a fast and accurate solution to speckle reduction targeted specifically at optical coherence tomography images. The proposed speckle removing filter is designed using a novel potential function based on the gradient of the local variance of intensity. After filtering, the spatially neighboring pixels with close values of intensities converge to uniform gray values, while the edges remain intact. This filtering process results in removal of speckle without destroying the edges of the desired object. The proposed filter also prevents the generation of any false edges. Detailed experimental analysis shows at least 1-dB improvement in the peak signal-to-noise ratio for spectral domain optical coherence tomography images. The method also shows superior edge preservation, contrast, and speed compared to the state of the art in speckle removing filters.

In the study of neurons, morphology influences function. The complexity in the structure of neurons poses a challenge in the identification and analysis of similar and dissimilar neuronal cells. Existing methodologies carry out structural and geometrical simplifications, which substantially change the morphological statistics. Using digitally-reconstructed neurons, we extend the work of Path2Path as ElasticPath2Path, which seamlessly integrates the graph-theoretic and differential-geometric frameworks. By decomposing a neuron into a set of paths, we derive graph metrics, which are path concurrence and path hierarchy. Next, we model each path as an elastic string to compute the geodesic distance between the paths of a pair of neurons. Later, we formulate the problem of finding the distance between two neurons as a path assignment problem with a cost function combining the graph metrics and the geodesic deformation of paths. ElasticPath2Path is shown to have superior performance over the state of the art.

We propose an automatic 3D segmentation algorithm for multiphoton microscopy images of microglia. Our method is capable of segmenting tubular and blob-like structures from noisy images. Current segmentation techniques and software fail to capture the fine processes and soma of the microglia cells, useful for the study of the microglia role in the brain during healthy and diseased states. Our coupled tubularity flow field (TuFF)-blob flow field (BFF) method evolves a level set towards the object boundary using the directional tubularity and blobness measure of 3D images. Our method found a 20% performance increase against state of the art segmentation methods on a dataset of 3D images of microglia even in images with intensity heterogeneity throughout the object. The coupled TuFF-BFF segmentation results also yielded 40% improvement in accuracy for the ramification index of the processes, which displays the efficacy of our method.

Reliable cell counting and segmentation of oligodendrocyte progenitor cells (OPCs) are critical image analysis steps that could potentially unlock mysteries regarding OPC function during pathology. We propose a saliency-based method to detect OPCS and use a marker-controlled watershed algorithm to segment the OPCS This method first implements frequency-tuned saliency detection on separate channels to obtain regions of cell candidates. Final detection results and internal markers can be computed by combining information from separate saliency maps. An optimal saliency level for OPCS (OSLO) is highlighted in this work. Here, watershed segmentation is performed efficiently with effective internal markers. Experiments show that our method outperforms existing methods in terms of accuracy.

A shape filter is presented to repair segmentation results obtained in calcium imaging of neurons in vivo. This post-segmentation algorithm can automatically smooth the shapes obtained from a preliminary segmentation, while precluding the cases where two neurons are counted as one combined component. The shape filter is realized using a square-root velocity to project the shapes on a shape manifold in which distances between shapes are based on elastic changes. Two data-driven weighting methods are proposed to achieve a trade-off between shape smoothness and consistency with the data. Intuitive comparisons of proposed methods via projection onto Cartesian maps demonstrate the smoothing ability of the shape filter. Quantitative measures also prove the superiority of our methods over models that do not employ any weighting criterion.

I have been honored to lead the flagship journal of image processing, the IEEE Transactions on Image Processing for three and half years. Since taking the reins of the IEEE Transactions on Image Processing in 2014, many things have changed.

In image classification, obtaining adequate data to learn a robust classifier has often proven to be difficult in several scenarios. Classification of histological tissue images for health care analysis is a notable application in this context due to the necessity of surgery, biopsy or autopsy. To adequately exploit limited training data in classification, we propose a saliency guided dictionary learning method and subsequently an image similarity technique for histo-pathological image classification. Salient object detection from images aids in the identification of discriminative image features. We leverage the saliency values for the local image regions to learn a dictionary and respective sparse codes for an image, such that the more salient features are reconstructed with smaller error. The dictionary learned from an image gives a compact representation of the image itself and is capable of representing images with similar content, with comparable sparse codes. We employ this idea to design a similarity measure between a pair of images, where local image features of one image, are encoded with the dictionary learned from the other and vice versa. To effectively utilize the learned dictionary, we take into account the contribution of each dictionary atom in the sparse codes to generate a global image representation for image comparison. The efficacy of the proposed method was evaluated using three tissue data sets that consist of mammalian kidney, lung and spleen tissue, breast cancer, and colon cancer tissue images. From the experiments, we observe that our methods outperform the state of the art with an increase of 14.2% in the average classification accuracy over all data sets.

Short-time event detection from videos obtained using handheld or car-mounted cameras is an overarching challenge in surveillance. The problem demands simultaneous spatiotemporal localization of the event along with removal of a dynamic background. Existing state-of-the-art techniques are sensitive to non-uniform jitter, changing background, and clutter. In this paper, we propose graph Laplacian assisted parametric dictionary learning, GraDED to account for the aforementioned variations. The temporal occurrence and duration of the event is determined from weights learned using a dynamic graph, while the spatial localization is performed by graph based dictionary learning. We demonstrate the efficacy of our approach by comparing with three state-of-the-art methods and achieve on average an overall increase of 0.08 in specificity and 0.6 in sensitivity for event detection.

In this paper, we propose an automated segmentation algorithm for human leg muscles from 3D MRI data using a deep convolutional neural network (CNN). Using a generalized cylinder model, a 3D human leg muscle is represented by two smooth 2D images. The CNN predicts these two images from raw 3D voxels. For our base CNN, we use a pre-trained AlexNet that is coupled with a principle components analysis (PCA)-head. The AlexNet predicts a compressed vector, which is then back-projected by the PCA-head into two 2D images representing a 3D leg muscle boundary. This structured-output CNN architecture is fine-tuned in an end-to-end fashion. Our proposed CNN outperforms the conventional model-based approach, the active appearance model (AAM) image segmentation algorithm. The average Dice score between the ground truth segmentation and the obtained segmentation image is 0.85 using our CNN model, whereas the AAM yields a Dice score of 0.60.

Neuron microscopy enables advances in the exploration of neural anatomy and function, and neuron image enhancement is typically necessary before higher-level analyses. Most existing neuron image enhancement methods exploit the unique tubular structure of neurons. However, such methods do not preserve neuron details. In this paper, we propose a content-aware neuron image enhancement (CaNE) method that combines the neuron image properties of gradient sparsity and tubular structure. Experiments show that neuron images enhanced by CaNE possess 20% more details in terms of correlation as compared to existing methods. With high-quality neuron images provided by CaNE, neuron analyses, such as segmentation and skeleton detection, are enabled at higher rates of success as described in this paper.

In microscopy, new super-resolution methods are emerging that produce three-dimensional images at resolutions ten times smaller than that provided by traditional light microscopy. Such technology is enabling the exploration of structure and function in living tissues such as bacterial biofilms that have mysterious interconnections and organization. Unfortunately, the standard tools used in the image analysis community to perform segmentation and other higher-level analyses cannot be applied naïvely to these data. This paper presents Bact-3D, a 3D method for segmenting super-resolution images of multi-leveled, living bacteria cultured in vitro. The method incorporates a novel initialization approach that exploits the geometry of the bacterial cells as well an iterative local level set evolution that is tailored to the biological application. In experiments where segmentation is used as precursor to cell detection, the Bact-3D matches or improves upon the Dice score and mean-squared error of two existing methods, while yielding a substantial improvement in cell detection accuracy. In addition to providing improvements in performance over the state-of-the-art, this report also characterizes the tradeoff between imaging resolution and segmentation quality.

Activity recognition and activity boundary detection are two separate long-standing challenges in the image processing literature. In activity recognition, a predefined set of activities is classified using features. Often, subjects do not perform meaningful activities in all the frames, thus requiring the identification of the beginning and the end of the set of contiguous frames containing the activity. This is known as activity boundary detection problem. We seamlessly integrate both tasks by leveraging a single graph-theoretical framework: UGrAD. First, we model the data as a graph and use the spectral properties of its complementary to identify the boundaries of an activity. Next, we introduce the concept of normalized flow between consecutive frames, encoded using a bipartite graph formalism, and use it to capture the temporal evolution of the detected activity. This is accomplished by deriving a robust and reliable feature descriptor from fitting a set of Legendre polynomials to the flow matrix. Lastly, the features are used for activity classification using a linear support vector machine (SVM). Our approach shows substantial improvement over the state-of-the-art activity classification and boundary detection techniques.

In this paper we present a novel idea of evaluating similarity between two images aided by a salient object detection framework. For computing similarity between images consisting of multiple objects and varying background, extracting features relevant to the object of interest is of cruicial importance. To accomplish this task, we employ a saliency guided dictionary learning framework for image similarity evaluation (SLIDE). The saliency detection framework emphasizes the image regions that attract human attention and is exploited to build a dictionary for generating sparse representation of the images. The compressibility of the sparse codes is exploited in computing the similarity measure. The SLIDE framework is used in the application of image retrieval for three military image datasets and shows in average an improvement of 14.8% in retrieval accuracy compared to two state-of-the-art similarity measures - information distance and sparse representation based compression distance.

We present Active-Rods, a technique for automatically detecting collimator shutter edges in digital radiographs. Unlike the active contour method designed for smooth boundary contouring, the Active-Rods approach seeks straight line open boundaries. The shutter detection problem is formulated as minimizing an energy functional of both local edges and regional statistics. Initial shutter positions are obtained through edge detection, modeling and grouping for desired collinear sets. Rod translations and rotations are driven by the forces and torques via energy minimization. This technique has demonstrated promising shutter detection on a broad range of clinical images.

We present a histogram-based real-time solution to detecting directly irradiated regions in digital fluoroscopic images. Our method leverages the power of model matching, machine learning and domain knowledge to characterize and segment images using histograms. The input image is automatically identified as containing partial, all, or null direct radiation. The regions with direct radiation are segmented out via global thresholding according to image characterizations. The algorithm involves only one-dimensional processing. The test results achieved 99.82% accurate detection rate on a dataset of 9256 clinical images.

Registration of an in vivo microscopy image sequence is necessary in many significant studies, including studies of atherosclerosis in large arteries and the heart. Significant cardiac and respiratory motion of the living subject, occasional spells of focal plane changes, drift in the field of view, and long image sequences are the principal roadblocks. The first step in such a registration process is the removal of translational and rotational motion. Next, a deformable registration can be performed. The focus of our study here is to remove the translation and/or rigid body motion that we refer to here as coarse alignment. The existing techniques for coarse alignment are unable to accommodate long sequences often consisting of periods of poor quality images (as quantified by a suitable perceptual measure). Many existing methods require the user to select an anchor image to which other images are registered. We propose a novel method for coarse image sequence alignment based on minimum weighted spanning trees (MISTICA) that overcomes these difficulties. The principal idea behind MISTICA is to reorder the images in shorter sequences, to demote nonconforming or poor quality images in the registration process, and to mitigate the error propagation. The anchor image is selected automatically making MISTICA completely automated. MISTICA is computationally efficient. It has a single tuning parameter that determines graph width, which can also be eliminated by the way of additional computation. MISTICA outperforms existing alignment methods when applied to microscopy image sequences of mouse arteries.

With the growing availability and wide distribution of low-cost, high-performance 3D imaging sensors, the image analysis community has witnessed an increased demand for solutions to the challenges of activity recognition and person identification. We propose an integrated framework, based on graph signal processing, that simultaneously performs both tasks using a single set of features. The novelty of our approach is based on the fact that the set of features used for activity recognition accommodates person identification without additional computation. The analysis is based on the extracted structure-invariant graph (skeleton). The Laplacian of the skeleton is used both to identify the person and recognize the performed activity. While person identification is achieved directly from the analysis of the Laplacian, activity recognition is obtained after transformation, into the graph spectral domain, of the vectorized form of the skeletal joints 3D coordinates. Feature vectors for activity recognition are then derived, in this domain, from the covariance matrices evaluated over fixed-length sequential video segments. Both classification tasks are implemented using linear support vector machines (SVM). When applied to real activity datasets, our approach shows an improved performance over the existing state-of-the-art.

We propose a novel region based segmentation method capable of segmenting objects in presence of significant intensity variation. Current solutions use some form of local processing to tackle intra-region inhomogeneity, which makes such methods susceptible to local minima. In this letter, we present a framework which generalizes the traditional Chan-Vese algorithm. In contrast to existing local techniques, we represent the illumination of the regions of interest in a lower dimensional subspace using a set of pre-specified basis functions. This representation enables us to accommodate heterogeneous objects, even in presence of noise. We compare our results with three state of the art techniques on a dataset focusing on biological/biomedical images with tubular or filamentous structures. Quantitatively, we achieve a 44% increase in performance, which demonstrates efficacy of the method.

We propose a novel region based segmentation technique using dictionary learning. In a previous work we have developed a method which uses a set of pre-specified Legendre basis functions to perform region based segmentation of an object in presence of heterogeneous illumination. We hypothesize that in problems where a set of training images for the object is available for analysis (such as depth image sequence of blood vessels via ultrasound imaging), segmentation accuracy can be significantly improved by learning the basis functions instead of specifying them implicitly. The salient idea of this letter is to compute the optimal set of functions to model the region intensities. Our solution to this problem involves the integration of a level set segmentation methodology with the dictionary learning framework. This provides an elegant solution to deal with intensity inhomogeneities prevalent in many imaging applications such as ultrasound and fluorescence microscopy. The proposed algorithm, Dictionary Learning Level Set (DL2S) is used to segment ultrasound images of blood vessels captured using low cost, portable ultrasound devices employed in a phlebotomy application. Qualitative and quantitative results obtained from this dataset suggest efficacy of D2LS with an associated improvement in the average Dice index of 12% over the relevant competitors.

A segmentation framework is proposed to trace neurons from confocal microscopy images. With an increasing demand for high throughput neuronal image analysis, we propose an automated scheme to perform segmentation in a variational framework. Our segmentation technique, called tubularity flow field (TuFF) performs directional regional growing guided by the direction of tubularity of the neurites. We further address the problem of sporadic signal variation in confocal microscopy by designing a local attraction force field, which is able to bridge the gaps between local neurite fragments, even in the case of complete signal loss. Segmentation is performed in an integrated fashion by incorporating the directional region growing and the attraction force-based motion in a single framework using level sets. This segmentation is accomplished without manual seed point selection; it is automated. The performance of TuFF is demonstrated over a set of 2D and 3D confocal microscopy images where we report an improvement of >75% in terms of mean absolute error over three extensively used neuron segmentation algorithms. Two novel features of the variational solution, the evolution force and the attraction force, hold promise as contributions that can be employed in a number of image analysis applications.

A global solution to the enhancement and segmentation of ultrasound images is proposed that is operable in both low contrast and high contrast imaging scenarios. The solution is based on two optimization processes: one that minimizes error with respect to the original image while minimizing the number of edge contours (above the number expected by the known topology), and the second that maximizes edge fidelity. The maximization of edge fidelity is achieved by way of connected filters that operate on connected components of a threshold-decomposed image. To test the algorithm, an application in the ultrasound imaging of human blood vessels is explored. The results show significant improvements (8X to 50X) in a vesselness measure of the segmented vessels as compared to that yielded by a traditional speckle reduction technique and to a diffusion-based technique.

Registration of in vivo microscopy image sequences is important for tracking of cells. Registering a long sequence of in vivo microscopy images is particularly challenging for several reasons, which include motion artifacts created by the cardiac cycle and breathing movements of the living subject, occasional defocussing, illumination change, and noise in image acquisition. To accommodate these variations, we sample time points redundantly during microscopic image acquisition. Second, we use dynamic programming to select image frames with tolerable motion and eliminate those with large motion. Third, we employ a novel method based on the minimum spanning tree algorithm to register the selected image frames. Testing on actual in vivo image sequences reveals that our approach excels over three existing registration methods in terms of structural image similarity of the registered images.

With increasing complexity of the dataset it becomes impractical to use a single feature to characterize all constituent images. In this paper we describe a method that will automatically select the appropriate image features that are relevant and efficacious for classification, without requiring modifications to the feature extracting methods or the classification algorithm. We first describe a method for designing class distinctive dictionaries using a dictionary learning technique, which yields class specific sparse codes and a linear classifier parameter. Then, we apply information theoretic measures to obtain the more informative feature relevant to a test image and use only that feature to obtain final classification results. With at least one of the features classifying the query accurately, our algorithm chooses the correct feature in 88.9% of the trials.

In this paper we propose a novel approach to the extraction of medial axis for grayscale objects. The method utilizes a computationally efficient vector field convolution to enhance the medialness feature. Local maxima of medialness are analyzed in scale space, yielding a robust medial axis for grayscale imagery. An important application of this work is the segmentation of neurons from noisy, cluttered microscopy images. Existing neuron segmentation methods depend heavily on accurate, noise-insensitive medial axis extraction. We propose the vector field convolution medialness operation as a first step in segmenting neurons. The proposed method requires no complex parameters or an initial binarization step. The efficacy of the method is demonstrated by a 60% reduction root mean squared error (2.9 pixels) as compared to an approach based on gradient vector flow.

In order to understand the brain, we need to first understand the morphology of neurons. In the neurobiology community, there have been recent pushes to analyze both neuron connectivity and the influence of structure on function. Currently, a technical road block that stands in the way of these studies is the inability to automatically trace neuronal structure from microscopy. On the image processing side, proposed tracing algorithms face difficulties in low contrast, indistinct boundaries, clutter, and complex branching structure. To tackle these difficulties, we develop Tree2Tree, a robust automatic neuron segmentation and morphology generation algorithm. Tree2Tree uses a local medial tree generation strategy in combination with a global tree linking to build a maximum likelihood global tree. Recasting the neuron tracing problem in a graph-theoretic context enables Tree2Tree to estimate bifurcations naturally, which is currently a challenge for current neuron tracing algorithms. Tests on cluttered confocal microscopy images of Drosophila neurons give results that correspond to ground truth within a margin of ±2.75% normalized mean absolute error.

Trackability is defined here as a numerical measure associated with a video for a given target that increases with decreasing difficulty of tracking the target. The method applied to quantify trackability is grounded in information theory. First, a measure of similarity between the target signal and the template is established by way of mutual information. This mutual information becomes a three-variable analysis as the influence of clutter is considered. The effects of video quality are included in a traditional Shannon-Hartley computation and the motion of the target and registration of the video are used to modify this quality-motion term. The sum of both terms, computed in bits per second, yields trackability. Fifteen tracking experiments show a promising Spearman rank correlation between the trackability and the actual tracking performance.

In this paper we present a new method to generate the external force field for the active contour. The new approach, multi-scale vector field convolution (MVFC), is based on the vector field convolution (VFC) method, and extends VFC to a multi-scale process. Via automatic scale selection, the proposed method constructs the external force field used by the active contour by varying the convolution kernel width according to local image structure. The result is an adaptive force generation method that accommodates detailed, high curvature features with a large capture range, and demonstrates improved performance in terms of local convergence and multi-target segmentation over traditional VFC. Meanwhile, the new method preserves the low computation complexity as supported by VFC and exemplifies as a more efficient force generation method than gradient vector flow (GVF). Synthetic and real experiments demonstrate the efficacy of MVFC in terms of detail preservation, capture range, noise resilience and computational cost.

Shape manifolds provide mathematically consistent and rigorous characterization of shapes and their variability. Continuous boundary based representations offer advantages of unambiguous reproduction of shape compared to landmark based methods and obviates the need for manual selection of landmark points. An important drawback of continuous shape representations is their inability to constrain shape to meaningful deformation modes, thus making template based shape retrievals impossible. We build a class of symmetric generators that yield a systematic methodology to constrain shape deformations within a submanifold of the infinite dimensional shape manifold. Geodesic distances on this submanifold estimate dissimilarities in physically meaningful deformation modes. Induced Riemannian metrics on the submanifold can be used to calculate directional derivatives, which can in turn be used to perform analysis of energy functions in the shape space. We finally describe a real valued match metric on the Riemannian shape manifold which is minimized to obtain projections of arbitrary shapes into the constrained submanifold and can be applied to shape based object retrieval.

Images produced by ultrasound systems are adversely hampered by a stochastic process known as speckle. A despeckling method based upon removing outlier is proposed. The method is developed to contrast enhance B-mode ultrasound images. The contrast enhancement is with respect to decreasing pixel variations in homogeneous regions while maintaining or improving differences in mean values of distinct regions. A comparison of the proposed despeckling filter is compared with the other well known despeckling filters. The evaluations of despeckling performance are based upon improvements to contrast enhancement, structural similarity, and segmentation results on a Field II simulated image and actual B-mode cardiac ultrasound images captured in vivo.

Recent investigations in estimating object shape in images and leveraging knowledge of expected shapes to perform object segmentation have necessitated the formalization of a rigorous mathematical theory of shape. Most of the existing theory in nonlinear shape manifolds lacks physically meaningful parameterization of the shape components, for e.g., pose. We build a novel pose-shape manifold in which manifold parameters signify physically meaningful pose/shape deformation modes. Geodesic distances on this manifold estimate dissimilarities in pose and shape. The segmentation method initializes a template point on the pose-shape manifold and navigates the manifold to converge on the correct pose and shape of the object to be segmented. We show that this method is superior to traditional active contour methods in robustness to edges from clutter. Application of this method to cell delineation of vascular myocytes from phase-contrast microscopy gives reliable segmentation (within ±5% RMS pixel error) of cell boundaries and reliable estimates of geodesic object deformation.

The availability of easily programmable manycore CPUs and GPUs has motivated investigations into how to best exploit their tremendous computational power for scientific computing. Here we demonstrate how a systems biology application - detection and tracking of white blood cells in video microscopy - can be accelerated by 200times using a CUDA-capable GPU. Because the algorithms and implementation challenges are common to a wide range of applications, we discuss general techniques that allow programmers to make efficient use of a manycore GPU.

Cardiovascular researchers are constantly developing new and innovative medical imaging technologies, striving to improve the understanding, diagnosis, and treatment of cardiovascular dysfunction. Combining these sophisticated imaging methods with advancements in image understanding via computational intelligence will continue to advance the frontier of cardiovascular medicine. Recently, researchers have turned to a new class of tissue motion imaging techniques, including displacement encoding with stimulated echoes (DENSE) in cardiac magnetic resonance (cMR) imaging, to directly quantify cardiac displacement and produce accurate spatiotemporal measurements of myocardial strain, twist, and torsion. The associated analysis of DENSE cMR and other tissue motion imagery, however, represents a major bottleneck in the study of intramyocardial mechanics. In the computational intelligence area of deformable models, this paper develops an automated motion recovery technique termed active trajectory field models (ATFMs) geared toward these new motion imaging protocols, offering quantitative physiological measurements without the pains of manual analyses. This novel generative deformable model exploits both image information and prior knowledge of cardiac motion, utilizing a point distribution model derived from a training set of myocardial trajectory fields to automatically recover cardiac motion from a noisy image sequence. The effectiveness of the ATFM method is demonstrated by quantifying myocardial motion in 2-D short-axis murine DENSE cMR image sequences both before and after myocardial infarction, producing results comparable to existing semiautomatic analysis methods.

Competing technologies have recently emerged across the spectrum of cardiovascular imaging modalities that study intra-myocardial function, probing promising new parameters such as cardiac strain, twist, and torsion. Displacement encoding with stimulated echoes (DENSE) in cardiac magnetic resonance (cMR) imaging achieves accurate spatiotemporal measurements of tissue motion, but remains expensive and time-consuming. Speckle tracking in echocardiography uses an inexpensive and ubiquitous imaging modality, but often fails to fully capture tissue motion. This project strives to combine the desirable aspects of DENSE cMR and speckle tracking via a novel automated analysis technique termed active trajectory field models (ATFMs). The proposed ATFM analysis characterizes cardiac motion within a training set of DENSE cMR data, and uses this characterization to subsequently recover motion from a noisy and incomplete echocardiographic sequence. We demonstrate the effectiveness of the proposed technique via 2D short-axis murine image acquisitions, achieving results comparable to existing semi-automatic echocardiographic motion analysis methods.

Atherosclerosis, the precursor to acute events such as myocardial infarction and stroke, is a focal inflammatory disease that is influenced by local hemodynamic forces such as wall shear stress (WSS). Phase contrast (PC) MRI is an established method that encodes blood velocity into the phase of an image. A requisite step for the calculation of WSS from PC-MRI data is delineation of the vasculature. Previous automatic segmentation algorithms have primarily relied on the image magnitude. Phase based segmentation algorithms have reduced the phase data to image features prior to segmentation. We outline a novel method that directly incorporates phase data as an additional external force in an active model. We demonstrate improved segmentation results over current methods on both synthetic and real data.

Oil sands may constitute two thirds of the world's oil reserves. To efficiently harvest this important resource, image analysis is required to quantify production related performance in terms of particle size distribution. We utilize connected filters to simplify the oil sand images and to generate a robust segmentation. Specifically, a self-dual operator called the inclusion filter is applied to the difficult segmentation problem. The inclusion filter removes minor interior regions and clutter based on the connected component relationships defined by the adjacency forest. We show that the use of the inclusion filter significantly improves the edge fidelity and the insensitivity to initialization for the oil sand application.

For medical and epidemiologic investigators and caregivers, one powerful functionality yet to be developed is the ability to group retinal images based upon common pathologic appearance. Such a tool would enable advances in evidence-based medicine and would accelerate automated or computer-assisted screening and diagnosis. In this report, we show that current, traditional content based image retrieval methods are insufficient to sort dichotomous images (age-related macular degeneration and Stargardt disease) and then propose novel feature extraction techniques that may improve retrieval performance. Prior to processing of the images, a specialized diffusion method to enhance the contrast, reduce the discontinuity, and eliminate edge artifacts is applied to facilitate segmentation. A robust statistic is applied to find abnormal areas and to differentiate AMD from SD. Two methods of analyzing the subretinal deposits are presented - a granulometry based on area morphology and an AM-FM model. Preliminary data show that the image analysis tools show promise as a useful retrieval tool.

Active models have been widely used in image processing applications. A crucial stage that affects the ultimate active model performance is initialization. This paper proposes a novel automatic initialization approach for parametric active models in both 2-D and 3-D. The PIG initialization method exploits a novel technique that essentially estimates the external energy field from the external force field and determines the most likely initial segmentation. Examples and comparisons with two state-of-the- art automatic initialization methods are presented to illustrate the advantages of this innovation, including the ability to choose the number of active models deployed, rapid convergence, accommodation of broken edges, superior noise robustness, and segmentation accuracy.

We introduce a theoretical framework for implicit evolution of an open ended curve in a two-dimensional image plane. This approach is particularly suitable for identifying thin filamentous structures present in 2D images. The open ended curve is represented as the of the level set of a function (called the*) defined on the curve. The iterative evolution of the curvature map is guided by a diffusion equation and constrained by the imaging force, such as the image intensity gradient. The centerline of the evolved curvature map provides the position of the curve in subsequent iterations. We have tested this new model on both synthetic and real images that contain structures including rivers/roads/arteries. Nine experiments show that our model is successful in identifying complex topological filaments with a low 9% RMSE pixel error, and that the technique withstands effects of shape irregularities such as kinks, bending, circularity and inconsistent edges.

Segmentation of echocardiographic imagery is central to the understanding, diagnosis, and treatment of cardiovascular disease. Although volumes of literature have been devoted to automated image segmentation, little work has been directed towards the validation of these techniques. An echocardiographic simulation has the advantage of exact knowledge of the myocardial borders, thus providing quantifiable measurements of an algorithms performance. Existing simulation tools either become intractable when generating multiple images, or accommodate only simplistic myocardial motion models. This paper proposes a novel tool for the simulation of short-axis echocardiographic image sequences towards the goal of automated segmentation algorithm validation. We consider a complete set of simulation concerns, including a realistic myocardial model, variable inter-frame speckle pattern correlation, and low computational cost for fast simulation. We demonstrate the value of the proposed tool by evaluating a speckle filtering algorithms effects on segmentation accuracy.

Accurate segmentation and volumetric assessment of the enlarged prostate is critical to assessment of cancer progression. Moreover, 3D segmentation is necessary for treatment in both radiotherapy and brachytherapy. We propose a 3D segmentation solution for ultrasound images of the prostate based on deformable surfaces. The deformable surfaces are propelled by the vector field convolution (VFC) external force model. This external force has both computational efficiency and solution quality advantages over existing techniques such as gradient vector flow (GVF). A salient aspect of the segmentation solution proposed here is the ability to automatically initialize the deformable surface in 3D. The initialization method exploits a novel Poisson inverse gradient technique that essentially solves the inverse problem from the external force field to the external energy and determines the most likely coarse segmentation. We validate our 3D segmentation on simulated images of the prostate. Furthermore, simulated data show that PIG initialization leads to a 60% reduction in segmentation error for high curvature contours.

In this letter, a novel method of land-surface classification using surface-reflected global positioning system (GPS) signals in combination with digital imagery is presented. Two GPS-derived classification features are merged with visible image data to create terrain moisture classes, defined here as visibly identifiable terrain or landcover classes containing a surface/soil moisture component. As compared to using surface imagery alone, classification accuracy is significantly improved for a number of visible classes when adding GPS-based signal features. Since the strength of the reflected GPS signal is proportional to the amount of moisture in the surface, the use of these GPS features provides information about the surface that is not obtainable using visible wavelengths alone. Application areas include hydrology, precision agriculture, and wetlands mapping

The bilateral filter represents a wide group of nonlinear filters for edge-preserving image smoothing. In this work, we study the convergence properties of the bilateral filter algorithm. The understanding is established that the bilateral filter is an optimization procedure. We demonstrate that the bilateral filter is equivalent to minimizing a robust cost criterion using iterative reweighting, which is a good approximation to the very fast but unstable Newton's method. Further, the results of the analysis allow us to derive an improved hybrid smoothing scheme with concerns of computational efficiency and edge preservation.

Snakes, or active contours, have been widely used in image processing applications. Typical roadblocks to consistent performance include limited capture range, noise sensitivity, and poor convergence to concavities. This paper proposes a new external force for active contours, called vector field convolution (VFC), to address these problems. VFC is calculated by convolving the edge map generated from the image with the user-defined vector field kernel. We propose two structures for the magnitude function of the vector field kernel, and we provide an analytical method to estimate the parameter of the magnitude function. Mixed VFC is introduced to alleviate the possible leakage problem caused by choosing inappropriate parameters. We also demonstrate that the standard external force and the gradient vector flow (GVF) external force are special cases of VFC in certain scenarios. Examples and comparisons with GVF are presented in this paper to show the advantages of this innovation, including superior noise robustness, reduced computational cost, and the flexibility of tailoring the force field.

Cardiac parameters such as end-systolic volume, ejection fraction and myocardial mass are essential to the diagnosis and treatment of cardiovascular disease (CVD). Traditionally, these parameters are calculated based on manual myocardial segmentation by a trained technician. Fast, accurate, and automatic segmentation would provide researchers with an increased subject pool, an enhanced understanding of CVD, and may lead to the development of new therapies. In this paper we propose an automated algorithm for myocardial segmentation. This method utilizes speckle reducing anisotropic diffusion to assist the automated contour initialization. Speckle tracking segmentation (STS) is then applied throughout the cardiac cycle to track the myocardial borders. This approach, compared to standard active contour techniques, reduces the RMSE to ground truth by an order of magnitude.

3D ultrasound provides clinicians and researchers with more intuitive anatomical visualization as well as accurate volumetric measurements. A novel freehand transducer constructed by Hossack et al. allows for 3D reconstruction of cross-sectional slices through the use of additional tracking arrays. These tracking arrays produce a coplanar image sequence, perpendicular to the cross-sectional slices, from which frame-to-frame motion can be determined. Accurate and precise motion estimation is required for successful volume reconstruction. In this paper we propose a novel volume reconstruction technique for the freehand 3D ultrasound transducer utilizing sub-pixel phase correlation. Using a synthetic data set with known volumetric characteristics, we compare our algorithm to standard correlation techniques.

Snakes, or active contours, have been widely used in image processing applications. Typical roadblocks to consistent performance include limited capture range, noise sensitivity, and poor convergence to concavities. This paper proposes a new design for the snake external force, called vector field convolution (VFC), to address these problems. Qualitative and quantitative comparisons with the gradient vector flow (GVF) external force are presented in this paper to show the advantages of this innovation.

A novel stochastically driven filtering method to despeckle B mode ultrasound images is presented. This method is motivated by viewing the pixel values as a stochastic process and removing outliers, where outliers are defined by local extrema. These outliers are removed by local averaging. This produces another image with new outliers (local extrema) and the process is iterated. With each iteration homogeneous regions become smoother while edges that defined these regions are preserved. By allowing a dynamically varying window to determine the local mean, we achieve equivalent results with fewer iterations.

Processing of bulk microscopy video data requires automated tracking of rolling leukocytes in the hundreds to compute a rolling velocity distribution, which is an indispensable descriptor in inflammation research and anti/pro-inflammatory drug testing. However, for any automated tracking method to be successful, an automated validation process must exist to accept or reject the output of tracking. In this paper, we propose an automated validation technique that first generates a spatiotemporal image from the cell locations output by a tracking method; then, it segments the spatiotemporal image to detect the presence or absence of a leukocyte by employing an edge-response filter followed by an active contour method. The proposed direction sensitive edge-response filter, the maximum absolute average directional derivative (MAADD), computes the magnitude of the mean directional derivative over an oriented line segment and chooses the maximum of all such values within a range of orientations of the line segment. Our validation experiments show that the proposed method is successful in 93% of the trials using manual tracking, in 83% using correlation tracking and in 84% using active contour tracking method.

In order to propel the analysis of medical ultrasound imagery from qualitative observation to quantitative measurement, the obstacles of distortion from speckle and from blurring due to the point spread function must be overcome. A recent partial differential equation (PDE) based enhancement technique has improved the ability to segment ultrasound images and to detect salient edges. However, this diffusion method often distorts the size of image features and may in fact efface subtle features. This paper proposes a new PDE that combines the enhancement of speckle reducing anisotropic diffusion (SRAD) with the mechanism of deconvolution. The resulting method, called deconvolutional speckle reducing anisotropic diffusion (DeSpeRADo), surpasses the edge localization ability of SRAD while yielding lower error in terms of area estimation and improved detection of fine features. A comparative study employs 100 experiments to contrast the quantification enabled by adaptive filtering, inverse filtering, diffusion and the new DeSpeRADo technique.

We propose a multi-target tracking (MTT) algorithm in a sequential Bayesian framework that computes cell velocities from video microscopy. Unlike the traditional tracking methods, our formulation does not involve the estimation of target states; instead, we estimate one-to-one target correspondences by way of a sequential Markov chain Monte Carlo (MCMC) algorithm. The proposed probabilistic framework also automatically accounts for a variable number of targets. We have tested the proposed tracking algorithm on two different in vitro and one in vivo microscopy experiments. The three experiments show that the method holds promise in terms of low false positive and false negative rates as well as low rates of correspondence error.

In this paper, we define a connected operator that either fills or retains the holes of the connected sets depending on application-specific criteria that are increasing in the set theoretic sense. We refer to this class of connected operators as inclusion filters, which are shown to be increasing, idempotent, and self dual (gray-level inversion invariance). We demonstrate self duality for 8-adjacency on a discrete Cartesian grid. Inclusion filters are defined first for binary-valued images, and then the definition is extended to grayscale imagery. It is also shown that inclusion filters are levelings, a larger class of connected operators. Several important applications of inclusion filters are demonstrated-automatic segmentation of the lung cavities from magnetic resonance imagery, user interactive shape delineation in content-based image retrieval, registration of intravital microscopic video sequences, and detection and tracking of cells from these sequences. The numerical performance measures on 100-cell tracking experiments show that the use of inclusion filter improves the total number of frames successfully tracked by five times and provides a threefold reduction in the overall position error.

The problem of identifying and counting rolling leukocytes within intravital microscopy is of both theoretical and practical interest. Currently, methods exist for tracking rolling leukocytes in vivo, but these methods rely on manual detection of the cells. In this paper we propose a technique for accurately detecting rolling leukocytes based on Bayesian classification. The classification depends on a feature score, the gradient inverse coefficient of variation (GICOV), which serves to discriminate rolling leukocytes from a cluttered environment. The leukocyte detection process consists of three sequential steps: the first step utilizes an ellipse matching algorithm to coarsely identify the leukocytes by finding the ellipses with a locally maximal GICOV. In the second step, starting from each of the ellipses found in the first step, a B-spline snake is evolved to refine the leukocytes boundaries by maximizing the associated GICOV score. The third and final step retains only the extracted contours that have a GICOV score above the analytically determined threshold. Experimental results using 327 rolling leukocytes were compared to those of human experts and currently used methods. The proposed GICOV method achieves 78.6% leukocyte detection accuracy with 13.1% false alarm rate.

In this paper, we propose an automatic approach for detecting particle tracers (microspheres) in microscopic imagery obtained from mouse cremaster venules in vivo. Measurements of the translational speed and radial position of individual microspheres provide the input data needed to extract velocity profiles from steady blood flow in venules. These profiles provide information about local hemodynamics that is critical to a broad range of fields in microvascular physiology, including endothelial-cell mechanotransduction, inflammation, and microvascular resistance. In the preprocessing stage, an active contour method based on dynamic programming is used for vessel region extraction. Each microsphere is then identified using a process of coarse segmentation followed by verification. Segmentation is achieved using a morphological method for microsphere detection while verification is achieved using an analytical model tailored to the microsphere. Experimental results are obtained using the proposed scheme and compared with previously published manually acquired data.

We first derive rigorously a partial differential equation (PDE) for speckle reduction from minimizing a cost functional of the instantaneous coefficient of variation. Then, we show that the piecewise exponential function is the solution of the derived PDE. Next, we express the derived PDE using log-compressed ultrasound data, followed by a numerical implementation scheme. Finally, we demonstrate the performance of the proposed PDE using examples and compare the results with those obtained from the speckle reducing anisotropic diffusion (SRAD) algorithm.

We present a novel approach to constraining the evolution of active contours used in image analysis. The proposed approach constrains the final curve obtained at convergence of curve evolution to be related to the initial curve from which evolution begins through an element of a desired Lie group of plane transformations. Constraining curve evolution in such a way is important in numerous tracking applications where the contour being tracked in a certain frame is known to be related to the contour in the previous frame through a geometric transformation such as translation, rotation, or affine transformation, for example. It is also of importance in segmentation applications where the region to be segmented is known up to a geometric transformation. Our approach is based on suitably modifying the Euler-Lagrange descent equations by using the correspondence between Lie groups of plane actions and their Lie algebras of infinitesimal generators, and thereby ensures that curve evolution takes place on an orbit of the chosen transformation group while remaining a descent equation of the original functional. The main advantage of our approach is that it does not necessitate any knowledge of nor any modification to the original curve functional and is extremely straightforward to implement. Our approach therefore stands in sharp contrast to other approaches where the curve functional is modified by the addition of geometric penalty terms. We illustrate our algorithm on numerous real and synthetic examples.

We propose a cell detection and tracking solution using image-level sets computed via threshold decomposition. In contrast to existing methods where manual initialization is required to track individual cells, the proposed approach can automatically identify and track multiple cells by exploiting the shape and intensity characteristics of the cells. The capture of the cell boundary is considered as an evolution of a closed curve that maximizes image gradient along the curve enclosing a homogeneous region. An energy functional dependent upon the gradient magnitude along the cell boundary, the region homogeneity within the cell boundary and the spatial overlap of the detected cells is minimized using a variational approach. For tracking between frames, this energy functional is modified considering the spatial and shape consistency of a cell as it moves in the video sequence. The integrated energy functional complements shape-based segmentation with a spatial consistency based tracking technique. We demonstrate that an acceptable, expedient solution of the energy functional is possible through a search of the image-level lines: boundaries of connected components within the level sets obtained by threshold decomposition. The level set analysis can also capture multiple cells in a single frame rather than iteratively computing a single active contour for each individual cell. Results of cell detection using the energy functional approach and the level set approach are presented along with the associated processing time. Results of successful tracking of rolling leukocytes from a number of digital video sequences are reported and compared with the results from a correlation tracking scheme.

The instantaneous coefficient of variation (ICOV) edge detector, based on normalized gradient and Laplacian operators, has been proposed for edge detection in ultrasound images. In this paper, the edge detection and localization performance of the ICOV-squared (ICOVS) detector are examined. First, a simplified version of the ICOVS detector, the normalized gradient magnitude squared, is scrutinized in order to reveal the statistical performance of edge detection and localization in speckled ultrasound imagery. Both the probability of detection and the probability of false alarm are evaluated for the detector. Edge localization is characterized by the position of the peak and the 3-dB width of the detector response. Then, the speckle-edge response of the ICOVS as applied to a realistic edge model is studied. Through theoretical analysis, we reveal the compensatory effects of the normalized Laplacian operator in the ICOV edge detector for edge-localization error. An ICOV-based edge-detection algorithm is implemented in which the ICOV detector is embedded in a diffusion coefficient in an anisotropic diffusion process. Experiments with real ultrasound images have shown that the proposed algorithm is effective in extracting edges in the presence of speckle. Quantitatively, the ICOVS provides a lower localization error, and qualitatively, a dramatic improvement in edge-detection performance over an existing edge-detection method for speckled imagery.

Recording rolling leukocyte velocities from intravital microscopic video imagery is a critical task in inflammation research and drug validation. Since manual tracking is excessively time consuming, an automated method is desired. This paper illustrates an active contour based automated tracking method, where we propose a novel external force to guide the active contour that takes the hemodynamic flow direction into account. The construction of the proposed force field, referred to as motion gradient vector flow (MGVF), is accomplished by minimizing an energy functional involving the motion direction, and the image gradient magnitude. The tracking experiments demonstrate that MGVF can be used to track both slow- and fast-rolling leukocytes, thus extending the capture range of previously designed cell tracking techniques.

Most of the existing watermarking processes become vulnerable when the attacker knows the watermark insertion algorithm. This paper presents an invisible spatial domain watermark insertion algorithm for which we show that the watermark can be recovered, even if the attacker tries to manipulate the watermark with the knowledge of the watermarking process. The process incorporates buyer specific watermarks within a single multimedia object, and the same multimedia object has different watermarks that differ from owner to owner. Therefore recovery of this watermark not only authenticates the particular owner of the multimedia object but also could be used to identify the buyer involved in the forging process. This is achieved after spatially dividing the multimedia signal randomly into a set of disjoint subsets (referred to as the image key) and then manipulating the intensity of these subsets differently depending on a buyer specific key. These buyer specific keys are generated using a secret permutation of error correcting codes so that exact keys are not known even with the knowledge of the error correcting scheme. During recovery process a manipulated buyer key (due to attack) is extracted from the knowledge of the image key. The recovered buyer key is matched with the exact buyer key in the database utilizing the principles of error correction. The survival of the watermark is demonstrated for a wide range of transformations and forging attempts on multimedia objects both in spatial and frequency domains. We have shown that quantitatively our watermarking survives rewatermarking attack using the knowledge of the watermarking process more efficiently compared to a spread spectrum based technique. The efficacy of the process increases in scenarios in which there exist fewer numbers of buyer keys for a specific multimedia object. We have also shown that a minor variation of the watermark insertion process can survive a "Stirmark" attack. By making the image key and the intensity manipulation proms specific for a buyer and with proper selection of error correcting codes, certain categories of collusion attacks can also be precluded.

In this paper, we propose a variational method for the detection of leukocytes observed in vivo. An adaptive threshold surface is constructed automatically using boundary information from the image. The surface is created using an objective functional that is minimized via a variational approach. This surface is constrained by an edge field that is also computed with a variational method. Objects extracted from background are pruned according to two geometric criteria. In the experiments, we find the false positive rate of the detector and show that the proposed approach can automatically and accurately identify multiple rolling leukocytes in vivo.

Using the structure of an adjacency-tree for binary-valued images, we define inclusion filters, a class of connected operators. Inclusion filters modify the image by filling or retaining the holes of the connected components of foreground and those of the background of a binary image depending on application-specific criteria, which are increasing in the set theoretic sense. We demonstrate a straightforward method to achieve self-duality (gray level inversion invariance) of inclusion filters on the discrete Cartesian domain by considering only 8-adjacency. Inclusion filters are extended to the grayscale images by the threshold decomposition principle. As an application, inclusion filters are shown to improve the performance of snake-based tracking of leukocytes observed in intravital microscopic video imagery. In this set of experiments the mean position error is reduced by a factor of 2.5 using the inclusion filter.

The detection of region boundaries in low SNR images using region based active contours is investigated. In this paper, we first show that the region based active contours models based on the region mean-difference binary flows suffer from two drawbacks when being applied to the detection of low SNR objects: (1) the contours give biased boundary locations if the image is smoothed with a linear filter; (2) the contours detect numerous false boundaries if the image is not prefiltered. Toward this end, we present a model that quantifies the contour location bias. Then, we derive a new region based model to detect the boundaries of objects in low SNR imagery, based on techniques of curve evolution, area-weighted mean-difference binary flows, and level sets. Finally, we present various experimental results to illustrate that the area-weighted technique can overcome the drawbacks of the existing method.

An image enhancement algorithm for images compressed using the JPEG standard is presented. The algorithm is based on a contrast measure defined within the discrete cosine transform (DCT) domain. The advantages of the psychophysically motivated algorithm are 1) the algorithm does not affect the compressibility of the original image because it enhances the images in the decompression stage and 2) the approach is characterized by low computational complexity. The proposed algorithm is applicable to any DCT-based image compression standard, such as JPEG, MPEG 2, and H. 261.

Inhaled hyperpolarized helium-3 (3He) gas is a new magnetic resonance (MR) contrast agent that is being used to study lung functionality. To evaluate the total lung ventilation from the hyperpolarized 3He MR images, it is necessary to segment the lung cavities. This is difficult to accomplish using only the hyperpolarized 3He MR images, so traditional proton (1H) MR images are frequently obtained concurrent with the hyperpolarized 3He MR examination. Segmentation of the lung cavities from traditional proton (1H) MRI is a necessary first step in the analysis of hyperpolarized 3He MR images. In this paper, we develop an active contour model that provides a smooth boundary and accurately captures the high curvature features of the lung cavities from the 1H MR images. This segmentation method is the first parametric active contour model that facilitates straightforward merging of multiple contours. The proposed method of merging computes an external force field that is based on the solution of partial differential equations with boundary condition defined by the initial positions of the evolving contours. A theoretical connection with fluid flow in porous media and the proposed force field is established. Then by using the properties of fluid flow we prove that the proposed method indeed achieves merging and the contours stop at the object boundary as well. Experimental results involving merging in synthetic images are provided. The segmentation technique has been employed in lung 1H MR imaging for segmenting the total lung air space. This technology plays a key role in computing the functional air space from MR images that use hyperpolarized 3He gas as a contrast agent.

We propose an image segmentation model using an agglomerative clustering technique. The clustering is performed within a feature matrix where intensity and boundary relations are defined between neighboring segments. The iterative segment agglomeration process satisfies a cluster aggregation property and does not utilize a priori knowledge of the number of clusters present in the image. The performance of the algorithm is demonstrated on a number of color images and compared with a similar algorithm.

We propose and demonstrate an active contour technique to track fast-rolling leukocytes observed in vivo from video microscopy. A rolling leukocyte is an activated white blood cell that interacts with the vessel wall (the endothelium) in the inflammatory process. Tracking is enhanced here to accommodate fast-moving cells. To tackle the task of tracking wherein only low temporal resolution is possible, we have introduced an energy-minimizing framework and obtained a partial differential equation (PDE) based active contour evolution technique. The proposed PDEs are shown to be an initialization-insensitive version of the gradient vector flow (GVF) proposed by Xu and Prince (1998). We modify the GVF-PDEs by adding a Dirichlet type boundary condition (BC) based on the initial position of the active contour and the direction of cell movement. Using actual intravital experiments, we compare the performance of the proposed active contour tracker with the Dirichlet BC, the active contour tracker without the BC, the correlation tracker and the centroid tracker. The comparative results provide evidence of the advantages of the proposed method in terms of increased number of frames successfully tracked and reduced localization error.

The problem of cloud tracking within a sequence of geo-stationary satellite images has direct relevance to the analysis of cloud life cycles and to the detection of cloud motion vectors (CMVs). The proposed approach first identifies a homogeneous consistent cloud mass for tracking and then establishes motion correspondence within an image sequence. In contrast to the crosscorrelation based approach as adopted in automatic CMV detection analysis, a scale space classifier is designed to detect cloud mass in the source image taken at time t and the destination image at time t+/spl delta/t. Boundaries of the extracted cloud segments are matched by computing a correspondence between high curvature points. This shape based method is capable of tracking in the cases of rotation, scaling, and shearing, while the correlation technique is limited to translational motion. The final tracking results provide motion magnitude and direction for each contour point, allowing reliable estimation of meteorological events and wind velocities aloft. With comparable computational expense, the scale space classification technique exceeds the performance of the traditional correlation-based approach in terms of reduced localization error and false matches.

This paper provides the derivation of speckle reducing anisotropic diffusion (SRAD), a diffusion method tailored to ultrasonic and radar imaging applications. SRAD is the edge-sensitive diffusion for speckled images, in the same way that conventional anisotropic diffusion is the edge-sensitive diffusion for images corrupted with additive noise. We first show that the Lee and Frost filters can be cast as partial differential equations, and then we derive SRAD by allowing edge-sensitive anisotropic diffusion within this context. Just as the Lee and Frost filters utilize the coefficient of variation in adaptive filtering, SRAD exploits the instantaneous coefficient of variation, which is shown to be a function of the local gradient magnitude and Laplacian operators. We validate the new algorithm using both synthetic and real linear scan ultrasonic imagery of the carotid artery. We also demonstrate the algorithm performance with real SAR data. The performance measures obtained by means of computer simulation of carotid artery images are compared with three existing speckle reduction schemes. In the presence of speckle noise, speckle reducing anisotropic diffusion excels over the traditional speckle removal filters and over the conventional anisotropic diffusion method in terms of mean preservation, variance reduction, and edge localization.

Inflammatory disease is initiated by leukocytes (white blood cells) rolling along the inner surface lining of small blood vessels called postcapillary venules. Studying the number and velocity of rolling leukocytes is essential to understanding and successfully treating inflammatory diseases. Potential inhibitors of leukocyte recruitment can be screened by leukocyte rolling assays and successful inhibitors validated by intravital microscopy. In this paper, we present an active contour or snake-based technique to automatically track the movement of the leukocytes. The novelty of the proposed method lies in the energy functional that constrains the shape and size of the active contour. This paper introduces a significant enhancement over existing gradient-based snakes in the form of a modified gradient vector flow. Using the gradient vector flow, we can track leukocytes rolling at high speeds that are not amenable to tracking with the existing edge-based techniques. We also propose a new energy-based implicit sampling method of the points on the active contour that replaces the computationally expensive explicit method. To enhance the performance of this shape and size constrained snake model, we have coupled it with Kalman filter so that during coasting (when the leukocytes are completely occluded or obscured), the tracker may infer the location of the center of the leukocyte. Finally, we have compared the performance of the proposed snake tracker with that of the correlation and centroid-based trackers. The proposed snake tracker results in superior performance measures, such as reduced error in locating the leukocyte under tracking and improvements in the percentage of frames successfully tracked. For screening and drug validation, the tracker shows promise as an automated data collection tool.

The study of inflammatory disease hinges upon the behavior and movement of leukocytes and their interaction with the endothelium. We put forth a method for tracking leukocytes in vivo, whereas tracking has been demonstrated previously only for in vitro experiments. The tracker is based on the enhancing capability of morphological anisotropic diffusion, a partial differential equation model for adaptively filtering imagery that retains structures of interest. Morphological anisotropic diffusion excels over standard diffusion in the ability to preserve objects of a certain shape and scale, and it improves upon standard morphological filters in terms of edge preservation and adaptive smoothing. We use the video frames enhanced by morphological diffusion for edge-based registration and background removal in the tracking process.

A novel technique that combines MR imaging of hyperpolarized helium gas and conventional MR imaging facilitates, high resolution imaging of lung functionality for the first time. We put forth a segmentation method for measuring the total lung air space and a classification approach to computing the functional air space. For segmentation, we introduce a parametric active contour that allows automated merging of multiple contours. The active contour technique uses gradient vector flow modified and strengthened by a boundary condition that inhibits contour crossover. The active contour approach is computationally inexpensive and is independent of initial contour placement. For classification of the functional lung air space in the helium images, a fuzzy c-means technique is applied. The classification results, in conjunction with the segmentation, allow the analysis of ventilation. The resultant biomedical image analysis tool can used in determining the efficacy of certain respiratory treatments.

For the first time, we explore the application of active contours in the modulation domain by computing snakes on image modulations. As we demonstrate in the examples, such snakes are able to utilize information inherent in the dominant image modulations to acquire and track visually and semantically meaningful structures within the image. We use nonlinear AM-FM image representations to capture regions that are homogeneous in intensity and in texture. A geometric snake approach utilizing a fuzzy classifier is then applied to the image modulations. The combination of AM-FM analysis and the active contour evolution produces an efficacious image partition. As a preliminary demonstration of this novel approach, we apply the modulation domain snakes to the classical texture segmentation problem.

This paper presents a robust technique for automatically detecting radioactive seeds that are used for prostate cancer therapy. The main innovation of the detection technique is the utilization of distributed constant false alarm rate (CFAR) processors and orientation-sensitive morphological filtering to locate the seeds in the ultrasound imagery. CFAR detection is utilized to detect the seed candidates with high signal-to-clutter ratios (SCR). The CFAR problem is posed as the detection of a fluctuating target against K-distributed clutter. Adaptive template matching is used to detect the weak seed signals and to discriminate the seed-like clutter from the seed candidates. To reduce the speckle noise, the adaptive template matching uses orientation-sensitive morphological filters. The complete detection algorithm has been tested using a set of phantom ultrasound images containing radioactive seeds. In the process of implantation of seeds for radiotherapy, the detection method can be used to evaluate seed placement before 3D reconstruction is accomplished.

We provide an automated method to repair broken, occluded oriented image textures. Our approach is based on partial differential equations (PDEs) and AM-FM image modeling. Reconstruction of the texture occurs via simultaneous PDE-generated diffusion and reaction. In the diffusion process, the image is adaptively smoothed, preserving important boundaries and features. The reaction process produces the reconstructed textural information in the occluded image regions. Gabor (1946) filters are designed and used in the reaction process using an AM-FM dominant component analysis. An AM-FM model of the texture image is constructed, making it possible to localize the reaction filters spatio-spectrally. In contrast to previous disocclusion techniques that depend on interpolation, on continuity of the connected components within the image level sets, or on texture estimation, the reaction-diffusion process proposed here yields a seamless transition between the recreated region and the unoccluded image regions. Using AM-FM dominant component analysis, we avoid the ad hoc parameter selection typified with other reaction-diffusion approaches. As a useful example, we focus on the repair of broken, occluded fingerprints. We also treat several exemplary natural textures to demonstrate the technique's generality.

This paper introduces an edge detection process based on area morphology. Area open-close and area close-open operators are used to generate scaled image representations for feature extraction. The edges are defined by the boundaries of the scaled objects in the area-filtered images. From the area open-close and close-open operators, thin closed contours suitable for image segmentation are produced. The edge maps allow exact specification of the minimum area for the extracted regions and are Euclidean invariant and causal through scale space. Results are given that demonstrate the effectiveness of the area operator-based edge detection. In contrast to traditional edge detectors, edge detection via area morphology provides well-localized boundaries and does not require thresholding.

A fast algorithm that approximates area morphological filters is presented. Area morphological filters enable the elimination of connected components within image level sets according to area alone. To date, area morphological filters have been limited in application due to the high computational expense involved with connected component analysis. The pyramidal implementation described here dramatically reduces the computational cost by simultaneously creating a marker image and performing reconstruction in a coarse-to-fine manner. The pyramidal algorithm reduces the computational cost by three orders of magnitude for standard-sized grayscale images. Image examples and an analysis of the computational expense are provided in the paper.

Content based retrieval (CBR) has emerged as a useful tool for management of digital image libraries. Traditional approaches based on metadata and global features are limited in effectiveness and suffer from various drawbacks involving the use of heuristics and postprocessing techniques. The automated segmentation technique presented is based on fuzzy clustering in a scaled texture space, which is generated via a bank of Gabor filters. Within the texture space, we scale the responses using the novel tool of area morphology. The segmentation algorithm provides segments satisfying a minimum scale requirement for use in CBR. The experiments indicate that our segmentation mechanism increases feature relevancy and reduces the computational costs of segment based matching.

This paper presents a new technique for partitioning polarimetric synthetic aperture radar (POL-SAR) imagery into regions of homogeneous polarimetric backscattering properties. The method consists of two cascaded steps: an initial texture segmentation of one image derived from the POL-SAR data, and a polarimetric statistical region merging process that refines and modifies the initial segmentation. In the first stage, a morphological region-based image partitioning technique, the watershed algorithm, plays the key role. While in the second stage, the region adjacency graph and the segment dissimilarity measures derived from the Wishart model and the K distribution are applied. The overall segmentation algorithm has been tested using real 4-look POL-SAR imagery and sample results are provided in the paper. The main innovation of this work is the utilization of the K distribution in the segmentation process.

We explore the application of area morphology to image classification. From the input image, a scale space is created by successive application of an area morphology operator. The pixels within the scale space corresponding to the same image location form a scale space vector. A scale space vector therefore contains the intensity of a particular pixel for a given set of scales, determined in this approach by image granulometry. Using the standard k-means algorithm or the fuzzy c-means algorithm, the image pixels can be classified by clustering the associated scale space vectors. The scale space classifier presented here is rooted in the novel area open-close and area close-open scale spaces. Unlike other scale generating filters, the area operators affect the image by removing connected components within the image level sets that do not satisfy the minimum area criterion. To show that the area open-close and area close-open scale spaces provide an effective multiscale structure for image classification, we demonstrate the fidelity, causality, and edge localization properties for the scale spaces. The analysis also reveals that the area open-close and area close-open scale spaces improve classification by clustering members of similar objects more effectively than the fixed scale classifier. Experimental results are provided that demonstrate the reduction in intra-region classification error and in overall classification error given by the scale space classifier for classification applications where object scale is important. In both visual and objective comparisons, the scale space approach outperforms the traditional fixed scale clustering algorithms and the parametric Bayesian classifier for classification tasks that depend on object scale.

We introduce binary locally monotonic regression as a first step in the study of the application of local monotonicity for image estimation. Given an algorithm that generates a similar locally monotonic image from a given image, we can specify both the scale of the image features retained and the image smoothness. In contrast to the median filter and to morphological filters, a locally monotonic regression produces the optimally similar locally monotonic image. Locally monotonic regression is a computationally expensive technique, and the restriction to binary-range signals allows the use of Viterbi-type algorithms. Binary locally monotonic regression is a powerful tool that can be used in the solution of image estimation, image enhancement, and image segmentation problems.

Definitions of locally monotonic images are introduced. The model definitions are complemented with algorithms that compute locally monotonic versions of a given image or video frame input. The property of local monotonicity provides a useful vehicle for image smoothing and denoising. Local monotonicity is also useful for scale space generation, wherein the degree of local monotonicity is the scale parameter. Currently, the property of local monotonicity is well defined for the 1-D case, but is not well defined for images or video. In this paper, models for multidimensional local monotonicity that extend the 1-D definition are rendered. Regression-based and diffusion-based processing methods are prescribed that yield meaningful locally monotonic images. The definitions and associated algorithms are applicable to image enhancement and a variety of multiscale tasks such as image segmentation and video coding.

Locally monotonic (lomo) images are defined as root signals of a morphological lomo filter. The morphological approach allows a multidimensional generalization of local monotonicity. This generalization is well motivated in that it retains the essential properties of one-dimensional (1D) local monotonicity. Repeated application of the lomo filter produces a lomo root signal of a specified scale. By filtering at multiple scales, a locally monotonic scale-space can be created and used in multiscale image applications such as segmentation, tracking, content based retrieval, and image coding. In contrast to existing linear and nonlinear scale-generating filters, the lomo filter has no spatial or graylevel bias and preserves edge localization through scale-space.

The paper details a multiscale clustering technique for document page segmentation. In contrast to existing hierarchical (coarse-to-fine), multi-resolution methods, this image segmentation technique simultaneously uses information from different scaled representations of the original image. The final clustering of image segments is achieved through a fuzzy c-means based similarity measure between vectors in scale space. The segmentation process reduces the effects of insignificant detail and noise. Furthermore, object integrity is preserved in the segmentation process.

In this paper we apply content based retrieval (CBR) to the automated inspection of printed circuit boards. Manual probing of faulty circuit boards is expensive and time consuming for electronics manufacturers. The proposed CBR system allows groups of similarly faulted boards to be identified and repaired simultaneously. We improve upon current matching techniques used in CBR, exploiting relative pairings of features rather than simple distance measures. The novel matching techniques are seen to improve upon traditional approach in cases of varying background conditions, environmental factors and inter-chip interactions.

We describe two broad classes of useful and physically meaningful image models that can be used to construct novel smoothing constraints for use in the regularized image restoration problem. The two classes, termed piecewise image models (PIMs) and focal image models (LIMs), respectively, capture unique image properties that can be adapted to the image and that reflect structurally significant surface characteristics. Members of the PIM and LIM classes are easily formed into regularization operators that replace differential-type constraints. We also develop an adaptive strategy for selecting the best PIM or LIM for a given problem (from among the defined class), and we explain the construction of the corresponding regularization operators. Considerable attention is also given to determining the regularization parameter via a cross-validation technique, and also to the selection of an optimization strategy for solving the problem. Several results are provided that illustrate the processes of model selection, parameter selection, and image restoration. The overall approach provides a new viewpoint on the restoration problem through the use of new image models that capture salient image features that are not well represented through traditional approaches.

This paper investigates the relationship between anisotropic diffusion and local monotonicity. A diffusion technique that has locally monotonic root signals is presented. The enhancement algorithm rapidly converges to a locally monotonic signal of the desired degree. It is shown that the diffusion coefficient used here is the only formation that guarantees idempotence for locally monotonic signals. The signals resulting from locally monotonic diffusion are closer to the original signals than the corresponding median root signals. Furthermore, the diffusion algorithm does not have a difficulty with alternating signals, as does the median filter. In contrast to other anisotropic diffusion techniques, the diffusion method given here does not preserve outliers and does not require a gradient magnitude threshold in the diffusion coefficient.

A new multi-scale image enhancement mechanism is presented. Derived from the differential representation of the morphological filters, it approximates a median filter and alleviates many of the image blotching and noise preserving characteristics of morphological filtering. In one-dimension, the process is shown to be idempotent and to converge. In two dimensions, experimental results demonstrate convergence and display the ability to remove impulsive noise. Gradient independent translation avoids the two dimensional convergence problems of the median filter and does not involve the expensive rank-ordering of pixel intensities. Describing a scale space with median filter characteristics, it provides a multi-scale analysis method suitable for compression, coding, and feature extraction.

A multigrid anisotropic diffusion algorithm for image processing is presented. The multigrid implementation provides an efficient hierarchical relaxation method that facilitates the application of anisotropic diffusion to time-critical processes. Through a multigrid V-cycle, the anisotropic diffusion equations are successively transferred to coarser grids and used in a coarse-to-fine error correction scheme. When a coarse grid with a trivial solution is reached, the coarse grid estimates of the residual error can be propagated to the original grid and used to refine the solution. The main benefits of the multigrid approach are rapid intraregion smoothing and reduction of artifacts due to the elimination of low-frequency error. The theory of multigrid anisotropic diffusion is developed. Then, the intergrid transfer functions, relaxation techniques, diffusion coefficients, and boundary conditions are discussed. The analysis includes the examination of the storage requirements, the computational cost, and the solution quality. Finally, experimental results are reported that demonstrate the effectiveness of the multigrid approach.

We introduce a new approach to image estimation based on a flexible constraint framework that encapsulates meaningful structural image assumptions. Piecewise image models (PIMs) and local image models (LIMs) are defined and utilized to estimate noise-corrupted images, PIMs and LIMs are defined by image sets obeying certain piecewise or local image properties, such as piecewise linearity, or local monotonicity. By optimizing local image characteristics imposed by the models, image estimates are produced with respect to the characteristic sets defined by the models. Thus, we propose a new general formulation for nonlinear set-theoretic image estimation. Detailed image estimation algorithms and examples are given using two PIMs: piecewise constant (PICO) and piecewise linear (PILI) models, and two LIMs: locally monotonic (LOMO) and locally convex/concave (LOCO) models. These models define properties that hold over local image neighborhoods, and the corresponding image estimates may be inexpensively computed by iterative optimization algorithms. Forcing the model constraints to hold at every image coordinate of the solution defines a nonlinear regression problem that is generally nonconvex and combinatorial. However, approximate solutions may be computed in reasonable time using the novel generalized deterministic annealing (GDA) optimization technique, which is particularly well suited for locally constrained problems of this type. Results are given for corrupted imagery with signal-to-noise ratio (SNR) as low as 2 dB, demonstrating high quality image estimation as measured by local feature integrity, and improvement in SNR.

Current formulations of anisotropic diffusion are unable to prevent feature drift and smooth small regions. These deficiencies reduce the effectiveness of the diffusion operation in many image processing tasks, including segmentation, edge detection, compression, and multiscale processing. This paper introduces a morphological diffusion coefficient capable of smoothing small objects while maintaining edge locality. Results are presented that demonstrate its efficacy in edge detection tasks.

In this paper, we present a multiresolution implementation of the watershed segmentation algorithm. Our approach uses the morphological pyramid to form a scale space representation, offering a significant reduction in computational cost. In addition to increased efficiency, the multiresolution approach avoids the over-segmentation problem of traditional fixed scale watershed algorithms. As shown in the examples, the watershed pyramids produce edge maps corresponding to the desired scale without sacrificing accuracy in edge location.

A novel approach to diffusion-based edge detection is presented. The method utilizes a multiresolution, multiscale pyramidal structure created via successive anisotropic diffusion and subsampling of the original image. Coarse, low resolution image representations of the anisotropic diffusion pyramid (ADP) guide boundary detection at higher resolutions. The edges computed by this hierarchical technique are resilient to image corruption and reflect features of a desired scale. The ADP edge detector offers the advantages of decreased edge localization error and region merging, as compared to Gaussian-based pyramid edge detectors. The ADP-based approach also improves upon the performance of fixed-resolution anisotropic diffusion in noise and alleviates the need for post-processing such as edge thinning and linking.

The problem of enhancing and detecting edges in infrared images is addressed in this paper. The proposed solution utilizes a multiresolution structure called the anisotropic diffusion pyramid (ADP), which is created by successive application of anisotropic diffusion and subsampling. A pyramid node linking process is used to segment the image and to detect edges. For the slowly varying sigmoidal edges in the IR imagery, it is shown that the diffusion process is effective in edge enhancement. Furthermore, system parameters are derived to maximize the edge sharpening ability of the ADP. The ADP avoids the "staircase" artifacts of single-resolution diffusion algorithms and avoids the localization and region merging problems of pyramids based on linear filters.

We develop a general formalism for computing high quality, low-cost solutions to nonconvex combinatorial optimization problems expressible as distributed interacting local constraints. For problems of this type, generalized deterministic annealing (GDA) avoids the performance-related sacrifices of current techniques. GDA exploits the localized structure of such problems by assigning K-state neurons to each optimization variable. The neuron values correspond to the probability densities of K-state local Markov chains and may be updated serially or in parallel; the Markov model is derived from the Markov model of simulated annealing (SA), although it is greatly simplified. Theorems are presented that firmly establish the convergence properties of GDA, as well as supplying practical guidelines for selecting the initial and final temperatures in the annealing process. A benchmark image enhancement application is provided where the performance of GDA is compared to other optimization methods. The empirical data taken in conjunction with the formal analytical results suggest that GDA enjoys significant performance advantages relative to current methods for combinatorial optimization.

In this paper, we present a new approach to image restoration based on a flexible constraint framework that encapsulates structural assumptions about the uncorrupted image. Piecewise and local class (PALC) models are defined and utilized to restore images degraded by linear blurring and additive noise. The restoration process is accomplished by iteratively deconvolving the solution image while simultaneously optimizing local image characteristics defined by the PALC models. Solution images to this ill-posed, combinatorial problem are computed using the novel generalized deterministic annealing (GDA) optimization technique. The results demonstrate high quality image restoration as measured by local feature integrity, improvement in signal-to-noise ratio, and reduction of restoration artifacts, especially in the presence of heavy-tailed additive noise.>

We introduce the Anisotropic Diffusion Pyramid (ADP), a structure for multiresolution image processing. We also develop the ADP for use in region-based segmentation. The pyramid is constructed using the anisotropic diffusion equations, creating an efficient scale-space representation. Segmentation is accomplished using pyramid node linking. Since anisotropic diffusion preserves edge localization as the scale is increased, the region boundaries in the coarse-to-fine ADP segmentation are accurately delineated. An application to segmentation of remotely sensed data is provided. The results of ADP segmentation are compared to Gaussian-based pyramidal segmentation. The examples show that the ADP has a superior ability to subdivide the image into integral groupings, minimizing the error in boundary localization and in pixel intensity.>

We introduce new classes of image enhancement techniques that are based on optimizing local characteristics of the image. Using a new optimization technique for nonconvex combinatorial optimization problems, generalized deterministic annealing (GDA), we compute fuzzy nonlinear regressions of noisy images with respect to characteristic image sets defined by certain local image models. The image enhancement results demonstrate the powerful approach of nonlinear regression and the low-cost, high-quality optimization of GDA.

An edge detection technique that optimizes edge localization while providing edge continuity and edge thinning is introduced. The solution is obtained by annealing a mean field neural network, providing inexpensive solutions with high parameter insensitivity. Anisotropic diffusion is used to provide localized edge data through the scale-space. Analysis of network parameters, diffusion parameters, network convergence, and scale-space equivalence is provided. Results are shown for real image data and compared with the results of other important edge detection schemes.>

We have developed a system, Generalized cylinder Recognition Using Perceptual Organization (GRUPO), that performs model-based recognition of the projections of generalized cylinders. Motivated by psychological theory, the approach uses perceptual organization, the grouping of structurally significant features, to limit the object and viewpoint search spaces in recognition. The system receives feature data from a segmentation based on perceptual organization and ranks the object space according to estimates of conditional object probabilities. Depth information is not used in the approach. To complete the recognition system, several problems were solved. For modeling, theoretical contributions include a proof for the invariance of discontinuities to projection, a method to find the axis of symmetry1, and a technique for determining self-occlusion. For the recognition process, solutions to search administration, feature matching, probabilistic search of the object space, and final template matching have been developed. The theory has been implemented and tested on synthetic data.