Anomaly detection for rail running bands, the pattern of wheel–rail contact area, is crucial to analyze composite rail irregularities. This paper presents an all-weather vision-based solution for running-band inspection. However, two major algorithmic challenges restrict inspection effectiveness: 1) the identification of high-quality features for diversified and imbalanced data subject to noise and outliers, and 2) inferring implicit anomaly co-occurrence patterns. We regard overall running-band anomaly detection as a multi-label classification problem and develop a novel deep multi-anomaly recognition network via recurrent attention graphs (RAGRN). The proposed RAGRN consists of two fundamental components, each directly addressing the two major challenges of this paper: 1) Class-specific features are extracted for fine-grained discrimination via split-channel and gradient-guided class-specific attention mechanisms; 2) We develop a multi-anomaly classifier, which effectively captures long-distance correlation features via a recurrent attention graph with visual and statistical guidance for graph propagation, containing prior statistical and image-specific information. The experiments and statistical analyses demonstrate that RAGRN outperforms all related state-of-the-art frameworks and has the potential to be applied to practical inspection.

As part of an ongoing worldwide effort to comprehend and monitor insect biodiversity, this paper presents the BIOSCAN-5M Insect dataset to the machine learning community and establish several benchmark tasks. BIOSCAN-5M is a comprehensive dataset containing multi-modal information for over 5 million insect specimens, and it significantly expands existing image-based biological datasets by including taxonomic labels, raw nucleotide barcode sequences, assigned barcode index numbers, geographical, and size information. We propose three benchmark experiments to demonstrate the impact of the multi-modal data types on the classification and clustering accuracy. First, we pretrain a masked language model on the DNA barcode sequences of the BIOSCAN-5M dataset, and demonstrate the impact of using this large reference library on species- and genus-level classification performance. Second, we propose a zero-shot transfer learning task applied to images and DNA barcodes to cluster feature embeddings obtained from self-supervised learning, to investigate whether meaningful clusters can be derived from these representation embeddings. Third, we benchmark multi-modality by performing contrastive learning on DNA barcodes, image data, and taxonomic information. This yields a general shared embedding space enabling taxonomic classification using multiple types of information and modalities. The code repository of the BIOSCAN-5M Insect dataset is available at https://github.com/bioscan-ml/BIOSCAN-5M.

Vision-based anomaly inspection plays a crucial role in the efficient maintenance of millions of kilometers of railway, with rail segmentation, a key step in such anomaly detection for providing localization prior. However multi-type rails, those involved in crossings and connections, have highly variable patterns, greatly restricting the performance of standard (straight) rail segmentation methods. Semantic segmentation helps to deal with complex railway scenes and variable patterns, however the noise sensitivity, intra-class differences, and inter-class similarities still challenge the segmentation. Superpixel segmentation can aggregate local similar pixels with precise boundaries, which can offer a weak prior for semantic segmentation for boundary information modeling, intra-class aggregation, and inter-class differentiation, however how to integrate superpixel-level guidance to advance rail segmentation is still challenging. This paper proposes a two-stage transformer-Convolutional Neural Network (CNN)-based segmentation framework. The first stage, Attention-Based Superpixel Segmentation Sub-Network via Boundary Calibration (BCASN), generates railway superpixels by the learning of intra-superpixel consistency and boundary calibration to effectively fit rail boundaries and guide the second-stage rail segmentation. The second stage, Superpixel-Guided Multi-Type Rail Segmentation Sub-Network via Contextual Information Aggregation (CIASSN), captures railway semantics via global and cross-scale context construction, aggregates rail features via directional guidance and structured prior, and makes comprehensive segmentation decisions at superpixel and pixel scales with the learning of superpixel-level context and classification. The experiments demonstrate that the proposed solution achieves 98.71% overall accuracy, 98.44% mIoU, and 87.33% boundary recall in multi-type rail segmentation, significantly extends applicable scenarios, and outperforms all related state-of-the-art methods in rail and road segmentation.

Vision-based ego-centric 3D human pose estimation (ego-HPE) is essential to support critical applications of xR-technologies. However, severe self-occlusions and strong distortion introduced by the fish-eye view from the head mounted camera, make ego-HPE extremely challenging. To address these challenges, we propose a domain-guided spatio-temporal transformer model that leverages information specific to ego-views. Powered by this domain-guided transformer, we build Egocentric Spatio-Temporal Self-Attention Network (Ego-STAN), which uses 2D image representations and spatio-temporal attention to address both distortions and self-occlusions in ego-HPE. Additionally, we introduce a spatial concept called feature map tokens (FMT) which endows Ego-STAN with the ability to draw complex spatio-temporal information encoded in ego-centric videos. Our quantitative evaluation on the contemporary xR-EgoPose dataset, achieves a 38.2% improvement on the highest error joints against the SOTA ego-HPE model, while accomplishing a 22% decrease in the number of parameters. Finally, we also demonstrate the generalization capabilities of our model to real-world HPE tasks beyond ego-views achieving 7.7% improvement on 2D human pose estimation with the Human3.6M dataset. Our code is also made available at: https://github.com/jmpark0808/Ego-STAN

In an effort to catalog insect biodiversity, we propose a new large dataset of hand-labelled insect images, the BIOSCAN-1M Insect Dataset. Each record is taxonomically classified by an expert, and also has associated genetic information including raw nucleotide barcode sequences and assigned barcode index numbers, which are genetic-based proxies for species classification. This paper presents a curated million-image dataset, primarily to train computer-vision models capable of providing image-based taxonomic assessment, however, the dataset also presents compelling characteristics, the study of which would be of interest to the broader machine learning community. Driven by the biological nature inherent to the dataset, a characteristic long-tailed class-imbalance distribution is exhibited. Furthermore, taxonomic labelling is a hierarchical classification scheme, presenting a highly fine-grained classification problem at lower levels. Beyond spurring interest in biodiversity research within the machine learning community, progress on creating an image-based taxonomic classifier will also further the ultimate goal of all BIOSCAN research: to lay the foundation for a comprehensive survey of global biodiversity. This paper introduces the dataset and explores the classification task through the implementation and analysis of a baseline classifier. The code repository of the BIOSCAN-1M-Insect dataset is available at https://github.com/zahrag/BIOSCAN-1M

Optimal performance is desired for decision-making in any field with binary classifiers and diagnostic tests, however common performance measures lack depth in information. The area under the receiver operating characteristic curve (AUC) and the area under the precision recall curve are too general because they evaluate all decision thresholds including unrealistic ones. Conversely, accuracy, sensitivity, specificity, positive predictive value and the F1 score are too specific-they are measured at a single threshold that is optimal for some instances, but not others, which is not equitable. In between both approaches, we propose deep ROC analysis to measure performance in multiple groups of predicted risk (like calibration), or groups of true positive rate or false positive rate. In each group, we measure the group AUC (properly), normalized group AUC, and averages of: sensitivity, specificity, positive and negative predictive value, and likelihood ratio positive and negative. The measurements can be compared between groups, to whole measures, to point measures and between models. We also provide a new interpretation of AUC in whole or part, as balanced average accuracy, relevant to individuals instead of pairs. We evaluate models in three case studies using our method and Python toolkit and confirm its utility.

In recent years a vast amount of visual content has been generated and shared from many fields, such as social media platforms, medical imaging, and robotics. This abundance of content creation and sharing has introduced new challenges, particularly that of searching databases for similar content - Content Based Image Retrieval (CBIR) - a long-established research area in which improved efficiency and accuracy are needed for real-time retrieval. Artificial intelligence has made progress in CBIR and has significantly facilitated the process of instance search. In this survey we review recent instance retrieval works that are developed based on deep learning algorithms and techniques, with the survey organized by deep feature extraction, feature embedding and aggregation methods, and network fine-tuning strategies. Our survey considers a wide variety of recent methods, whereby we identify milestone work, reveal connections among various methods and present the commonly used benchmarks, evaluation results, common challenges, and propose promising future directions.

Many measurement modalities which perform imaging by probing an object pixel-by-pixel, such as via Photoacoustic Microscopy, produce a multi-dimensional feature (typically a time-domain signal) at each pixel. In principle, the many degrees of freedom in the time-domain signal would admit the possibility of significant multi-modal information being implicitly present, much more than a single scalar “brightness”, regarding the underlying targets being observed. However, the measured signal is neither a weighted-sum of basis functions (such as principal components) nor one of a set of prototypes (K-means), which has motivated the novel clustering method proposed here. Signals are clustered based on their shape, but not amplitude, via angular distance, and centroids are calculated as the direction of maximal intra-cluster variance, resulting in a clustering algorithm capable of learning centroids (signal shapes) that are related to the underlying, albeit unknown, target characteristics in a scalable and noise-robust manner.

Abstract Object detection, one of the most fundamental and challenging problems in computer vision, seeks to locate object instances from a large number of predefined categories in natural images. Deep learning techniques have emerged as a powerful strategy for learning feature representations directly from data and have led to remarkable breakthroughs in the field of generic object detection. Given this period of rapid evolution, the goal of this paper is to provide a comprehensive survey of the recent achievements in this field brought about by deep learning techniques. More than 300 research contributions are included in this survey, covering many aspects of generic object detection: detection frameworks, object feature representation, object proposal generation, context modeling, training strategies, and evaluation metrics. We finish the survey by identifying promising directions for future research.

Texture is a fundamental characteristic of many types of images, and texture representation is one of the essential and challenging problems in computer vision and pattern recognition which has attracted extensive research attention over several decades. Since 2000, texture representations based on Bag of Words and on Convolutional Neural Networks have been extensively studied with impressive performance. Given this period of remarkable evolution, this paper aims to present a comprehensive survey of advances in texture representation over the last two decades. More than 250 major publications are cited in this survey covering different aspects of the research, including benchmark datasets and state of the art results. In retrospect of what has been achieved so far, the survey discusses open challenges and directions for future research.

Research in texture recognition often concentrates on recognizing textures with intraclass variations, such as illumination, rotation, viewpoint, and small-scale changes. In contrast, in real-world applications, a change in scale can have a dramatic impact on texture appearance to the point of changing completely from one texture category to another. As a result, texture variations due to changes in scale are among the hardest to handle. In this paper, we conduct the first study of classifying textures with extreme variations in scale. To address this issue, we first propose and then reduce scale proposals on the basis of dominant texture patterns. Motivated by the challenges posed by this problem, we propose a new GANet network where we use a genetic algorithm to change the filters in the hidden layers during network training in order to promote the learning of more informative semantic texture patterns. Finally, we adopt a Fisher vector pooling of a convolutional neural network filter bank feature encoder for global texture representation. Because extreme scale variations are not necessarily present in most standard texture databases, to support the proposed extreme-scale aspects of texture understanding, we are developing a new dataset, the extreme scale variation textures (ESVaT), to test the performance of our framework. It is demonstrated that the proposed framework significantly outperforms the gold-standard texture features by more than 10% on ESVaT. We also test the performance of our proposed approach on the KTHTIPS2b and OS datasets and a further dataset synthetically derived from Forrest, showing the superior performance compared with the state-of-the-art.

With the growing number of automated welding systems present throughout manufacturing, achieving high precision is naturally a key objective. The alignment of weld tip to weld seam, particularly in very long welds (such as in pipes), is a technical challenge in which computer vision has much to offer. This paper introduces a real-time methodology for weld-seam tracking. The key challenge associated with the weld tracking problem is to accurately locate the seam and welding tip; that is, to define image features which allow the seam and tip edges to be identified. An efficient edge detection method based on steerable filters is proposed to have a robust edge extraction in different conditions of noise and illumination. The image processing precision is experimentally verified through comparison with ground truth data.

In medical image analysis, multi-modal registration has been a challenging task due to the complex intensity relationship between images to be aligned. Conventional multi-modal approaches tend to assess the accuracy of the alignment by measuring a similarity based on statistical dependency of the intensity values between images. However, measuring statistical similarity measures, such as mutual information, is not promising, especially in those cases with complex and spatially dependent intensity relations. A new similarity measure is proposed based on the concept of self-similarity, the similarity of patches within an image, motivated by the fact that similar structures are more probable to undergo similar intensity transformations. The method is applied to the registration framework to align simulated and real brain images from different modalities and compared to the conventional multi-modal registration method. Quantitative evaluation of the method demonstrates that better accuracy can be achieved.

In this letter, a novel approach to visual saliency detection via neural response divergence (NeRD) is proposed, where synaptic portions of deep neural networks, previously trained for complex object recognition, are leveraged to compute low-level cues that can be used to compute image region distinctiveness. Based on this concept, an efficient visual saliency detection framework is proposed using deep convolutional StochasticNets. Experimental results using complex scene saliency dataset and MSRA10k natural image datasets show that the proposed NeRD approach can achieve improved performance when compared to state-of-the-art image saliency approaches, while attaining low computational complexity necessary for near-real-time computer vision applications.

Local binary patterns (LBP) are considered among the most computationally efficient high-performance texture features. However, the LBP method is very sensitive to image noise and is unable to capture macrostructure information. To best address these disadvantages, in this paper, we introduce a novel descriptor for texture classification, the median robust extended LBP (MRELBP). Different from the traditional LBP and many LBP variants, MRELBP compares regional image medians rather than raw image intensities. A multiscale LBP type descriptor is computed by efficiently comparing image medians over a novel sampling scheme, which can capture both microstructure and macrostructure texture information. A comprehensive evaluation on benchmark data sets reveals MRELBP's high performance-robust to gray scale variations, rotation changes and noise-but at a low computational cost. MRELBP produces the best classification scores of 99.82%, 99.38%, and 99.77% on three popular Outex test suites. More importantly, MRELBP is shown to be highly robust to image noise, including Gaussian noise, Gaussian blur, salt-and-pepper noise, and random pixel corruption.

A limitation of active contours models (both parametric and geometric) is their sensitivity to noise. Many solutions to noise sensitivity have been proposed in the literature, with the current state-of-the-art based on image blurring and multiresolution processing. However a significant drawback of both approaches is the side effect of edge delocalization. In this paper, gradient information extracted from all resolutions of the undecimated wavelet transform is used to build the external force map for the active contour. The new map accurately drives the active contour and improves edge localization. The proposed method builds on both Gradient Vector Flow and Vector Field Convolution active contours. Comparisons to classical and state-of-the-art methods show a dramatic improvement in active contour convergence for all levels of noise.

Local Binary Patterns (LBP) are among the most computationally efficient amongst high-performance texture features. However, LBP is very sensitive to image noise and is unable to capture macrostructure information. To best address these disadvantages, in this paper we introduce a novel descriptor for texture classification, the Median Robust Extended Local Binary Pattern (MRELBP). In contrast to traditional LBP and many LBP variants, MRELBP compares local image medians instead of raw image intensities. We develop a multiscale LBP-type descriptor by efficiently comparing image medians over a novel sampling scheme, which can capture both microstructure and macrostructure. A comprehensive evaluation on benchmark datasets reveals MRELBP's remarkable performance (robust to gray scale variations, rotation changes and noise) relative to state-of-the-art algorithms, but nevertheless at a low computational cost, producing the best classification scores of 99.82%, 99.38% and 99.77% on three popular Outex test suites. Furthermore, MRELBP is also shown to be highly robust to image noise including Gaussian noise, Gaussian blur, Salt-and-Pepper noise and random pixel corruption.

We describe a new model for the detection of edges in a given image. The model takes the invariance of local features of the image w.r.t translational symmetry operations into account. This is done by expressing the symmetries as a local Lie group and their associated Lie algebras in the regularizer of our model. Central to our work is the formulation of an energy density for the regularizer which itself is invariant under the action of a Lie algebra. Formulated as a Gaussian Markov Random Field, the parameters of the model are estimated by the EM principle.

Over the last decade Carbon-Fiber reinforced polymers (CFRP) have become increasingly important in industry and consumer oriented products due to their physical robustness and shape adaptivity. Defect localization and characterization is key to assuring the quality and durability of CFRPs. Active thermography has become an important non-invasive technology for the detection of hidden defects within CFRP materials. However current active thermographers exhibit resolutions of only up to 640 × 512 pixels, whereas visible spectrum imagers routinely have resolutions of 10 megapixels and higher. We therefore propose a fusion of active thermography and visible imagery data, capable of producing higher-resolution thermography images based on the physical boundaries in the visible domain. Our approach handles the disparity between the two imagers without the need for stereo or individual calibration.

Markov random fields (MRFs) and conditional random fields (CRFs) are influential tools in image modeling, particularly for applications such as image segmentation. Local MRFs and CRFs utilize local nodal interactions when modeling, leading to excessive smoothness on boundaries (i.e., the short-boundary bias problem). Recently, the concept of fully connected conditional random fields with stochastic cliques (SFCRF) was proposed to enable long-range nodal interactions while addressing the computational complexity associated with fully connected random fields. While SFCRF was shown to provide significant improvements in segmentation accuracy, there were still limitations with the preservation of fine structure boundaries. To address these limitations, we propose a new approach to stochastic clique formation for fully connected random fields (G-SFCRF) that is guided by the structural characteristics of different nodes within the random field. In particular, fine structures surrounding a node are modeled statistically by probability distributions, and stochastic cliques are formed by considering the statistical similarities between nodes within the random fields. Experimental results show that G-SFCRF outperforms existing fully connected CRF frameworks, SFCRF, and the principled deep random field framework for image segmentation.

Synthetic Aperture Radar (SAR) images of sea ice have proven to be very useful toward classification of ice cover into ice types. However, using SAR images to separate the marginal ice zone (MIZ) from consolidated ice and open water has not been explicitly considered before. One typical feature of MIZ is that it is more dynamic than consolidated ice, and includes floes, fast and thin ice or ice eddies. The current paper utilizes the dynamic feature of MIZ to investigate a curvelet-based feature extraction method in order to classify a SAR image into open water, dynamic ice and consolidated ice, as a first step toward using SAR imagery to identify the MIZ. An experiment of 10-fold cross validation is conducted to demonstrate that the proposed feature extraction method is effective. Finally, an SVM classifier is used on a SAR image to test the performance of the curvelet-based feature. The result shows that curvelet-based feature can classify the dynamic ice accurately.

This paper presents a conceptually simple, and robust, yet highly effective, approach to both texture classification and material categorization. The proposed system is composed of three components: 1) local, highly discriminative, and robust features based on sorted random projections (RPs), built on the universal and information-preserving properties of RPs; 2) an effective bag-of-words global model; and 3) a novel approach for combining multiple features in a support vector machine classifier. The proposed approach encompasses the simplicity, broad applicability, and efficiency of the three methods. We have tested the proposed approach on eight popular texture databases, including Flickr Materials Database, a highly challenging materials database. We compare our method with 13 recent state-of-the-art methods, and the experimental results show that our texture classification system yields the best classification rates of which we are aware of 99.37% for Columbia-Utrecht, 97.16% for Brodatz, 99.30% for University of Maryland Database, and 99.29% for Kungliga Tekniska högskolan-textures under varying illumination, pose, and scale. Moreover, the proposed approach significantly outperforms the current state-of-the-art approach in materials categorization, with an improvement to classification accuracy of 67%.

A promising, recently explored, alternative to ultra-high b-value diffusion weighted imaging (UHB-DWI) is apparent ultra-high b-value diffusion-weighted image reconstruction (AUHB-DWR), where a computational model is used to assist in the reconstruction of apparent DW images at ultra-high b -values. Firstly, we present a novel approach to AUHB-DWR that aims to improve image quality. We formulate the reconstruction of an apparent DW image as a hidden conditional random field (HCRF) in which tissue model diffusion parameters act as hidden states in this random field. The second contribution of this paper is a new generation of fully connected conditional random fields, called the hidden stochastically fully connected conditional random fields (HSFCRF) that allows for efficient inference with significantly reduced computational complexity via stochastic clique structures. The proposed AUHB-DWR algorithms, HCRF and HSFCRF, are evaluated quantitatively in nine different patient cases using Fisher's criteria, probability of error, and coefficient of variation metrics to validate its effectiveness for the purpose of improving intensity delineation between expert identified suspected cancerous and healthy tissue within the prostate gland. The proposed methods are also examined using a prostate phantom, where the apparent ultra-high b-value DW images reconstructed using the tested AUHB-DWR methods are compared with real captured UHB-DWI. The results illustrate that the proposed AUHB-DWR methods has improved reconstruction quality and improved intensity delineation compared with existing AUHB-DWR approaches.

We propose Eigen-RBM, a scalable Restricted Boltzmann Machine (RBM) for visual recognition in which the number of free parameters to learn is independent of the image size. Eigen-RBM exploits the global structure of the image and does not impose any locality or translation-invariance assumption, and regularizes the network weights to be a linear combination of a set of predefined filters. We show that, compared to basic RBM, Eigen-RBM can achieve similar or better performance in both recognition and sample generation with significantly less training time.

A limitation of Optical Coherence Tomography (OCT) image segmentation is the poor signal-to-noise ratio of the imaging process, particularly because images are sampled quickly, at high resolutions, and in-vivo. Furthermore, speckle noise is generated by the reflections of the OCT LASER. Because OCT is widely used in imaging the cornea, retina, and skin, OCT layer segmentation is of key interest in all applications. In this paper, a multi-resolution parametric active contour is used for OCT segmentation. The proposed method uses an undecimated wavelet transform to obtain scale-dependent noise reduction, while the active contour is initialized with a generalized Hough transform. Experimental results show that the proposed method outperforms classical as well as state-of-the-art methods and segments OCT images with high level of accuracy.

Multi-atlas label fusion is a widely used approach in medical image analysis that has improved the accuracy of segmentation. Majority voting, as the most common combination strategy, weighs each candidate in the atlas database equally. More sophisticated methods rely on the intensity similarity of each atlas to the target volume. However, these methods cannot handle those cases in which the atlases and the target image are in different modalities. A new method for label fusion is proposed, based on a structural similarity measure, relying on the structural relationships of features extracted from an undecimated wavelet transform instead of explicit image intensities. The new label fusion method has been tested on simulated and real MR images; segmentation results are promising, and open the door to a wider range of multi-modal approaches.

This paper presents a simple, novel, yet highly effective approach for robust face recognition. Given LBP-like descriptors based on local accumulated pixel differences, Angular Differences (AD) and Radial Differences (RD), the local differences are decomposed into complementary components of signs and magnitudes. The proposed descriptors have desirable features: (1) robustness to lighting, pose, and expression; (2) computation efficiency; (3) encoding of both microstructures and macrostructures; (4) consistent in form with traditional LBP, thus inheriting the merits of LBP; and (5) no required training, improving generalizability. From a given face image, we obtain six histogram features, each of which is obtained by concatenating spatial histograms extracted from nonoverlapping subregions. The Whitened PCA technique is used for dimensionality reduction, followed by Nearest Neighbor classification. We have evaluated the effectiveness of the proposed method on the Extended Yale B and CAS-PEAL-R1 databases. The proposed method impressively outperforms other well known systems, including what we believe to be the best reported performance for the the CAS-PEAL-R1 lighting probe set with a recognition rate of 72.3%.

A multi-scale statistical non-redundancy modeling approach is introduced for saliency detection in images. The statistical non-redundancy of pixels at different wavelet sub-bands is characterized using a multi-dimensional lattice of non-parametric statistical models, thus taking into account image saliency at multiple scales. This identifies saliency in image attributes at multiple scales, and makes saliency detection strongly robust against noisy input images. Results based on images from a public database show that the proposed approach outperforms existing single and multi-scale approaches, particularly when dealing with noisy images.

Conditional random fields (CRFs) are one of the most powerful frameworks in image modeling. However practical CRFs typically have edges only between nearby nodes; using more interactions and expressive relations among nodes make these methods impractical for large-scale applications, due to the high computational complexity. Recent work has shown that fully connected CRFs can be tractable by defining specific potential functions. In this paper, we present a novel framework to tackle the computational complexity of a fully connected graph without requiring specific potential functions. Instead, inspired by random graph theory and sampling methods, we propose a new clique structure called stochastic cliques. The stochastically fully connected CRF (SFCRF) is a marriage between random graphs and random fields, benefiting from the advantages of fully connected graphs while maintaining computational tractability. The effectiveness of SFCRF was examined by binary image labeling of highly noisy images. The results show that the proposed framework outperforms an adjacency CRF and a CRF with a large neighborhood size.

In this paper, we propose a simple, efficient, yet robust multiresolution approach to texture classification-binary rotation invariant and noise tolerant (BRINT). The proposed approach is very fast to build, very compact while remaining robust to illumination variations, rotation changes, and noise. We develop a novel and simple strategy to compute a local binary descriptor based on the conventional local binary pattern (LBP) approach, preserving the advantageous characteristics of uniform LBP. Points are sampled in a circular neighborhood, but keeping the number of bins in a single-scale LBP histogram constant and small, such that arbitrarily large circular neighborhoods can be sampled and compactly encoded over a number of scales. There is no necessity to learn a texton dictionary, as in methods based on clustering, and no tuning of parameters is required to deal with different data sets. Extensive experimental results on representative texture databases show that the proposed BRINT not only demonstrates superior performance to a number of recent state-of-the-art LBP variants under normal conditions, but also performs significantly and consistently better in presence of noise due to its high distinctiveness and robustness. This noise robustness characteristic of the proposed BRINT is evaluated quantitatively with different artificially generated types and levels of noise (including Gaussian, salt and pepper, and speckle noise) in natural texture images.

Local Binary Pattern (LBP) and its variants are effective and popular descriptors for texture classification. Most LBP like descriptors have disadvantages including sensitiveness to noise and inability to capture long distance texture information. In this paper we propose a simple, efficient, yet robust multi-resolution descriptor to texture classification - Binary Rotation Invariant and Noise Tolerant (BRINT). The proposed descriptor is very fast to build, very compact while remaining robust to illumination variations, rotation changes and noise. We develop a novel and simple strategy - averaging before binarization - to compute a local binary descriptor based on the conventional LBP approach. Points are sampled in a circular neighborhood, but keeping the number of bins in a single-scale LBP histogram constant and small by averaging over several contiguous pixels in the circle. There is no need for pre-training, no texton dictionary, and no tuning of parameters to deal with different datasets. Experiments on the Outex test suite demonstrate that the proposed approach is very robust to noise and significantly outperforms the state-of-the-art in terms of classifying noise corrupted textures.

The significant role dust aerosols play in the earth's climate system and microbial nutrition cycles have lead to increased efforts in employing remote sensing to monitor their genesis, transport and deposition. In this contribution we considerably refine our earlier statistical models for aerosol detection and atmospheric transport that rely on latent Gaussian Markov random fields for inference. Based on explicitly satisfying the so-called integrated continuity equation we develop a Bayesian generalized linear model intrinsically expressing the divergence of the field as a multiplicative factor covering physical aspects such as compressibility and column projection. Alongside employing surface emissivity estimates for improved genesis detection, we conduct a simulation study clearly showing a reduction of errors in the estimated flow field. We conclude with a case study that relates this experimental finding back to a dust event over northern Africa.

There is significant interest in the synthesis of discrete-state random fields, particularly those possessing structure over a wide range of scales. However, given a model on some finest, pixellated scale, it is computationally very difficult to synthesize both large- and small-scale structures, motivating research into hierarchical methods. In this paper, we propose a frozen-state approach to hierarchical modeling, in which simulated annealing is performed on each scale, constrained by the state estimates at the parent scale. This approach leads to significant advantages in both modeling flexibility and computational complexity. In particular, a complex structure can be realized with very simple, local, scale-dependent models, and by constraining the domain to be annealed at finer scales to only the uncertain portions of coarser scales; the approach leads to huge improvements in computational complexity. Results are shown for a synthesis problem in porous media.

A novel algorithm based on statistical non-redundancy is proposed for saliency detection in natural images. By modeling site neighbourhoods as realizations of other site neighbourhoods under a Gaussian process, the saliency of any arbitrary site can be characterized by the statistical non-redundancy of its site neighbourhood with respect to other site neighbourhoods in a given image. Preliminary results using natural images show that the proposed method provides improved precision vs. recall characteristics over previous methods such as spectral residuals and spectral whitening.

Two major challenges faced in active contours are poor capture range and high sensitivity towards noise. Recently, the concept of tensor vector convolution (TVF) was introduced and shown to be promising in handling these challenges. However, in the presence of high noise levels, TVF may have difficulty in converging to the desired object boundary, particularly if the distance is great between the initial contour and the object boundary. To tackle this challenge, the concept of a multi-scale tensor vector field (MTVF) active contour is introduced to further reduce noise sensitivity. Comparing the performance of MTVF with multi-scale gradient vector field and multi-scale vector field convolution demonstrates that MTVF is more resilient to high noise levels as well as significantly reducing computation time.

In this paper, a novel adaptive post-filtering method is introduced for effective video coding. Specifically, an enhanced filtering capability is obtained by partitioning a reconstructed video frame into non-overlapping segments based on local pattern information. Experimental results show that the proposed approach has the potential to produce an enhanced video frame reconstruction with more implementable filters when compared to existing adaptive post-filtering.

The significant role dust aerosols play in the earth's climate system and microbial nutrition cycles have lead to increased efforts of employing remote sensing to monitor their genesis, transport and deposition. This contribution extends earlier approaches of using Bayesian hierarchical models to extract dust activity from multi-spectral MSG-SEVIRI measurements by focusing on the signal-to-noise ratio with respect to post hoc motion analysis via optical flow. While interpreting also the latter in a completely Bayesian fashion, we show that our novel dust indication scheme reduces background noise and thereby renders the optical flow more decisive in terms of detecting even faint dust plumes. As a side effect of the indicators stability in case of dust absence, we point out the potential usage of its temporal variance to characterize dust at an early stage of the genesis and thus close to the corresponding source region.

Mineral dust aerosols can influence the Earth's climate system to a significant degree and have a strong effect on terrestrial and oceanic biogeochemical cycles. As one step in quantifying dust sources, sinks, and transport, this paper seeks to quantify the presence of dust storms in the Sahara desert, which is the most active worldwide source of dust. Our work is based on the SEVIRI infrared imager on-board the geostationary Meteosat-8 satellite, providing three separate channels at a 3km by 3km resolution. The significant challenge is that the infrared channels are highly influenced by the presence of water clouds and surface temperatures, which complicate the identification of dust-cloud anomalies. This paper develops a method of spatio-temporal background estimation from sparse data as a way of recovering dust images and presents results on real data.

Inspired by theories of sparse representation and compressed sensing, this paper presents a simple, novel, yet very powerful approach for texture classification based on random projection, suitable for large texture database applications. At the feature extraction stage, a small set of random features is extracted from local image patches. The random features are embedded into a bag-of-words model to perform texture classification; thus, learning and classification are carried out in a compressed domain. The proposed unconventional random feature extraction is simple, yet by leveraging the sparse nature of texture images, our approach outperforms traditional feature extraction methods which involve careful design and complex steps. We have conducted extensive experiments on each of the CUReT, the Brodatz, and the MSRC databases, comparing the proposed approach to four state-of-the-art texture classification methods: Patch, Patch-MRF, MR8, and LBP. We show that our approach leads to significant improvements in classification accuracy and reductions in feature dimensionality.

This paper presents a novel approach for texture classification, generalizing the wellknown local binary patterns (LBP). In the proposed approach, two different and complementary types of features are extracted from local patches, based on pixel intensities and differences. Inspired by the LBP approach, two intensity-based and two difference-based descriptors are developed. All four descriptors have the same form as the conventional LBP codes, thus they can be readily combined to form joint histograms to represent textured images. The proposed approach is computationally simple and is training-free: there is no need to learn a texton dictionary and no tuning of parameters. Extensive experimental results on two challenging texture databases (Outex and KTHTIPS2b) show that the proposed approach significantly outperforms the classical LBP approach and other state-of-the-art methods with a nearest neighbor classifier.

Identifying the surfaces of three-dimensional static objects or of two-dimensional objects over time are key to a variety of applications throughout computer vision. Active surface techniques have been widely applied to such tasks, such that a deformable spline surface evolves by the influence of internal and external (typically opposing) energies until the model converges to the desired surface. Present deformable model surface extraction techniques are computationally expensive and are not able to reliably identify surfaces in the presence of noise, high curvature, or clutter. This paper proposes a novel active surface technique, decoupled active surfaces, with the specific objectives of robustness and computational efficiency. Motivated by recent results in two-dimensional object segmentation, the internal and external energies are treated separately, which leads to much faster convergence. A truncated maximum likelihood estimator is applied to generate a surface consistent with the measurements (external energy), and a Bayesian linear least squares estimator is asserted to enforce the prior (internal energy). To maintain tractability for typical three-dimensional problems, the density of vertices is dynamically resampled based on curvature, a novel quasi-random search is used as a substitute for the ML estimator, and sparse conjugate-gradient is used to execute the Bayesian estimator. The performance of the proposed method is presented using two natural and two synthetic image volumes.

This paper presents a simple and highly effective system for robust texture classification, based on (1) random local features, (2) a simple global Bag-of-Words (BoW) representation, and (3) Support Vector Machines (SVMs) based classification. The key contribution in this work is to apply a sorting strategy to a universal yet information-preserving random projection (RP) technique, then comparing two different texture image representations (histograms and signatures) with various kernels in the SVMs. We have tested our texture classification system on six popular and challenging texture databases for exemplar based texture classification, comparing with 12 recent state-of-the-art methods. Experimental results show that our texture classification system yields the best classification rates of which we are aware of 99.37% for CUReT, 97.16% for Brodatz, 99.30% for UMD and 99.29% for KTH-TIPS. Moreover, combining random features significantly outperforms the state-of-the-art descriptors in material categorization.

Among the main limitations of active contours are their high noise sensitivity and poor capture range from the target object. One of the most promising approaches for addressing these limitations is the concept of Vector Field Convolution (VFC). However, due to its isotropic vector field kernel, VFC does not take full advantage of the underlying image structural characteristics. By specifically addressing this idea, a novel local tensor vector field approach is developed to adaptively account for these structural characteristics. Experimental results demonstrate that the proposed adaptive method leads to more accurate segmentation.

This paper explores the combining of powerful local texture descriptors and the advantages over single descriptors for texture classification. The proposed system is composed of three components: (i) highly discriminative and robust sorted random projections (SRP) features; (ii) a global Bag-of-Words (BoW) model; and (iii) the use of multiple kernel Support Vector Machines (SVMs) combining multiple features. The proposed system is also very simple, stemming from (1) the effortless extraction of the SRP features, (2) the simple orderless histogramming in the BoW model, (3) a strategy with low computational complexity for multiple kernel SVMs. We have tested our texture classification system on three popular and challenging texture databases and find that the SVMs combining of SRP features produces outstanding classification results, out-performing the state-of-the-art for CUReT (99.37%) and KTH-TIPS (99.29%), and with highly competitive results for UIUC (98.56%).

In this paper, a novel approach is introduced to video resolution enhancement based on a structure-guided conditional sampling model. The proposed conditional sampling model is composed of hybrid models to handle strongly structured and weakly structured regions separately. To accomplish this, a binary hidden field indicating image structure is introduced into the conditional reconstruction process for estimating the high resolution video frames from the low resolution video frames. Preliminary results show that the proposed approach has the potential to produce high resolution video frames with improved image quality when compared to existing structure-guided video resolution enhancement approaches.

A novel nonlinear volumetric scale-space framework is proposed for multi-scale volumetric data representation. The problem is formulated as a Bayesian least-squares estimator, and a quasi-random density estimation approach is introduced for estimating the posterior distribution between consecutive volumetric scale space realizations. Experimental results using both synthetic and real MR volumetric data demonstrate the effectiveness of the proposed scale-space framework for three-dimensional representation with significantly better structural separation and localization across all scales when compared to existing volumetric scale-space frameworks such as volumetric anisotropic diffusion and volumetric linear Gaussian scale-space, especially under scenarios with high noise levels.

This paper presents a novel tracker, based on combining a linear chain conditional random field (CRF) adaptive multi-resolution segmentation with an unscented Kalman filter (UKF). Specifically, the proposed method combines multiple features and multiple resolutions to facilitate video tracking. The advantages of our method lie in its speed and robustness. Speed is dramatically improved by taking into account multiple resolutions in one dimensional CRF-based segmentation. Robustness is achieved by using multiple cues. The performance of the proposed method is demonstrated in human head tracking with a non-stationary camera. Results show that we are able to maintain real-time processing on quite generous video sequences. The paper argues that our approach is faster, more efficient and more robust than the conventional UKF.

In this paper, the effects of employing different noise estimation strategies on the performance of noise artifact suppression techniques in achieving high image quality has been investigated. Most literature on the subject tends to use the true noise level of the noisy image when performing noise artifact suppression. However, this approach does not reflect how such techniques would be used in practical situations where the true noise level is unknown, which is common in most image and video processing applications. Therefore, in practical situations, the noise level must first be estimated before a noise artifact suppression technique can be applied using the estimated noise level. Through a comprehensive analysis of different noise estimation strategies using empirical testing on a variety of images with different characteristics, the MAD wavelet noise estimation technique was found to be the overall preferred noise estimation technique for all popular noise artifact suppression techniques investigated (BM3D, bilateral, Neigh Shrink, BLS-GSM and non-local means). Furthermore, the BM3D noise artifact suppression technique, combined with the MAD wavelet noise estimation technique, was found to offer the best performance in achieving high image quality in situations where the noise level is unknown and must be estimated. The outcome of this research is clear recommendations that can be used in practise when suppressing noise artifacts exhibited in digital imagery and video.

In this paper, a hierarchical model-based approach to frame rate-up conversion is presented. Given a sequence of consecutive video frames, a Spatio-Temporal Conditional Random Field (ST-CRF) is trained to capture both the motion and shape characteristics of objects within consecutive frames. A hierarchical tree is then constructed via hierarchical segmentation that sub-divides frames into regions based on color intensity and regional velocity. A hierarchical sampling approach is then introduced to construct new intermediate frames between adjacent video frames, where estimated intermediate frames are constructed at each level of a hierarchical tree constructed such that the probability of the ST-CRF is maximized. Preliminary results using videos with different motion characteristics show that the proposed approach has potential for producing intermediate frames with high visual quality.

A novel energy functional for automatic registration of remote sensing imagery based on quasi-random scale-space structural correlation is presented. The structural correlation energy functional takes advantage of the fact that, for many types of remote sensing imagery, there exist common structures at different scales even if the acquired images have very different intensity characteristics. The proposed energy functional also takes advantage of the noise robustness and feature localization properties of quasi-random scale-space theory. An efficient globally exhaustive optimization strategy in the frequency domain is developed for registering remote sensing imagery based on the proposed energy functional. Promising test results on interband, intraband, and intermodal remote sensing image sets show that the proposed method has the advantage of being robust to differing sensing conditions and large misalignments.

The accurate detection of object boundaries via active contours is an ongoing research topic in computer vision. Most active contours converge toward some desired contour by minimizing a sum of internal (prior) and external (image measurement) energy terms. Such an approach is elegant, but suffers from a slow convergence rate and frequently misconverges in the presence of noise or complex contours. To address these limitations, a decoupled active contour (DAC) is developed which applies the two energy terms separately. Essentially, the DAC consists of a measurement update step, employing a Hidden Markov Model (HMM) and Viterbi search, and then a separate prior step, which modifies the updated curve based on the relative strengths of the measurement uncertainty and the nonstationary prior. By separating the measurement and prior steps, the algorithm is less likely to misconverge; furthermore, the use of a Viterbi optimizer allows the method to converge far more rapidly than energy-based iterative solvers. The results clearly demonstrate that the proposed approach is robust to noise, can capture regions of very high curvature, and exhibits limited dependence on contour initialization or parameter settings. Compared to five other published methods and across many image sets, the DAC is found to be faster with better or comparable segmentation accuracy.

The systematic evaluation of synthetic aperture radar (SAR) data analysis tools, such as segmentation and classification algorithms for geographic information systems, is difficult given the unavailability of ground-truth data in most cases. Therefore, testing is typically limited to small sets of pseudoground-truth data collected manually by trained experts, or primitive synthetic sets composed of simple geometries. To address this issue, we investigate the potential of employing an alternative approach, which involves the synthesis of SAR data and corresponding label fields from real SAR data for use as a reliable evaluation testbed. Given the scale-dependent nonstationary nature of SAR data, a new modeling approach that combines a resolution-oriented hierarchical method with a region-oriented binary tree structure is introduced to synthesize such complex data in a realistic manner. Experimental results using operational RADARSAT SAR sea-ice data and SIR-C/X-SAR land-mass data show that the proposed hierarchical approach can better model complex nonstationary scale structures than local MRF approaches and existing nonparametric methods, thus making it well suited for synthesizing SAR data and the corresponding label fields for potential use in the systematic evaluation of SAR data analysis tools.

An automatic method for segmenting skin lesions in conventional macroscopic images is presented. The images are acquired with conventional cameras, without the use of a dermoscope. Automatic segmentation of skin lesions from macroscopic images is a very challenging problem due to factors such as illumination variations, irregular structural and color variations, the presence of hair, as well as the occurrence of multiple unhealthy skin regions. To address these factors, a novel iterative stochastic region-merging approach is employed to segment the regions corresponding to skin lesions from the macroscopic images, where stochastic region merging is initialized first on a pixel level, and subsequently on a region level until convergence. A region merging likelihood function based on the regional statistics is introduced to determine the merger of regions in a stochastic manner. Experimental results show that the proposed system achieves overall segmentation error of under 10% for skin lesions in macroscopic images, which is lower than that achieved by existing methods.

Methods of scientific imaging and image analysis have become pervasive in a great variety of fields, including the properties of porous media. To study the large-scale morphological properties of porous media, high resolution random (Monte Carlo) samples are required. The purpose of this paper is to propose a novel approach for the statistical synthesis of scientific images, based on the concept of Conditional Random Fields. We explore two different sets of potential functions are used to model the pore-structure characteristics, and Monte Carlo Markov chain methods are also used to sample the high resolution images from the trained model. The resulting images are of high quality, and show the performance of the proposed framework.

We present Temporal Conditional Random Fields, a probabilistic framework for modeling object motion. The state-of-the-art discriminative approach for tracking is known as dynamic conditional random fields. This method models an event based on spatial and temporal relation between pixels in an image sequence without any prediction. To facilitate such a powerful graphical model with prediction and come up with a CRF-based predictor, we propose a set of new temporal relations for object tracking, with feature functions such as optical flow (calculated among consequent frames) and line filed features. We validate our proposed method with real data sequences and will show that the TCRF prediction is nearly equivalent with result of template matching. Experimental results indicate that our TCRF can predict future state of any maneuvering target with nearly zero error during its constant motion. Not only the proposed TCRF has a simple and easy to implement structure, but also it outperforms the state-of-the-art predictors such as Kalman filter.

For remote sensing data, the testing analysis tools is difficult since the ground-truth data are not available in many cases. To address this issue, a novel method for image synthesis is presented for use as a evaluation test-bed. Given the scale-dependent, non-stationary nature of remotely sensed data, a new modeling approach that combines a resolution-oriented hierarchical method with a regional label-oriented binary tree structure is introduced to synthesize such complex data. In this paper, we are proposing on first synthesizing a label field, which contains the complex structural characteristics, then synthesizing the texture based on the generated label field for a more accurate modeling. Experimental results using operational RADARSAT SAR sea-ice image data show that the proposed method is capable of modeling complex, nonstationary scale structures, thus making it well-suitable to produce reliable, realistic remote sensing imagery.

In this paper, a novel approach for performing blind decorrelation of SAR data is proposed. A patch-wise computation of the point-spread function (PSF) is performed directly from the SAR data to account for spatial nonstationarities present in SAR. The problem of estimating the PSF is formulated as an additive source separation problem in the frequency domain, and is subsequently solved using a Bayesian least squares estimation approach based on a Fisher-Tippett log-scatter model. Experimental results using both simulated SAR data and real RADARSAT-2 SAR sea-ice data showed that the proposed decorrelation approach can successfully learn the correct PSF and significantly reduce the correlation in SAR data.

Braille is the most common means of reading and study for visually handicapped people. The need for converting Braille documents into a computer-readable format has motivated research into the implementation of Braille recognition systems. The most fundamental steps in such recognition are estimating the scaling, spacing, and skewness properties of a given Braille document. In this paper we propose a statistical method for estimating these parameters, where we model an entire Braille image using a parameterized probability density function and find the maximum-likelihood (ML) estimates using Expectation-Maximization (EM) algorithm. The proposed method is robust against noise and other image artifacts, and has a modest computational complexity for straightforward execution on ordinary personal computers.

This paper examines non-linear shrinkage methods specifically taking into account the correlation structure of the multiresolution wavelet coefficients. In contrast to hidden Markov trees, which model the relationship of wavelet variance from scale to scale, here we wish to take advantage of coefficient correlation. A linear shrinkage based on the LLS (Linear Least Square) estimator, employing a sample correlation scheme, is tested and verified to have an aesthetic denoising performance. Then, state-of-the-art independent shrinkage functions are applied to exploit the efficiency of such techniques and to introduce non-linearity into the algorithm to compensate for non-Gaussianity of the wavelet statistics. The performance of the non-linear shrinkage technique, as used individually and together with the linear correlated approach, are illustrated.

The extraction of contours using deformable models, such as snakes, is a problem of great interest in computer vision, particular in areas of medical imaging and tracking. Snakes have been widely studied and many methods are available. In most cases, the snake converges towards the optimal contour by minimizing a sum of internal (prior) and external (image measurement) energy terms. This approach is elegant, but frequently mis-converges in the presence of noise or complex contours. To address these limitations, a novel discrete snake is proposed which treats the two energy terms separately. Essentially, the proposed method is a deterministic iterative statistical data fusion approach, in which the visual boundaries of the object are extracted, ignoring any prior, employing a Hidden Markov Model (HMM) and Viterbi search, and then applying importance sampling to the boundary points, on which the shape prior is asserted. The proposed implementation is straightforward and achieves dramatic speed and accuracy improvement compared to other methods.

Porous media are an important class of heterogeneous materials possessing complex random structures. Due to the limitation in measuring high resolution real samples, studying different physical properties of porous media requires the reconstruction of artificial samples. In many cases of significant interest, we have a two-scale reconstruction, in which only the large scales are resolved by low resolution measurements, leaving fine scales to be inferred from statistical models. In this paper we propose a statistical fusion approach for a two-scale porous media reconstruction, in which low resolution measurements are fused with high resolution samples, with synthetic realizations generated by posterior sampling.

Background modelling is a key task in tracking applications. Our interest in this paper is the accurate estimation of static backgrounds in scientific imaging, such as those in automated stem cell tracking. In this paper, an effective background estimation method is proposed. First, the segmentation results are used to remove the foreground objects, then the background is robustly estimated over the resultant 3-Dimensional residual image sequence. We do spatio-temporal background estimation over a local neighbourhood with a robust trimmed mean. The experimental results generated by the proposed method are quite promising.

This paper introduces a novel perceptually adaptive approach to image denoising using anisotropic non-local means. In the classical non-local means image denoising approach, the value of a pixel is determined based on the weighted average of other pixels, where the weights are determined based on a fixed isotropically weighted similarity function between the local neighborhoods. In the proposed algorithm, we demonstrate that noticeably improved perceptual quality can be achieved through the use of adaptive anisotropically weighted similarity functions between local neighborhoods. This is accomplished by adapting the similarity weighing function in an anisotropic manner based on the perceptual characteristics of the underlying image content derived efficiently based on the Mexican Hat wavelet. Experimental results show that the proposed method can be used to provide improved perceptual quality in the denoised image both quantitatively and qualitatively when compared to existing methods.

This paper presents a novel method for medical image signal fusion using complex-valued wavelets to enhance the information content in the fused signal from a perceptual manner. The proposed method introduces an adaptively weighted aggregation of signal characteristics based on the phase characteristics of medical image signals. The proposed method exploits the phase characteristics of the image signals to adaptively accentuate important anatomical and functional characteristics captured by each image signal during the signal fusion process. Experimental results show that the proposed method can improve the visualization of important anatomical and functional characteristics from different medical image signals in the fused image signal when compared with non-adaptive image signal fusion methods.

This paper introduces a novel stochastic approach to image denoising using an adaptive Monte Carlo scheme. Random samples are generated from the image field using a spatially-adaptive importance sampling approach. Samples are then represented using Gaussian probability distributions and a sample rejection scheme is performed based on a chi2statistical hypothesis test. The remaining samples are then aggregated based on Pearson Type VII statistics to create a non-linear estimate of the denoised image. The proposed method exploits global information redundancy to suppress noise in an image. Experimental results show that the proposed method provides superior noise suppression performance both quantitatively and qualitatively when compared to the state-of-the-art image denoising methods.

There are ever increasing number of applications of multi-target tracking and considerable research has been conducted to solve this problem. Multi-target tracking is a NP-hard problem and almost all of the present multi-target tracking algorithms are sub-optimal by finding the solution in a reduced hypothesis space. In this paper we introduce a new approach toward finding the optimal single frame solution for general multi-target tracking problem. Our proposed method finds the optimal solution using linear programming optimization method. The proposed method has been successfully applied to synthetic and real data.

In this paper a combined Bayesshrink wavelet-ridgelet de-noising method is presented. In our previous work we have showed that Bayesshrink ridgelet performs better than Visushrink ridgelet and Visushrink wavelet. Although our Bayesshrink ridgelet technique performs somewhat poorer in comparison with Bayesshrink wavelet, based on SNR, visually it produces smoother results, especially for images with straight lines. In the proposed method Bayesshrink wavelet is combined with Bayesshrink ridgelet denoising method which performs better than each filter individually. The proposed combined denoising method gains the advantage of each filter in its specific domain, i.e., wavelet for natural and ridgelet for straight regions, and produces better and smoother results, both visually and in terms of SNR

This paper proposes a novel correlated shrinkage method based on wavelet joint statistics. Our objective is to demonstrate effectiveness of the wavelet correlation models [Z. Azimifar et al., 2004] in estimating the original signal from a noising observation. Simulation results are given to show the advantage of the new correlated shrinkage function. In comparison with the popular nonlinear shrinkage algorithms, it improves the denoised results.

A better understanding of cell behavior is very important in drug and disease research. Cell size, shape, and motility may play a key role in stem-cell specialization or cancer development. However the traditional method of inferring these values from image sequences manually is such an onerous task that automated methods of cell tracking and segmentation are in high demanded, especially given the increasing amount of cell data being collected. In this paper, a novel probabilistic cell model is designed to segment the individual hematopoietic stem cells (HSCs) extracted from mice bone marrow cells. The proposed cell model has been successfully applied to HSC segmentation, identifying the most probable cell locations in the image on the basis of cell brightness and morphology.

The dynamic estimation of large-scale stochastic image sequences, as frequently encountered in remote sensing, is important in a variety of scientific applications. However, the size of such images makes conventional dynamic estimation methods, for example, the Kalman and related filters, impractical. In this paper, we present an approach that emulates the Kalman filter, but with considerably reduced computational and storage requirements. Our approach is illustrated in the context of a 512 x 512 image sequence of ocean surface temperature. The static estimation step, the primary contribution here, uses a mixture of stationary models to accurately mimic the effect of a nonstationary prior, simplifying both computational complexity and modeling. Our approach provides an efficient, stable, positive-definite model which is consistent with the given correlation structure. Thus, the methods of this paper may find application in modeling and single-frame estimation.

The computational complexity of simulated annealing makes it an impractical tool in many applications, particularly for complex, non-local models on very large 2D and 3D domains as desired in many scientific contexts. In particular, it is very difficult to produce large scale structure from a fine, pixellated lattice. Thus a hierarchical approach is intuitively attractive. However, existing approaches are few and limited. Motivated by a current problem in porous media, we develop a hierarchical approach to complex model sampling. In experiments, this approach results in 1-2 orders of magnitude computational gain, and significant gains in convergence as well.

Interest in the problem of shape matching has been recently reignited by the huge proliferation of images on the Internet, particularly huge computer databases containing thousands or millions of images. Although many typical image database algorithms rely on a variety of shading, texture, and colour attributes, there are significant opportunities for the use of shape as a discriminator, particularly for binary or high-contrast images. Fourier shape descriptors have been studied extensively for shape comparison, however the descriptor phases have been mostly neglected - either ignored entirely or treated simplistically. This paper formalizes the use of phase in shape matching, and derives shape discriminators, applicable to both simple and complex shapes.

An important general model for discrete-time signal processing is the switching state space (SSS) model, which generalizes the hidden Markov model and the Gaussian state space model. Inference and parameter estimation in this model are known to be computationally intractable. This paper presents a powerful new approximation to the SSS model. The approximation is based on a variational technique that preserves the multimodal nature of the continuous state posterior distribution. Furthermore, by incorporating a windowing technique, the resulting EM algorithm has complexity that is just linear in the length of the time series. An alternative Viterbi decoding with frame-based likelihood is also presented which is crucial for the speech application that originally motivates this work. Our experiments focus on demonstrating the effectiveness of the algorithm by extensive simulations. A typical example in speech processing is also included to show the potential of this approach for practical applications.

The estimation of images and random fields from sparse and/or noisy data is highly-developed, to the point where methods such as least-squares estimation, simulated annealing, and wavelet shrinkage are quite standardized. The key problem, however, is that the estimates are not a realistic version of the random field, and do not represent a typical or representative sample of the system being studied. Instead, what is often desired is that we find a random sample from the posterior distribution, a much more subtle and difficult problem than estimation. Typically this is solved using Markov-Chain Monte-Carlo / simulated annealing approaches, however these may be computationally challenging and slow to converge. In this paper we use hierarchical models to formulate a novel, fast posterior sampler.

Virtually all implementations of simulated annealing are simplified by assuming discrete unknowns, however continuous-parameter annealing has many potential applications to image processing. Widely scattered problems such as formant tracking, boundary estimation and phase- unwrapping can all be approached as the annealed minimizations of continuous B-spline parameters. The benefits of simulated annealing are well known, including an insensitivity to initial conditions and the ability to solve problems with many local minima. Discrete variable annealing has seen broad application, however continuous-variable annealing is limited by the computational challenge of Gibbs sampling. In this paper we develop efficient approaches to sampling, illustrated in the context of contour tracking in noisy images.

In this work, we consider the adaptive Wiener filtering of noisy images and image sequences. We begin by using an adaptive weighted averaging (AWA) approach to estimate the second-order statistics required by the Wiener filter. Experimentally, the resulting Wiener filter is improved by about 1 dB in the sense of peak-to-peak SNR (PSNR). Also, the subjective improvement is significant in that the annoying boundary noise, common with the traditional Wiener filter, has been greatly suppressed. The second, and more substantial, part of this paper extends the AWA concept to the wavelet domain. The proposed AWA wavelet Wiener filter is superior to the traditional wavelet Wiener filter by about 0.5 dB (PSNR). Furthermore, an interesting method to effectively combine the denoising results from both wavelet and spatial domains is shown and discussed. Our experimental results outperform or are comparable to state-of-art methods.

A new probabilistic image segmentation model based on hypothesis testing and Gibbs random fields is introduced. First, a probabilistic difference measure derived from a set of hypothesis tests is introduced. Next, a Gibbs/Markov random field model endowed with the new measure is then applied to the image segmentation problem to determine the segmented image directly through energy minimization. The Gibbs/Markov random fields approach permits us to construct a rigorous computational framework where local and regional constraints can be globally optimized. Results on grayscale and color images are encouraging.

Among existing ocean data assimilation methodologies, reduced-state Kalman filters are a widely studied compromise between resolution, optimality, error specification, and computational feasibility. In such reduced-state filters, the measurement update takes place on a coarser grid than that of the general circulation model (GCM); therefore, these filters require mapping operators from the GCM grid to the reduced state and vice versa. The general requirements are that the state-reduction and interpolation operators be pseudoinverses of each other, that the coarse state define a closed dynamical system, that the mapping operations be insensitive to noise, and that they be appropriate for regions with irregular coastlines and bathymetry. In this paper, we describe three efficient algorithms for computing the pseudoinverse: a fast Fourier transform algorithm that serves for illustration purposes, an exact implicit method that is recommended for most applications, and an efficient iterative algorithm that can be used for the largest problems. The mapping performance of 11 interpolation kernels is evaluated. Surprisingly, common kernels such as bilinear, exponential, Gaussian, and sinc perform only moderately well. We recommend instead three kernels, smooth, thin-plate, and optimal interpolation, which have superior properties. This study removes the computational bottleneck of mapping and pseudoinverse algorithms and makes possible the application of reduced-state filters to global problems at state-of-the-art resolutions.

The paper investigates the statistical characterization of signals and images in the wavelet domain. In particular, in contrast to common decorrelated-coefficient models, we find that the correlation between wavelet scales can be surprisingly substantial, even across several scales. We investigate possible choices of statistical-interaction models. One efficient and fast strategy which describes the wavelet-based statistical correlations is illustrated. Finally, the effectiveness of the proposed tool towards an efficient hierarchical MRF (Markov random field) modeling of within-scale neighborhoods and across-scale dependencies is demonstrated.

A new framework for color image segmentation is introduced generalizing the concepts of point-based and spatially-based methods. This framework is based on Markov random fields using a continuous Gibbs sampler. The Markov random fields approach allows for a rigorous computational framework where local and global spatial constraints can be globally optimized. Using a continuous Gibbs sampler enables the algorithm to adapt continuous-valued regional prototypes in a manner analogous to vector quantization while the discrete Gibbs sampler is used to adjust region boundaries.

Among existing ocean data assimilation methodologies, reduced-state Kalman filters are a widely-studied compromise between resolution, optimality, error specification, and computational feasibility. In such reduced-state filters, the measurement update takes place on a coarser grid than that of the general circulation model (GCM); therefore, these filters require mapping operators from the GCM grid to the reduced state and vice-versa. The general requirements are that the state-reduction and interpolation operators be pseudo-inverses of each other, that the coarse state defines a closed dynamical system, that the mapping operations be insensitive to noise, and that they be appropriate for regions with irregular coastlines and bathymetry. In this paper we investigate a variety of approaches, including computing the pseudoinverse by brute force, using the FFT, subsampling methods, implicit methods, and finally develop a novel iterative approach. We also evaluate the mapping performance of eleven interpolation kernels; surprisingly, common kernels such as bilinear, exponential, Gaussian, and sinc, performed only moderately well. This comprehensive study greatly reduces the computational bottleneck and guesswork of pseudo-inverse algorithms, making possible the application of reduced-state filters to global problems at state-of-the-art resolution.

Dynamic estimation of large-scale remote-sensing image sequences is important in a variety of scientific applications. However, the growing size of such sensed images makes conventional dynamic estimation methods, for example the Kalman and related filters, impractical. In this paper we present an approach that emulates the Kalman filter, but with considerably reduced computational and storage requirements. Our approach is illustrated in the context of a large (512/spl times/512) image sequence of ocean surface temperature.

This paper presents an approach to the fusion of SSM/I (Special Sensor Microwave/Imager) data from different resolutions, based on the prior statistical information about the data. The result is an estimated field that lives in a finer scale than any of the measurements. We apply a Wavelet Transform that increases speed and decreases memory requirements by sparsifying and preconditioning the statistics. This approach makes feasible the use of reprogrammable FPGA implementations for onboard satellite data processing, which greatly enhances flexibility and, most importantly, reduces communication burdens by limiting the extent to which raw, unprocessed data are transmitted to the ground.

The well-studied interferometric synthetic aperture radar (InSAR) problem for digital elevation map generation involves the derivation of topography from the radar phase. The topography is a function of the full phase, whereas the measured phase is known modulo 2/spl pi/, necessitating the process of recovering full phase values via phase unwrapping. This mathematical process becomes difficult through the presence of noise and phase discontinuities. This paper is motivated by research which models phase unwrapping as a network-flow minimization problem. A major limitation is that often a substantial computational effort is required to find solutions. Commonly, these phase images are huge (/spl Gt/10 million pixels), and obviously the sheer size of the problem itself makes phase unwrapping challenging. This paper addresses the development of a computationally efficient hierarchical algorithm, based on a "divide-and-conquer" approach. We have shown that the phase-unwrapping problem can first be partitioned into independent phase-unwrapping subproblems, which can further be recombined to produce the unwrapped phase. Interestingly, the recombination step itself can be interpreted as an unwrapping problem, for which a modified network-flow solution applies! In short, this paper develops a parallelization of the network-flow algorithm, allowing images of virtually unlimited size to be unwrapped and leading to dramatic decreases in the algorithm execution time.

Introduces a statistical speech production model. The model synthesizes natural speech by modeling some key dynamic properties of vocal articulators in a linear/nonlinear state-space framework. The goal-oriented movements of the articulators (tongue tip, tongue dorsum, upper lip, lower lip, and jaw) are described in a linear dynamic state equation. The resulting articulatory trajectories, combined with the effects of the velum and larynx, are nonlinearly mapped into the acoustic feature space (MFCCs). The key challenges in this model are the development of a nonlinear parameter estimation methodology, and the incorporation of appropriate prior assumptions to assert in the articulatory dynamic structure. Such a model can also be directly applied to speech recognition to better account for coarticulation and phonetic reduction phenomena with considerably fewer parameters than HMM based approaches.

This paper investigates the statistical characterization of multiscale wavelet coefficients corresponding to random signals and images. Virtually all approaches to wavelet shrinkage model the wavelet coefficients as independent; we challenge that assumption and demonstrate several cases where substantial correlations may be present in the wavelet domain. In particular, the correlation between scales can be surprisingly substantial, even for pixels separated by several scales. Our goal is to develop an efficient random field model describing these statistical correlations, and demonstrate its effectiveness in the context of Bayesian wavelet shrinkage for signal and image denoising.

Divide-and-conquer or multiscale techniques have become popular for solving large statistical estimation problems. The methods rely on defining a state which conditionally decorrelates the large problem into multiple subproblems, each more straightforward than the original. However this step cannot be carried out for asymptotically large problems since the dimension of the state grows without bound, leading to problems of computational complexity and numerical stability. In this paper, we propose a new approach to hierarchical estimation in which the conditional decorrelation of arbitrarily large regions is avoided, and the problem is instead addressed piece-by-piece. The approach possesses promising attributes: it is not a local method-the estimate at every point is based on all measurements; it is numerically stable for problems of arbitrary size; and the approach retains the benefits of the multiscale framework on which it is based: a broad class of statistical models, a stochastic realization theory, an algorithm to calculate statistical likelihoods, and the ability to fuse local and nonlocal measurements.

Contact lens wearers are often affected by a condition known as conjuctival hyperaemia. This condition is characterised by dilated blood vessels visible in the sclera (the white portion of the eye). Optometrists need to be able to detect changes in the level of redness and to repeatably assess the absolute levels of redness found in a patient's eye in order to prescribe and evaluate treatment. The goal of this paper is to develop an automated method for grading the level of redness in human eyes. The selected features examine the average redness and length of arterial edges. The computed features are correlated with expert data collected to train the redness estimator.

Fast image retrieval is the key to success for operations on large image databases, and a great many techniques have been developed for efficient retrieval. However, most of these methods are tailored to visual scenes or to images having limited variations. We investigate the searching of enormous databases (of up to 10/sup 7/ images) for the matching and identification of precious stones (principally diamonds). Because of the size of the database, we propose a hierarchy of classifiers, which successively prune candidate images such that the more complex classifiers are required to test only tiny portions of the data. The new classifier developed here applies a wavelet transform to image histograms and is capable of rejecting 99.9% of bad matches.

The limitations of estimating structure from either stereo or motion alone can be addressed by the use of stereo image sequences; however, many existing techniques for processing stereo sequences rely on having accurate feature correspondences initially. We present an iterative feature matching and shape reconstruction algorithm. The proposed method simultaneously resolves matching ambiguities by multiple hypothesis testing and develops an incrementally accurate and dense representation of the reconstructed object. This approach uses minimal domain specific constraints and can be easily generalised. The method's potential is demonstrated in a space vision application.

The well-studied interferometric synthetic aperture radar (InSAR) problem for digital elevation map generation involves the derivation of topography from radar phase. The topography is a function of the full phase, whereas the measured phase is known module 2/spl pi/, necessitating the process of recovering full phase values via phase unwrapping. This mathematical process becomes difficult through the presence of noise and phase discontinuities. The authors' research is motivated by recent research that models phase unwrapping as a network flow minimization problem. The cost function to be optimized is a weighted L/sup 1/-norm of the phase discontinuities. Determining these cost weights is critical, yet past work in the literature does not reflect the statistics of the unwrapping problem. The purpose of this paper is to propose a new method to compute the flow weights from a theoretical foundation. Specifically, they formulate phase unwrapping as a maximum likelihood (ML) estimation problem, which they mathematically rewrite as a network flow problem with a specific choice of weights. The approach is based on estimating the probability of phase discontinuities, which can be derived as a function of coherence and topographic slope from the known statistical properties of SAR phase.

This paper addresses the problem of both segmenting and reconstructing a noisy signal or image. The work is motivated by large problems arising in certain scientific applications, such as medical imaging. Two objectives for a segmentation and denoising algorithm are laid out: it should be computationally efficient and capable of generating statistics for the errors in the reconstruction and estimates of the boundary locations. The starting point for the development of a suitable algorithm is a variational approach to segmentation (Shah 1992). This paper then develops a precise statistical interpretation of a one dimensional (1-D) version of this variational approach to segmentation. The 1-D algorithm that arises as a result of this analysis is computationally efficient and capable of generating error statistics. A straightforward extension of this algorithm to two dimensions would incorporate recursive procedures for computing estimates of inhomogeneous Gaussian Markov random fields. Such procedures require an unacceptably large number of operations. To meet the objective of developing a computationally efficient algorithm, the use of previously developed multiscale statistical methods is investigated. This results in the development of an algorithm for segmenting and denoising which is not only computationally efficient but also capable of generating error statistics, as desired.

The development of user interfaces based on vision and speech requires the solution of a challenging statistical inference problem: The intentions and actions of multiple individuals must be inferred from noisy and ambiguous data. We argue that Bayesian network models are an attractive statistical framework for cue fusion in these applications. Bayes nets combine a natural mechanism for expressing contextual information with efficient algorithms for learning and inference. We illustrate these points through the development of a Bayes net model for detecting when a user is speaking. The model combines four simple vision sensors: face detection, skin color, skin texture, and mouth motion. We present some promising experimental results.

Autoregressive (AR) models have been shown to be effective models of the human vocal tract during voicing. However the most common model of speech for enhancement purposes, AR process excited by white noise, fails to capture the periodic nature of voiced speech. Speech synthesis researchers have long recognized this problem and have developed a variety of sophisticated excitation models, however these models have yet to make an impact in speech enhancement. We have chosen one of the most common excitation models, the four-parameter LF model of Fant, Liljencrants and Lin (1985), and applied it to the enhancement of individual voiced phonemes. Comparing the performance of the conventional white-noise-driven AR, an impulsive-driven AR, and AR based on the LF model shows that the LF model yields a substantial improvement, on the order of 1.3 dB.

Efficient, large-scale estimation methods such as nested dissection or multiscale estimation rely on a divide-and-conquer strategy, in which a statistical problem is conditionally broken into smaller pieces. This conditional decorrelation is not possible for arbitrarily large problems due to issues of computational complexity and numerical stability. Given the growing interest in global-scale-remote sensing problems (or even three-dimensional problems), in this summary we develop a class of estimators with more promising asymptotic computational properties.

Closed circuit television (CCTV) surveys are used widely in North America to assess the structural integrity of underground pipes. The video images are examined visually and classified into grades according to degrees of damage. The human eye is extremely effective at recognition and classification, but it is not suitable for assessing pipe defects in thousands of miles of pipeline images due to fatigue, subjectivity and cost. This paper presents ongoing research into the automatic assessment of the structural condition of underground pipes for the purpose of preventive maintenance by municipalities.

We recently developed a multiscale recursive estimation procedure for the estimation of large-scale dynamic systems. The procedure propagates multiscale models for the estimation errors more efficiently than the Kalman filter's propagation of the error covariances, with a resulting computational complexity of 𝒪(N) and 𝒪(N3/2), where N is the number of variables estimated, for 1-D and 2-D dynamic systems, respectively. To further reduce the computational cost, we introduce in this paper a new class of reduced-order spatially-interpolated multiscale models, and demonstrate their use in remote.

This paper discusses efficient solutions to large-scale two-dimensional estimation problems, using reduced state methods motivated by the multipole method of mathematical physics. The work is mainly exploratory, building on past efforts in multiscale statistical signal modeling and estimation. We will illustrate applications to the estimation of Markov random field textures, with the motivation and goal the estimation of remotely-sensed fields.

Variational methods have been employed with considerable success in computer vision, particularly for surface reconstruction problems. Formulations of this type require the solution of computationally complex Euler–Lagrange partial differential equations (PDEs) to obtain the desired reconstructions. Further, the calculation of reconstruction error covariances for such approaches are usually neglected. In this paper we describe a computationally efficient multiscale approach to surface reconstruction which differs fundamentally from other multiresolution methods that are used to solve the Euler–Lagrange PDEs. Instead, we interpret the variational problem as a statistical estimation problem in order to define a nearby, but slightly different , multiscale estimation problem that admits efficient solutions for both surface reconstruction and the calculation of error statistics. In particular, the membrane and thin-plate variational models for surfaces are interpreted as 1/ f 2 prior statistical models for the surface and its gradients, respectively. Such 1/ f 2 behavior is then achieved using a recently introduced class of multiresolution models that admits algorithms with constant per-pixel computational complexity.

We develop a simple and very fast method for object tracking based exclusively on color information in digitized video images. Running on a Silicon Graphics R4600 Indy system with an IndyCam, our algorithm is capable of simultaneously tracking objects at full frame size (640/spl times/480 pixels) and video frame rate (30 fps). Robustness with respect to occlusion is achieved via can explicit hypothesis-tree model of the occlusion process. We demonstrate the efficacy of our technique in the challenging task of tracking people, especially tracking human heads and hands.

Recently, a class of multiscale stochastic models has been introduced in which random processes and fields are described by scale-recursive dynamic trees. A major advantage of this framework is that it leads to an extremely efficient, statistically optimal algorithm for least-squares estimation. In certain applications, however, estimates based on the types of multiscale models previously proposed may not be adequate, as they have tended to exhibit a visually distracting blockiness. We eliminate this blockiness by discarding the standard assumption that distinct nodes on a given level of the multiscale process correspond to disjoint portions of the image domain; instead, we allow a correspondence to overlapping portions of the image domain. We use these so-called overlapping-tree models for both modeling and estimation. In particular, we develop an efficient multiscale algorithm for generating sample paths of a random field whose second-order statistics match a prespecified covariance structure, to any desired degree of fidelity. Furthermore, we demonstrate that under easily satisfied conditions, we can "lift" a random field estimation problem to one defined on an overlapped tree, resulting in an estimation algorithm that is computationally efficient, directly produces estimation error covariances, and eliminates blockiness in the reconstructed imagery without any sacrifice in the resolution of fine-scale detail.

The successful application of a previously developed tree-based multiscale estimation framework to large-scale static estimation problems has proven the statistical flexibility and the computational efficiency of the framework. However, the estimation of processes evolving in time remains a considerable challenge. We address this challenge by investigating multiscale models for the steady-state estimation of dynamic systems. In particular, we build such models for 1-D and 2-D heat diffusion processes, making use of a canonical correlations realization technique. The problem of the assimilation of satellite altimetric measurements of the dynamic ocean surface over time is also addressed.

This work presents a method for segmenting images based on gradients in the intensity function. Past approaches have centered on formulating the problem in the context of variational calculus as the minimization of a functional involving the image intensity and edge functions. Computational methods for finding the minima of such variational problems are prone to two shortfalls: they are often computationally intensive and almost always incapable of computing error statistics associated with the segmentation. Using a particular variational formulation as a starting point, this paper presents a derivation of an associated statistical formulation using multiscale models. The result is an algorithm which is fast and capable of computing error statistics.

A fast multiscale optimal interpolation algorithm has been adapted to the mapping of hydrographic and other oceanographic data. This multiscale algorithm produces solution and error estimates which are consistent with those obtained from exact least-squares methods, but at a small fraction of the computational cost. Problems whose solution would be completely impractical using exact least-squares, that is problems with tens or hundreds of thousands of measurements and estimation grid points, can easily be solved on a small workstation using the multiscale algorithm. Contrary to methods previously proposed for solving large least-squares problems, the multiscale approach provides error statistics while permitting long-range correlations, using all measurements, and permitting arbitrary measurement locations.

A significant problem in oceanographic remote sensing is the dense gridding or smoothing of sparsely sampled altimetric data. The smoothing of altimetric measurements has an application much broader than just the regular production of elevation maps for oceanographers, however. In particular, the ability to estimate ocean circulation patterns from altimetric data can serve as an important measure for the verification of ocean acoustic tomographic results. The authors present a multiscale technique capable of extremely efficient interpolation of altimetric data: about 256000 estimates and estimation error variances are computed in one minute on a Sun Sparc-10. They also demonstrate how similar techniques may be used to directly estimate the surface gradient and biases in the geoid-model error.

A class of multiscale stochastic models has been introduced in which Gaussian random processes are described by scale-recursive dynamics that are indexed by the nodes of a tree. One of the primary reasons the framework is useful is that it leads to an extremely fast, statistically optimal algorithm for least-squares estimation in the context of 2-D images. We refine this approach to estimation by eliminating the visually distracting blockiness that has been observed in the previous work. We eliminate the blockiness by discarding the standard assumption that distinct nodes at a given level of our tree correspond to disjoint portions of the image domain; as a consequence of this simple idea, a given image pixel may now correspond to several tree nodes. We develop tools for systematically building overlapping-tree multiscale representations of prespecified statistics, and we develop a corresponding estimation algorithm for this processes. In this way, we achieve nearly optimal estimation results, we generate corresponding error covariance information, and we eliminate blockiness without sacrificing the resolution of fine-scale detail.

A recently developed multiresolution estimation framework offers the possibility of highly efficient statistical analysis, interpolation, and smoothing of extremely large data sets in a multiscale fashion. This framework enjoys a number of advantages not shared by other statistically-based methods. In particular, the algorithms resulting from this framework have complexity that scales only linearly with problem size, yielding constant complexity load per grid point independent of problem size. Furthermore these algorithms directly provide interpolated estimates at multiple resolutions, accompanying error variance statistics of use in assessing resolutionlaccuracy tradeoffs and in detecting statistically significant anomalies, and maximum likelihood estimates of parameters such as spectral power law coefficients. Moreover, the efficiency of these algorithms is completely insensitive to irregularities in the sampling or spatial distribution of measurements and to heterogeneities in measurement errors or model parameters. For these reasons this approach has the potential of being an effective tool in a variety of remote sensing problems. In this paper, we demonstrate a realization of this potential by applying the multiresolution framework to a problem of considerable current interest-the interpolation and statistical analysis of ocean surface data from the TOPEXPOSEIDON altimeter

Large data assimilation problems present a number of challenges, many of which the authors' multiscale estimation framework is capable of addressing. They demonstrate the application of their estimation framework to an oceanographic data assimilation problem of considerable interest: the smoothing of Topex/Poseidon ocean altimetric data. they produce surface estimates and error statistics on a 512/spl times/512 grid in one minute on a Sparc-10. Using a generic surface reconstruction problem they demonstrate alternative uses of the multiscale framework: higher order state models, combining estimates from an ensemble of trees, and overlapping tree models.>