Classification of different object surface material types can play a significant role in the decision-making algorithms for mobile robots and autonomous vehicles. RGB-based scene-level semantic segmentation has been well-addressed in the literature. However, improving material recognition using the depth modality and its integration with SLAM algorithms for 3D semantic mapping could unlock new potential benefits in the robotics perception pipeline. To this end, we propose a complementarity-aware deep learning approach for RGB-D-based material classification built on top of an object-oriented pipeline. The approach further integrates the ORB-SLAM2 method for 3D scene mapping with multiscale clustering of the detected material semantics in the point cloud map generated by the visual SLAM algorithm. Extensive experimental results with existing public datasets and newly contributed real-world robot datasets demonstrate a significant improvement in material classification and 3D clustering accuracy compared to state-of-the-art approaches for 3D semantic scene mapping.

Coral reefs are biologically diverse and structurally complex ecosystems that have been severely affected by natural and anthropogenic stressors. Consequently, there is a need for rapid and accurate ecological assessment of coral reefs, but current approaches entail time-consuming manual data acquisition and analysis. We propose a scheme to identify and localize individual entities within the coral reef ecosystem as distinct 3D objects and assess its performance. Given 2D region proposals in an RGB image, our method generates, for each 2D region proposal, a 3D region proposal based on an existing annotated 3D reef reconstruction and the intrinsic and extrinsic camera parameters associated with the RGB image. The annotated 3D reef reconstruction is generated using a commercial Structure-from-Motion (SfM) software and a previously designed multiview convolutional neural network (CNN) for 3D semantic segmentation. As individual coral reef entities are often viewed in multiple images, a 3D bounding-box merging strategy coupled with an overlap criterion are used to combine multiple 3D region proposals into a single 3D object prediction for the purpose of classification and localization. Experimental results and comparison with the Frustum PointNet architecture show the efficacy of the proposed scheme on coral reef survey images.

Structure from Motion (SfM) techniques are increasingly being used to create 3D maps from images in many domains including environmental monitoring. However, SfM techniques are often confounded in visually repetitive environments as they rely primarily on globally distinct image features. Simultaneous Localization and Mapping (SLAM) techniques offer a potential solution in visually repetitive environments since they use local feature matching; however, SLAM approaches work best with wide-angle cameras that are often unsuitable for documenting the environmental system of interest. We resolve this issue by proposing a dual-camera SLAM approach that uses a forward facing wide-angle camera for localization and a downward facing narrower-angle, high-resolution camera for documentation. Video frames acquired by the forward facing camera are processed using a standard SLAM approach providing a trajectory of the imaging system through the environment which is then used to guide registration of the documentation camera images. Fragmentary maps, initially produced from the documentation camera images via monocular SLAM, are subsequently scaled and aligned with the localization camera trajectory and finally processed using a global optimization procedure to produce a unified, refined map. An experimental comparison with several state-of-the-art SfM approaches shows the dual-camera SLAM approach to perform better in repetitive environmental systems based on select samples of ground control point markers.

Recent advances in computer vision and machine learning, most notably deep convolutional neural networks (CNNs), are exploited to identify and localize various plant species in salt marsh images. Three different approaches are explored that provide estimations of abundance and spatial distribution at varying levels of granularity defined by spatial resolution. In the coarsest-grained approach, CNNs are tasked with identifying which of six plant species are present/absent in large patches within the salt marsh images. CNNs with diverse topological properties and attention mechanisms are shown capable of providing accurate estimations with > 90 % precision and recall for the more abundant plant species and reduced performance for less common plant species. Estimation of percent cover of each plant species is performed at a finer spatial resolution, where smaller image patches are extracted and the CNNs tasked with identifying the plant species or substrate at the center of the image patch. For the percent cover estimation task, the CNNs are observed to exhibit a performance profile similar to that for the presence/absence estimation task, but with an ≈ 5%-10% reduction in precision and recall. Finally, fine-grained estimation of the spatial distribution of the various plant species is performed via semantic segmentation. The DeepLab-V3 semantic segmentation architecture is observed to provide very accurate estimations for abundant plant species, but with significant performance degradation for less abundant plant species; in extreme cases, rare plant classes are seen to be ignored entirely. Overall, a clear trade-off is observed between the CNN estimation quality and the spatial resolution of the underlying estimation thereby offering guidance for ecological applications of CNN-based approaches to automated plant identification and localization in salt marsh images.

MOTIVATION: Time-series nuclear magnetic resonance (NMR) has advanced our knowledge about metabolic dynamics. Before analyzing compounds through modeling or statistical methods, chemical features need to be tracked and quantified. However, because of peak overlap and peak shifting, the available protocols are time consuming at best or even impossible for some regions in NMR spectra. RESULTS: We introduce Ridge Tracking-based Extract (RTExtract), a computer vision-based algorithm, to quantify time-series NMR spectra. The NMR spectra of multiple time points were formulated as a 3D surface. Candidate points were first filtered using local curvature and optima, then connected into ridges by a greedy algorithm. Interactive steps were implemented to refine results. Among 173 simulated ridges, 115 can be tracked (RMSD < 0.001). For reproducing previous results, RTExtract took less than 2 h instead of ∼48 h, and two instead of seven parameters need tuning. Multiple regions with overlapping and changing chemical shifts are accurately tracked. AVAILABILITY AND IMPLEMENTATION: Source code is freely available within Metabolomics toolbox GitHub repository (https://github.com/artedison/Edison_Lab_Shared_Metabolomics_UGA/tree/master/metabolomics_toolbox/code/ridge_tracking) and is implemented in MATLAB and R. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

An end-to-end trainable ConvNet architecture, that learns to harness the power of shape representation for matching disparate image pairs, is proposed. Disparate image pairs are deemed those that exhibit strong affine variations in scale, viewpoint and projection parameters accompanied by the presence of partial or complete occlusion of objects and extreme variations in ambient illumination. Under these challenging conditions, neither local nor global feature-based image matching methods, when used in isolation, have been observed to be effective. The proposed correspondence determination scheme for matching disparate images exploits high-level shape cues that are derived from low-level local feature descriptors, thus combining the best of both worlds. A graph-based representation for the disparate image pair is generated by constructing an affnity matrix that embeds the distances between feature points in two images, thus modeling the correspondence determination problem as one of graph matching. The eigen-spectrum of the affnity matrix, i.e., the learned global shape representation, is then used to further regress the transformation or homography that defnes the correspondence between the source image and target image. The proposed scheme is shown to yield state-of-the-art results for both, coarse-level shape matching as well as fine point-wise correspondence determination.

Cyanobacterial Harmful Algal Blooms (CyanoHABs) are a major water quality and public health issue in inland waters as they hamper recreational activities, degrade aquatic habitats, and potentially affect human health via toxic contamination. Despite the risks posed to environment, human and animal health, currently, there is lack of rapid monitoring program to periodically evaluate the spatial distribution of cyanobacteria in inland waters. This study integrated multiple clouds including community cloud (via social media data), sensor cloud (wireless hyperspectral sensor and satellite sensor) and computational cloud to design and implement techniques for early detection of CyanoHABs in inland waters. Social cloud data helped to identify the geographical locations frequently affected by CyanoHABs and sensor clouds helped in verifying those locations. This integrated monitoring system would be very useful for lake resource managements and state agencies by reducing their budget cost for rapid detection and frequent monitoring of CyanoHABs across inland waters.

Effective and cost-efficient monitoring is indispensable for ensuring environmental sustainability. Cyanobacterial Harmful Algal Blooms (CyanoHABs) are a major water quality and public health issue in inland water bodies. The recent popularity of online social media (OSM) platforms coupled with advances in cloud computing and data analytics has given rise to citizen science-based approaches to environmental monitoring. These approaches involve the lay community in the acquisition, collection and transmission of relevant data in the form of tweets, images, voice recordings and videos typically acquired using low-cost mobile devices such as smartphones or tablet computers. While cost effective, citizen science-based approaches are highly susceptible to noise, inaccuracies and missing data. In this paper we address the problem of automated detection of harmful algal blooms (HABs) via analysis of image data of inland water bodies. These image data are acquired using a variety of smartphones and communicated via popular OSM platforms such as Facebook, Twitter and Instagram. To account for the wide variations in imaging parameters and ambient environmental parameters we propose a deep learning approach to image feature extraction and classification for the purpose of HAB detection. The current system is a first step in the design of an automated early detection, warning and rapid response system that can be adopted to mitigate the detrimental effects of CyanoHAB contamination of inland water bodies.

From scientific research to commercial applications, eye tracking is an important tool across many domains. Despite its range of applications, eye tracking has yet to become a pervasive technology. We believe that we can put the power of eye tracking in everyone's palm by building eye tracking software that works on commodity hardware such as mobile phones and tablets, without the need for additional sensors or devices. We tackle this problem by introducing GazeCapture, the first large-scale dataset for eye tracking, containing data from over 1450 people consisting of almost 2:5M frames. Using GazeCapture, we train iTracker, a convolutional neural network for eye tracking, which achieves a significant reduction in error over previous approaches while running in real time (10-15fps) on a modern mobile device. Our model achieves a prediction error of 1.71cm and 2.53cm without calibration on mobile phones and tablets respectively. With calibration, this is reduced to 1.34cm and 2.12cm. Further, we demonstrate that the features learned by iTracker generalize well to other datasets, achieving state-of-the-art results. The code, data, and models are available at http://gazecapture.csail.mit.edu.

A comprehensive framework for detection and characterization of partial intrinsic symmetry over 3D shapes is proposed. To identify prominent symmetric regions which overlap in space and vary in form, the proposed framework is decoupled into a Correspondence Space Voting (CSV) procedure followed by a Transformation Space Mapping (TSM) procedure. In the CSV procedure, significant symmetries are first detected by identifying surface point pairs on the input shape that exhibit local similarity in terms of their intrinsic geometry while simultaneously maintaining an intrinsic distance structure at a global level. To allow detection of potentially overlapping symmetric shape regions, a global intrinsic distance-based voting scheme is employed to ensure the inclusion of only those point pairs that exhibit significant intrinsic symmetry. In the TSM procedure, the Functional Map framework is employed to generate the final map of symmetries between point pairs. The TSM procedure ensures the retrieval of the underlying dense correspondence map throughout the 3D shape that follows a particular symmetry. The TSM procedure is also shown to result in the formulation of a metric symmetry space where each point in the space represents a specific symmetry transformation and the distance between points represents the complexity between the corresponding transformations. Experimental results show that the proposed framework can successfully analyze complex 3D shapes that possess rich symmetries.

Level set-based contour tracking methods have generated recent interest in the computer vision community. In this paper, we propose a novel level set-based algorithm for tracking dynamic implicit contours that utilizes minimal prior information. Our solution consists of two main steps. In the first step, a simple first-order Markov chain model is employed for the coarse localization of a target object. In the second step, we evolve level sets within a narrow band to accurately track the target contour. Narrow band curve evolution is guided through color- and region-based terms in the standard Chan-Vese framework. Comprehensive experimentation on a dataset comprising of several publicly available video sequences clearly demonstrate the advantage of the proposed tracking algorithm.

A novel multi-criteria optimization framework for matching of partially visible shapes in multiple images using joint geometric graph embedding is proposed. The proposed framework achieves matching of partial shapes in images that exhibit extreme variations in scale, orientation, viewpoint and illumination and also instances of occlusion; conditions which render impractical the use of global contour-based descriptors or local pixel-level features for shape matching. The proposed technique is based on optimization of the embedding distances of geometric features obtained from the eigenspectrum of the joint image graph, coupled with regularization over values of the mean pixel intensity or histogram of oriented gradients. It is shown to obtain successfully the correspondences denoting partial shape similarities as well as correspondences between feature points in the images. A new benchmark dataset is proposed which contains disparate image pairs with extremely challenging variations in viewing conditions when compared to an existing dataset [18]. The proposed technique is shown to significantly outperform several state-of-the-art partial shape matching techniques on both datasets.

The limitations of conventional imaging techniques have hitherto precluded a thorough and formal investigation of the complex morphology of the left ventricular (LV) endocardial surface and its relation to the severity of coronary artery disease (CAD). However, recent developments in high-resolution multirow-detector computed tomography (MDCT) scanner technology have enabled the imaging of the complex LV endocardial surface morphology in a single heartbeat. Analysis of high-resolution computed tomography images from a 320-MDCT scanner allows for the noninvasive study of the relationship between the percent diameter stenosis (DS) values of the major coronary arteries and localization of the cardiac segments affected by coronary arterial stenosis. In this paper, a novel approach for the analysis of the nonrigid LV endocardial surface from MDCT images, using a combination of rigid body transformation-invariant shape descriptors and a more generalized isometry-invariant Bag-of-Features descriptor, is proposed and implemented. The proposed approach is shown to be successful in identifying, localizing, and quantifying the incidence and extent of CAD and, thus, is seen to have a potentially significant clinical impact. Specifically, the association between the incidence and extent of CAD, determined via the percent DS measurements of the major coronary arteries, and the alterations in the endocardial surface morphology is formally quantified. The results of the proposed approach on 16 normal datasets and 16 abnormal datasets exhibiting CAD with varying levels of severity are presented. A multivariable regression test is employed to test the effectiveness of the proposed morphological analysis approach. Experiments performed on a strictly leave-one-out basis are shown to exhibit a distinct and interesting pattern in terms of the correlation coefficient values within the cardiac segments, where the incidence of coronary arterial stenosis is localized.

A novel intrinsic geometric scale space formulation for 3D deformable surfaces termed as the Biharmonic Density Estimate (BDE) is proposed. The proposed BDE signature allows for multiscale surface feature-based representation of deformable 3D shapes for subsequent image and scene analysis. It is shown to provide an underlying theoretical framework for the concept of intrinsic geometric scale space, resulting in a highly descriptive characterization of both, the local surface structure and the global metric of the 3D shape. The compactness and robustness of the proposed BDE signature are demonstrated via a series of experiments and a key components detection application.

The ever increasing deployment of broadband networks and simultaneous proliferation of low cost video capturing and multimedia enabled mobile devices have triggered a wave of novel mobile multimedia applications, resulting in the development of large scale systems for delivery of video streams to heterogeneous resource constrained mobile clients. Invariably, the video streams need to be personalized to provide a resource constrained mobile device with video content that is most relevant to the client's request while simultaneously satisfying the client-side and system-wide resource constraints. In this paper we present the design and implementation of a distributed system, consisting of several geographically distributed video personalization servers and proxy caches, for efficient dissemination of personalized video in a resource constrained mobile environment. With the objective of optimizing cache performance, a novel cache replacement policy and multi-stage client request aggregation strategy, both of which are specifically tailored for personalized video content, are proposed. A novel latency-biased collaborative caching protocol based on counting Bloom filters is designed for further enhancing the scalability and efficiency of disseminating personalized video content. The benefits and costs associated with collaborative caching for disseminating personalized video content to resource constrained and geographically distributed clients are analyzed and experimentally verified. The impact of different levels of collaboration amongst the caches and, the advantages of using multiple video personalization servers with varying degrees of mirrored content on the efficiency of personalized video delivery, are also studied. Experimental results demonstrate that the proposed collaborative caching scheme, coupled with the proposed personalization-aware cache replacement and client request aggregation strategies, provides a means for efficient dissemination of personalized video streams in resource constrained environments.

Video streaming on mobile devices such as PDA's, laptop PCs, pocket PCs and cell phones is becoming increasingly popular. These mobile devices are typically constrained by their battery capacity, bandwidth, screen resolution and video decoding and rendering capabilities. Consequently, video personal

The design and implementation of a client-centered multimedia content adaptation system suitable for a mobile environment comprising of resource-constrained handheld devices or clients is described. The primary contributions of this work are: (1) the overall architecture of the client-centered content adaptation system, (2) a data-driven multi-level Hidden Markov model (HMM)-based approach to perform both video segmentation and video indexing in a single pass, and (3) the formulation and implementation of a Multiple-choice Multidimensional Knapsack Problem (MMKP)-based video personalization strategy. In order to segment and index video data, a video stream is modeled at both the semantic unit level and video program level. These models are learned entirely from training data and no domain-dependent knowledge about the structure of video programs is used. This makes the system capable of handling various kinds of videos without having to manually redefine the program model. The proposed MMKP-based personalization strategy is shown to include more relevant video content in response to the client's request than the existing 0/1 knapsack problem and fractional knapsack problem-based strategies, and is capable of satisfying multiple client-side constraints simultaneously. Experimental results on CNN news videos and Major League Soccer (MLS) videos are presented and analyzed.

An automated system for planning and optimization of lumber production using Machine Vision and Computed Tomography (CT) is proposed. Cross-sectional CT images of hardwood logs are analyzed using machine vision algorithms. Internal defects in the hardwood logs pockets are identified and localized. A virtual in silico 3-D reconstruction of the hardwood log and its internal defects is generated using Kalman filter-based tracking algorithms. Various sawing operations are simulated on the virtual 3-D reconstruction of the log and the resulting virtual lumber products automatically graded using rules stipulated by the National Hardwood Lumber Association (NHLA). Knowledge of the internal log defects is suitably exploited to formulate sawing strategies that optimize the value yield recovery of the resulting lumber products. A prototype implementation shows significant gains in value yield recovery when compared with lumber processing strategies that use only the information derived from the external log structure.

Semantic video indexing is the first step towards automatic video retrieval and personalization. We propose a data-driven stochastic modeling approach to perform both video segmentation and video indexing in a single pass. Compared with the existing hidden Markov model (HMM)-based video segmentation and indexing techniques, the advantages of the proposed approach are as follows: (1) the probabilistic grammar defining the video program is generated entirely from the training data allowing the proposed approach to handle various kinds of videos without having to manually redefine the program model; (2) the proposed use of the Tamura features improves the accuracy of temporal segmentation and indexing; (3) the need to use an HMM to model the video edit effects is obviated thus simplifying the processing and collection of training data and ensuring that all video segments in the database are labeled with concepts that have clear semantic meanings in order to facilitate semantics-based video retrieval. Experimental results on broadcast news video are presented.

Digital video playback on mobile devices is fast becoming widespread and popular. Since mobile devices are typically resource constrained in terms of network bandwidth, battery power and available screen resolution, it is often necessary to formulate special encoding techniques in order to optimize power consumption during video streaming and playback. The existing H.264 standard is popular for video encoding on mobile devices, since it results in a low-bitrate video with visual clarity that is adequate for video playback on mobile devices. However, due to the complexity of the H.264 representation, the video decoding procedure is typically computationally intensive. In this paper, we propose a novel lossy video representation termed as Ligne-Claire (LC) video. LC videos are obtained via graphics overlay of outlines or silhouettes of objects in the video over an approximated texture video. Since the playback of LC video is typically meant for mobile devices, the visual quality of video is adequate for most mobile applications wherein the semantic content of the video can be characterized by object shapes and approximate texture information. Experimental results presented in the paper demonstrate that the proposed lossy LC video encoding scheme results in power savings of 50% or more during video playback compared to standard H.264-encoded videos, of similar video file size. In order to evaluate the visual quality of the LC video, we compare the performance of LC videos with H.264-encoded videos in the context of some typical computer vision tasks. Our results indicate that the performance of the computer vision algorithms on these videos is similar. This fact, coupled with subjective evaluation, and the resulting significant power savings, indicates that the proposed LC representation can be used effectively to encode video for power-constrained mobile devices.

Progressive download of multimedia objects over the Internet (e.g. www.youtube.com), where the video is downloaded and viewed during the download process, has become an increasingly popular alternative to multimedia streaming. Due to the fluctuating bandwidth and latency of the Internet, progressive download is often not fast enough, often resulting in intermittent stalling of the video. In this paper, we first propose a variation of the existing MPEG Fine Grained Scalability (FGS) profile to create a layered video representation that is suitable for progressive download in an environment characterized by varying bitrate. We also propose an efficient caching scheme that is specifically tailored for the proposed layered video representation. The proposed layered version of the Greedy-Dual-Size cache replacement policy is shown to reduce the latency observed by the client during progressive download of video in a varying bitrate environment. Experimental results demonstrate that the proposed caching scheme improves the latency of progressive video downloads as well as the server efficiency.

Multimedia data, especially video data, is being increasingly transmitted to, transmitted from and viewed on mobile devices such as PDA's, laptop PCs, pocket PCs and cell phones. One of the natural limitations of these multimedia-capable, mobile devices is that they are constrained by their battery power capacity, viewing time limit, amount of data received, and in many situations, by available network bandwidth connecting these devices with video servers. The video server is typically also constrained by its computing power and connection bandwidth. In order to provide a resource-constrained mobile client with its desired video content, it is necessary to adapt or personalize the video content while simultaneously satisfying the aforementioned constraints. Also, in order to limit the client-experienced latency, it is necessary to perform client request aggregation on the server end. To this end, a video personalization strategy is proposed to provide mobile, resource-constrained clients with personalized video content that is most relevant to the client's request while simultaneously satisfying multiple client-side system-level resource constraints. A client request aggregation strategy is also proposed to cluster client requests with similar video content preferences and similar client-side resource constraints such that the number of requests the server needs to process and the client-experienced latency are both reduced.

Dynamic wettability of a nanostructured surface is an important property for many liquid-related applications of nanostructures. The dynamic wettability analysis is performed by measuring the evolution of the precursor (outer rim) contour of a water droplet as it spreads on a nanostructured surface. A video-based metrological system based on the snake active contour model which is capable of precisely tracking the precursor contour of a spreading water droplet in a high frame-rate video is developed. The radius of the precursor contour is empirically observed to obey a power law with respect to time. Experiments show reasonable agreement between the results of the metrological system and those obtained via manual measurement

Detection of hairline fractures, representing points or areas of discontinuity in the bone, is a clinically challenging task, especially in presence of noise. The above problem is equally appealing from a computer vision or pattern recognition perspective since (a) traditional techniques for detection of corners, denoting points of surface discontinuity, typically fail in such cases and, (b) one needs to implicitly handle unknown local degradation in the image. A novel two-phase scheme for hairline mandibular fracture detection, that is robust to noise, is proposed. In the first phase, the hairline fractures are coarsely localized using statistical correlation and by exploiting the bilateral symmetry of the human mandible. In the second phase, the fractures are precisely identified and highlighted using a Markov random field (MRF) modeling approach coupled with maximum a posteriori probability (MAP) estimation. Gibbs sampling is used to maximize the posterior probability. Experimental results on computer tomography (CT) scans from real patients are presented

MPEG-4 body animation parameters (BAP) are used for animation of MPEG-4 compliant virtual human-like characters. Distributed virtual reality applications and networked games on mobile computers require access to locally stored or streamed compressed BAP data. Existing MPEG-4 BAP compression techniques are inefficient for streaming, or storing, BAP data on mobile computers, because: 1) MPEG-4 compressed BAP data entails a significant number of CPU cycles, hence significant, unacceptable power consumption, for the purpose of decompression, 2) the lossy MPEG-4 technique of frame dropping to reduce network throughput during streaming leads to unacceptable animation degradation, and 3) lossy MPEG-4 compression does not exploit structural information in the virtual human model. In this article, we propose two novel algorithms for lossy compression of BAP data, termed as BAP-Indexing and BAP-Sparsing. We demonstrate how an efficient combination of the two algorithms results in a lower network bandwidth requirement and reduced power for data decompression at the client end when compared to MPEG-4 compression. The algorithm exploits the structural information in the virtual human model, thus maintaining visually acceptable quality of the resulting animation upon decompression. Consequently, the hybrid algorithm for BAP data compression is ideal for streaming of motion animation data to power- and network-constrained mobile computers

Human Motion Capture (MoCap) data can be used for animation of virtual human-like characters in distributed virtual reality applications and networked games. MoCap data compressed using the standard MPEG-4 encoding pipeline comprising of predictive encoding (and/or DCT decorrelation), quantization, and arithmetic/Huffman encoding, entails significant power consumption for the purpose of decompression. In this paper, we propose a novel algorithm for compression of MoCap data, which is based on smart indexing of the MoCap data by exploiting structural information derived from the skeletal virtual human model. The indexing algorithm can be fine-controlled using three predefined quality control parameters (QCPs). We demonstrate how an efficient combination of the three QCPs results in a lower network bandwidth requirement and reduced power consumption for data decompression at the client end when compared to standard MPEG-4 compression. Since the proposed algorithm exploits structural information derived from the skeletal virtual human model, it is observed to result in virtual human animation of visually acceptable quality upon decompression.

Background modeling for dynamic scenes is an important problem in the context of real time video surveillance systems. Several nonparametric background models have been proposed to model dynamic scenes and promising results have been reported. However, a critical problem with existing nonparametric models is their high computational requirement because a large set of background samples is usually needed to model the background. In this paper, a nonparametric background model that uses an importance sampling method is proposed to overcome the problem of high computational complexity of conventional nonparametric background models. Instead of using a large number of samples to model the background probability densities, much fewer background samples are maintained and updated using the CONDENSATION algorithm. A Markov Random Field model is used to enhance the foreground detection results by imposing spatial constraints. Experimental results show that the proposed method is much faster and computationally more efficient than existing nonparametric background models. The proposed technique is observed to match the capabilities of existing nonparametric background models in terms of being able to effectively model dynamic backgrounds but with greatly reduced computational complexity.

Multi-resolution quad-tree based image representations are useful for image and video encoding at varying bit rates. Existing algorithms use a difference measure of color values to trim branches of the quad-tree, which results in the various spatial regions of the image being represented at different resolutions. However, color difference-based quad-tree branch pruning often leads to sub-optimal multi-resolution images, where interesting features of the image are represented at low resolution, and/or uninteresting image regions are represented at high resolution. In this paper, we present a mask-based method for pruning a quadtree representation of a multi-resolution image. The masks highlight various features of the image which require the best resolution, allowing tightly controlled pruning of the quad-tree branches. We suggest three main groups of masks, which are suitable for most types of applications requiring multi-resolution images and videos.

A novel procedure for in-silico (virtual) craniofacial reconstruction of human mandibles with multiple fractures from a sequence of Computed Tomography (CT) images is presented. The problem is formulated as one of combinatorial pattern matching and solved in two stages. First, the opposable fracture surfaces are identified using a maximum weight graph matching algorithm where the fracture surfaces are modeled as the vertices of a weighted graph. The edge weights between pairs of vertices are treated as elements of a score matrix, whose values are a linear combination of (a) the Hausdorff distance, and (b) a score function based on fracture surface characteristics. Second, the pairs of opposable fracture surfaces identified in the first stage are actually registered using the Iterative Closest Point (ICP) algorithm enhanced with a graph theoretic improvisation. The correctness of the registration in the second stage is constantly monitored by volumetric matching of the reconstructed mandible with an intact mandible. Experimental results on simulated CT image sequences of broken human mandibles are presented.

A novel algorithm, termed a boosted adaptive particle filter (BAPF), for integrated face detection and face tracking is proposed. The proposed algorithm is based on the synthesis of an adaptive particle filtering algorithm and an AdaBoost face detection algorithm. A novel adaptive particle filter (APF), based on a new sampling technique, is proposed to obtain accurate estimates of the proposal distribution and the posterior distribution to enable accurate tracking in video sequences. The AdaBoost algorithm is used to detect faces in input image frames, while the APF algorithm is designed to track faces in video sequences. The proposed BAPF algorithm is employed for face detection, face verification, and face tracking in video sequences. Experimental results show that the proposed BAPF algorithm provides a means for robust face detection and accurate face tracking under various tracking scenarios.

The MPEG-4 Fine Grained Scalability (FGS) profile aims at scalable video encoding, in order to ensure efficient video streaming in networks with fluctuating bandwidth. In order to allow very low bit rate streaming, the Base Layer of an FGS video is encoded at a very low bit rate, resulting in very low video quality. In this paper, we propose FGS-MR, which uses content aware multi-resolution video frames to obtain better video quality for a target bit rate, compared to existing MPEG-4 FGS Base Layer video encoding schemes. FGS-MR is an integrated approach that requires only encoder side modification, and is transparent to the decoder. In addition, FGS-MR can be used with any existing MPEG-4 codec which supports FGS, since it entails smart video preprocessing and does not involve any components from the MPEG-4 compression pipeline. FGS-MR is a mask based technique. We have demonstrated an unsupervised algorithm to automatically create the mask from a given video sequence.

The recent proliferation of streaming multimedia on a variety of mobile devices has severely tested their battery lifetime. The long running nature of typical streaming applications results in significant energy consumption by the wireless network interface card (WNIC) in these mobile devices. In this paper we explore linear prediction-based client-side strategies that reduce the WNIC energy consumption to receive multimedia streams by judiciously transitioning the WNIC to a lower power consuming sleep state during the no-data intervals in the multimedia stream, without explicit support from the multimedia servers themselves. Experimental results on popular streaming formats such as Microsoft Media, Real and Apple QuickTime show that a linear prediction-based strategy performs better than history-based strategies that use simple temporal averaging.

A novel region-based multiple object tracking framework based on Kalman filtering and elastic matching is proposed. The proposed Kalman filtering-elastic matching model is general in two significant ways. First, it is suitable for tracking of both, rigid and elastic objects. Second, it is suitable for tracking using both, fixed cameras and moving cameras since the method does not rely on background subtraction. The elastic matching algorithm exploits both the spectral features and structural features of the tracked objects, making it more robust and general in the context of object tracking. The proposed tracking framework can be viewed as a generalized Kalman filter where the elastic matching algorithm is used to measure the velocity field which is then approximated using B-spline surfaces. The control points of the B-spline surfaces are directly used as the tracking variables in a grid-based Kalman filtering model. The limitations of the Gaussian distribution assumption in the Kalman filter are overcome by the large capture range of the elastic matching algorithm. The B-spline approximation of the velocity field is used to update the spectral features of the tracked objects in the grid-based Kalman filter model. The dynamic nature of these spectral features are subsequently used to reason about occlusion. Experimental results on tracking of multiple objects in real-time video are presented.

The problem of appearance-based recognition of faces and facial expressions is addressed. Previous work on sliced inverse regression (SIR) resulted in the formulation of an appearance-based face recognition technique termed as Sirface that is insensitive to large variation in lighting direction and facial expression. Sirface was shown to be superior to the well known Fisherface technique, that is based on Fisher's linear discriminant analysis (LDA), in terms of both, dimensionality reduction and classification accuracy. However, Sirface, which relies only on first-order statistics, is shown to be poor at discriminating between facial expressions. A novel statistical data dimensionality reduction technique based on sliced average variance estimation (SAVE) is shown to be effective in distinguishing between different facial expressions of the same face. SAVE, which exploits the difference in second-order statistics between the pattern classes, is shown to result in an optimal reduced dimensional subspace for quadratic discriminant analysis (QDA). The resulting appearance-based technique for recognition effaces and facial expressions, termed as Saveface, is experimentally compared to Sirface in terms of classification accuracy and data dimensionality reduction.

Animation of Virtual Humans (avatars) is done typically using motion data files that are stored on a client or streaming motion data from a server. Several modern applications require avatar animation in mobile networked virtual environments comprising of power constrained clients such as PDAs, Pocket-PCs and notebook PCs operating in battery mode. These applications call for efficient compression of the motion animation data in order to conserve network bandwidth, and save power at the client side during data reception and motion data reconstruction from the compressed file. In this paper, we have proposed and implemented a novel file format, termed the Quantized Motion Data (QMD) format, which enables significant, though lossy, compression of the motion data. The motion distortion resulting from the reconstructed motion from the QMD file is minimized by intelligent use of the hierarchical structure of the skeletal avatar model. The compression gained by using the QMD files for the motion data is more than twice achieved via standard MPEG-4 compression using a pipeline comprising of quantization, predictive encoding and arithmetic coding. In addition, considerably fewer CPU cycles are needed to reconstruct the motion data from the QMD files compared to motion data compressed using the MPEG-4 standard.

The proliferation of multimedia-capable mobile devices and ubiquitous high-speed network technologies to deliver multimedia objects has fueled the demand of mobile streaming multimedia. A necessary criterion for the mass acceptance of mobile devices is acceptable battery life of these devices. This paper explores linear prediction-based client-side strategies to reduce the wireless network interface card (WNIC) energy consumption by transitioning the WNIC to a lower power consuming sleep state. The basic idea of this strategy is to selectively choose proper periods of time to suspend communication by switching the WNIC to sleep state. A linear prediction-based time series forecasting technique is used to predict future no-data intervals. Simulation results show that linear prediction-based strategy gives better results than those based on simple averaging [Surendar Chandra et al., (2002)].

An interactive tool for the segmentation, visualization and navigation of magnetic resonance (MR) images is presented. Previous work has shown the hierarchical self-organizing map (HSOM) to be highly effective in segmenting MR images at multiple scales or levels of abstraction. The resulting abstraction tree represents the multiscale segmentation of the MR image. A segmented MR image at any desired scale or level of detail can be obtained by appropriate traversal of the abstraction tree. The interactive tool permits traversal of the abstraction tree using a user-friendly graphical user interface (GUI) allowing the user to view the segmented MR image or any portion thereof at the desired level of detail. The tool could be used by radiologists to effectively sift through large amounts of MR image data to arrive at accurate diagnoses in an expeditious manner.

Using a novel data dimension reduction method proposed in statistics, we develop an appearance-based face recognition algorithm which is insensitive to large variation in lighting direction and facial expression. Taking a pattern classification approach, we consider each pixel in an image as coordinate in a high-dimensional space. However, since faces are not truly Lambertian surfaces and indeed produce self-shadowing, images deviates from this linear subspace. Rather than explicitly modeling this deviation, we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation using Sliced inverse regression (SIR) [K.C. Li, 1991]. Our face recognition algorithm termed as Sirface produces well-separated classes in a low-dimensional subspace, even under severe variation in lighting and facial expression. Sirface is shown to be equivalent to the well known Fisherface algorithm [P.N. Belhumeur, et al., 1997] in the subspace sense. However, Sirface is shown to produce the optimal reduced subspace (with the fewest dimensions) resulting in a lower error rate and reduced computational expense. Experimental results comparing Sirface to Fisherface on the Yale face database are presented.

MOTIVATION: Physical mapping of chromosomes using the maximum likelihood (ML) model is a problem of high computational complexity entailing both discrete optimization to recover the optimal probe order as well as continuous optimization to recover the optimal inter-probe spacings. In this paper, two versions of the genetic algorithm (GA) are proposed, one with heuristic crossover and deterministic replacement and the other with heuristic crossover and stochastic replacement, for the physical mapping problem under the maximum likelihood model. The genetic algorithms are compared with two other discrete optimization approaches, namely simulated annealing (SA) and large-step Markov chains (LSMC), in terms of solution quality and runtime efficiency. RESULTS: The physical mapping algorithms based on the GA, SA and LSMC have been tested using synthetic datasets and real datasets derived from cosmid libraries of the fungus Neurospora crassa. The GA, especially the version with heuristic crossover and stochastic replacement, is shown to consistently outperform the SA-based and LSMC-based physical mapping algorithms in terms of runtime and final solution quality. Experimental results on real datasets and simulated datasets are presented. Further improvements to the GA in the context of physical mapping under the maximum likelihood model are proposed. AVAILABILITY: The software is available upon request from the first author.

The application of a hierarchical self-organizing map (HSOM) to the problem of segmentation of multispectral magnetic resonance (MR) images is investigated. The HSOM is composed of several layers of self-organizing maps (SOMs) organized in a pyramidal fashion. SOMs have previously been used for the segmentation of multispectral MR images, but the results often suffer from under-segmentation or over-segmentation. By combining the concepts of self-organization and topographic mapping with multi-scale image segmentation, the HSOM is shown to overcome the major drawbacks of the SOM. The segmentation results of the HSOM are compared with those of the SOM and the k-means clustering algorithm on multispectral MR images of the human brain representing both normal conditions and pathological conditions, such as multiple sclerosis. The multi-scale segmentation results of the HSOM are shown to have interesting consequences from the viewpoint of the clinical diagnosis of pathological conditions.

The design and implementation of a computer vision system called DNAScan for the automated analysis of DNA hybridization images is presented. The hybridization of a DNA clone with a radioactively tagged probe manifests itself as a spot on the hybridization membrane. A recursive segmentation procedure is designed and implemented to extract spot-like features in the hybridization images in the presence of a highly inhomogeneous background. Positive hybridization signals (hits) are extracted from the spot-like features using grouping and decomposition algorithms based on computational geometry. A mathematical model for the positive hybridization patterns and a pattern classifier based on shape-based moments are proposed and implemented to distinguish between the clone-probe hybridization signals.

Video parsing refers to the detection and classification of abrupt and gradual scene changes in a video stream and constitutes an important preprocessing step in applications that treat video streams as sources of information. Parallel processing is proposed as a means of dealing with the high computational demands of video parsing. Parallel versions of two algorithms that detect scene transitions in compressed video streams are proposed. Three granularities of parallelism are investigated; Group of Pictures (GOP), frame and slice. Results show that the GOP-level implementation, which represents the coarsest granularity of task and data decomposition, always performs the best. The slice and frame levels of granularity take the second and third place respectively. The speedup a's shown to be almost linear in the case of the GOP level of granularity, whereas the Synchronization overheads are seen to be high for the frame and slice levels of granularity.

MOTIVATION: Contig maps are a type of physical map that show the native order of a set of overlapping genomic clones. Overlaps between clones can be detected by finding common sequences using a number of experimental protocols including hybridization of probes. All current mapping algorithms of which we are aware require that hybridizations be scored using a fixed number of discrete values (typically 0/1 or high/medium/low). When hybridization data is captured automatically using digital equipment, this provides the opportunity for hybridization intensities to be used in map construction. More fine-grained distinctions in the levels of hybridization may be exploited by algorithms to generate more accurate physical maps. RESULTS: We describe an approach to creating contig maps that uses measured hybridization intensities instead of data scored with a fixed number of discrete values. We describe and compare four algorithms for creating physical maps with hybridization intensities. Simulations using measured intensities sampled from actual data on Aspergillus nidulans indicate that using hybridization intensities rather than data that is automatically scored with respect to threshold values may yield more accurate physical maps.

Reconstructing a physical map of a chromosome from a genomic library presents a central computational problem in genetics. Physical map reconstruction in the presence of errors is a problem of high computational complexity which provides the motivation for parallel computing. Parallelization strategies for a maximum likelihood estimation-based approach to physical map reconstruction are presented. The estimation procedure entails gradient descent search for determining the optimal spacings between probes for a given probe ordering. The optimal probe ordering is determined using a stochastic optimization algorithm. A two-tier parallelization strategy is proposed wherein the gradient descent search is parallelized at the lower level and the stochastic optimization algorithm is simultaneously parallelized at the higher level. Implementation and experimental results on a distributed memory multiprocessor cluster running the Parallel Virtual Machine (PVM) environment are presented.

The design and implementation of a computer vision system called DNAScan for the automated analysis of DNA hybridization images is presented. The hybridization of a DNA clone with a radioactively tagged probe manifests itself as a spot on the hybridization membrane. The imaging of the hybridization membranes and the automated analysis of the resulting images is imperative for high-throughput genomics experiments. A recursive segmentation procedure is designed and implemented to extract spot-like features in the hybridization images in the presence of a highly inhomogeneous background. Positive hybridization signals (hits) are extracted from the spot-like features using grouping and decomposition algorithms based on computational geometry. A mathematical model for the positive hybridization patterns and a pattern classifier based on shape-based moments are proposed and implemented to distinguish between the clone-probe hybridization signals. Experimental results on real DNA hybridization images are presented.

Image segmentation denotes a process by which a raw input image is partitioned into nonoverlapping regions such that each region is homogeneous and the union of any two adjacent regions is heterogenous. A segmented image is considered to be the highest domain-independent abstraction of an input image. In this paper, the image segmentation problem is treated as one of combinatorial optimization. A cost function which incorporates both, edge information and region gray-scale variances is defined. The cost function is shown to be multivariate with several local minima. Three stochastic optimization techniques, namely, simulated annealing (SA), microcanonical annealing (MCA), and the random cost algorithm (RCA) are investigated and compared in the context of minimization of the aforementioned cost function for image segmentation. Experimental results on gray-scale images are presented. © 2000 John Wiley & Sons, Inc.

Image segmentation denotes a process by which a raw input image is partitioned into nonoverlapping regions such that each region is homogeneous and the union of any two adjacent regions is heterogeneous. A segmented image is considered to be the highest domain-independent abstraction of an input image. The image segmentation problem is treated as one of combinatorial optimization. A cost function which incorporates both edge information and region gray-scale uniformity is defined. The cost function is shown to be multivariate with several local minima. The genetic algorithm, a stochastic optimization technique based on evolutionary computation, is explored in the context of image segmentation. A class of hybrid evolutionary optimization algorithms based on a combination of the genetic algorithm and stochastic annealing algorithms such as simulated annealing, microcanonical annealing, and the random cost algorithm is shown to exhibit superior performance as compared with the canonical genetic algorithm. Experimental results on gray-scale images are presented.

A computer vision-based system for lumber production planning is described. Computer axial tomography (CT or CAT) images of hardwood logs are analyzed for identification and classification of internal log defects. Individual CT image slices are analyzed for detection of 2-D defects which are correlated across CT image slices in order to establish 3-D support and identify true 3-D defects. Currently, the system is capable of 3-D reconstruction and rendering of the log and its internal defects from the individual CT image slices. It is also capable of simulation and rendering of key machining operations such as sawing and veneering on the 3-D reconstructions of the logs. From the 3-D reconstruction of the log and knowledge of its internal defects, the system can formulate sawing strategies to optimize the yield and grade of the resulting lumber. The system is intended as a decision aid for lumber production planning and an interactive training tool for novice sawyers and machinists.

Multiscale structures and algorithms that unify the treatment of local and global scene information are of particular importance in image segmentation. Vector quantization, owing to its versatility, has proved to be an effective means of image segmentation. Although vector quantization can be achieved using self-organizing maps with competitive learning, self-organizing maps in their original single-layer structure, are inadequate for image segmentation. A hierarchical self-organizing neural network for image segmentation is presented. The Hierarchical Self-Organizing Map (HSOM) is an extension of the conventional (single-layer) Self-Organizing Map (SOM). The problem of image segmentation is formulated as one of vector quantization and mapped onto the HSOM. By combining the concepts of self-organization and topographic mapping with those of multiscale image segmentation the HSOM alleviates the shortcomings of the conventional SOM in the context of image segmentation.

A novel reconfigurable architecture based on a multiring multiprocessor network is described. The reconfigurability of the architecture is shown to result in a low network diameter and also a low degree of connectivity for each node in the network. The mathematical properties of the network topology and the hardware for the reconfiguration switch are described. Primitive parallel operations on the network topology are described and analyzed. The architecture is shown to contain 2D mesh topologies of varying sizes and also a single one factor of the Boolean hypercube in any given configuration. A large class of algorithms for the 2D mesh and the Boolean n-cube are shown to map efficiently on the proposed architecture without loss of performance. The architecture is shown to be well suited for a number of problems in low and intermediate level computer vision such as the FFT, edge detection, template matching, and the Hough transform. Timing results for typical low and intermediate level vision algorithms on a transputer based prototype are presented.

The paper presents a system for detection of some important internal log defects via analysis of axial CT images. Two major procedures are used. The first is the segmentation of a single computer tomography (CT) image slice which extracts defect-like regions from the image slice, the second is correlation analysis of the defect-like regions across CT image slices. The segmentation algorithm for a single CT image is basically a complex form of multiple thresholding that exploits both the prior knowledge of wood structure and gray value characteristics of the image. The defect-like region extraction algorithm first locates the pith, groups the pixels in the segmented image on the basis of their connectivity and classifies each region as either a defect-like region or a defect-free region using shape, orientation and morphological features. Each defect-like region is classified as a defect or non-defect via correlation analysis across corresponding defect-like regions in neighboring CT image slices.

A suite of parallel algorithms for ordering DNA sequences (termed PARODS) is presented. The algorithms in PARODS are based on an earlier serial algorithm, ODS, which is a physical mapping algorithm based on simulated annealing. Parallel algorithms for simulated annealing based on Markov chain decomposition are proposed and applied to the problem of physical mapping. Perturbation methods and problem-specific annealing heuristics are proposed and described. Implementations of parallel Single Instruction Multiple Data (SIMD) algorithms on a 2048 processor MasPar MP-2 system and implementations of parallel Multiple Instruction Multiple Data (MIMD) algorithms on an 8 processor Intel iPSC/860 system are presented. The convergence, speedup and scalability characteristics of the aforementioned algorithms are analyzed and discussed. The best SIMD algorithm is shown to have a speedup of approximately 1000 on the 2048 processor MasPar MP-2 system, whereas the best MIMD algorithm is shown to have a speedup of approximately 5 on the 8 processor Intel iPSC/860 system.

A surface feature hypergraph (SFAHG) representation is proposed for the recognition and localization of three-dimensional objects. The hypergraph representation is shown to be viewpoint independent thus resulting in substantial savings in terms of memory for the object model database. The resulting hypergraph matching algorithm integrates both, relational and the rigid pose constraint in a consistent unified manner. The matching algorithm is also shown to have a polynomial order of complexity even in multiple-object scenes with instances of objects partially occluding each other. An algorithm for incrementally constructing the hypergraph representation of an object model from range images of the object taken from different viewpoints is also presented. The hypergraph matching and the hypergraph construction algorithms are shown to be capable of correcting errors in the initial segmentation of the range image. The hypergraph construction algorithm and the matching algorithm are tested on range images of scenes containing multiple three-dimensional objects with partial occlusion.

In this paper we describe a reconfigurable architecture for image processing and computer vision based on a multi-ring network which we call a Reconfigurable Multi-Ring System (RMRS). We describe the reconfiguration switch for the RMRS and also describe its VLSI implementation. The RMRS topology is shown to be regular and scalable and hence well-suited for VLSI implementation. We prove some important properties of the RMRS topology and show that a broad class of algorithms for the n-cube can be mapped to the RMRS in a simple and elegant manner. We design and analyze a class of procedural primitives for the SIMD RMRS and show how these primitives can be used as building blocks for more complex parallel operations. We demonstrate the usefulness of the RMRS for problems in image processing and computer vision by considering two important operations—the Fast Fourier Transform (FFT) and the Hough transform for detection of linear features in an image. Parallel algorithms for the FFT and the Hough transform on the SIMD RMRS are designed using the aforementioned procedural primitives. The analysis of the complexity of these algorithms shows that the SIMD RMRS is a viable architecture for problems in computer vision and image processing.

This paper proposes and describes a hierarchical self-organizing neural network for range image segmentation. The multilayer self-organizing feature map (MLSOFM), which is an extension of the traditional (single-layer) self-organizing feature map (SOFM) is seen to alleviate the shortcomings of the latter in the context of range image segmentation. The problem of range image segmentation is formulated as one of vector quantization and is mapped onto the MLSOFM. The MLSOFM combines the ideas of self-organization and topographic mapping with those of multiscale image segmentation. Experimental results using real range images are presented.

A novel reconfigurable architecture based on a multi-ring multiprocessor network is described. The reconfigurable architecture is shown to combine low network diameter with a low degree of connectivity for each node in the network. The mathematical properties of the network topology and the hardware for the reconfiguration switch are described. Primitive parallel operations on the network topology are described and analyzed. A large class of algorithms for the Boolean n-cube and the 2-D mesh is shown to map efficiently on the proposed architecture without loss of performance. The architecture is shown to be well suited for a number of problems in computer vision.

A novel reconfigurable architecture based on a multi-ring multiprocessor network is described. The reconfigurability of the architecture is shown to result in a low network diameter and also a low degree of connectivity for each node in the network. The mathematical properties of the network topology and the hardware for the reconfiguration switch are described. Primitive parallel operations on the network topology are described and analyzed. The architecture is shown to contain a single 1-factor of the Boolean hypercube in any given configuration. A large class of algorithms for the Boolean n-cube and the 2-D mesh is shown to map efficiently on the proposed architecture without loss of performance. The architecture is shown to be well suited for a number of problems in robot vision.>

A fuzzy-probabilistic model of the generalized Hough transform (GHT) based on qualitative labeling of scene features is presented. Qualitative labeling of scene features is shown to be effective in pruning the search space of possible scene interpretations and also reducing the number of spurious interpretations explored by the GHT. Qualitative labeling of scene features is shown to result in the formulation of a weighted generalized Hough transform (WGHT) where each match of a scene feature with a model feature is assigned a weight based on the qualitative attributes assigned to the scene feature. These weights are looked upon as membership function values for the fuzzy sets defined by these qualitative attributes. A fuzzy-probabilistic model for the WGHT is presented. Analytical expressions for the probability of accumulation of random votes are derived for the WGHT and compared with the corresponding expressions for the conventional GHT. The WGHT is shown to perform better than the conventional GHT. Experimental results on intensity and range images are presented.>

The design and implementation of INTEGRA is described. INTEGRA is a range image understanding system that attempts to exploit the synergy between the various stages in the image understanding process i.e. segmentation, feature extraction, recognition and localization. Two prominent features of INTEGRA are: a synergetic combination of edge- and surface-based segmentation processes that results in more accurate segmentation than would have been possible with either of them alone; and the ability to correct errors made during segmentation in the matching and localization stages. At present, INTEGRA is capable of recognition and localization of polyhedral objects and objects that can be treated as piecewise combinations of curved surfaces of quadratic order, typically spherical, ellipsoidal, cylindrical and conical surfaces.>

Segmentation, feature extraction, recognition and localization are the four stages in range image understanding. Conventional approaches to range image understanding have treated these stages in isolation with a largely bottom-up flow of control and data through these various stages. Strictly bottom-up approaches have proved to be fragile in the face of errors in segmentation due to noise and limitations on sensor resolution and accuracy. Synergetic interaction of these various stages is essential for an image understanding system to exhibit robust behavior. This paper describes the design and implementation of INTEGRA, a range image understanding system that attempts to exploit the synergy between the various stages in the image understanding process. The salient features of INTEGRA are: (i) A synergetic combination of edge- and surface-based segmentation processes that results in more accurate segmentation than would have been possible with either of them alone and (ii) the ability to correct errors made during segmentation in the matching and localization stages. INTEGRA at this time, is limited to recognition and localization of polyhedral objects and is in the process of being enhanced to handle objects with curved surfaces of quadratic order such as spherical, ellipsoidal, cylindrical, and conical surfaces. Experimental results on real range images containing single and multiple polyhedral objects are presented. Future enhancements to INTEGRA are discussed.