This study introduces an innovative framework designed specifically for accurate colon segmentation in abdomen CT scans, tackling the distinct challenges inherent to this task. Building upon well-established 2D segmentation models, our architecture adeptly incorporates 3D contextual information via a novel method that generates an attention map for a given slice by considering its neighboring slices in a sequence. Our approach accomplishes effective colon segmentation without requiring complex 3D convolutional neural networks (CNNs) or Long Short-Term Memory (LSTM) networks by combining 2D CNNs. Validated on a dataset of 98 CT scans from 49 patients, the architecture exhibits notable performance, successfully capturing nuanced details crucial for precise colon segmentation. The experiments encompass a thorough examination of model selection and cross-validation, providing valuable insights into the efficacy of our proposed approach. The outcomes underscore the potential for streamlined colon segmentation in medical imaging by judiciously integrating 2D and 3D information, employing solely 2D networks, and mitigating challenges associated with 3D networks. The code for model architecture is available at: https://github.com/Samir-Farag/ICIP2024.git

Early diagnosis of colorectal polyps, before they turn into cancer, is one of the main keys to treatment. In this work, we propose a framework to help radiologists in reading CT scans and identifying candidate CT slices that have polyps. We propose a colorectal polyps detection approach which consists of two cascaded stages. In the first stage, a CNN-based model is trained and validated to detect polyps in axial CT slices. To narrow down the effective receptive field of the detector neurons, the colon regions are segmented and then fed into the network instead of the original CT slice. This drastically improves the detection and localization results, e.g., the mAP is increased by 36%. To reduce the false positives generated by the detector, in the second stage, we propose a multi-view network (MVN) that classifies polyp candidates. The proposed MVN classifier is trained using sagittal and coronal views corresponding to the detected axial views. The approach is tested in 50 CTC-annotated cases, and the experimental results confirm that after the classification stage, polyps can be detected with an AUC $\sim 95.27 \%$.

Colon cancer, also known as colorectal cancer, is a significant health concern, with increasing incidence rates, particularly among individuals under 50. This rise has led experts to recommend the introduction of regular screenings at 45 years of age for adults at average risk. Early detection through such screenings can identify precancerous polyps, allowing their removal before they develop into cancer. This proactive approach has the potential to reduce colorectal cancer deaths by up to 60%. In addition, research indicates that people diagnosed before age 50 have better survival rates, which emphasizes the importance of early diagnosis. Therefore, adhering to recommended screening guidelines and promptly addressing concerns can significantly improve outcomes and save lives. This study proposes a framework to detect and localize colon cancer indicators (polyps) in 3D medical images, which has been applied to help radiologists read computed tomography (CT) scans and identify candidate CT slices with colonic polyps. This work’s main benefit is introducing AI-based colon abnormalities detection framework that the radiologist could miss by using different approaches and modalities. In this work, we propose an automatic detection approach for colorectal polyps consisting of two cascade stages. In the first stage, a Convolutional Neural Network (CNN) model is trained to detect polyps in axial CT slices; a CNN model has been fed by the segmented colon wall CT slices instead of the original CT slices. Using segmented images as input to the CNN model has drastically improved detection and localization results; for example, the mean average precision (mAP) for detection increases by 36%. To reduce false positives generated by the detector, the second stage classifier is deployed to exploit the different views of CT scans instead of the axial view only. So, the classifier is trained using the 2D images of axial views, i.e., the candidate polyps generated by the detector, as well as their corresponding 2D images of sagittal and coronal views. Different experiments were conducted to evaluate the proposed work, including the Fly-In approach to visualize and detect polyps within the 3D colon model, which was successfully used to train a CNN-based model that detects polyps with mAP∼ 97.1%. The second approach is to use the axial CT view after removing other organs except the colon and using A.I model to detect polyps within the 2D axial view with mAP∼ 86%. Finally, the MVN classification approach successfully identified polyps after the classification stage with an area

In this paper, we propose a novel framework that measures the engagement level of students either in a class environment or in an e-learning environment. The proposed framework captures the user's video and tracks their faces' through the video's frames. Different features are extracted from the user's face e.g., facial fiducial points, head pose, eye gaze, learned features, etc. These features are then used to detect the Facial Action Coding System (FACS), which decomposes facial expressions in terms of the fundamental actions of individual muscles or groups of muscles (i.e., action units). The decoded action units (AUs) are then used to measures the student's willingness to participate in the learning process (i.e., behavioral engagement) and his/her emotional attitude towards learning (i.e., emotional engagement). This framework will allow the lecturer to receive a real-time feedback from facial features, gaze, and other body kinesics. The framework is robust and can be utilized in numerous applications including but not limited to the monitoring the progress of students with various degrees of learning disabilities, and the analysis of nerve palsy and its effects on facial expression and social interactions.

Intra-oral cameras are essentially enabling dentists to capture images of difficult-to-reach areas in the mouth. Oral dental applications based on visual data pose various challenges such as low lighting conditions and saliva. We introduce an approach to stitch images of human teeth that are captured by an intra-oral camera. In such monocular image matching, a low rate of features on teeth surfaces causes a problem leading to a mismatch between teeth images. In this paper, we propose an approach to improve the matching in these low-texture regions. First, normals of tooth surface is extracted using a shape from shading. Due to the oral environment, the surface normals impact many of imprecise values; hence we formulate an algorithm to rectify these values and generate normal maps. The normal maps reveals the impacted geometric properties of the images inside an area, boundary, and shape. Second, the normal maps are used to detect, extract and match the corresponding features. Finally, to enhance the stitching process for these unidealized data, normal maps are used to estimate as-projective-as-possible warps. The proposed approach outperforms the state-of-the-art auto-stitching approach and shows a better performance in such cases of low-texture regions.

In this paper, a new visualization method for tubular inner surfaces is proposed, denoted by “Fly-In”. The approach uses a virtual camera that moves along the inner surface's centerline, obtaining projections of the surrounding view, formed of small 3D topological rings, within the tube that is rendered as a 2D rectangular image. A new visualization loss measure is also presented, which utilizes the projection direction of the camera axis, the surface normal and the ratio of the camera focal length and the surface distance. A comparative study of Fly-in versus two state-of-the-art methods (Fly-Through and Fly-Over) is conducted on 3D reconstructed colons from CT Colonography (CTC) patient data. Quantitative results are computed using the new loss measure and visualized using color coding. Results show high performance for the proposed Fly-In approach with respect to the state-of-art.

A set of facial Action Units (AUs) is activated due to the movement of facial muscles in response to a person's internal emotion. The activated facial action units appear around sparse regions on the face e.g., the mouth and eyes. This group occurrence of AUs reveals the semantic relationships among them. These relationships should be considered in the detection process of AUs. Therefore, we propose an approach that learns these semantic relationships using a multi-label deep learning architecture. The sparse patches of the AUs are used in the proposed approach instead of the whole facial region. To handle these sparse patches, we propose a region-based network instead of the well-known convolutional network and the locally connected network. Moreover, unlike the current approaches, which define facial regions using a uniform grid, the proposed region-based architecture, define patches around facial landmarks. This overcomes the region displacement problem of the uniform grid. Usually, the AUs classification suffers from the high skewness factor problem. To overcome this problem, we use a weighted loss function. We conduct our experiments on two standard benchmarks: BP4D and FERA17. f1score, the area under the ROC (AROC) and the area under the precision-recall curve (APR) are used as performance measures. Compared to the standard convolutional layer, the proposed patch-based layer is more effective in capturing the required structural features of the face and learns the correlations among AUs under pose variations. Also, it has been shown that the proposed approach outperforms the state-of-the-art methods.

Student behavior and lecturer oversight in the classroom is known to modulate study behaviors and impact performance and learning outcomes, but cannot at present be managed for distance learning students. Quantifying and automatically measuring student engagement during lectures in a scalable and accessible manner for these students is essential for improving academic success, but has not been studied widely in natural distance learning environments. We collect video recordings from a screen-mounted camera of students studying online lectures in a mostly unstructured setting and gather annotations from a panel of humans for assessing student engagement levels. We present results on the prediction of different representations of engagement, both with subject-independent and individual-specific models, and quantify the performance gap between the generalized and personalized models for engagement prediction. While the subject-independent performance is challenged by data sparsity, results show that the individual-specific models can predict engagement well even with very few labeled examples.

This paper proposes an approach for modeling textured 3D non-rigid models based on Weighted Heat Kernel Signature(W-HKS). As a first contribution, we show how to include photometric information as a weight over the shape manifold, we also propose a novel formulation for heat diffusion over weighted manifolds. As a second contribution we present a new discretization method for the proposed equation using finite element approximation. Finally, the weighted heat kernel signature is used as a shape descriptor. The proposed descriptor encodes both the photometric, and geometric information based on the solution of one equation. We also propose a new method to introduce the scale invariance for the weighted heat kernel signature. The performance is tested on two benchmark datasets. The results have indeed confirmed the high performance of the proposed approach on the textured shape retrieval problem, and showed that the proposed method is useful in coping with different challenges of shape analysis where pure geometric and pure photometric methods fail.

Osteoporosis is a bone disease characterised by a reduction in bone mass, resulting in an increased risk of fractures. Doctors need the bone mineral density (BMD) measurements of vertebral bodies in order to diagnose and treat osteoporosis. The authors' objective is to segment the VBs as accurately as possible and hence to increase the accuracy of the BMD measurements and fracture analysis. Three pieces of information (intensity, spatial interaction and shape) are modelled to optimise a probabilistic energy functional. A universal shape prior, which is modelled using the cervical, thoracic and lumbar spinal regions, is proposed. Volumetric computed tomography data sets with various challenges are used in this study. The authors classify data sets based on some features related to the anatomy, imaging modality and level of the bone health. The proposed framework is one of only a few reported in the literature tested on the data obtained from different imaging devices. The experimental results reveal that the proposed method is robust under various noise levels, less variant to the initialisation and faster than existing vertebrae segmentation reports in the literature.

This study revisits a visualisation technique for virtual colonoscopy, known as virtual fly‐over (FO). The method views the entire colon anatomy from above the centreline. It assigns two cameras located on opposite sides of the centreline, each of which is responsible for viewing one half of the colon. This approach has several advantages over related colon visualisation methods with regards to visibility coverage and polyp detection rate. However, the traditional FO implementation created a few drawbacks that hinder complete visualisation. For example, it could overlook polyps located on the line between the two halves, and it has no data‐specific initialisation for cutting planes. The authors enhance the FO method in a number of respects in this study: resolve the cutting issue and improve the virtual camera setup in order to better visualise both the colon surface's hidden structures and polyps that are difficult to locate. Quantitative validation of the revamped FO on 30 actual clinical datasets with complicated shapes and large volumes demonstrated that the average surface visualisation rate is equal to 99.5 ± 0.2%. Also, true and synthetic polyps with various shapes and sizes were used to clinically validate the proposed method. Detection rate is up to 100% on the tested sets.

In this study, the authors learn a similarity measure that discriminates between inter‐class and intra‐class samples based on a statistical inference perspective. A non‐linear combination of Mahalanobis is proposed to reflect the properties of a likelihood ratio test. Since an object's appearance is influenced by the identity of the object and variations in the capturing process, the authors represent the feature vector, which is the difference between two samples in the differences space, as a sample that is drawn from a mixture of many distributions. This mixture consists of the identities distribution and other distributions of the variations in the capturing process, in case of dissimilar samples. However, in the case of similar samples, the mixture consists of the variations in the capturing process distributions only. Using this representation, the proposed similarity measure accurately discriminates between inter‐class and intra‐class samples. To highlight the good performance of the proposed similarity measure, it is tested on different computer vision applications: face verification and person re‐identification. To illustrate how the proposed learning method can easily be used on large scale datasets, experiments are conducted on different challenging datasets: labelled faces in the wild (LFW), public figures face database, ETHZ and VIPeR. Moreover, in these experiments, the authors evaluate different stages, for example, features detector, descriptor type and descriptor dimension, which constitute the face verification pipeline. The experimental results confirm that the learning method outperforms the state‐of‐the‐art.

The basic idea of shape from shading is to infer the shape of a surface from its shading information in a single image. Since this problem is ill-posed, a number of simplifying assumptions have been often used. However they rarely hold in practice. This paper presents a simple shading-correction algorithm that transforms the image to a new image that better satisfies the assumptions typically needed by existing algorithms, thus improving the accuracy of shape recovery. The algorithm takes advantage of some local shading measures that have been driven under these assumptions. The method is successfully evaluated on real data of human teeth with ground-truth 3D shapes.

This paper describes a unique method for tubular object visualization. The method involves rendering the exterior of the tube invisible while keeping the interior visible. This “One-sided-transparency” technique renders a more complete view of the tube's interior. When applied to virtual colonoscopy (VC), it compares favorably to existing methods. It provides more complete images, reduces computational time, and reduces memory requirements while preserving VCs benefits for patients and practitioners. The approach also has various potential uses outside of VC.

Faces in the wild have recently captured the focus of researchers for all facial analysis problems. Partial occlusion is a major problem for analyzing faces captured in unconstrained non-cooperative conditions. Even detecting the faces in such conditions is a challenging problem that needs to be solved before any further analysis of such faces can be done. In this paper, we propose modelling the face as a collection of parts that can be selected from the visible regular facial features and some other objects that can possibly occlude faces such as sunglasses, caps and hands. The proposed algorithm is more toward scene understanding in the sense that it is not only detecting faces but it also suggests the visible parts of these faces and even some of the occluding objects which can help in any further analysis. Experimental results show a state of the art performance on the challenging FDDB database with a thorough analysis of the performance with different types of partial occlusion.

Recently, several approaches for pedestrian detection have been investigated using discriminatively trained part based models with which Histogram of Oriented Gradients (HOG) showed to be a robust feature. In this paper, we propose a new feature based on HOG to be used with the discriminatively trained part framework for pedestrian detection. Our method is based on computing the image mixed partial derivatives to be used to redefine the gradients of some pixels and to reweigh the vote at all pixels with respect to the original HOG. Our approach was tested on the PASCAL2007 and INRIA person dataset and showed to have an outstanding performance.

While traditional dental fillings are molded during a dental visit, dental restoration (e.g. inlays and onlays) are fabricated in a dental lab to offer a long lasting reparative solution to tooth decay or similar structural damage. Such process requires dental technicians who are highly trained experts in tooth anatomy to pick an appropriate standard tooth model from a tooth database. The success of a restoration process primarily relies on the acquisition and modeling of an accurate 3D shape of the occlusal surface of interest for manufacturing purposes. Based on a single optical image, this paper provides an economical and automated solution for tooth restoration where user intervention is kept at the minimal. The inherit relation between the photometric information and the underlying 3D shape is formulated as a coupled statistical model where the effect of illumination is modeled using Spherical Harmonics. Moreover, shape and texture alignment is accomplished using a proposed definition of anatomical jaw landmarks which are automatically detected. The system is evaluated on database of 32 jaws for crown, inlay, and onlay restoration. Results shows a promising performance for using the proposed approach in clinical application.

Image formation involves understanding the sensors characteristics and object reflectance. In dentistry, for example an accurate three‐dimensional (3D) representation of the human jaw may be used for diagnostic and treatment purposes. Photogrammetry can offer a flexible, cost‐effective solution in that regard. Nonetheless there are several challenges, such as non‐friendly image acquisition environment inside the human mouth, problems with lighting (specularity effects because of saliva, gum discolourisation, and occlusion because of the tongue in the lower jaw), and errors because of the data acquisition sensors (e.g. camera calibration errors, lens distortion and so on). In this study, the authors focus on the 3D surface reconstruction aspect for human jaw modelling based on physical surface characteristics and sensor properties. Owing to apparent lens distortion imposed by near‐field imaging, the authors propose a new flexible calibration for lens radial distortion based on a single image of a sphere. The authors propose a non‐Lambertian shape‐from‐shading (SFS) algorithm under perspective projection which benefits from camera calibration parameters. Our experiments provide quantitative metric results for the proposed approach. The reflectance of the tooth surface is modelled by the Oren–Nayar reflectance model for rough surfaces whose roughness parameter is physically computed from an optical surface profiler measurements. As compared to state‐of‐the‐art SFS approaches, our approach is able to recover geometric details of tooth occlusal surface. This work is fundamental for establishing an optical‐based approach for reconstructing the human jaw, that is inexpensive and does not use ionising radiation.

In this study, a novel probabilistic, geometric and dynamic shape‐based level sets method is proposed. The shape prior is coupled with the intensity information to enhance the segmentation results. The two‐dimensional principal component analysis method is applied on the training shapes to represent the shape variation with enough number of shape projections in the training step. The shape model is constructed using the implicit representation of the projected shapes. A new energy functional is proposed (i) to embed the shape model into the image domain and (ii) to estimate the shape coefficients. The proposed method is validated on synthetic and clinical images with various challenges such as the noise, occlusion and missing information. The authors compare their method with some of related works. Experiments show that the proposed segmentation method is more accurate and robust than other alternatives under different challenges. ∗ Note: Colour figures are available in the online version of this paper.

Phenomenological appearance models capture surface appearance through mathematical modelling of the reflection process. Theoretically, the space of all possible images of a fixed‐pose object under all possible illumination conditions is infinite dimensional. Nonetheless, because of their low‐frequency nature, irradiance signals can be represented using low‐order basis functions. Discounting subsurface scattering and surface emittance, this work seeks to address the question; how to compactly and accurately represent image irradiance under unknown general illumination, given that a surface point sees its surrounding world through the local upper hemisphere oriented by the surface normal at this point. In this study, we formulate the image formation process of isotropic surface reflectance under arbitrary distant illumination in the frequency space while addressing the physical compliance of hemispherical basis for representing surface reflectance, for example, Helmholtz reciprocity and isotropy. The term ‘irradiance harmonics’ is also defined which enables decoupling illumination and reflectance from the underlying geometry and pose. We provide a closed form of the energy content being maintained by different reflectance modes of the proposed irradiance harmonics. Since specular materials tend to require more basis functions when compared with diffuse ones, the presented harmonics captures same cumulative energy content, by providing larger number of orthogonal irradiance basis, at lower illumination orders when compared to similar basis in literature.

Precise knowledge of the 3D shape of clinical crowns is crucial for the treatment of malocclusion problems as well as several endodontic procedures. While computed tomography would present such information, it is believed there is no threshold radiation dose below which it is considered safe. In this study, the authors propose an appearance based approach which allows for the reconstruction of plausible human jaw 3D models given a single optical image with unknown illumination. Appearance bases are analytically constructed using the frequency‐based representation of the irradiance equation while incorporating prior information about natural illumination and teeth reflectance. The inherent relation between the photometric information and the underlying 3D shape is formulated as a statistical model where the coupled effect of illumination and reflectance is modelled using the Helmholtz hemispherical harmonics‐based irradiance harmonics whereas the principle component regression is deployed to carry out the estimation of 3D shapes. The authors' approach relaxes limiting assumptions of conventional shape‐from‐shading approaches while being able to reconstruct tooth occlusal surface with challenging conditions, such as scattered specular spots and significant changes in colour and albedo characteristics resulting from tooth filling. Vis‐à‐vis dental applications, the results demonstrate a significant increase in accuracy in favour of the proposed approach.

Conventional subspace construction approaches suffer from the need of "large-enough" image ensemble rendering numerical methods intractable. In this paper, we propose an analytic formulation for low-dimensional subspace construction in which shading cues lie while preserving the natural structure of an image sample. Using the frequency-space representation of the image irradiance equation, the process of finding such subspace is cast as establishing a relation between its principal components and that of a deterministic set of basis functions, termed as irradiance harmonics. Representing images as matrices further lessen the number of parameters to be estimated to define a bilinear projection which maps the image sample to a lower-dimensional bilinear subspace. Results show significant impact on dimensionality reduction with minimal loss of information as well as robustness against noise.

Deformable models are common in image modeling and analysis. Random objects provide major challenges as shapes and appearances are hard to quantify; hence, formulation of deformable models are much harder to construct and validate. In this paper, we examine the effect of randomness on building the shape and appearance models for small-size lung nodules (≤ 1cm) which appear in computed tomography (CT) of the human chest. We devise an approach for annotation, which lends a standard mechanism for building traditional active appearance (AAM), active shape (ASM) and active tensor models (ATM). We illustrate the effectiveness of AAM for nodule detection.

In this paper, We propose a new approach for solving the Lambertian shape from shading (SFS) problem. The proposed technique employs image information as well as camera calibration parameters. We claim that the incorporation of camera parameters results in a better surface reconstruction. In this work, we slightly modify this more realistic model in order to take into account the attenuation of the illumination due to distance. We propose a new variational formulation that relates an evolving surface model with image information, taking into consideration that the image is taken by a perspective camera with known parameters. A new energy functional is formulated to incorporate brightness, smoothness, and integrability constraints. All of these terms assume a hyper surface that evolves in time to meet their criteria. Gradient descent optimization with Euler-Lagrange is used for optimization. We constructed a database of real surfaces as a ground-truth for the experiments. The proposed approach shows superior performance when compared with other approaches in the literature.

Accurate 3D modeling of the human teeth/jaw helps patients avoid the discomfort of the mold process, and improves the data accuracy for oral orthodontists and dental care personnel. Since the surface of the human tooth is almost textureless, Shape from Shading (SFS) has been successfully adopted in solving this problem. In this paper, we attempt to improve the limitations of previous 3D tooth reconstruction algorithms by developing a new approach for shape reconstruction from single image shading with Two-Dimensional Principle Component Analysis (2D-PCA) shape priors. The surface reflectance is modeled using Oren-Nayar-Wolff model which accounts for the retro-reflection case and we experimentally prove that the teeth surface follows the microfacet theory. Our formulation exploits the shape priors as extracted from a set of training CT scans of real human jaws. Our experiments provide promising quantitative metric results for the proposed approach. This work is fundamental for establishing an optical-based approach for reconstructing the human jaw that is inexpensive and does not use ionizing radiation.

In this paper we propose an educational robotic system for teaching kids at nursery schools how to write and draw simple shapes. Our system uses the humanoid robot NAO to draw shapes appearing on a computer screen. The system uses light polarization property for fast detection of the screen despite of its content. We also propose a mapping from the image domain to the robot space.

Under local illumination assumption, phenomenological appearance models capture surface appearance through the mathematical modelling of the reflection process. Theoretically, due to the arbitrariness of the lighting function, the space of all possible images of a fixed‐pose object under all possible illumination conditions is infinite dimensional. Nonetheless, due to their low‐ frequency nature, irradiance signals can be represented using low‐order basis functions, where spherical harmonics (SH) has been extensively adopted. When capturing image irradiance from a single viewpoint, the visible part of the object's surface constructs the upper hemisphere of the surface normals where the SH is no longer orthonormal. In this paper, we propose the use of hemispherical harmonics (HSH) to model image irradiance of convex Lambertian objects perceived from single viewpoint under unknown distant as well as near illumination. We prove analytically, and validate experimentally, that the Lambertian reflectance kernel has a more compact harmonic expansion in the hemispherical domain when compared to its spherical counterpart. Our experiments illustrate that, despite of having poor approximation accuracy under very close lights, such behavior improves exponentially with little increase in the distance to the light source relative to the object size.

In this paper, a novel method to solve the shape registration problem covering both global and local deformations is proposed. The vector distance function (VDF) is used to represent source and target shapes. The problem is formulated as an energy optimization process by matching the VDFs of the source and target shapes. The minimization process results in estimating the transformation parameters for the global and local deformation cases. Gradient descent optimization handles the computation of scaling, rotation, and translation matrices used to minimize the global differences between source and target shapes. Nonrigid deformations require a large number of parameters which make the use of the gradient descent minimization a very time-consuming process. We propose to compute the local deformation parameters using a closed-form solution as a linear system of equations derived from approximating an objective function. Extensive experimental validations and comparisons performed on generalized 2D shape data demonstrate the robustness and effectiveness of the method.

A new variational level set approach is proposed for lung nodule segmentation in lung CT scans. A general lung nodule shape model is proposed using implicit spaces as a signed distance function. The shape model is fused with the image intensity statistical information in a variational segmentation framework. The nodule shape model is mapped to the image domain by a global transformation that includes inhomogeneous scales, rotation, and translation parameters. A matching criteria between the shape model and the image implicit representations is employed to handle the alignment process. Transformation parameters evolve through gradient descent optimization to handle the shape alignment process and hence mark the boundaries of the nodule “head.” The embedding process takes into consideration the image intensity as well as prior shape information. A nonparametric density estimation approach is employed to handle the statistical intensity representation of the nodule and background regions. The proposed technique does not depend on nodule type or location. Exhaustive experimental and validation results are demonstrated on 742 nodules obtained from four different CT lung databases, illustrating the robustness of the approach.

Image formation involves understanding sensor characteristics and object reflectance. In dentistry, an accurate 3-D representation of the human jaw may be used for diagnostic and treatment purposes. Photogrammetry can offer a flexible, cost effective solution for accurate 3-D representation of the human teeth, which can be used for diagnostic and treatment purposes. Nonetheless there are several challenges, such as the non-friendly image acquisition environment inside the human mouth, problems with lighting and errors due to the data acquisition sensors. In this paper, we focus on the 3D surface reconstruction aspect for human teeth based on a single image. We introduce a more realistic formulation of the shape-from-shading (SFS) problem by considering the image formation components; the camera, the light source, and the surface reflectance. We propose a non-Lambertian SFS algorithm under perspective projection which benefits from camera calibration parameters. We take into account the attenuation of illumination due to near-field imaging. The surface reflectance is modeled using Oren-Nayar-Wolff model which accounts for the retro-reflection case. Our experiments provide promising quantitative metric results for the proposed approach.

This paper addresses the problem of lens distortion calibration from a single image of a smooth, textureless surface. To the best of our knowledge, this has not been addressed before in the literature. We show that this is possible, both theoretically and practically, taking advantage of some local shading measures that vary nonlinearly as a function of lens distortion. The proposed method is easy to use and requires no specific feature extraction from images. Experiments on simulations and real data are reported and compared to well-known techniques.

Discounting subsurface scattering and surface emittance, this work seeks to address the question; how to accurately represent image irradiance under unknown general illumination and reflectance, given that a surface point sees its surrounding world through the local upper hemisphere oriented by the surface normal at this point. In this paper, we formulate the image formation process of isotropic surfaces under arbitrary distant illumination in the frequency space while addressing the physical compliance of hemispherical basis for representing surface reflectance, e.g. Helmholtz reciprocity and isotropy. This representation is further reduced in dimensionality through analytically deriving the principal components of the image irradiance. A database of natural illumination and real world surface materials are used to compute a general-purpose basis which captures the full behavior of illumination and reflectance in a lower-dimensional subspace for image irradiance representation. Compared to similar basis in literature, our proposed basis achieves higher accuracy levels at lower illumination orders.

Thermal images are often paired with visible spectrum images captured simultaneously-as surface features are easier to identify with proper illumination-to determine the thermal feature locations. Not all thermal images are necessarily recorded this way, yet determining salient feature regions is beneficial to other tasks. The forehead region is useful to the field of vital signs analysis and the goal of this work is automatically identifying the forehead region solely from thermal images. The anatomy of the forehead and its thermo-dynamic properties lend itself to minimizing the variance of a facial thermogram. Experiments conducted indoors on 32 subjects under normal, exercise and pain conditions. The proposed solution identifies the forehead region with 90-95% accuracy in 80ms or less, without initialization or training data, and can be easily implemented in parallel.

Polarization imaging can give information about surface shape, and roughness. Polarization has been used for shape recovery, but with convex/concave reconstruction ambiguity. In this paper, we present a direct method to shape recovery using both polarization and shading that resolves this ambiguity, without the need for nonlinear optimization routines. Several experiments on synthetic and real datasets are reported to evaluate the proposed method. The method consistently outperforms some well-known methods based on polarization information alone.

In this paper, we propose a novel approach for 3D surface reconstruction of the human jaw. Due to the difficulties of setting up a data acquisition system inside the mouth, we use an intra-oral camera to capture a sequence of calibrated images. These images are registered together to build a panoramic view of the jaw. We incorporate a shape from shading(SFS) algorithm that benefits from camera calibration parameters to build a 3D model from the panoramic image obtained from the previous stage. Our approach results in a 3D surface which has more fine details compared with those resulting from other literature techniques. We will demonstrate different artificial jaws surfaces reconstruction to show the efficiency of our system. 1

The visual appearance of real-world surfaces is the net result of surface reflectance characteristics when exposed to illumination. Appearance models can be constructed using phenomenological models which capture surface appearance through mathematical modeling of the reflection process. This yields an integral equation, known as reflectance equation, describing the surface radiance, which depends on the interaction between the incident light field and the surface bidirectional reflectance distribution function (BRDF). The BRDF is a function defined on the cartesian product of two hemispheres corresponding to the incident and outgoing directions; the natural way to represent such a hemispherical function is to use hemispherical basis. However, due to their compactness in the frequency space, spherical harmonics (SH) have been extensively used for this purpose. In this paper, we address the geometrical compliance of hemispherical basis for representing surface BRDF. We propose a tensor product of the hemispherical harmonics (HSH) to provide a compact and efficient representation for arbitrary BRDFs, while satisfying the Helmholtz reciprocity property. We provide an analytical analysis and experimental justification that for a given approximation order, our proposed hemispherical basis provide better approximation accuracy when compared to Zernike-based basis, while avoiding the high computational complexity inherited from such polynomials. We validate our proposed Helmholtz HSH-based basis functions on Oren-Nayar and Cook-Torrance BRDF physical models.

Face and eye detection algorithms are deployed in a wide variety of applications. Unfortunately, there has been no quantitative comparison of how these detectors perform under difficult circumstances. We created a dataset of low light and long distance images which possess some of the problems encountered by face and eye detectors solving real world problems. The dataset we created is composed of reimaged images (photohead) and semi-synthetic heads imaged under varying conditions of low light, atmospheric blur, and distances of 3m, 50m, 80m, and 200m. This paper analyzes the detection and localization performance of the participating face and eye algorithms compared with the Viola Jones detector and four leading commercial face detectors. Performance is characterized under the different conditions and parameterized by per-image brightness and contrast. In localization accuracy for eyes, the groups/companies focusing on long-range face detection outperform leading commercial applications.

This paper proposes a model-based approach for 3D facial shape recovery using a small set of feature points from an input image of unknown pose and illumination. Previous model-based approaches usually require both texture (shading) and shape information from the input image in order to perform 3D facial shape recovery. However, the methods discussed here need only the 2D feature points from a single input image to reconstruct the 3D shape. Experimental results show acceptable reconstructed shapes when compared to the ground truth and previous approaches. This work has potential value in applications such face recognition at-a distance (FRAD), where the classical shape-from-X (e.g., stereo, motion and shading) algorithms are not feasible due to input image quality.

Surface irradiance signals are turned into outgoing radiance through the surface reflectance function, which can be significantly perturbed by the illumination conditions. Due to their low-frequency nature, irradiance signals can be represented using low-order basis functions, where spherical harmonics (SH) have been extensively used to provide such basis. When capturing image irradiance from a single viewpoint, the visible part of the object's surface constructs the upper hemisphere of the surface normals where the SH are no longer orthonormal. This reduced domain paves the way for even lower-dimensional approximation since full spherical representation is not needed. While harmonic basis are known to be optimal under distant light, light coming from near-by objects and indoor environments are common near light scenarios; it is essential to relax distant light assumption. Considering light source(s) distributed uniformly over the upper hemisphere, we propose the use of hemispherical harmonics (HSH) to model image irradiance of convex Lambertian objects perceived from single viewpoint under unknown near illumination. We prove analytically, and experimentally validated, that the Lambertian kernel has a more compact harmonic expansion in the hemispherical domain when compared to its spherical counterpart. We illustrate that HSH provide an efficient and accurate low-dimensional representation of image irradiance of Lambertian objects under near lighting conditions in contrast to SH.

We propose a new system for building a 3D database for human teeth. Real extracted human teeth are scanned using a cone-beam CT scanner to build. The teeth models are segmented from the resulting images by means of level sets. The database includes 3D models of these teeth after minimizing global differences between the models. An affine transformation function is used to model the differences between the models. The transformation parameters are estimated by minimizing a least squares energy. A total of 280 teeth 3-D models have been created to build the real database and demonstrate the performance of the proposed framework.

Our paper handles the elastic shape registration by combining the incremental free form deformation (IFFD) with the point- based registration technique using the sum of least squares method. The iterative closest point (ICP) algorithm is used as criteria to establish point correspondences in each level of the IFFD framework. The IFFD control lattice resolution is in- creased step by step to achieve a satisfactory deformation of the source shape to exactly match the target boundaries. Our point-based registration is based on least squares that measure the Euclidean distance between source and target boundaries in addition to the shape constrains. The minimization gives a closed form solution of the lattice control points positions. Promising results will be demonstrated for closed and open shapes and structures. The approach can also work for structures that contain multiple parts without any problems.

In this paper, we propose a new shape based segmentation and registration of the vertebral bodies (VBs) in clinical computed tomography (CT) images. The VB and surrounding organs have very close gray level information and there are no strong edges in some CT images. To overcome these challenges, image appearance and shape information of VBs are used. There are three phases of our experiments: i) the detection of the VB region using the Matched filter, ii) initial segmentation using the graph cuts which integrates the intensity and spatial interaction models, iii) registration of the shape priors and initially segmented region to obtain the final segmentation. Preliminary results show that our proposed algorithm gives very encouraging results and can solve many segmentation and registration problems.

We propose a new shape-based segmentation approach using the statistical shape prior and level sets method. The segmentation depends on the image information and shape prior. Training shapes are grouped to form a probabilistic model. The shape model is embedded into the image domain taking in consideration the evolution of a contour represented by a level set function. The evolution of the front gathers information from the image intensities and shape prior. The segmentation approach is applied in segmenting the vertebral bodies in CT images. Our results shows that the technique is accurate and robust compared with the other alternative in the literature.

This paper introduces a fuzzy framework for the simultaneous segmentation and registration in addition to bias field correction of MRI datasets. The framework utilizes prior information which may be available about the tissues' mean intensities and tissues' distribution through the datasets. Moreover it works on the given, original image intensities without any logarithmic transformation and thus produces more accurate results and faster performance. The algorithm is evaluated using simulated and real brain MRI data. The results show that the algorithm has indeed improved the segmentation accuracy.

A surface reflectance function represents the process of turning irradiance signals into outgoing radiance. Irradiance signals can be represented using low-order basis functions due to their low-frequency nature. Spherical harmonics (SH) have been used to provide such basis. However the incident light at any surface point is defined on the upper hemisphere; full spherical representation is not needed. We propose the use of hemispherical harmonics (HSH) to model images of convex Lambertian objects under distant illumination. We formulate and prove the addition theorem for HSH in order to provide an analytical expression of the reflectance function in the HSH domain. We prove that the Lambertian kernel has a more compact harmonic expansion in the HSH domain when compared to its SH counterpart. Our experiments illustrate that the 1st order HSH outperforms 1st and 2nd order SH in the process of image reconstruction as the number of light sources grows.

Lung nodules from low dose CT (LDCT) scans may be used for early detection of lung cancer. However, these nodules vary in size, shape, texture, location, and may suffer from occlusion within the tissue. This paper presents an approach for segmentation of lung nodules detected by a prior step. First, regions around the detected nodules are segmented; using automatic seed point placement levels sets. The outline of the nodule region is further improved using the curvature characteristics of the segmentation boundary. We illustrate the effectiveness of this method for automatic segmentation of the Juxta-pleural nodules.

Bone mineral density (BMD) measurements and fracture analysis of the spine bones are restricted to the Vertebral bodies (VBs), especially the trabecular bones (TBs). In this paper, we propose a novel, fast, and robust 3D framework to segment VBs and trabecular bones in clinical computed tomography (CT) images without any user intervention. The Matched filter is employed to detect the VB region automatically. To segment the whole VB, the graph cuts method which integrates a linear combination of Gaussians (LCG) and Markov Gibbs Random Field (MGRF) is used. Then, the cortical and trabecular bones are segmented using local volume growing methods. Validity was analyzed using ground truths of data sets (expert segmentation) and the European Spine Phantom (ESP) as a known reference. Experiments on the data sets show that the proposed segmentation approach is more accurate than other known alternatives.

Osteoporosis is a bone disease characterized by a reduction in bone mass, resulting in an increased risk of fractures. To diagnose the osteoporosis accurately, bone mineral density (BMD) measurements and fracture analysis (FA) of the Vertebral bodies (VBs) are required. In this paper, we propose a robust and 3D shape based method to segment VBs in clinical computed tomography (CT) images in order to make BMD measurements and FA accurately. In this experiment, image appearance and shape information of VBs are used. In the training step, 3D shape information is obtained from a set of data sets. Then, we estimate the shape variations using a distance probabilistic model which approximates the marginal densities of the VB and background in the variability region. In the segmentation step, the Matched filter is used to detect the VB region automatically. We align the detected volume with 3D shape prior in order to be used in distance probabilistic model. Then, the graph cuts method which integrates the linear combination of Gaussians (LCG), Markov Gibbs Random Field (MGRF), and distance probabilistic model obtained from 3D shape prior is used.

In this paper, we propose a new 3D framework to identify and segment VBs and TBs in clinical computed tomography (CT) images without any user intervention. The Matched filter is employed to detect the VB region automatically on axial axis. To identify the VB on coronal and sagittal axis, we use a new developed approach based on 4 points automatically placed on cortical shell. To segment the identified VB, the graph cuts method which integrates a linear combination of Gaussians (LCG) and Markov Gibbs Random Field (MGRF) are used. Then, the cortical and trabecular bones are segmented using local volume growing methods. Experiments on the data sets show that the proposed segmentation approach is more accurate than other known alternatives.

Accurate 3-D modeling of the human teeth helps patients avoid the discomfort of the mold process, and improves the data accuracy for oral surgeons, orthodontists and dental care personnel. Since the surface of the human tooth is almost textureless, Shape from Shading (SFS) has been successfully adopted in solving this problem. This paper evaluates 3-D tooth reconstruction using three SFS models. Based on quantitative error analysis and curvature analysis using Robust Point Matching (RPM) and visualization results, the assumption of a perspective camera projection and an Oren-Nayar reflectance model has been proved to be the most ideal for extracting crown information from an image of a human tooth. Three reconstructed teeth have been shown to demonstrate the performance of this model.

This work presents a novel approach in automatic detection of the lung nodules and is compared with respect to parametric nodule models in terms of sensitivity and specificity. A Statistical method is used for generating data driven models of the nodules appearing in low dose CT (LDCT) scans of the human chest. Four types of common lung nodules are analyzed using the Procrustes based AAM method to create descriptive lung nodules. Performance of the new nodule models on clinical datasets is significant over parametric nodule models in both sensitivity and specificity. The new nodule modeling approach is also applicable for automatic classification of nodules into pathologies given a descriptive database. This approach is a major step forward for early diagnosis of lung cancer.

This paper deals with segmentation of the lung tissues from low dose CT (LDCT) scans of the chest. Goal is correct segmentation as well as maintaining the details of the lung region in the chest cavity. In particular, it is essential that the lung nodules inside the lungs as well as on the boundary regions be maintained for subsequent steps that aim at automatic detection and classification of nodules from LDCT scanning; a step for early diagnosis of lung cancer. An approach for segmentation based on combination of EM algorithm and morphological operations is presented. This algorithm is compared with respect two other approaches that are based on level sets and energy optimization by the Graph Cuts technique. Performance evaluation is conducted on a labeled data set from the Early Lung Cancer Action Program (ELCAP) database. The new segmentation approach provides comparable results to Level sets and Graph-Cuts, with the advantage of faster execution time, and minimal user interception.

We discuss a statistical shape-from-shading framework for images of general and unknown illumination. To overcome arbitrary illumination, the framework makes use of the fact that general lighting can be expressed using low-order spherical harmonics for convex Lambertian objects. We cast the classical shape-from-shading equation as a Partial Least Squares (PLS) regression problem, which allows for the rapid computation of the solution. Results show accurate shape recovery with respect to ground truth data.

Exposure levels (X-ray tube amperage and peak kilovoltage) are associated with various noise levels and radiation dose. When higher exposure levels are applied, the images have higher signal to noise ratio (SNR) in the CT images. However, the patient receives higher radiation dose in this case. In this paper, we use our robust 3D framework to segment vertebral bodies (VBs) in clinical computed tomography (CT) images with different noise levels. The Matched filter is employed to detect the VB region automatically. In the graph cuts method, a VB (object) and surrounding organs (background) are represented using a gray level distribution models which are approximated by a linear combination of Gaussians (LCG). Initial segmentation based on the LCG models is then iteratively refined by using Markov Gibbs random field(MGRF) with analytically estimated potentials. Experiments on the data sets show that the proposed segmentation approach is more accurate and robust than other known alternatives.

Bone mineral density (BMD) measurements and fracture analysis of the spine bones are restricted to the Vertebral bodies (VBs). In this paper, we propose a novel 3D shape based method to segment VBs in clinical computed tomography (CT) images without any user intervention. The proposed method depends on both image appearance and shape information. 3D shape information is obtained from a set of training data sets. Then, we estimate the shape variations using a distance probabilistic model which approximates the marginal densities of the VB and background in the variability region. To segment a VB, the Matched filter is used to detect the VB region automatically. We align the detected volume with 3D shape prior in order to be used in distance probabilistic model. Then, the graph cuts method which integrates the linear combination of Gaussians (LCG), Markov Gibbs Random Field (MGRF), and distance probabilistic model obtained from 3D shape prior is used. Experiments on the data sets show that the proposed segmentation approach is more accurate than other known alternatives.

The quality of the lung nodule models determines the success of lung nodule detection. This paper describes aspects of our data-driven approach for modeling lung nodules using the texture and shape properties of real nodules to form an average model template per nodule type. The ELCAP low dose CT (LDCT) scans database is used to create the required statistics for the models based on modern computer vision techniques. These models suit various machine learning approaches for nodule detection including Bayesian methods, SVM and Neural Networks, and computations may be enhanced through genetic algorithms and Adaboost. The eminence of the new nodule models are studied with respect to parametric models showing significant improvements in both sensitivity and specificity.

This paper introduces a framework for long-distance face recognition using dense and sparse stereo reconstruction, with texture of the facial region. Two methods to determine correspondences of the stereo pair are used in this paper: (a) dense global stereo-matching using maximum-a-posteriori Markov Random Fields (MAP-MRF) algorithms and (b) Active Appearance Model (AAM) fitting of both images of the stereo pair and using the fitted AAM mesh as the sparse correspondences. Experiments are performed using combinations of different features extracted from the dense and sparse reconstructions, as well as facial texture. The cumulative rank curves (CMC), which are generated using the proposed framework, confirms the feasibility of the proposed work for long distance recognition of human faces.

We present a 3D segmentation technique of trabecular (cancellous) bones in CT images of Vertebral bodies (VBs). In order to be used for Bone Mineral Density (BMD) measurements, the cortical and trabecular bones are subsequently segmented using graph cuts method and local volume growing methods separately. In the final step, we measure our segmentation accuracy for each method. Validity was analyzed using ground truths of data sets and the European Spine Phantom (ESP). Preliminary results are very encouraging and a reproducibility of the results was achieved for 16 data sets. The average segmentation error is below 2.0% for both methods.

Colorectal cancer includes cancer of the colon, rectum, anus and appendix. Since it is largely preventable, it is extremely important to detect and treat the colorectal cancer in the earliest stage. Virtual colonoscopy is an emerging screening technique for colon cancer. One component of virtual colonoscopy, image pre-processing, is important for colonic polyp detection/ diagnosis, feature extraction and classification. This paper introduces a general variational approach based framework for a computer aided diagnosis system for colorectal cancer. It includes techniques for 3D colon segmentation, 3D colon object reconstruction by iso-surface generation, and 3D centerline extraction. The proposed framework has been validated on 22 real CT colonography datasets.

In this paper, we aim to provide the further evaluation on the recently proposed visualization technique known as Virtual Fly-Over for virtual colonoscopy by using clinical CT colonography datasets. The proposed method mainly consists of two steps: splitting the entire colon anatomy into exactly two halves, and assigning a virtual camera to each half to perform fly-over navigation. We have quantitatively validated the effectiveness of the proposed technique by using 22 real CT colonography datasets. Experimental results have shown that the average surface visibility coverage is 97.49±1.02%.

We introduce a framework for face recognition at a distance based on sparse-stereo reconstruction. We develop a 3D acquisition system that consists of two CCD stereo cameras mounted on pan-tilt units with adjustable baseline. We first detect the facial region and extract its landmark points, which are used to initialize an AAM mesh fitting algorithm. The fitted mesh vertices provide point correspondences between the left and right images of a stereo pair; stereo-based reconstruction is then used to infer the 3D information of the mesh vertices. We perform experiments regarding the use of different features extracted from these vertices for face recognition. The cumulative rank curves (CMC), which are generated using the proposed framework, confirms the feasibility of the proposed work for long distance recognition of human faces with respect to the state-of-the-art [3].

We present a new statistical shape-from-shading framework for images of unknown illumination. The object (e.g., face) to be reconstructed is described by a parametric model. To deal with arbitrary illumination, the framework makes use of recent results that general lighting can be expressed using low-order spherical harmonics for convex Lambertian objects. The classical shape-from-shading equation is modified according to this framework. Results show accurate shape recovery with respect to ground truth data.

Representing a 3D shape by a set of 1D curves that are locally symmetric with respect to its boundary (i.e., curve skeletons) is of importance in several machine intelligence tasks. This paper presents a fast, automatic, and robust variational framework for computing continuous, subvoxel accurate curve skeletons from volumetric objects. A reference point inside the object is considered a point source that transmits two wave fronts of different energies. The first front (beta-front) converts the object into a graph, from which the object salient topological nodes are determined. Curve skeletons are tracked from these nodes along the cost field constructed by the second front (alpha-front) until the point source is reached. The accuracy and robustness of the proposed work are validated against competing techniques as well as a database of 3D objects. Unlike other state-of-the-art techniques, the proposed framework is highly robust because it avoids locating and classifying skeletal junction nodes, employs a new energy that does not form medial surfaces, and finally extracts curve skeletons that correspond to the most prominent parts of the shape and hence are less sensitive to noise.

This paper presents a new symmetric shape from shading (SFS) algorithm where the self-ratio image irradiance equation proposed by Zhao and Chellappa is formulated as a partial differential equation (PDE) with a Dirichlet boundary condition. This PDE is solved using the Lax-Friedrichs sweeping method. The potential of the proposed symmetric SFS algorithm is demonstrated by several experiments on synthetic and real data. As an application, the proposed algorithm is used for 3-D face reconstruction.

This work proposes a new MAP-based segmentation framework of multimodal images. In this work a joint MGRF model is used to describe the image. The main focus here is a more accurate model identification. For a known number of classes in the given image, the empirical distributions of this image signals are precisely approximated by a LCG distributions with positive and negative components. Gibbs potential, which is used to identify the spatial interaction between the neighboring pixels, is analytically estimated. Finally, an energy function using the previous models is formulated and is globally minimized using graph cuts. Experiments show that the developed technique gives promising accurate results compared to other known algorithms.

We propose a new approach that approximates an empirical probability density function of scalar data with a linear combination of Gaussians (LCG). The proposed algorithm approximates the marginal density of each class using a Gaussian distribution. Number of the classes and their distributions parameters are estimated using a new Akaike Information Criterion (AlC)-type criterion and the Expectation- Maximization (EM) approach. Each class does not follow perfect Gaussian distribution so we refine the initial LCG model using a modified EM algorithm. The modified EM algorithm approximates the marginal density of each class using a LCG with positive and negative components. Experiments in segmenting multimodal medical images show that the developed technique gives promising accurate results.

In this paper, we propose a novel anisotropic 3D surface evolution model for detecting protrusion shape based colonic polyp on the curved surface. The important feature of the proposed model is that it can detect protrusions with both convex and concave shapes. Protrusion shapes are defined as the extension beyond the usual limits or above a plane surface. Based on Gaussian and mean curvature flows, the approach works by locally deforming the convex or concave surface until the second principal curvature goes to zero. The diffusion directions are changed to prevent convex surfaces from converting into concave shapes, and vice versa. The deformation field quantitatively measures the amount of protrudeness. The proposed method has been evaluated by using synthetic phantoms and real colon datasets.

This paper addresses the shape-based segmentation problem using level sets. In particular, we propose a fast algorithm to solve the piece-wise constant Chan-Vese segmentation model with shape priors and labeling functions. Instead of directly solving the underlying PDE, we calculate the energy and check how it changes when we move image points from inside the region enclosed by the evolving interface to the outside region and vice-vera. This algorithm is then extended to the case of multi-phase Chan-Vese model, with multiple selective shape priors and a corresponding labeling function for each prior. This makes our algorithm different from that in [1] and other similar works in different aspects. On one hand, our algorithm is not restricted to two regions, but allows segmentation into several regions. On the other hand, more than one shape prior can be taken into account in our implementation. In addition, the proposed algorithm improves dramatically the computational speed. Experimental results, on both synthetic and real images, demonstrate the performance of our algorithm and the computational improvements it offers.

In local invariant features, the detected interest points using scale-space representations are considered the most robust to many variations in the imaging conditions. The existing approaches extract interest points at scale-space differential singularities. In these approaches, all detected interest points are considered equally robust without taking in consideration the variable sensitivities of these interest points with respect to noise effects. In this paper, we analyze the robustness of detected interest points at scale-space singularities against noise effects. Also, we propose a novel quantitative stability measure of these interest points. The evaluation results show the effectiveness of the proposed stability measure in estimating the robustness of the detected interest points to noise effects.

This paper proposes a new analytical method for estimating parameters of a homogeneous isotropic Potts model with an asymmetric Gibbs potential function. The model is generalized by including both pairwise and triple cliques. The maximum likelihood estimates of the cliques potentials are obtained by a further elaboration of the approximate analytical estimator proposed in. Experiments with synthetic textures have shown that our potential estimates are more accurate and practicable than their counterparts obtained with classical methods.

Most of the shape from shading (SFS) algorithms have been developed under the simplifying assumptions of a Lambertian surface, an orthographic projection, and a distant light source. Due to the difficulty of the SFS problem, only a small number of algorithms have been proposed for surfaces with non-Lambertian reflectance, and among those, only very few algorithms are applicable for surfaces with specular and diffuse reflectance. In this paper we propose a unified framework that is capable of solving the SFS problem under various settings of imaging conditions i.e., Lambertian or non-Lambertian, orthographic or perspective projection, and distant or nearby light source. The proposed algorithm represents the image irradiance equation of each setting as an explicit Partial Differential Equation (PDF). In our implementation we use the Lax-Friedrichs sweeping method to solve this PDF. To demonstrate the efficiency of the proposed algorithm, several comparisons with the state of the art of the SFS literature are given.

We present a novel model for measurement of the arterial pulse from the superficial temporal artery (STA) using passive thermal infrared (IR) sensors. The proposed approach has a physical and physiological basis and as such is of fundamental nature. Thermal IR camera is used to capture the heat pattern from superficial arteries, and a blood vessel model is used to describe the pulsatile nature of the blood flow. A multiresolution wavelet-based signal analysis approach is used to extract the arterial pulse waveform, which lends itself to various physiological measurements. We validate the results using a traditional contact vital-sign monitor as a ground truth. Eight people of different age, race and gender have been tested in our study consistent with IRB approval. The resultant arterial pulse waveforms exactly matched the ground-truth readings. The essence of our approach is the automatic detection of region of arterial pulse measurement (ROM), from which the arterial pulse waveform is extracted. To the best of our knowledge, the correspondence between non-contact thermal IR imaging based measurements of the arterial pulse in the time domain and traditional contact approaches has never been reported in the literature.

This paper introduces a new method for shape registration by matching vector distance functions. The vector distance function representation is more flexible than the conventional signed distance map since it enables us to better control the shapes registration process by using more general transformations. Based on this model, a variational frame work is proposed for the global and local registration of shapes which does not need any point correspondences. The optimization criterion can handle efficiently the estimation of the global registration parameters. A closed form solution is provided to handle an incremental free form deformation model for covering the local deformations. This is an advantage over the gradient descent optimization which is biased towards the initialization and is more time consuming. Results of real shapes registration will be demonstrated to show the efficiency of the proposed approach with small and large global/local deformations.

In this paper, we propose a new variational framework for computing continuous curve skeletons from discrete objects that are suitable for structural shape representation. We have derived a new energy function, which is proportional to some medialness function, such that the minimum cost path between any two medial voxels in the shape is a curve skeleton. We have employed two different medialness functions; the Euclidean distance field and a variant of the magnitude of the gradient vector flow (GVF), resulting in two different energy functions. The first energy controls the identification of the shape topological nodes from which curve skeletons start, while the second one controls the extraction of curve skeletons. The accuracy and robustness of the proposed framework are validated both quantitatively and qualitatively against competing techniques as well as several 3D shapes of different complexity.

This paper presents a new method for recovering the shape of hybrid surfaces that have both diffuse reflection and specular reflection using shape from shading (SFS). The image irradiance equation has been derived as an explicit partial differential equation (PDE) under the assumptions of orthographic camera projection and distant point light source. The reflectance model of Ward has been used to express the hybrid reflection as a linear combination from the diffuse and specular components. The resulting PDE is solved using the Lax-Friedrichs sweeping method. The proposed algorithm is evaluated by using both synthetic data and real images and the experimental results show the efficiency of the approach.

Curvature-based geometric features have been proven to be important for colonic polyp detection. In this paper, we present an automatic detection framework and color coding scheme to highlight the detected polyps. The key idea is to place the detected polyps at the same locations in a newly created polygonal dataset with the same topology and geometry properties as the triangulated mesh surface of real colon dataset, and assign different colors to the two separated datasets to highlight the polyps. Finally, we validate the proposed framework by computer simulated and real colon datasets. For fifteen synthetic polyps with different shapes and different sizes, the sensitivity is 100%, and false positive is 0. For four real colon datasets, the proposed algorithm has achieved the sensitivity of 75%.

This paper addresses the issue of assessing the quality of the clusters found by fuzzy and hard clustering algorithms. In particular, it seeks an answer to the question on how well cluster validity indexes can automatically determine the appropriate number of clusters that represent the data. The paper surveys several key existing solutions for cluster validity in the domain of image segmentation. In addition, it suggests two new indexes. The first one is based on Akaike's information criterion (AIC). While AIC was devoted to other domains such as statistical estimation of model fitting, it is implemented here for the first time as a validation index. The second index is developed from the well-established idea of cross-validation. The existing and new indexes are evaluated and compared on several synthetic images corrupted with noise of varying levels and volumetric MR data.

Geometric analysis of postmortem normal and autistic human subjects reveal distinctions in deformations in the corpus callosum (CC) that may be used for image analysis-based studies of autism. Preliminary studies showed that the CC of autistic patients is quite distinct from normal controls. We use an implicit vector representation of CC to carry out the registration process which measures the differences between different CC's. This paper introduces a new method for the 2D shape registration problem by matching vector distance functions. A variational frame work is proposed for the global and local registration of CC's. A gradient descent optimization is used which can efficiently handle both the rigid and the non-rigid operations together. The registration of real CC extracted from postmortem data sets demonstrates the potential of the proposed approach.

This work deals with the problem of shape registration. Broadly speaking, the problem is that of establishing point-wise correspondences in two different shapes of arbitrary dimension and topology. This problem is a fundamental component in numerous image and vision applications. We propose a variational framework for a dense global-to-local 2D shape registration. Affine transformations are accounted for using vector distance functions. Based on this representation, a dissimilarity measure between the two shapes is minimized to recover the global matching parameters. The local coordinate transformation between the two shapes is explicitly estimated by solving a regularized non-linear PDE-based motion model. Various experimental results are presented and discussed to show the potential of the proposed framework with a finite element (FE)-based validation of its performance.

A wide range of computer vision applications require an accurate solution of a particular Hamilton- Jacobi (HJ) equation, known as the Eikonal equation. In this paper, we propose an improved version of the fast marching method (FMM) that is highly accurate for both 2D and 3D Cartesian domains. The new method is called multi-stencils fast marching (MSFM), which computes the solution at each grid point by solving the Eikonal equation along several stencils and then picks the solution that satisfies the upwind condition. The stencils are centered at each grid point and cover its entire nearest neighbors. In 2D space, 2 stencils cover the 8-neighbors of the point, while in 3D space, 6 stencils cover its 26-neighbors. For those stencils that are not aligned with the natural coordinate system, the Eikonal equation is derived using directional derivatives and then solved using higher order finite difference schemes. The accuracy of the proposed method over the state-of-the-art FMM-based techniques has been demonstrated through comprehensive numerical experiments.

In this paper, we revisit the implicit front representation and evolution using the vector level set function (VLSF) proposed in [1]. Unlike conventional scalar level sets, this function is designed to have a vector form. The distance from any point to the nearest point on the front has components (projections) in the coordinate directions included in the vector function. This kind of representation is used to evolve closed planar curves and 3D surfaces as well. Maintaining the VLSF property as the distance projections through evolution will be considered together with a detailed derivation of the vector partial differential equation (PDE) for such evolution. A shape-based segmentation framework will be demonstrated as an application of the given implicit representation. The proposed level set function system will be used to represent shapes to give a dissimilarity measure in a variational object registration process. This kind of formulation permits us to better control the process of shape registration, which is an important part in the shape-based segmentation framework. The method depends on a set of training shapes used to build a parametric shape model. The color is taken into consideration besides the shape prior information. The shape model is fitted to the image volume by registration through an energy minimization problem. The approach overcomes the conventional methods problems like point correspondences and weighing coefficients tuning of the evolution (PDEs). It is also suitable for multidimensional data and computationally efficient. Results in 2D and 3D of real and synthetic data will demonstrate the efficiency of the framework.

SIFT has been proven to be the most robust local invariant feature descriptor. SIFT is designed mainly for gray images. However, color provides valuable information in object description and matching tasks. Many objects can be misclassified if their color contents are ignored. This paper addresses this problem and proposes a novel colored local invariant feature descriptor. Instead of using the gray space to represent the input image, the proposed approach builds the SIFT descriptors in a color invariant space. The built Colored SIFT (CSIFT) is more robust than the conventional SIFT with respect to color and photometrical variations. The evaluation results support the potential of the proposed approach.

Lambert’s model for diffuse reflection is a main assumption in most of shape from shading (SFS) literature. Even with this simplified model, the SFS is still a difficult problem. Nevertheless, Lambert’s model has been proven to be an inaccurate approximation of the diffuse component of the surface reflectance. In this paper, we propose a new solution of the SFS problem based on a more comprehensive diffuse reflectance model: the Oren and Nayar model. In this work, we slightly modify this more realistic model in order to take into account the attenuation of the illumination due to distance. Using the modified non-Lambertian reflectance, we design a new explicit Partial Differential Equation (PDE) and then solve it using Lax-Friedrichs Sweeping method. Our experiments on synthetic data show that the proposed modeling gives a unique solution without any information about the height at the singular points of the surface. Additional results for real data are presented to show the efficiency of the proposed method . To the best of our knowledge, this is the first non-Lambertian SFS formulation that eliminates the concave/convex ambiguity which is a well known problem in SFS.

A wide range of computer vision applications such as distance field computation, shape from shading, and shape representation require an accurate solution of a particular Hamilton-Jacobi (HJ) equation, known as the Eikonal equation. Although the fast marching method (FMM) is the most stable and consistent method among existing techniques for solving such equation, it suffers from large numerical error along diagonal directions as well as its computational complexity is not optimal. In this paper, we propose an improved version of the FMMthat is both highly accurate and computationally efficient for Cartesian domains. The new method is called the multi-stencils fast marching (MSFM), which computes the solution at each grid point by solving the Eikonal equation along several stencils and then picks the solution that satisfies the fast marching causality relationship. The stencils are centered at each grid point x and cover its entire nearest neighbors. In 2D space, 2 stencils cover the 8-neighbors of x, while in 3D space, 6 stencils cover its 26-neighbors. For those stencils that are not aligned with the natural coordinate system, the Eikonal equation is derived using directional derivatives and then solved using a higher order finite difference scheme.

A new approach to align an image of a textured object with a given prototype is proposed. Visual appearance of the images, after equalizing their signals, is modeled with a Markov-Gibbs random field with pairwise interaction. Similarity to the prototype is measured by a Gibbs energy of signal cooccurrences in a characteristic subset of pixel pairs derived automatically from the prototype. An object is aligned by an affine transformation maximizing the similarity by using an automatic initialization followed by gradient search. Experiments confirm that our approach aligns complex objects better than popular conventional algorithms.

A new physically justified adaptive probabilistic model of blood vessels on magnetic resonance angiography (MRA) images is proposed. The model accounts for both laminar (for normal subjects) and turbulent blood flow (in abnormal cases like anemia or stenosis) and results in a fast algorithm for extracting a 3D cerebrovascular system from the MRA data. Experiments with real data sets confirm the high accuracy of the proposed approach.

This paper presents a statistical learning-based solution to the camera calibration problem in which the support vector machines (SVM) are used for the estimation of the projection matrix elements. The projection matrix is obtained explicitly by using a dot product kernel in the formulation of the SVM algorithm. The mean field theory is used to approximate an efficient learning procedure for the SVM algorithm. In order to assess the robustness of the proposed approach against noise, the experiments using synthetic data are carried out at different noise levels. The proposed approach is evaluated also with real 3D reconstruction experiments. The experimental results illustrate that the proposed calibration approach is efficient and more robust against noise than other known approaches for camera calibration.

The main focus of this paper is the shape representation and registration using vector level set functions. This powerful representation is more flexible than conventional signed distance level sets since it enables us to control the shapes registration process by using more complicated transforms. Based on this model, a variational frame work is proposed for the rigid and non-rigid registration of shapes which can be extended to the higher dimensional case in a straightforward manner and also does not need any point correspondences. The optimization criterion presented can handle efficiently both the rigid and the non-rigid operations together. Experimental results in 2D for real shapes registration are discussed to show the efficiency of the approach with small and large global deformations of shapes.

We propose new techniques for unsupervised segmentation of multimodal grayscale images such that each region-of-interest relates to a single dominant mode of the empirical marginal probability distribution of grey levels. We follow the most conventional approaches in that initial images and desired maps of regions are described by a joint Markov-Gibbs random field (MGRF) model of independent image signals and interdependent region labels. However, our focus is on more accurate model identification. To better specify region borders, each empirical distribution of image signals is precisely approximated by a linear combination of Gaussians (LCG) with positive and negative components. We modify an expectation-maximization (EM) algorithm to deal with the LCGs and also propose a novel EM-based sequential technique to get a close initial LCG approximation with which the modified EM algorithm should start. The proposed technique identifies individual LCG models in a mixed empirical distribution, including the number of positive and negative Gaussians. Initial segmentation based on the LCG models is then iteratively refined by using the MGRF with analytically estimated potentials. The convergence of the overall segmentation algorithm at each stage is discussed. Experiments show that the developed techniques segment different types of complex multimodal medical images more accurately than other known algorithms.

Deformable or active contour, and surface models are powerful image segmentation techniques. We introduce a novel fast and robust bi-directional parametric deformable model which is able to segment regions of intricate shape in multi-modal greyscale images. The power of the algorithm in terms of computation time and robustness is owing to the use of joint probabilities of the signals and region labels in individual points as external forces guiding the model evolution. These joint probabilities are derived from a Markov– Gibbs random field (MGRF) image model considering an image as a sample of two interrelated spatial stochastic processes. The low level process with conditionally independent and arbitrarily distributed signals relates to the observed image whereas its hidden map of regions is represented with the high level MGRF of interdependent region labels. Marginal probability distributions of signals in each region are recovered from a mixed empirical signal distribution over the whole image. In so doing, each marginal is approximated with a linear combination of Gaussians (LCG) having both positive and negative components. The LCG parameters are estimated using our previously proposed modification of the EM algorithm, and the high-level Gibbs potentials are computed analytically. Comparative experiments show that the proposed model outlines complicated boundaries of different modal objects much more accurately than other known counterparts. 1

In this paper, we present a novel framework for computing centerlines for both 2D and 3D shape analysis. The framework works as follows: an object centerline point is selected automatically as the point of global maximum Euclidean distance from the boundary, and is considered a point source (Ps) that transmits a wave front that evolves over time and traverses the object domain. The front propagates at each object point with a speed that is proportional to its Euclidean distance from the boundary. The motion of the front is governed by a nonlinear partial differential equation whose solution is computed efficiently using level set methods. Initially, the P/sub S/ transmits a moderate speed wave to explore the object domain and extract its topological information such as merging and extreme points. Then, it transmits a new front that is much faster at centerline points than non central ones. As a consequence, centerlines intersect the propagating fronts at those points of maximum positive curvature. Centerlines are computed by tracking them, starting from each topological point until the Ps is reached, by solving an ordinary differential equation using an efficient numerical scheme. The proposed method is computationally inexpensive, handles efficiently objects with complex topology, and computes centerlines that are centered, connected, one point thick, and less sensitive to boundary noise. In addition, the extracted paths form a tree graph without additional cost. We have extensively validated the robustness of the proposed method both quantitatively and qualitatively against several 2D and 3D shapes.

In this paper, we propose a variational method to segment image objects, which have a given parametric shape based on a level-set formulation of the Mumford-Shah functional, and the shape parameters. We define an energy functional composed by two complementary terms. The first one detects object boundaries using a Chan-Vese-like method. The second term constrains the contour to find a shape compatible with the parametric shape. The segmentation of the object of interest is given by the minimum of our energy functional. This minimum is computed with the calculus of variation and the gradient descent method that provide a system of evolution equations solved with the well-known level set method. We focus in this paper on the parametric category of image linear objects. Applications of the proposed model are presented on synthetic and real images.

We propose a novel approach for shape-based segmentation based on a specially designed level set function format. This format permits us to better control the process of object registration which is an important part in the shape-based segmentation framework. The method depends on a set of training shapes used to build a parametric shape model. The color is taken into consideration besides the shape prior information. The shape model is fitted to the image volume by registration through an energy minimization problem. The approach overcomes the conventional methods problems like point correspondences and weighing coefficients tuning of the partial differential equations (PDE's). Also it is suitable for multidimensional data and computationally efficient. Results of extracting the 2D star fish and the brain ventricles in 3D demonstrate the efficiency of the approach.

Data registration is a crucial step in performance evaluation procedures that aim at localizing errors in given measured data. Under the performance evaluation topic, the probably corrupted measured data should be accurately registered/aligned to the ground truth data such that the registration process would not affect the accuracy of the consequent evaluation steps. To cope with this problem, another registration methodology that can go beyond the conventional ones should be used. Here we propose a novel approach for 3-D data registration. The performance of the approach is totally independent of the measured data set. The key advantage of this approach is that it keeps the registration process away from being affected by the probably corrupted data sets, hence it permits confidential evaluation results.

In this paper, we propose a general and robust robotic path planning framework for both planar and terrain environments using level set methods. The framework is general in the sense that it can be used for both 2D and 3D environments. It generates a collision-free optimum paths for the entire or a portion of the configuration space. The optimum planned path can be controlled to follow the safest, shortest, or hybrid path. We have demonstrated the robustness of the proposed framework by correctly extracting all planned paths of complex maps with several obstacles.

In this paper, we present an automatic statistical approach for extracting 3D blood vessels from time-of-flight (TOF) magnetic resonance angiography (MRA) data. The voxels of the dataset are classified as either blood vessels or background noise. The observed volume data is modeled by two stochastic processes. The low level process characterizes the intensity distribution of the data across the volume, while the high level process characterizes the statistical dependence among neighboring voxels. 3D Markov random field (MRF) has been employed to model the high level process, whose parameters are estimated using the maximum pseudo likelihood estimator (MPLE). Our proposed model exhibits a good fit to the clinical data and is extensively tested on different synthetic vessel phantoms and several TOF datasets. Experimental results showed that the proposed model is capable of delineating vessels down to 3 voxel diameters.

We have recently developed a level set based-framework for computing medial curves or curve skeletons CS for arbitrary 2D shapes as well as tubular and articulated 3D objects. The proposed framework is robust, fully automatic, computationally efficient, and produces curve skeletons that are connected, centered, thin, and less sensitive to boundary noise. In this paper, we introduce two improvements to the framework in order to enhance its performance in terms of stability and topology preserving.

This paper proposes advanced algorithms for building different components of the Bayesian classification setup, with focus in high dimensional spaces. The mean field (MF) theory is used with support vector machines (SVM) to overcome the curse of dimensionality problem associated with the estimation of class conditional probabilities in high dimensional spaces. The Markov random fields (MRF) model is used to implement the contextual interaction of classes in an image. A new algorithm which uses SVM, in a regression prospective, is proposed for estimating the parameters of the MRF model. An iterative setup for the Bayesian image segmentation setup is proposed which maximizes the overall likelihoods of the defined classes in the image. Experimental results using a real hyperspectral remote sensing image illustrate the outstanding performance of the proposed algorithms.

A complete framework is proposed for applying the maximum a posteriori (MAP) estimation principle in remote sensing image segmentation. The MAP principle provides an estimate for the segmented image by maximizing the posterior probabilities of the classes defined in the image. The posterior probability can be represented as the product of the class conditional probability (CCP) and the class prior probability (CPP). In this paper, novel supervised algorithms for the CCP and the CPP estimations are proposed which are appropriate for remote sensing images where the estimation process might to be done in high-dimensional spaces. For the CCP, a supervised algorithm which uses the support vector machines (SVM) density estimation approach is proposed. This algorithm uses a novel learning procedure, derived from the main field theory, which avoids the (hard) quadratic optimization problem arising from the traditional formulation of the SVM density estimation. For the CPP estimation, Markov random field (MRF) is a common choice which incorporates contextual and geometrical information in the estimation process. Instead of using predefined values for the parameters of the MRF, an analytical algorithm is proposed which automatically identifies the values of the MRF parameters. The proposed framework is built in an iterative setup which refines the estimated image to get the optimum solution. Experiments using both synthetic and real remote sensing data (multispectral and hyperspectral) show the powerful performance of the proposed framework. The results show that the proposed density estimation algorithm outperforms other algorithms for remote sensing data over a wide range of spectral dimensions. The MRF modeling raises the segmentation accuracy by up to 10% in remote sensing images.

This paper addresses the problem of calibrating camera lens distortion, which can be significant in medium to wide angle lenses. Our approach is based on the analysis of distorted images of straight lines. We derive new distortion measures that can be optimized using nonlinear search techniques to find the best distortion parameters that straighten these lines. Unlike the other existing approaches, we also provide fast, closed-form solutions to the distortion coefficients. We prove that including both the distortion center and the decentering coefficients in the nonlinear optimization step may lead to instability of the estimation algorithm. Our approach provides a way to get around this, and, at the same time, it reduces the search space of the calibration problem without sacrificing the accuracy and produces more stable and noise-robust results. In addition, while almost all existing nonmetric distortion calibration methods needs user involvement in one form or another, we present a robust approach to distortion calibration based on the least-median-of-squares estimator. Our approach is, thus, able to proceed in a fully automatic manner while being less sensitive to erroneous input data such as image curves that are mistakenly considered projections of three-dimensional linear segments. Experiments to evaluate the performance of this approach on synthetic and real data are reported.

In this paper, we present a sensor planning approach for a mobile trinocular active vision system. At the stationary state (i.e., no motion) the sensor planning system calculates the generalized cameras' parameters (i.e., translational distance from the center, zoom, focus and vergence) using deterministic geometric specifications of both the sensors and the objects in their field of view. Some of these geometric parameters are difficult to be predetermined for the mobile system operation. In this paper, a new sensor planning approach, based on processing the content of the captured images, is proposed. The approach uses a combination of a closed-form solution for the translation between the three cameras, the vergence angle of the cameras as well as zoom and focus settings with the results of the correspondences between the acquired images and a predefined target object(s) obtained using the SIFT algorithm. We demonstrate the accuracy of the new approach using practical experiments.

In this paper, we propose a novel algorithm for isolating lung abnormalities (nodules) from low dose spiral chest CT scans. The proposed algorithm consists of three main steps. The first step isolates the lung nodules, arteries, veins, bronchi, and bronchioles from the surrounding anatomical structures. The second step detects lung nodules using deformable 2D and 3D templates describing typical geometry and gray level distribution within the nodules of the same type. The detection combines the normalized cross-correlation template matching and genetic optimization algorithm. The final step eliminates the false positive nodules (FPNs) using three features that robustly define the true lung nodules. Accurate density estimation for these three features is obtained using logistic regression model and linear combination of Gaussians (LCG) with positive and negative components. This paper focuses on the second and third steps. Experiments with 200 patients' CT scans demonstrate the accuracy of our approach.

This paper focuses on evaluating a number of statistical and neural methods for supervised, pixel-wise remote-sensing image classification and decision fusion. Despite the enormous progress in the analysis of remote sensing imagery over the past three decades, still much is desired in the area of image classification as no specific algorithm is known to provide accurate results under all circumstances. Decision fusion may be pursued to combine the outputs of different classifiers applied on the same data, in the hope of combining the best of what each approach provides. We report the results of the comparison between several classification and fusion methods on two real datasets, one of which is the standard benchmark Satimage dataset. It is shown that the fusion approaches can indeed outperform the performance of the best classifier.

The paper introduces an adaptive technique for 3D object identification and recognition in 3D scanned scenes. This technique uses neural learning of the 3D free-form surface representation of the object in study. This representation scheme captures the 3D curvature information of any free-form surface and encodes it into a 2D image corresponding to a certain point on the surface. This image represents a "surface signature" because it is unique for this point and is independent of the object translation or orientation in space.

Our aim is to develop a fully automatic computer-assisted diagnosis (CAD) system for lung cancer screening using chest spiral CT scans. A screening program on 1000 subjects aims at quantification of the effectiveness of low dose spiral CT scans for early diagnosis of lung cancer, and evaluation of its possible impact on improving the mortality rate of cancer patients. The paper presents an image analysis system for 3D reconstruction of the lungs and trachea, detection of lung abnormalities, identification/classification of these abnormalities with respect to specific diagnosis, and distributed visualization of the results over computer networks. We present two novel approaches for segmentation of the lung tissues from the surrounding structures in the chest cavity, and detection of abnormalities in the lungs. The segmentation algorithm is hierarchical, first isolating the background from the chest cavity, then isolating the lungs from surrounding structures (e.g., ribs, liver, and other organs). Abnormalities in the lungs are detected by analyzing the segmented lung tissues and extracting the isolated lumps that appear in various connected regions. 3D reconstructions are also generated for these abnormalities, to be used for subsequent identification/classification steps. Results on 50 subjects are shown, and have been evaluated against radiologists. Our image analysis approach has provided comparable results with respect to the experts. The approach is quite fast, and lends itself to distributed visualization over computer networks.

We address the problem of calibrating camera lens distortion, which can be significant in medium to wide angle lenses. While almost all existing nonmetric distortion calibration methods need user involvement in one form or another, we present an automatic approach based on the robust the-least-median-of-squares (LMedS) estimator. Our approach is thus less sensitive to erroneous input data such as image curves that are mistakenly considered as projections of 3D linear segments. Our approach uniquely uses fast, closed-form solutions to the distortion coefficients, which serve as an initial point for a nonlinear optimization algorithm to straighten imaged lines. Moreover we propose a method for distortion model selection based on geometrical inference. Successful experiments to evaluate the performance of this approach on synthetic and real data are reported.

Stochastic models of images are commonly represented in terms of three random processes (random fields) defined on the region of support of the image. The observed image process G is considered as a composite of two random process: a high level process G/sup h/, which represents the regions (or classes) that form the observed image; and a low level process G/sup l/, which describes the statistical characteristics of each region (or class). The representation G = (G/sup h/, G/sup l/) has been widely used in the image processing literature in the past two decades. In this paper, we consider the low level process G/sup l/ as mixture of normal distributions, and we use the expectation-maximization (EM) algorithm to estimate the mean, the variance, and proportion for each distribution. A popular model for the high level process G/sup h/ has been the Gibbs-Markov random field (GMRF) model. We introduce a novel unsupervised approach to estimate the parameters of a GMRF model. In this approach, we estimate the model parameters that maximize the posteriori probability of each pixel in a given image. The MAP estimate is obtained using a combination of genetic search and deterministic optimization using the iterated conditional mode (ICM) approach of Besag. The desired estimate of the GMRF parameters is the one corresponding to the MAP estimate. The approach has been applied on real images (Spiral CT slices) and provides satisfactory results.

In this paper, we use a level set based segmentation algorithm to extract the vascular tree from magnetic resonance angiography, "MRA". Classification model finds an optimal partition of homogeneous classes with regular interfaces. Regions and their interfaces are represented by level set functions. The algorithm initializes level sets in each image slice using automatic seed initialization and then iteratively, each level set approaches the steady state and contains the vessel or nonvessel area. The algorithm is applied on each slice of the volume to build up the tree. The results are validated using a phantom that simulates the "MRA". The approach is fast and accurate. Results on various cases demonstrate the accuracy of the approach.

Correspondence is one of the major problems that must be solved in stereo vision. Correlation has been commonly used in the past for this problem. However, most classical linear correlation methods fail near depth discontinuities and in the presence of occlusions. Many robust methods have been proposed that claim to effectively deal with some or all of these issues. Many of these robust methods are transformation-based, however, other robust methods are non-transformation based. This paper gives five requirements that should be met by a transformation-based robust correlation method. We compare some of the robust correspondence methods and demonstrate their utility on different data sets. Based on these results, we propose a solution to the correspondence problem which represents a compromise between the speed of classical correlation and the improved results obtained from a more robust correspondence method. Also, we propose a median filtering technique that removes noise from the disparity maps while preserving certain image features usually removed by ordinary median filtering.

The paper discusses the matching of two multispectral data sets using the genetic algorithm (GA) as a search technique for the global optimum estimates of the transformation parameters. It uses the simplest form of the affine transformation, where the scaling and rotation are ignored, to explain how the GAs, which are used in the registrations' processes for the different bands, can cooperate. This cooperation between the GAs improves the speed of the matching process. The cooperation depends on the exchange of the intermediate results between the GAs. Therefore, the proposed approach demands the parallel realization of GAs. In addition, the proposed approach can be considered as a fusion method for the different decisions resulting form the registrations for the different bands. Landsat 7-band and 4-band aerial data set types are considered. The aerial data set type is used to explain how to the proposed approach speeds the matching process.

This paper describes design and implementation of a vision based platform for automated refueling tasks. The platform is an autonomous docking system in principle, with the specific application - refueling of vehicles. The system is. based on monochromatic, monocular vision, and it utilizes very specialized image processing schemes. Image processing consists of very fast filtering and segmenting algorithms, as well as moment's computation. A robotic arm with 6 joints (FANUC M-6i), and a controller unit (R-B), does the physical work. A serial interface, with very high-level commands, connects a supercomputing machine and the robot's controller. A practical setup would probably be scaled down to a special design robot, and a single processor, controller with special VLSI chips for image processing. Results are very promising; the robot can identify the cap position, orientation, and height in real time with acceptable accuracy and reliability.

This paper introduces anew free-form surface representation scheme for the purpose of fast and accurate registration and matching. Accurate registration of surfaces is a common task in computer vision. The proposed representation scheme captures the surface curvature information (seen from certain points) and produces images, called "surface signatures," at these points. Matching signatures of different surfaces enables the recovery of the transformation parameters between these surfaces. We propose using template matching to compare the signature images. To enable partial matching, another criterion, the overlap ratio is used. This representation scheme can be used as a global representation of the surface as well as a local one and performs near real-time registration. We show that the signature representation can be used to recover scaling transformation as well as matching objects in 3D scenes in the presence of clutter and occlusion. Applications presented include: free-form object matching, multimodal medical volumes registration, and dental teeth reconstruction from intraoral images.

In this paper, we present a novel algorithm for fuzzy segmentation of magnetic resonance imaging (MRI) data and estimation of intensity inhomogeneities using fuzzy logic. MRI intensity inhomogeneities can be attributed to imperfections in the radio-frequency coils or to problems associated with the acquisition sequences. The result is a slowly varying shading artifact over the image that can produce errors with conventional intensity-based classification. Our algorithm is formulated by modifying the objective function of the standard fuzzy c-means (FCM) algorithm to compensate for such inhomogeneities and to allow the labeling of a pixel (voxel) to be influenced by the labels in its immediate neighborhood. The neighborhood effect acts as a regularizer and biases the solution toward piecewise-homogeneous labelings. Such a regularization is useful in segmenting scans corrupted by salt and pepper noise. Experimental results on both synthetic images and MR data are given to demonstrate the effectiveness and efficiency of the proposed algorithm.

This paper addresses the problem of calibrating camera lens distortion, which can be significant in medium to wide angle lenses. Our approach is based on the analysis of distorted images of straight lines. We derive new distortion measures that can be optimized using non-linear search techniques to find the best distortion parameters that straighten these lines. Unlike other approaches, we also provide fast, closed-form solutions to the distortion coefficients. Experiments to evaluate the performance of this approach on synthetic and real data are reported.

This paper presents a model-based vision system for dentistry that will assist in diagnosis, treatment planning and surgical simulation. Dentistry requires an accurate 3D representation of the teeth and jaws for diagnostic and treatment purposes. The proposed integrated computer vision system constructs a 3D model of the patient's dental occlusion using an intra-oral video camera. The space carving algorithm is used to reconstruct the shape of the human jaw. This algorithm provides more flexibility and eliminates several constraints imposed by other approaches like stereo. The system performance is investigated, and the results show acceptable reconstruction accuracy.

This paper addresses the problem of calibrating camera lens distortion, which can be significant in medium to wide angle lenses. Our approach is based on the analysis of distorted images of straight lines. We derive a new distortion measure that can be optimized using nonlinear search techniques to find the best distortion parameters that straighten these lines. Unlike the other existing approaches, we also show how to use this measure to find fast, closed-form solutions to the distortion coefficients. Some experiments to evaluate the performance of this approach on synthetic and real data are reported.

Camera systems with zoom lenses are inherently more useful than those with passive lenses due to their \nexibility and controllability. However, their calibration raises several challenges. In this paper, we present a neural framework for zoom-lens camera calibration that can capture complex variations in the camera model parameters across continuous ranges in the lens control space, while minimizing the calibration error over all the calibration data. To automate the tedious process of col-lecting calibration data, the calibration approach should be prepared to handle possible outliers in the data. We demonstrate how the cal-ibration approach can be robust and less sensitive to outliers. The validity and performance of our approach are tested using both syn-thetic data with outliers, and with real experiments to calibrate two Hitachi CCD cameras with H10x11E Fujinon active lenses. 1

Camera systems with zoom lenses are inherently more useful than those with passive lenses due to their flexibility and controllability. However, calibration techniques for active-cameras, still, lag behind those developed for calibration of passive-lens cameras. In this paper, we present a neural framework for zoom-lens camera calibration based on our proposed neurocalibration approach, which maps the classical problem of geometric camera calibration into a learning problem of a multi-layered feedforward neural network (MLFN). After discussing the features and advantages of the neurocalibration network, we present how this neural framework can capture the complex variations in the camera model parameters, both intrinsic and extrinsic, while minimizing the calibration error over all the calibration data across continuous ranges in the lens control space. The framework consists of a number of MLFNs learning concurrently, independently and cooperatively, the perspective projection transformation of the camera over its optical setting ranges. The calibration results of this technique applied to Hitachi CCD cameras with H10x11E Fujinon active lenses are reported.

Intra-operative MRI (iMRI) is a new technology that allows near real time updates of scans during a surgical procedure under the magnet. From an initial comprehensive scan, 3D volumes are obtained. During the surgery, only the portion of the organ under surgery needs to be re-scan (region of interest-ROI) in order to guide the progress of the surgery, and validate the procedure. We describe a registration approach which can be used for iMRI applications. Registration is performed through a hybrid of the surface point signature approach (SPS) and the maximization of the mutual information (MI) criterion. We demonstrate the approach on some real data.

We consider the problem of computing the 3D shape of an unknown, arbitrary-shaped object. Starting with an initial surface, larger than the scene, we refine it using the object's projections taken at known but arbitrarily distributed viewpoints. These projections are used independently in a recursive manner. Thus, using more projections improves the reconstruction with a slight increase in the computation time. The technique generates an approximate voxelized representation of the object. We describe the proposed technique and outline the algorithm. Rendered images of voxel spaces recovered from synthetic and real observation images are shown.

This paper presents a supervised fuzzy c-mean (SFCM) classifier for the classification of high dimensional data. The proposed SFCM classifier can be iterative or non iterative to reduce the computational time. Comparison with the conventional FCM clustering technique and the Bayesian classification technique is also presented. Performance results of the three algorithms are presented on simulated and real remote sensing multispectral data, which show improvement in the classification accuracy using the SFCM technique.

This paper presents a new concept for 3D coding of freeform surfaces. The proposed coding technique uses the surface signature representation scheme. This representation scheme captures the 3D curvature information of any freeform surface and encodes it into a 2D image corresponding to a certain point on the surface. This image is unique for this point and is independent of the object translation or orientation in space. For this reason this image is called "surface point signature" (SPS). Using SPS in 3D coding has many applications in 3D compression, 3D pose estimation and in 3D object recognition.

The goal of this paper is twofold. First, we present a supervised fuzzy c-mean (SFCM) classifier for the classification of high dimensional data. Comparisons of the conventional FCM clustering technique and Bayesian classification technique are also presented. Next, we present a two-step classifier in which the proposed SFCM and Bayesian algorithms are used in a cooperative way such that classification results of the SFCM algorithm are used to compute the prior probabilities required for the Bayesian classifier. Classification results of the three algorithms are presented on simulated and real remote sensing multispectral data. The results obtained show improvements in the classification accuracy and reliability using the two-step algorithm.

This paper presents a new concept for 3D coding of free-form surfaces. The proposed coding technique uses the surface signature representation scheme. This representation scheme captures the 3D curvature information of many free-form surface and encodes it into a 2D image corresponding to a certain point on the surface. This image is unique for this point and is independent of the the object translation or orientation in space. For this reason this image is called "surface point signature" (SPS). Using SPS in 3D coding has many applications, such as in 3D compression, 3D pose estimation and 3D object recognition.

This paper presents a model-based vision system for dentistry that will assist in diagnosis, treatment planning, and surgical simulation. Dentistry requires an accurate three-dimensional (3-D) representation of the teeth and jaws for diagnostic and treatment purposes. The proposed integrated computer vision system constructs a 3-D model of the patient's dental occlusion using an intraoral video camera. A modified shape from shading (SFS) technique, using perspective projection and camera calibration, extracts the 3-D information from a sequence of two-dimensional (2-D) images of the jaw. Data fusion of range data and 3-D registration techniques develop the complete jaw model. Triangulation is then performed, and a solid 3-D model is reconstructed. The system performance is investigated using ground truth data, and the results show acceptable reconstruction accuracy.

In this paper, we present a novel algorithm for adaptive fuzzy segmentation of MRI data and estimation of intensity inhomogeneities using fuzzy logic. MRI intensity inhomogeneities can be attributed to imperfections in the RF coils or some problems associated with the acquisition sequences. The result is a slowly-varying shading artifact over the image that can produce errors with conventional intensity-based classification. Our algorithm is formulated by modifying the objective function of the standard fuzzy c-means (FCM) algorithm to compensate for such inhomogeneities and to allow the labeling of a pixel (voxel) to be influenced by the labels in its immediate neighborhood. The neighborhood effect acts as a regularizer and biases the solution towards piecewise-homogeneous labelings. Such a regularization is useful in segmenting scans corrupted by salt and pepper noise. Experimental results on both synthetic images and MR data are given to demonstrate the effectiveness and efficiency of the proposed algorithm.

We present an algorithm to solve the sensor planning problem for a trinocular, active vision system. This algorithm uses an iterative optimization method to first solve for the translation between the three cameras and then uses this result to solve for parameters such as pan, tilt angles of the cameras and zoom setting.

This paper presents a framework for integrating multiple sensory data, sparse range data and dense depth maps from shape from shading in order to improve the 3D reconstruction of visible surfaces of 3D objects. The integration process is based on propagating the error difference between the two data sets by fitting a surface to that difference and using it to correct the visible surface obtained from shape from shading. A feedforward neural network is used to fit a surface to the sparse data. We also study the use of the extended Kalman filter for supervised learning and compare it with the backpropagation algorithm. A performance analysis is done to obtain the best neural network architecture and learning algorithm. It is found that the integration of sparse depth measurements has greatly enhanced the 3D visible surface obtained from shape from shading in terms of metric measurements.

Motion can be estimated by detecting the edges of a moving object using active contours, and registering them together to obtain the motion model parameters. This idea can be applied to patient motion during the acquisition of an MRI to eliminate motion artifacts in the image. The data obtained during the MRI acquisition, the k-space, can be divided into several subbands such that each subband is acquired in a small fraction of the full imaging time. These subbands create invariant tissue feature maps called subband images. Using active contours, the relative motion is analyzed across the different subband images to determine the motion parameters. Using these motion parameters it is possible to correct the subbands, thus correcting the k-space. This has the potential to yield clear, noise-free MR images.

This paper describes the application of fuzzy set theory in medical imaging, namely the segmentation of brain images. We propose a fully automatic technique to obtain image clusters. A modified fuzzy c-mean (FCM) classification algorithm is used to provide a fuzzy partition. Our new method, inspired from the Markov random field (MRF), is less sensitive to noise as it filters the image while clustering it, and the filter parameters are enhanced in each iteration by the clustering process. We applied the new method on a noisy CT scan and on a single channel MRI scan. We recommend using a methodology of over segmentation to the textured MRI scan and a user guided-interface to obtain the final clusters. One of the applications of this technique is TBI recovery prediction in which it is important to consider the partial volume. It is shown that the system stabilizes after a number of iterations with the membership value of the region contours reflecting the partial volume value. The final stage of the process is devoted to decision making or the defuzzification process.

Camera calibration is a primary crucial step in many computer vision tasks. We present a new neural approach for camera calibration. Unlike some existing neural approaches, our calibrating network can tell the perspective-projection-transformation matrix between the world 3D points and the corresponding 2D image pixels. Starting from random initial weights, the net can specify the camera model parameters satisfying the orthogonality constraints on the rotational transformation. The neurocalibration technique is shown to solve four different types of calibration problems that are found in computer vision applications. Moreover, it can be extended to the more difficult problem of calibrating cameras with automated active lenses. The validity and performance of our technique are tested with both synthetic data under different noise conditions and with real images. Experiments have shown the accuracy and the efficiency of our neurocalibration technique.

The paper introduces a new free-form surface representation scheme for the purpose of fast and accurate registration and matching. Accurate registration of surfaces is a common task in computer vision. The proposed representation scheme captures the surface curvature information seen from certain points and produces images called surface signatures at these points. Matching signatures of different surfaces enables the recovery of the transformation parameters between these surfaces. We propose to use template matching to compare the signature images. To enable partial matching, another criterion, the overlap ratio, is used. This representation scheme can be used as a global representation of the surface as well as a local one and performs near real time registration. We show that the signature representation can be used to match objects in 3D scenes in the presence of clutter and occlusion. Applications presented include free-form object matching, multimodal medical volume registration and dental teeth reconstruction from intra-oral images.

This paper presents the neurocalibration approach as a new neural-based solution for the problem of camera calibration. Unlike some existing neural approaches, our calibrating network can match the perspective-projection-transformation matrix between the world 3D points and the corresponding 2D image pixels. Starting from random initial weights, the net can specify the camera model parameters satisfying the orthogonality constraints on the rotational transformation. In order to improve the accuracy of calibration results, the paper demonstrates the application of the neurocalibration technique to multi-image camera calibration. In such a case, many images are taken by the same camera but from different (rotated and/or translated) positions. Experiments have shown the accuracy and the efficiency of our neurocalibration technique.

To estimate motion, the edges of a moving object can be detected using active contours. After detecting the edges throughout the duration of the motion, the contours are registered together in order to obtain the motion model parameters. This idea can be applied to motion artifacts in Magnetic Resonance Images (MRI) which are caused by patient motion during the acquisition of an MRI. Thus, if the patient motion can be determined, it will be possible to correct for the motion in the images. The proposed method eliminates motion artifacts by dividing the acquired MRI data into several subbands. These subbands create invariant tissue feature maps called subband images. Using active contours, the edges of each subband image are detected and the relative motion is analyzed across the different subband images, thus determining the motion parameters. From these motion parameters it is possible to correct the subband images, thus correcting the k-space yielding clear, noise-free MR images.

This paper presents a new method for integrating different low level vision modules, stereo and shape from shading, in order to improve the 3D reconstruction of visible surfaces of objects from intensity images. The integration process is based on correcting the 3D visible surface obtained from shape from shading using the sparse depth measurements from the stereo module by fitting a surface into the difference between the two data sets. A feedforward neural network is used to fit a surface to the error difference. An extended Kalman filter algorithm is used for the network learning. It is found that the integration of sparse depth measurements has greatly enhanced the 3D visible surface obtained from shape from shading in terms of metric measurements.

This paper presents a new concept for 3-D free-form surface registration and object recognition using a new surface representation scheme. This representation scheme captures the 3-D curvature information of any free-form surface and encodes it into a 2-D image corresponding to a certain point on the surface. This image is unique for this point and is independent from the object translation or orientation in space. For this reason we called this image “Surface Point Signature” (SPS). This scheme can be used as a global representation of the surface as well as a local one and also in a scale independent surface matching. It performs faster registration than existing registration approaches. We applied this technique in object registration, multimodal medical image registration and the recognition of multiple objects in a 3-D scene.

We present a concept for 3D free-form object recognition using a surface representation scheme. This representation scheme captures the 3D curvature information of any free-form surface and encodes it into a 2D image corresponding to a certain point on the surface. This image is unique for this point and is independent from the object translation or orientation in space. For this reason we called this image the surface point signature (SPS). Using specially designed neural networks; the SPS images are used in the matching and recognition of 3D objects in a 3D scanned scene.

We present a trinocular stereo vision technique that can be used to build a fast, robust description of indoor objects. The proposed technique utilizes the geometrical constraints of the trinocular vision system and considers the uncertainty of image measurements. The technique has been successfully applied to several indoor scenes. The reliability and speed of the proposed system make it particularly suitable for real-time applications such as robot navigation.

A novel approach is proposed to obtain a record of the patient's occlusion using computer vision. Data acquisition is obtained using intra-oral video cameras. The technique utilizes shape from shading to extract 3D information from 2D views of the jaw, and a novel technique for 3D data registration using genetic algorithms. The resulting 3D model can be used for diagnosis, treatment planning, and implant purposes. The overall purpose of this research is to develop a model-based vision system for orthodontics to replace traditional approaches. This system will be flexible, accurate, and will reduce the cost of orthodontic treatments.

A composite of two random field models is used to describe the observed image: a high-level process that describes the various regions in the images is modeled by a Gibbs-Markov model, and a low-level process that describes each particular region is modeled by a simultaneous autoregressive model. Using this composition, a recursive maximum a posterior (MAP) segmentation algorithm is formulated and various issues related to parameter estimation are discussed.>