In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in http://medicaldecathlon.com/. In addition, both data and online evaluation are accessible via https://competitions.codalab.org/competitions/17094.

The recent success of learning-based algorithms can be greatly attributed to the immense amount of annotated data used for training. Yet, many datasets lack annotations due to the high costs associated with labeling, resulting in degraded performances of deep learning methods. Self-supervised learning is frequently adopted to mitigate the reliance on massive labeled datasets since it exploits unlabeled data to learn relevant feature representations. In this work, we propose SS-StyleGAN, a self-supervised approach for image annotation and classification suitable for extremely small annotated datasets. This novel framework adds self-supervision to the StyleGAN architecture by integrating an encoder that learns the embedding to the StyleGAN latent space, which is well-known for its disentangled properties. The learned latent space enables the smart selection of representatives from the data to be labeled for improved classification performance. We show that the proposed method attains strong classification results using small labeled datasets of sizes 50 and even 10. We demonstrate the superiority of our approach for the tasks of COVID-19 and liver tumor pathology identification.

Since its renaissance, deep learning has been widely used in various medical imaging tasks and has achieved remarkable success in many medical imaging applications, thereby propelling us into the so-called artificial intelligence (AI) era. It is known that the success of AI is mostly attributed to the availability of big data with annotations for a single task and the advances in high performance computing. However, medical imaging presents unique challenges that confront deep learning approaches. In this survey paper, we first present traits of medical imaging, highlight both clinical needs and technical challenges in medical imaging, and describe how emerging trends in deep learning are addressing these issues. We cover the topics of network architecture, sparse and noisy labels, federating learning, interpretability, uncertainty quantification, etc. Then, we present several case studies that are commonly found in clinical practice, including digital pathology and chest, brain, cardiovascular, and abdominal imaging. Rather than presenting an exhaustive literature survey, we instead describe some prominent research highlights related to these case study applications. We conclude with a discussion and presentation of promising future directions.

This work estimates the severity of pneumonia in COVID-19 patients and reports the findings of a longitudinal study of disease progression. It presents a deep learning model for simultaneous detection and localization of pneumonia in chest Xray (CXR) images, which is shown to generalize to COVID-19 pneumonia. The localization maps are utilized to calculate a "Pneumonia Ratio" which indicates disease severity. The assessment of disease severity serves to build a temporal disease extent profile for hospitalized patients. To validate the model's applicability to the patient monitoring task, we developed a validation strategy which involves a synthesis of Digital Reconstructed Radiographs (DRRs - synthetic Xray) from serial CT scans; we then compared the disease progression profiles that were generated from the DRRs to those that were generated from CT volumes.

The identification of lesion within medical image data is necessary for diagnosis, treatment and prognosis. Segmentation and classification approaches are mainly based on supervised learning with well-paired image-level or voxel-level labels. However, labeling the lesion in medical images is laborious requiring highly specialized knowledge. We propose a medical image synthesis model named abnormal-to-normal translation generative adversarial network (ANT-GAN) to generate a normal-looking medical image based on its abnormal-looking counterpart without the need for paired training data. Unlike typical GANs, whose aim is to generate realistic samples with variations, our more restrictive model aims at producing a normal-looking image corresponding to one containing lesions, and thus requires a special design. Being able to provide a "normal" counterpart to a medical image can provide useful side information for medical imaging tasks like lesion segmentation or classification validated by our experiments. In the other aspect, the ANT-GAN model is also capable of producing highly realistic lesion-containing image corresponding to the healthy one, which shows the potential in data augmentation verified in our experiments.

The bag-of-visual-words (BoVW) method with construction of a single dictionary of visual words has been used previously for a variety of classification tasks in medical imaging, including the diagnosis of liver lesions. In this paper, we describe a novel method for automated diagnosis of liver lesions in portal-phase computed tomography (CT) images that improves over single-dictionary BoVW methods by using an image patch representation of the interior and boundary regions of the lesions. Our approach captures characteristics of the lesion margin and of the lesion interior by creating two separate dictionaries for the margin and the interior regions of lesions ("dual dictionaries" of visual words). Based on these dictionaries, visual word histograms are generated for each region of interest within the lesion and its margin. For validation of our approach, we used two datasets from two different institutions, containing CT images of 194 liver lesions (61 cysts, 80 metastasis, and 53 hemangiomas). The final diagnosis of each lesion was established by radiologists. The classification accuracy for the images from the two institutions was 99% and 88%, respectively, and 93% for a combined dataset. Our new BoVW approach that uses dual dictionaries shows promising results. We believe the benefits of our approach may generalize to other application domains within radiology.

Classification of clustered breast microcalcifications into benign and malignant categories is an extremely challenging task for computerized algorithms and expert radiologists alike. In this paper we apply a multi-view-classifier for the task. We describe a two-step classification method that is based on a view-level decision, implemented by a logistic regression classifier, followed by a stochastic combination of the two view-level indications into a single benign or malignant decision. The proposed method was evaluated on a large number of cases from a standardized digital database for screening mammography (DDSM). Experimental results demonstrate the advantage of the proposed multi-view classification algorithm that automatically learns the best way to combine the views.

The papers in this special section focus on the technology and applications supported by deep learning. Deep learning is a growing trend in general data analysis and has been termed one of the 10 breakthrough technologies of 2013. Deep learning is an improvement of artificial neural networks, consisting of more layers that permit higher levels of abstraction and improved predictions from data. To date, it is emerging as the leading machine-learning tool in the general imaging and computer vision domains. In particular, convolutional neural networks (CNNs) have proven to be powerful tools for a broad range of computer vision tasks. Deep CNNs automatically learn mid-level and high-level abstractions obtained from raw data (e.g., images). Recent results indicate that the generic descriptors extracted from CNNs are extremely effective in object recognition and localization in natural images. Medical image analysis groups across the world are quickly entering the field and applying CNNs and other deep learning methodologies to a wide variety of applications.

In this study we present an efficient image categorization and retrieval system applied to medical image databases, in particular large radiograph archives. The methodology is based on local patch representation of the image content, using a "bag of visual words" approach. We explore the effects of various parameters on system performance, and show best results using dense sampling of simple features with spatial content, and a nonlinear kernel-based support vector machine (SVM) classifier. In a recent international competition the system was ranked first in discriminating orientation and body regions in X-ray images. In addition to organ-level discrimination, we show an application to pathology-level categorization of chest X-ray data, the most popular examination in radiology. The system discriminates between healthy and pathological cases, and is also shown to successfully identify specific pathologies in a set of chest radiographs taken from a routine hospital examination. This is a first step towards similarity-based categorization, which has a major clinical implications for computer-assisted diagnostics.

A supervised framework is presented for the automatic registration and segmentation of white matter (WM) tractographies extracted from brain DT-MRI. The framework relies on the direct registration between the fibers, without requiring any intensity-based registration as preprocessing. An affine transform is recovered together with a set of segmented fibers. A recently introduced probabilistic boosting tree classifier is used in a segmentation refinement step to improve the precision of the target tract segmentation. The proposed method compares favorably with a state-of-the-art intensity-based algorithm for affine registration of DTI tractographies. Segmentation results for 12 major WM tracts are demonstrated. Quantitative results are also provided for the segmentation of a particularly difficult case, the optic radiation tract. An average precision of 80% and recall of 55% were obtained for the optimal configuration of the presented method.

Flow Doppler imaging has become an integral part of an echocardiographic exam. Automated interpretation of flow doppler imaging has so far been restricted to obtaining hemodynamic information from velocity-time profiles depicted in these images. In this paper we exploit the shape patterns in Doppler images to infer the similarity in valvular disease labels for purposes of automated clinical decision support. Specifically, we model the similarity in appearance of Doppler images from the same disease class as a constrained non-rigid translation transform of the velocity envelopes embedded in these images. The shape similarity between two Doppler images is then judged by recovering the alignment transform using a variant of dynamic shape warping. Results of similarity retrieval of doppler images for cardiac decision support on a large database of images are presented.

This paper presents a procedure for automatic extraction and segmentation of a class-specific object (or region) by learning class-specific boundaries. We describe and evaluate the method with a specific focus on the detection of lesion regions in uterine cervix images. The watershed segmentation map of the input image is modeled using a Markov random field (MRF) in which watershed regions correspond to binary random variables indicating whether the region is part of the lesion tissue or not. The local pairwise factors on the arcs of the watershed map indicate whether the arc is part of the object boundary. The factors are based on supervised learning of a visual word distribution. The final lesion region segmentation is obtained using a loopy belief propagation applied to the watershed arc-level MRF. Experimental results on real data show state-of-the-art segmentation results on this very challenging task that, if necessary, can be interactively enhanced.

In this paper, we present a robust approach to the registration of white matter tractographies extracted from diffusion tensor-magnetic resonance imaging scans. The fibers are projected into a high dimensional feature space based on the sequence of their 3-D coordinates. Adaptive mean-shift clustering is applied to extract a compact set of representative fiber-modes (FM). Each FM is assigned to a multivariate Gaussian distribution according to its population thereby leading to a Gaussian mixture model (GMM) representation for the entire set of fibers. The registration between two fiber sets is treated as the alignment of two GMMs and is performed by maximizing their correlation ratio. A nine-parameters affine transform is recovered and eventually refined to a twelve-parameters affine transform using an innovative mean-shift based registration refinement scheme presented in this paper. The validation of the algorithm on synthetic intrasubject data demonstrates its robustness to interrupted and deviating fiber artifacts as well as outliers. Using real intrasubject data, a comparison is conducted to other intensity based and fiber-based registration algorithms, demonstrating competitive results. An option for tracking-in-time, on specific white matter fiber tracts, is also demonstrated on the real data.

This paper provides an overview on super-resolution (SR) research in medical imaging applications. Many imaging modalities exist. Some provide anatomical information and reveal information about the structure of the human body, and others provide functional information, locations of activity for specific activities and specified tasks. Each imaging system has a characteristic resolution, which is determined based on physical constraints of the system detectors that are in turn tuned to signal-to-noise and timing considerations. A common goal across systems is to increase the resolution, and as much as possible achieve true isotropic 3-D imaging. SR technology can serve to advance this goal. Research on SR in key medical imaging modalities, including MRI, fMRI and PET, has started to emerge in recent years and is reviewed herein. The algorithms used are mostly based on standard SR algorithms. Results demonstrate the potential in introducing SR techniques into practical medical applications.

The work focuses on a unique medical repository of digital cervicographic images ("Cervigrams") collected by the National Cancer Institute (NCI) in longitudinal multiyear studies. NCI, together with the National Library of Medicine (NLM), is developing a unique web-accessible database of the digitized cervix images to study the evolution of lesions related to cervical cancer. Tools are needed for automated analysis of the cervigram content to support cancer research. We present a multistage scheme for segmenting and labeling regions of anatomical interest within the cervigrams. In particular, we focus on the extraction of the cervix region and fine detection of the cervix boundary; specular reflection is eliminated as an important preprocessing step; in addition, the entrance to the endocervical canal (the "os"), is detected. Segmentation results are evaluated on three image sets of cervigrams that were manually labeled by NCI experts.

An automated scheme for magnetic resonance imaging (MRI) brain segmentation is proposed. An adaptive mean-shift methodology is utilized in order to classify brain voxels into one of three main tissue types: gray matter, white matter, and Cerebro-spinal fluid. The MRI image space is represented by a high-dimensional feature space that includes multimodal intensity features as well as spatial features. An adaptive mean-shift algorithm clusters the joint spatial-intensity feature space, thus extracting a representative set of high-density points within the feature space, otherwise known as modes. Tissue segmentation is obtained by a follow-up phase of intensity-based mode clustering into the three tissue categories. By its nonparametric nature, adaptive mean-shift can deal successfully with nonconvex clusters and produce convergence modes that are better candidates for intensity based classification than the initial voxels. The proposed method is validated on 3-D single and multimodal datasets, for both simulated and real MRI data. It is shown to perform well in comparison to other state-of-the-art methods without the use of a preregistered statistical brain atlas.

Mixture of Gaussians (MoG) model is a useful tool in statistical learning. In many learning processes that are based on mixture models, computational requirements are very demanding due to the large number of components involved in the model. We propose a novel algorithm for learning a simplified representation of a Gaussian mixture, that is based on the Unscented Transform which was introduced for filtering nonlinear dynamical systems. The superiority of the proposed method is validated on both simulation experiments and categorization of a real image database. The proposed categorization methodology is based on modeling each image using a Gaussian mixture model. A category model is obtained by learning a simplified mixture model from all the images in the category.

This work focuses on a general framework for image categorization, classification and retrieval that may be appropriate for medical image archives. The proposed methodology is comprised of a continuous and probabilistic image representation scheme using Gaussian mixture modeling (MoG) along with information-theoretic image matching measures (KL). A category model is obtained by learning a reduced model from all the images in the category. We propose a novel algorithm for learning a reduced representation of a MoG, that is based on the unscented-transform. The superiority of the proposed method is validated on both simulation experiments and categorization of a real medical image database.

In this paper we propose a new segmentation algorithm which combines patch-based information with edge cues under a probabilistic framework. We use a mixture of multiple Gaussians for building the statistical model with color and spatial features, and we incorporate edge information based on texture, color and brightness differences into the EM algorithm. We evaluate our results qualitatively and quantitatively on a large data-set of natural images and compare our results to other state-of-the-art methods.

The Nomarski differential interference contrast (DIC) microscopy is of widespread use for observing live biological specimens. In fertility clinics the DIC microscope is used for evaluating human embryo cells. An image formation model for DIC imaging is needed for reconstruction and quantification of the visualized specimens. This calls for a complicated analysis of the interaction of light waves with biological matter. Most works express the solution via the first Born approximation, yet a theoretical bound is known that limits the validity of such approximation to very small objects. We show in this work that the theoretical bound is not directly relevant to microscopic imaging and is far too limiting. We derive a more realistic bound and show that it may justify in many cases the use of the Born approximation in biological cell microscopic imaging. It also provides limits on the validity of the Born expansion that several works violate.

This paper presents an image representation and matching framework for image categorization in medical image archives. Categorization enables one to determine automatically, based on the image content, the examined body region and imaging modality. It is a basic step in content-based image retrieval (CBIR) systems, the goal of which is to augment text-based search with visual information analysis. CBIR systems are currently being integrated with picture archiving and communication systems for increasing the overall search capabilities and tools available to radiologists. The proposed methodology is comprised of a continuous and probabilistic image representation scheme using Gaussian mixture modeling (GMM) along with information-theoretic image matching via the Kullback-Leibler (KL) measure. The GMM-KL framework is used for matching and categorizing X-ray images by body regions. A multidimensional feature space is used to represent the image input, including intensity, texture, and spatial information. Unsupervised clustering via the GMM is used to extract coherent regions in feature space that are then used in the matching process. A dominant characteristic of the radiological images is their poor contrast and large intensity variations. This presents a challenge to matching among the images, and is handled via an illumination-invariant representation. The GMM-KL framework is evaluated for image categorization and image retrieval on a dataset of 1500 radiological images. A classification rate of 97.5% was achieved. The classification results compare favorably with reported global and local representation schemes. Precision versus recall curves indicate a strong retrieval result as compared with other state-of-the-art retrieval techniques. Finally, category models are learned and results are presented for comparing images to learned category models.

We describe an algorithm for context-based segmentation of visual data. New frames in an image sequence (video) are segmented based on the prior segmentation of earlier frames in the sequence. The segmentation is performed by adapting a probabilistic model learned on previous frames, according to the content of the new frame. We utilize the maximum a posteriori version of the EM algorithm to segment the new image. The Gaussian mixture distribution that is used to model the current frame is transformed into a conjugate-prior distribution for the parametric model describing the segmentation of the new frame. This semisupervised method improves the segmentation quality and consistency and enables a propagation of segments along the segmented images. The performance of the proposed approach is illustrated on both simulated and real image data.

In this paper, we combine discrete and continuous image models with information-theoretic-based criteria for unsupervised hierarchical image-set clustering. The continuous image modeling is based on mixture of Gaussian densities. The unsupervised image-set clustering is based on a generalized version of a recently introduced information-theoretic principle, the information bottleneck principle. Images are clustered such that the mutual information between the clusters and the image content is maximally preserved. Experimental results demonstrate the performance of the proposed framework for image clustering on a large image set. Information theoretic tools are used to evaluate cluster quality. Particular emphasis is placed on the application of the clustering for efficient image search and retrieval.

An automated algorithm for tissue segmentation of noisy, low-contrast magnetic resonance (MR) images of the brain is presented. A mixture model composed of a large number of Gaussians is used to represent the brain image. Each tissue is represented by a large number of Gaussian components to capture the complex tissue spatial layout. The intensity of a tissue is considered a global feature and is incorporated into the model through tying of all the related Gaussian parameters. The expectation-maximization (EM) algorithm is utilized to learn the parameter-tied, constrained Gaussian mixture model. An elaborate initialization scheme is suggested to link the set of Gaussians per tissue type, such that each Gaussian in the set has similar intensity characteristics with minimal overlapping spatial supports. Segmentation of the brain image is achieved by the affiliation of each voxel to the component of the model that maximized the a posteriori probability. The presented algorithm is used to segment three-dimensional, T1-weighted, simulated and real MR images of the brain into three different tissues, under varying noise conditions. Results are compared with state-of-the-art algorithms in the literature. The algorithm does not use an atlas for initialization or parameter learning. Registration processes are therefore not required and the applicability of the framework can be extended to diseased brains and neonatal brains.

The National Cancer Institute has collected a large database of digitized 35 mm slides of the uterine cervix, the idea being to build a system enabling to study the evolution of lesions related to cervical cancer. In taking the first few steps towards this goal, the objective of this work is to develop and evaluate methodologies required for visual-based (i.e. content-based) indexing and retrieval that substantially improve information management of such a database. In this paper we model the properties of three tissue types using color and texture features, and use these models for image segmentation. Statistical modeling and segmentation tools are used for the task.

In this paper, we describe a statistical video representation and modeling scheme. Video representation schemes are needed to segment a video stream into meaningful video-objects, useful for later indexing and retrieval applications. In the proposed methodology, unsupervised clustering via Gaussian mixture modeling extracts coherent space-time regions in feature space, and corresponding coherent segments (video-regions) in the video content. A key feature of the system is the analysis of video input as a single entity as opposed to a sequence of separate frames. Space and time are treated uniformly. The probabilistic space-time video representation scheme is extended to a piecewise GMM framework in which a succession of GMMs are extracted for the video sequence, instead of a single global model for the entire sequence. The piecewise GMM framework allows for the analysis of extended video sequences and the description of nonlinear, nonconvex motion patterns. The extracted space-time regions allow for the detection and recognition of video events. Results of segmenting video content into static versus dynamic video regions and video content editing are presented.

The objective of the current study is to develop an automatic tool to identify microbiological data types using computer-vision and statistical modeling techniques. Bacteriophage (phage) typing methods are used to identify and extract representative profiles of bacterial types out of species such as the Staphylococcus aureus. Current systems rely on the subjective reading of profiles by a human expert. This process is time-consuming and prone to errors, especially as technology is enabling the increase in the number of phages used for typing. The statistical methodology presented in this work, provides for an automated, objective and robust analysis of visual data, along with the ability to cope with increasing data volumes.

We present two new methods for approximating the Kullback-Liebler (KL) divergence between two mixtures of Gaussians. The first method is based on matching between the Gaussian elements of the two Gaussian mixture densities. The second method is based on the unscented transform. The proposed methods are utilized for image retrieval tasks. Continuous probabilistic image modeling based on mixtures of Gaussians together with KL measure for image similarity, can be used for image retrieval tasks with remarkable performance. The efficiency and the performance of the KL approximation methods proposed are demonstrated on both simulated data and real image data sets. The experimental results indicate that our proposed approximations outperform previously suggested methods.

We present a method for unsupervised clustering of image databases. The method is based on a recently introduced information-theoretic principle, the information bottleneck (IB) principle. Image archives are clustered such that the mutual information between the clusters and the image content is maximally preserved. The IB principle is applied to both discrete and continuous image representations, using discrete image histograms and probabilistic continuous image modeling based on mixture of Gaussian densities, respectively. Experimental results demonstrate the performance of the proposed method for image clustering on a large image database. Several clustering algorithms derived from the IB principle are explored and compared.

Retrieving images from large and varied collections using image content as a key is a challenging and important problem. We present a new image representation that provides a transformation from the raw pixel data to a small set of image regions that are coherent in color and texture. This "Blobworld" representation is created by clustering pixels in a joint color-texture-position feature space. The segmentation algorithm is fully automatic and has been run on a collection of 10,000 natural images. We describe a system that uses the Blobworld representation to retrieve images from this collection. An important aspect of the system is that the user is allowed to view the internal representation of the submitted image and the query results. Similar systems do not offer the user this view into the workings of the system; consequently, query results from these systems can be inexplicable, despite the availability of knobs for adjusting the similarity metrics. By finding image regions that roughly correspond to objects, we allow querying at the level of objects rather than global image properties. We present results indicating that querying for images using Blobworld produces higher precision than does querying using color and texture histograms of the entire image in cases where the image contains distinctive objects.

Objective testing of centerline extraction accuracy in quantitative coronary angiography (QCA) algorithms is a very difficult task. Standard tools for this task are not yet available. We present a simulation tool that generates synthetic angiographic images of a single coronary artery with predetermined centerline and diameter function. This simulation tool was used creating a library of images for the objective comparison and evaluation of QCA algorithms. This technique also provides the means for understanding the relationship between the algorithms' performance and limitations and the vessel's geometrical parameters. In this paper, two algorithms are evaluated and the results are presented.

A technique for enhancing the perceptual sharpness of an image is described. The enhancement algorithm augments the frequency content of the image using shape-invariant properties of edges across scale by using a nonlinearity that generates phase coherent higher harmonics. The procedure utilizes the Laplacian transform and the Laplacian pyramid image representation. Results are presented depicting the power-spectra augmentation and the visual enhancement of several images. Simplicity of computations and ease of implementation allow for real-time applications such as high-definition television (HDTV).

Retrieving images from large and varied collections using image content as a key is a challenging and important problem. In this paper we present a new image representation which provides a transformation from the raw pixel data to a small set of image regions which are coherent in color and texture space. This so-called "blobworld" representation is based on segmentation using the expectation-maximization algorithm on combined color and texture features. The texture features we use for the segmentation arise from a new approach to texture description and scale selection. We describe a system that uses the blobworld representation to retrieve images. An important and unique aspect of the system is that, in the context of similarity-based querying, the user is allowed to view the internal representation of the submitted image and the query results. Similar systems do not offer the user this view into the workings of the system; consequently, the outcome of many queries on these systems can be quite inexplicable, despite the availability of knobs for adjusting the similarity metric.

Retrieving images from very large collections, using image content as a key, is becoming an important problem. Finding objects in image databases is a big challenge in the field. The paper describes our approach to object recognition, which is distinguished by: a rich involvement of early visual primitives, including color and texture; hierarchical grouping and learning strategies in the classification process; the ability to deal with rather general objects in uncontrolled configurations and contexts. We illustrate these properties with three case studies: one demonstrating the use of color and texture descriptors; one learning scenery concepts using grouped features; and one demonstrating a possible application domain in detecting naked people in a scene.

The article work extends a nonlinear, image enhancement scheme proposed by Greenspan and Anderson (see Proceedings of SPIE on Image and Video Processing II, vol.2182, p 2-13, 1993). We concentrate on parameter sensitivity in a global scheme, as well as motivate and discuss adaptive local parametrization in the enhancement algorithm. The enhancement scheme uses nonlinear filtering to augment the frequency spectrum of a blurred input (creating new high-spatial frequencies). As a result of the enhancement, images are created with higher resolution than the sampling rate would allow. This property is highly valuable in application domains which require limited bandwidth to represent or transmit images, including the HDTV and videophone markets.

A given (overcomplete) discrete oriented pyramid may be converted into a steerable pyramid by interpolation. We present a technique for deriving the optimal interpolation functions (otherwise called 'steering coefficients'). The proposed scheme is demonstrated on a computationally efficient oriented pyramid, which is a variation on the Burt and Adelson (1983) pyramid. We apply the generated steerable pyramid to orientation-invariant texture analysis in order to demonstrate its excellent rotational isotropy. High classification rates and precise rotation identification are demonstrated.>

A rotation-invariant texture recognition system is presented. A steerable oriented pyramid is used to extract representative features for the input textures. The steerability of the filter set allows a shift to an invariant representation via a DFT-encoding step. Supervised classification follows. State-of-the-art recognition results are presented on a 30 texture database with a comparison across the performance of the k-NN, backpropagation and rule-based classifiers. In addition, high accuracy estimation of the input rotation angle is demonstrated.

We describe a texture analysis system in which informative discrimination rules are learned from a multiresolution representation of time textured input. The system incorporates unsupervised and supervised learning via statistical machine learning and rule-based neural networks, respectively. The textured input is represented in the frequency-orientation space via a log-Gabor pyramidal decomposition. In the unsupervised learning stage a statistical clustering scheme is used for the quantization of the feature-vector attributes. A supervised stage follows in which labeling of the textured map is achieved using a rule-based network. Simulation results for the texture classification task are given. An application of the system to real-world problems is demonstrated.>

A method for image analysis, based on the formalism of functional analysis is proposed. This approach, which is employed for pattern recognition in the combined position-orientation space, is motivated by a variety of neurophysiological findings that emphasize the importance of the orientation feature in visual image processing and analysis. The approach is applied to the so-called Glass patterns, which are special cases of images characterized uniquely by their orientational feature of local correlations between pairs of corresponding dots. The approach can be of interest in the analysis of optical flow.>