As artificial intelligence and machine learning algorithms become increasingly prevalent in society, multiple stakeholders are calling for these algorithms to provide explanations. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, have different explanation needs. To address these needs, in 2019, we created AI Explainability 360, an open source software toolkit featuring ten diverse and state-of-the-art explainability methods and two evaluation metrics. This paper examines the impact of the toolkit with several case studies, statistics, and community feedback. The different ways in which users have experienced AI Explainability 360 have resulted in multiple types of impact and improvements in multiple metrics, highlighted by the adoption of the toolkit by the independent LF AI & Data Foundation. The paper also describes the flexible design of the toolkit, examples of its use, and the significant educational material and documentation available to its users.

The novel nature of SARS-CoV-2 calls for the development of efficient de novo drug design approaches. In this study, we propose an end-to-end framework, named CogMol (Controlled Generation of Molecules), for designing new drug-like small molecules targeting novel viral proteins with high affinity and off-target selectivity. CogMol combines adaptive pre-training of a molecular SMILES Variational Autoencoder (VAE) and an efficient multi-attribute controlled sampling scheme that uses guidance from attribute predictors trained on latent features. To generate novel and optimal drug-like molecules for unseen viral targets, CogMol leverages a protein-molecule binding affinity predictor that is trained using SMILES VAE embeddings and protein sequence embeddings learned unsupervised from a large corpus. We applied the CogMol framework to three SARS-CoV-2 target proteins: main protease, receptor-binding domain of the spike protein, and non-structural protein 9 replicase. The generated candidates are novel at both the molecular and chemical scaffold levels when compared to the training data. CogMol also includes insilico screening for assessing toxicity of parent molecules and their metabolites with a multi-task toxicity classifier, synthetic feasibility with a chemical retrosynthesis predictor, and target structure binding with docking simulations. Docking reveals favorable binding of generated molecules to the target protein structure, where 87--95\% of high affinity molecules showed docking free energy $<$ -6 kcal/mol. When compared to approved drugs, the majority of designed compounds show low predicted parent molecule and metabolite toxicity and high predicted synthetic feasibility. In summary, CogMol can handle multi-constraint design of synthesizable, low-toxic, drug-like molecules with high target specificity and selectivity, even to novel protein target sequences, and does not need target-dependent fine-tuning of the framework or target structure information.

As artificial intelligence algorithms make further inroads in high-stakes societal applications, there are increasing calls from multiple stakeholders for these algorithms to explain their outputs. To make matters more challenging, different personas of consumers of explanations have different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360, an open-source Python toolkit featuring ten diverse and state-of-the-art explainability methods and two evaluation metrics. Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of interpretation and explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. The toolkit is not only the software, but also guidance material, tutorials, and an interactive web demo to introduce AI explainability to different audiences. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.

Artificial intelligence systems are being increasingly deployed due to their potential to increase the efficiency, scale, consistency, fairness, and accuracy of decisions. However, as many of these systems are opaque in their operation, there is a growing demand for such systems to provide explanations for their decisions. Conventional approaches to this problem attempt to expose or discover the inner workings of a machine learning model with the hope that the resulting explanations will be meaningful to the consumer. In contrast, this paper suggests a new approach to this problem. It introduces a simple, practical framework, called Teaching Explanations for Decisions (TED), that provides meaningful explanations that match the mental model of the consumer. We illustrate the generality and effectiveness of this approach with two different examples, resulting in highly accurate explanations with no loss of prediction accuracy for these two examples.

In recent years there has been an increasing trend in which data scientists and domain experts work together to tackle complex scientific questions. However, such collaborations often face challenges. In this paper, we aim to decipher this collaboration complexity through a semi-structured interview study with 22 interviewees from teams of bio-medical scientists collaborating with data scientists. In the analysis, we adopt the Olsons' four-dimensions framework proposed in Distance Matters to code interview transcripts. Our findings suggest that besides the glitches in the collaboration readiness, technology readiness, and coupling of work dimensions, the tensions that exist in the common ground building process influence the collaboration outcomes, and then persist in the actual collaboration process. In contrast to prior works' general account of building a high level of common ground, the breakdowns of content common ground together with the strengthen of process common ground in this process is more beneficial for scientific discovery. We discuss why that is and what the design suggestions are, and conclude the paper with future directions and limitations.

Your computer is continuously executing programs, but does it really understand them? Not in any meaningful sense. That burden falls upon human knowledge workers, who are increasingly asked to write and understand code. They would benefit greatly from intelligent tools that reveal the connections between their code and its subject matter. Towards this prospect, we present an AI system that forms semantic representations of computer programs, using techniques from knowledge representation and program analysis. These representations are created through a novel algorithm for the semantic enrichment of dataflow graphs. We illustrate its workings with examples from the field of data science. The algorithm is undergirded by a new ontology language for modeling computer programs and a new ontology about data science, written in this language.

Some of the largest worldwide employers today are knowledge-based enterprises whose most important asset is human capital. Knowledge workers are unique, each having individualized skills, competencies and expertise, which constantly evolve and expand. Managing and planning for such a workforce critically depends on the ability to construct complete, accurate, and real-time representation and inventory of the expertise of employees in a form that integrates with business processes. In this session Saška will describe how enterprise expertise assessment process can be posed as predictive modeling and recommendation problem, and will present results and lessons learned from an actual deployment of IBM Expertise, a corporate-wide expertise recommendation and management system.

Strategic planning and talent management in large enterprises composed of knowledge workers requires complete, accurate, and up-to-date representation of the expertise of employees in a form that integrates with business processes. Like other similar organizations operating in dynamic environments, the IBM Corporation strives to maintain such current and correct information, specifically assessments of employees against job roles and skill sets from its expertise taxonomy. In this work, we deploy an analytics-driven solution that infers the expertise of employees through the mining of enterprise and social data that is not specifically generated and collected for expertise inference. We consider job role and specialty prediction and pose them as supervised classification problems. We evaluate a large number of feature sets, predictive models and postprocessing algorithms, and choose a combination for deployment. This expertise analytics system has been deployed for key employee population segments, yielding large reductions in manual effort and the ability to continually and consistently serve up-to-date and accurate data for several business functions. This expertise management system is in the process of being deployed throughout the corporation.

In this article, we propose a random walk-based model to predict legislators' votes on a set of bills. In particular, we first convert roll call data, i.e. the recorded votes and the corresponding deliberative bodies, to a heterogeneous graph, where both the legislators and bills are treated as vertices. Three types of weighted edges are then computed accordingly, representing legislators' social and political relations, bills' semantic similarity, and legislator-bill vote relations. Through performing two-stage random walks over this heterogeneous graph, we can estimate legislative votes on past and future bills. We apply this proposed method on real legislative roll call data of the United States Congress and compare to state-of-the-art approaches. The experimental results demonstrate the superior performance and unique prediction power of the proposed model.

Signals in response to time-localized events of a common phenomenon tend to exhibit a common shape, but with variable time scale, amplitude, and delay across trials in many domains. We develop a new formulation to learn the common shape and variables from noisy signal samples with a Bayesian signal model and a Markov chain Monte Carlo inference scheme involving Gibbs sampling and independent Metropolis-Hastings. Our experiments with generated and real-world data show that the algorithm is robust to missing data, outperforms the existing approaches and produces easily interpretable outputs.

Consider a typical recommendation problem. A company has historical records of products sold to a large customer base. These records may be compactly represented as a sparse customer-times-product ``who-bought-what" binary matrix. Given this matrix, the goal is to build a model that provides recommendations for which products should be sold next to the existing customer base. Such problems may naturally be formulated as collaborative filtering tasks. However, this is a {\it one-class} setting, that is, the only known entries in the matrix are one-valued. If a customer has not bought a product yet, it does not imply that the customer has a low propensity to {\it potentially} be interested in that product. In the absence of entries explicitly labeled as negative examples, one may resort to considering unobserved customer-product pairs as either missing data or as surrogate negative instances. In this paper, we propose an approach to explicitly deal with this kind of ambiguity by instead treating the unobserved entries as optimization variables. These variables are optimized in conjunction with learning a weighted, low-rank non-negative matrix factorization (NMF) of the customer-product matrix, similar to how Transductive SVMs implement the low-density separation principle for semi-supervised learning. Experimental results show that our approach gives significantly better recommendations in comparison to various competing alternatives on one-class collaborative filtering tasks.

Many applications require matching objects to a predefined, yet highly dynamic set of categories accompanied by category descriptions. We present a novel approach to solving this class of categorization problems by formulating it in a semi-supervised clustering framework. Text-based matching is performed to generate ldquosoftrdquo seeds, which are then used to guide clustering in the basic feature space. We introduce a new variation of the k-means algorithm, called Soft Seeded k-means, which can effectively incorporate seeds that are of varying degrees of confidence, while allowing for incomplete coverage of the pre-defined categories. The algorithm is applied to real-world data from a business analytics application, and we demonstrate that it leads to superior performance compared to previous approaches.

By attempting to simultaneously partition both the rows (examples) and columns (features) of a data matrix, Co-clustering algorithms often demonstrate surpris- ingly impressive performance improvements over traditional one-sided (row) clustering techniques. A good clustering of features may be seen as a combinatorial transformation of the data matrix, effectively enforcing a form of regularization that may lead to a better clustering of examples (and vice-versa). In many applications, partial supervision in the form of a few row labels as well as column labels may be available to potentially assist co-clustering. In this paper, we develop two novel semi-supervised multi-class classification algorithms motivated respectively by spectral bipartite graph partitioning and matrix approximation (e.g., non-negative matrix factorization) formulations for co-clustering. These algorithms (i) support dual supervision in the form of labels for both examples and/or features, (ii) provide principled predictive capability on out-of-sample test data, and (iii) arise naturally from the classical Representer theorem applied to regularization problems posed on a collection of Reproducing Kernel Hilbert Spaces. Empirical results demonstrate the effectiveness and utility of our algorithms.

We consider the problem of deriving compressed perceptual representation of multivariate time series and using it for efficient indexing and similarity search. Our algorithm is based on the identification of perceptual skeletons in multidimensional space and the use of these "simplifications" in similarity measurements. We illustrate the performance of the algorithm in a financial modeling application. Our results indicate that the skeleton representation outperforms the traditional approaches and is robust enough to be used even with the simplest distance metrics.

Despite the wide use of data mining techniques in client segmentation and market analysis applications, so far there have been no algorithms that allow for the discovery of strategically important combinations of products (or offerings) – the ones that have the highest impact on the performance of the company. We present a novel algorithm for analyzing a multiproduct environment and identifying strategically important combinations of offerings with respect to a predefined criterion, such as revenue impact, profit impact, inventory turnover etc. In contrast to the traditional association rule and frequent item mining techniques, the goal of the new algorithm is to find segments of data, defined through combinations of products (rules), which satisfy certain conditions as a group. We present a novel algorithm to derive specialized partition threes, called High Yield Partition Trees, which lead to such segments, and investigate different splitting strategies. The algorithm has been tested on real-world data, and achieved very good performance.

Logistic regression is one of the frequently used models in pattern recognition, especially in binary classification tasks. We focus on a class of small-sample classification problems where logistic regression seems to be a "natural" choice for the classifier, yet its direct application yields sub-optimal results. Specifically, we consider cases when: (1) input-output relationships are nonlinear; (2) there is a need to estimate hidden states or auxiliary variables in the model; and (3) the training set is small preventing the use of more sophisticated techniques. We first describe an approach to compute the parameters of the regression, which addresses the issue of estimating hidden variables. We then describe a recursive adaptation procedure that identifies the most significant nonlinear relationships in the data and adapts the model by introducing corresponding higher-order terms. The performance of the method is tested in a business modeling application, demonstrating significant improvements over the traditional classifiers.

We propose a new approach for image segmentation that is based on low-level features for color and texture. It is aimed at segmentation of natural scenes, in which the color and texture of each segment does not typically exhibit uniform statistical characteristics. The proposed approach combines knowledge of human perception with an understanding of signal characteristics in order to segment natural scenes into perceptually/semantically uniform regions. The proposed approach is based on two types of spatially adaptive low-level features. The first describes the local color composition in terms of spatially adaptive dominant colors, and the second describes the spatial characteristics of the grayscale component of the texture. Together, they provide a simple and effective characterization of texture that the proposed algorithm uses to obtain robust and, at the same time, accurate and precise segmentations. The resulting segmentations convey semantic information that can be used for content-based retrieval. The performance of the proposed algorithms is demonstrated in the domain of photographic images, including low-resolution, degraded, and compressed images.

The extraction of high-level color descriptors is an increasingly important problem, as these descriptions often provide links to image content. When combined with image segmentation, color naming can be used to select objects by color, describe the appearance of the image, and generate semantic annotations. This paper presents a computational model for color categorization and naming and extraction of color composition. In this paper, we start from the National Bureau of Standards' recommendation for color names, and through subjective experiments, we develop our color vocabulary and syntax. To assign a color name from the vocabulary to an arbitrary input color, we then design a perceptually based color-naming metric. The proposed algorithm follows relevant neurophysiological findings and studies on human color categorization. Finally, we extend the algorithm and develop a scheme for extracting the color composition of a complex image. According to our results, the proposed method identifies known color regions in different color spaces accurately, the color names assigned to randomly selected colors agree with human judgments, and the description of the color composition of complex scenes is consistent with human observations.

We present a perceptually-tuned multiscale image segmentation algorithm that is based on spatially adaptive color and texture features. The proposed algorithm extends a previously proposed approach to include multiple texture scales. The determination of the multiscale texture features is based on perceptual considerations. We also examine the perceptual tuning of the algorithm and how it is affected by the presence of different texture scales. The multiscale extension is necessary for segmenting higher resolution images and is particularly effective in segmenting objects shown in different perspectives. The performance of the proposed algorithm is demonstrated in the domain of photographic images.

We propose a method for semantic categorization and retrieval of photographic images based on low-level image descriptors derived from perceptual experiments. The method applies multidimensional scaling and hierarchical clustering analysis to identify candidate semantic categories into which human observers organize images. Through a series of subjective experiments we refine our definition of these categories and select a set of low-level image features that uniquely describe them. We then devise a new image similarity metric and develop a prototype system, which identifies the semantic category of the image and retrieves similar images from the database. We tested the metric on a set of new images and compared the categorization results with that of human observers. Our results provide a good match to human performance, thus validating the use of human judgments to develop semantic descriptors. Our method can be used for the enhancement of current retrieval methods, better organization of image/video databases, and the development of more intuitive navigation schemes, browsing methods and user interfaces.

An image segmentation algorithm that is based on spatially adaptive color and texture features is presented. The proposed algorithm is based on a previously proposed algorithm but introduces a number of new elements. We use a new set of texture features based on a steerable filter decomposition. The steerable filters combined with a new spatial texture segmentation scheme provide a finer and more robust segmentation into texture classes. The proposed algorithm includes an elaborate border estimation procedure, which extends the idea of Pappas (1992) adaptive clustering segmentation algorithm to color texture. The performance of the proposed algorithm is demonstrated in the domain of photographic images, including low resolution compressed images.

We propose an image segmentation algorithm that is based on spatially adaptive color and texture features. The features are first developed independently, and then combined to obtain an overall segmentation. Texture feature estimation requires a finite neighborhood which limits the spatial resolution of texture segmentation, while color segmentation provides accurate and precise edge localization. We combine a previously proposed adaptive clustering algorithm for color segmentation with a simple but effective texture segmentation approach to obtain an overall image segmentation. Our focus is in the domain of photographic images with an essentially unlimited range of topics. The images are assumed to be of relatively low resolution and may be degraded or compressed.

Content-based retrieval (CBIR) methods in medical databases have been designed to support specific tasks, such as retrieval of digital mammograms or 3D MRI images. These methods cannot be transferred to other medical applications since different imaging modalities require different types of processing. To enable content-based queries in diverse collections of medical images, the retrieval system must be familiar with the current image class prior to the query processing. We describe a novel approach for the automatic categorization of medical images according to their modalities. We propose a semantically based set of visual features, their relevance and organization for capturing the semantics of different imaging modalities. The features are used in conjunction with a new categorization metric, enabling "intelligent" annotation, browsing/searching of medical databases. Our algorithm provides basic semantic knowledge about the image, and may serve as a front-end to the domain specific medical image analysis methods. To demonstrate the effectiveness of our approach, we have designed and implemented an Internet portal for browsing/querying online medical databases, and applied it to a large number of images. Our results demonstrate that accurate categorization can be achieved by exploiting the important visual properties of each modality.

Color is one of the main visual cues and has been frequently used in image processing, analysis and retrieval. The extraction of high-level color descriptors is an increasingly important problem, as these descriptions often provide link to image content. When combined with image segmentation color naming can be used to select objects by color, describe the appearance of the image and even generate semantic annotations. For example, regions labeled as light blue and strong green may represent sky and grass, vivid colors are typically found in man-made objects, and modifiers such as brownish, grayish and dark convey the impression of the atmosphere in the scene. This paper presents a computational model for color categorization, naming and extraction of color composition. In this work we start from the National Bureau of Standards recommendation for color names, and through subjective experiments develop our color vocabulary and syntax. Next, to attach the color name to an arbitrary input color, we design a perceptually based color naming metric. Finally, we extend the method and develop a scheme for extracting the color composition of a complex image. The algorithm follows the relevant neurophysiological findings and studies on human color categorization. In testing the method the known color regions in different color spaces were identified accurately, the color names assigned to randomly selected colors agreed with human judgments, and the color composition extracted from natural images was consistent with human observations.

Color descriptors are among the most important features used in image analysis and retrieval. Due to its compact representation and low complexity, direct histogram comparison is a commonly used technique for measuring the color similarity. However, it has many serious drawbacks, including a high degree of dependency on color codebook design, sensitivity to quantization boundaries, and inefficiency in representing images with few dominant colors. In this paper, we present a new algorithm for color matching that models behavior of the human visual system in capturing color appearance of an image. We first develop a new method for color codebook design in the Lab space. The method is well suited for creating small fixed color codebooks; for image analysis, matching, and retrieval. Then we introduce a statistical technique to extract perceptually relevant colors. We also propose a new color distance measure that is based on the optimal mapping between two sets of color components representing two images. Experiments comparing the new algorithm to some existing techniques show that these novel elements lead to better match to human perception in judging image similarity in terms of color composition.

We propose a method for semantic categorization and retrieval of photographic images based on low-level image descriptors. In this method, we first use multidimensional scaling (MDS) and hierarchical cluster analysis (HCA) to model the semantic categories into which human observers organize images. Through a series of psychophysical experiments and analyses, we refine our definition of these semantic categories, and use these results to discover a set of low-level image features to describe each category. We then devise an image similarity metric that embodies our results, and develop a prototype system, which identifies the semantic category of the image and retrieves the most similar images from the database. We tested the metric on a new set of images, and compared the categorization results with that of human observers. Our results provide a good match to human performance, thus validating the use of human judgments to develop semantic descriptors.

Color quantization is sampling of three-dimensional (3-D) color spaces (such as RGB or Lab) which results in a discrete subset of colors known as a color codebook or palette. It is extensively used for display, transfer, and storage of natural images in Internet-based applications, computer graphics, and animation. We propose a sampling scheme which provides a uniform quantization of the Lab space. The idea is based on several results from number theory and phyllotaxy. The sampling algorithm is very much systematic and allows easy design of universal (image-independent) color codebooks for a given set of parameters. The codebook structure allows fast quantization and ordered dither of color images. The display quality of images quantized by the proposed color codebooks is comparable with that of image-dependent quantizers. Most importantly, the quantized images are more amenable to the type of processing used for grayscale ones. Methods for processing grayscale images cannot be simply extended to color images because they rely on the fact that each gray-level is described by a single number and the fact that a relation of full order can be easily established on the set of those numbers. Color spaces (such as RGB or Lab) are, on the other hand, 3-D. The proposed color quantization, i.e., color space sampling and numbering of sampled points, makes methods for processing grayscale images extendible to color images. We illustrate possible processing of color images by first introducing the basic average and difference operations and then implementing edge detection and compression of color quantized images.

We present a color matching algorithm that models the behavior of the human visual system in capturing color appearance of an image. We first develop a new method for color codebook design. The method is well suited for creating small color codebooks used in image analysis and retrieval. We then introduce a statistical technique to extract perceptually relevant colors. We also propose a new color metric that guarantees optimality in matching different color components of two images. Experiments comparing the new algorithm to several existing ones show that it provides a better match to human perception of color similarity.

Color features are among the most important features used in image database retrieval, especially in cases where no additional semantic information is available. Due to its compact representation and low complexity, direct histogram comparison is the most commonly used technique in comparing color similarity of images. However, it has many serious drawbacks, including a high degree of dependency on color codebook design, sensitivity to quantization boundaries, and inefficiency in representing images with few dominant colors. We present an algorithm for color matching. We describe a statistical technique to extract perceptually relevant colors. We propose a new color distance measure that guarantees optimality in matching different color components of two images. Finally, experimental results are presented comparing this new algorithm to some existing techniques.

We propose a perceptually based system for pattern retrieval and matching. The central idea is that similarity judgment has to be modeled along perceptual dimensions. Hence, we detect basic visual categories that people use in their judgment of similarity, and design a computational model that accepts patterns as input and, depending on the query, produces a set of choices that follow human behavior in pattern matching. There are two major research aspects to our work. The first one addresses the issue of how humans perceive and measure similarity within the domain of color patterns. To understand and describe this mechanism, we performed a subjective experiment which yielded five perceptual criteria used in comparison between color patterns (vocabulary), as well as a set of rules governing the use of these criteria in similarity judgment (grammar). The second research aspect is the implementation of the perceptual criteria and rules in an image retrieval system. Following the processing typical for human vision, we design a system to: (1) extract perceptual features from the vocabulary and (2) perform the comparison between the patterns according to the grammar rules. The modeling of human perception of color patterns is new--starting with a new color codebook design, compact color representation, and texture description through multi-scale edge distribution along different directions. Moreover, we propose new color and texture distance functions that correlate with human performance. The performance of the system is illustrated with numerous examples from image databases from different application domains.

We determine the basic categories and the hierarchy of rules used by humans in judging similarity and matching of color patterns. The categories are: (1) overall color; (2) directionality and orientation; (3) regularity and placement; (4) color purity; (5) complexity and heaviness. These categories form the pattern vocabulary which is governed by the grammar rules. Both the vocabulary and the grammar were obtained as a result of a subjective experiment. Experimental data were interpreted using multidimensional scaling techniques yielding the vocabulary and the hierarchical clustering analysis, yielding the grammar rules. Finally, we give a short overview of the existing techniques that can be used to extract and measure the elements of the vocabulary.

Although most of the theoretical and implementation aspects of wavelet based algorithms in texture characterization are well studied and understood, many issues related to the choice of filter bank in texture processing remain unresolved. The impact of the wavelet basis has been mentioned in a few papers, whereas other more detailed investigations have considered only the choice of the wavelet basis in image coding. For example, regularity, number of vanishing moments and a shift variance degree have been used for the filter evaluation in image coding, but the utility of all these metrics in texture analysis has not yet been established. Therefore, the scope of this paper was to investigate whether the properties of decomposition filters play an important role in texture description, and which feature is dominant in the selection of an optimal filter bank. We performed classification experiments with 23 Brodatz textures. Our investigation shows that the selection of the decomposition filters has a significant influence on the result of texture characterization. Finally, the paper ranks 19 orthogonal and biorthogonal filters, and establishes the most relevant criteria for choice of decomposition filters in wavelet-based texture characterization algorithms.

In this paper we present a new wavelet domain technique for texture analysis and test of pattern replicability. The main property of the proposed features is that they measure texture quality along the most important perceptual dimensions. In other words, we quantify and classify textures according to their directionality, symmetry regularity and type of regularity. After the feature extraction, texture classification is performed by traversing a tree. The algorithm is tested on a database with 340 images demonstrating an excellent classification accuracy.

We propose a perceptually-based system for pattern retrieval and matching. The central idea of the work is that similarity judgment has to be modeled along perceptual dimensions. Hence, we detect basic visual categories that people use in judgment of similarity, and design a computational model which accepts patterns as input, and depending on the query, produces a set of choices that follow human behavior in pattern matching. To understand how humans perceive color patterns we performed a subjective experiment The experiment yielded five perceptual criteria used in comparison between color patterns (vocabulary), as well as a set of rules governing the use of these criteria in similarity judgment (grammar). This paper describes the actual implementation of the perceptual criteria and rules in the image retrieval system. Following the processing typical for human vision, we designed a system to: (a) extract perceptual features from the vocabulary and (b) perform the comparison between the patterns according to the grammar rules. We propose new color and texture features, as well as new distance functions that correlate with human performance. The performance of the system is illustrated with numerous examples from image databases from different application domains.

This paper describes a new approach for texture characterization, based on nonseparable wavelet decomposition, and its application for the discrimination of visually similar diffuse diseases of liver. The proposed feature-extraction algorithm applies nonseparable quincunx wavelet transform and uses energies of the transformed regions to characterize textures. Classification experiments on a set of three different tissue types show that the scale/frequency approach, particularly one based on the nonseparable wavelet transform, could be a reliable method for a texture characterization and analysis of B-scan liver images. Comparison between the quincunx and the traditional wavelet decomposition suggests that the quincunx transform is more appropriate for characterization of noisy data, and practical applications, requiring description with lower rotational sensitivity.

This paper investigates the application of nonseparable wavelet transform for the texture characterization. On a set of 21 Brodatz textures we have performed traditional dyadic wavelet decomposition (in two levels) and nonseparable quincunx decomposition (in six levels), testing their ability to classify textures in: (a) normal working conditions, (b) noisy environment, and (c) with respect to rotation. Our experiments have shown that the quincunx transform is appropriate for characterization of noisy data, small number of resolution levels and shorter feature vectors and rotationally invariant description.

This paper describes the application of the nonseparable wavelet decomposition for the discrimination of diffuse diseases of liver. The proposed feature extraction algorithm uses the filter bank performing the quincunx transform and characterizes textures by a set of channel variances estimated at the output of each filter. Classification experiments on a set of three different tissue types show that this approach could be a reliable method for analysis of B-scan liver images.

The authors propose a new separable extension of the 1-D wavelet transform to the 2-D case and describe its application to the texture characterization problem. Comparing to previous decompositions, with the same resolution levels for each subband in horizontal and vertical directions, the new method has different resolutions for different directions. The new algorithm is applied to 84 ultrasound liver images, to detect liver cirrhosis in it's early stage. The classification accuracy was 92%. The method was compared to other texture description methods (gray level cooccurrence, Laws filters, pyramid and tree structured wavelet decompositions). The proposed 2N/spl times/1-D wavelet decomposition, gave the highest classification rate, showing its applicability for the approach based analysis of the large class of natural textures.

Some computer applications for tissue characterization in medicine operate with tissue samples taken from small area of interest so that only few methods can be used. These methods should be insensitive to noise and image distortions and yet reliable enough. Here, the authors propose a new approach for texture analysis, based on the wavelet transform. The idea of this method is to decompose analyzed image with the filter bank derived from an orthonormal wavelet basis and to form image approximations with higher resolution. Energies calculated at the outputs of the filter bank are used as texture features in an unsupervised classification procedure based on a modification of the statistical T-test. The method is tested clinically, for characterization of infarcted myocardium and reported classification results are very promising.