Real-time semantic segmentation is playing a more important role in computer vision, due to the growing demand for mobile devices and autonomous driving. Therefore, it is very important to achieve a good trade-off among performance, model size and inference speed. In this paper, we propose a Channel-wise Feature Pyramid (CFP) module to balance those factors. Based on the CFP module, we built CFPNet for real-time semantic segmentation which applied a series of dilated convolution channels to extract effective features. Experiments on Cityscapes and CamVid datasets show that the proposed CFPNet achieves an effective combination of those factors. For the Cityscapes test dataset, CFPNet achieves 70.1% class-wise mIoU with only 0.55 million parameters and 2.5 MB memory. The inference speed can reach 30 FPS on a single RTX 2080Ti GPU with a $1024\times 2048$-pixel image.

The Generative Adversarial Network (GAN) is a state-of-the-art technique in the field of deep learning. A number of recent papers address the theory and applications of GANs in various fields of image processing. Fewer studies, however, have directly evaluated GAN outputs. Those that have been conducted focused on using classification performance, e.g., Inception Score (IS) and statistical metrics, e.g., Fr\'echet Inception Distance (FID). Here, we consider a fundamental way to evaluate GANs by directly analyzing the images they generate, instead of using them as inputs to other classifiers. We characterize the performance of a GAN as an image generator according to three aspects: 1) Creativity: non-duplication of the real images. 2) Inheritance: generated images should have the same style, which retains key features of the real images. 3) Diversity: generated images are different from each other. A GAN should not generate a few different images repeatedly. Based on the three aspects of ideal GANs, we have designed the Likeness Score (LS) to evaluate GAN performance, and have applied it to evaluate several typical GANs. We compared our proposed measure with two commonly used GAN evaluation methods: IS and FID, and four additional measures. Furthermore, we discuss how these evaluations could help us deepen our understanding of GANs and improve their performance.

Coronavirus disease 2019 (COVID-19) is an ongoing global pandemic that has spread rapidly since December 2019. Real-time reverse transcription polymerase chain reaction (rRT-PCR) and chest computed tomography (CT) imaging both play an important role in COVID-19 diagnosis. Chest CT imaging offers the benefits of quick reporting, a low cost, and high sensitivity for the detection of pulmonary infection. Recently, deep-learning-based computer vision methods have demonstrated great promise for use in medical imaging applications, including X-rays, magnetic resonance imaging, and CT imaging. However, training a deep-learning model requires large volumes of data, and medical staff faces a high risk when collecting COVID-19 CT data due to the high infectivity of the disease. Another issue is the lack of experts available for data labeling. In order to meet the data requirements for COVID-19 CT imaging, we propose a CT image synthesis approach based on a conditional generative adversarial network that can effectively generate high-quality and realistic COVID-19 CT images for use in deep-learning-based medical imaging tasks. Experimental results show that the proposed method outperforms other state-of-the-art image synthesis methods with the generated COVID-19 CT images and indicates promising for various machine learning applications including semantic segmentation and classification.

For supervised learning models, the analysis of generalization ability (generalizability) is vital because the generalizability expresses how well a model will perform on new data. Traditional generalization measures, such as the VC dimension, do not apply to deep neural network (DNN) models. Thus, new theories to explain the generalizability of DNNs are required. In this study, we hypothesize that the DNN with a simpler decision boundary has better generalizability by the law of parsimony (Occam's Razor). We create the decision boundary complexity (DBC) score to define and measure the complexity of decision boundary of DNNs. The idea of the DBC score is to generate data points (called adversarial examples) on or near the decision boundary. Our new approach then measures the complexity of the boundary using the entropy of the eigenvalues of those data. The method works equally well for high-dimensional data. We use training data and the trained model to compute the DBC score. And, the ground truth for a model's generalizability is its test accuracy. Experiments based on the DBC score have verified our hypothesis. The DBC is shown to provide an effective method to measure the complexity of a decision boundary and gives a quantitative measure of the generalizability of DNNs.

To evaluate clustering results is a significant part of cluster analysis. There are no true class labels for clustering in typical unsupervised learning. Thus, a number of internal evaluations, which use predicted labels and data, have been created. They are also named internal cluster validity indices (CVIs). Without true labels, to design an effective CVI is not simple because it is similar to create a clustering method. And, to have more CVIs is crucial because there is no universal CVI that can be used to measure all datasets, and no specific method for selecting a proper CVI for clusters without true labels. Therefore, to apply more CVIs to evaluate clustering results is necessary. In this paper, we propose a novel CVI - called Distance-based Separability Index (DSI), based on a data separability measure. We applied the DSI and eight other internal CVIs including early studies from Dunn (1974) to most recent studies CVDD (2019) as comparison. We used an external CVI as ground truth for clustering results of five clustering algorithms on 12 real and 97 synthetic datasets. Results show DSI is an effective, unique, and competitive CVI to other compared CVIs. In addition, we summarized the general process to evaluate CVIs and created a new method - rank difference - to compare the results of CVIs.

The diagnostic value of a medical image after it has been decompressed is a very practical measure of the image quality and the compression method. A computational observer can be used to make decisions based on an image and thus can provide an assessment of diagnostic value. This paper provides a closed-form estimate of the channelized Hotelling observer's (CHO) output. In medical applications, CHO has been used successfully to predict human observer performance and to evaluate image quality for detection tasks in various backgrounds. To date, it has been used only empirically - i.e., on a set of images - to evaluate compressed image quality. Using a representation of compression noise this paper derives in closed-form the corresponding CHO test statistics and performance. The performance (as measured by the receiver operating characteristic [ROC]) is verified using decompressed JPEG lumpy-background images. The verification results show that the derived ROCs predict the estimated ROCs very well.

This paper considers binary classification. We assess a classifier in terms of the Area Under the ROC Curve (AUC). We estimate three important parameters, the conditional AUC (conditional on a particular training set) and the mean and variance of this AUC. We derive, as well, a closed form expression of the variance of the estimator of the AUC. This expression exhibits several components of variance that facilitate an understanding for the sources of uncertainty of that estimate. In addition, we estimate this variance, i.e., the variance of the conditional AUC estimator. Our approach is nonparametric and based on general methods from U-statistics; it addresses the case where the data distribution is neither known nor modeled and where there are only two available data sets, the training and testing sets. Finally, we illustrate some simulation results for these estimators.

In many cases the combined assessment of three-dimensional anatomical and functional images [single photon emission computed tomography (SPECT), positron emission tomography (PET), magnetic resonance imaging (MRI), and computed tomography (CT)] is necessary to determine the precise nature and extent of lesions. It is important, prior to performing the addition, subtraction, or any other combination of the images, that they be adequately aligned and registered either by experienced radiologists via visual inspection, mental reorientation and overlap of slices, or by an automated registration algorithm. To be useful clinically, the latter case requires validation. The human capacity to evaluate registration results visually is limited and time consuming. This paper describes an algorithmic procedure to provide proxy measures for human assessment that discriminate between badly misregistered pairs of brain images and those likely to be clinically useful. The new algorithm consists of four major steps: segmentation of brain and skin/air boundaries, contour extraction, computation of the principal axes, and computation of the registration quality measures from the contour volumes. The test data were MR and CT brain images. The results of the present study indicate that the use of a measure based on the combination of brain and skin contours and a principal axis function is a good first step to reduce the number of badly registered images reaching the clinician.

Optical methods for detecting tumors in tissue are desirable in part because optical photons are non-ionizing. The highly scattering nature of tissue makes traditional, and even time-gated, optical imaging impractical for sites deeper than a few millimeters. Scattering in tissue causes dispersion in the path lengths of traversing photons which blurs images. With a theoretical description of path length dispersion, however, the perturbation caused by a localized anomaly may be identified from time-resolved intensity data. Random walk theory has been used to show a quantitative, closed-form relationship between the perturbation and the inclusion size and scattering properties relative to the background. Using Monte Carlo data, the authors show a method for analyzing the sensitivity of the relationship to signatures of small, deeply embedded, abnormally scattering inclusions.

Feature analysis for classification is based on the discriminately power of features. In our previous research (1997), we presented a method for measuring the non-parametric discriminatory power of features, called relative feature importance (RFI). RFI has been shown to correctly rank features for a variety of artificial data sets. In this research, we validate RFI on natural data using a multiclass natural data set.

Two one-day workshops were held in 1997 that brought together a group of 38 experts from nine countries to address several aspects of the issues surrounding multimodality medical image registration. The first workshop produced an agenda that defined the significant problems and assigned them to subgroups for study. The second reviewed the results, limited the scope of the problem and proposed an action plan that focused initially on the development of standard data sets. This paper discusses the nature of the problem (why it's interesting and why it's difficult) and the consensus plan that is emerging.

Blood vessels overlying one another at distinct depths (and hence appearing to intersect) in the conjunctiva of the eye can be distinguished reliably from those that in fact do branch within the same depth, using only the information contained in a single photograph of the conjunctiva. That conclusion arises from extension of earlier work that qualitatively inferred relative depth of vessels. The current research was motivated by the need to quantify such inferences in terms of their sensitivities and robustness. Features extracted from an image are shown to be useful in that effort; their utility is verified with phantoms that mimic the behavior of the conjunctiva and the backing sclera. Because no special preparations are needed, the method works as well on archived images as on newly-acquired ones, and thus can be used in retrospective studies of images of the eye and other diffuse media.

Fractal dimension (fd) is a feature which is widely used to characterize medical images. Previously, researchers have shown that fd separates important classes of images and provides distinctive information about texture. We analyze limitations of two principal methods of estimating fd: box-counting (BC) and power spectrum (PS). BC is ineffective when applied to data-limited, low-resolution images; PS is based on a fractional Brownian motion (fBm) model-a model which is not universally applicable. We also present background information on the use of fractal interpolation function (FIF) models to estimate fd of data which can be represented in the form of a function. We present a new method of estimating fd in which multiple FIF models are constructed. The mean of the fd's of the FIF models is taken as the estimate of the fd of the original data. The standard deviation of the fd's of the FIF models is used as a confidence measure of the estimate. We demonstrate how the new method can be used to characterize fractal texture of medical images. In a pilot study, we generated plots of curvature values around the perimeters of images of red blood cells from normal and sickle cell subjects. The new method showed improved separation of the image classes when compared to BC and PS methods.

In this paper, classification problems with N non-mutually exclusive classes are discussed. We introduce a method which can systematically formulate 2/sup n/ mutually exclusive classes without additional training data. We also show that the error rate of the new method is lower than that of the traditional approach when patterns are not mutually exclusive.

Measurement and comparison of medical images is of growing importance for several reasons: (1) as medical imaging becomes ever more digital, and networks and archives of images proliferate, the opportunity will create the need; (2) automated approaches to treatment design, dose measurement, and surgery planning are emerging from the laboratory and entering limited clinical use; and (3) a greater variety of users wants to employ the reliable and repeatable methodology that seems to be offered by the automated methods. Image-processing tools give us some characterization of shape, size, texture, color, depth and three-dimensionality. Combined with properties of the imaging modality and knowledge of anatomy, they yield quantitative descriptions that are useful in differential diagnosis. What has received little attention, however, is the need for benchmarking and evaluation of the various methods available. Almost nothing has been done to ensure the comparability of reported results. The user interface, which may be the clinician's only contact with the application system, has not claimed appreciably more study by system designers than benchmarking. As the need grows to justify the expense of imaging, analysis of its benefits will have to be measured, but until good criteria exist for assessing the imaging system, there cannot be a reliable way to measure outcomes. Equally, use of images from other sites for teaching and research will be impeded in the absence of such metrics. This paper outlines the problem and suggests some steps that can be taken to bring real quantification to medical imaging.>

In this preliminary study the authors analyzed eyetrack data that was recorded while radiologists reviewed mammography images. The method of collecting the eyetrack data has been previously described by Barrett et al. (Acad. Radiol., vol. 1, p.40-5, 1994). The goal of this effort was to identify areas of microcalcifications from the eyetrack information. The identification of areas of microcalcification was accomplished using a Bayes classifier trained on images for which the presence of microcalcifications was verified by biopsies.

Discusses and debates the role played by 3D visualization in medicine as a set of methods and techniques for displaying 3D spatial information related to the anatomy and the physiology of the human body.>

The conjunctiva is a thin membrane that covers the surface of the eye and reveals the small blood vessels of the microcirculation for detailed non-invasive study. The morphology of these vessels is of interest because structural changes in the vascular bed have been shown to occur coincident with certain diseases. Detecting these changes could be used as an early indication, leading to more timely treatment. Our efforts have been to find a reliable method to automate the currently manual methods of analyzing images of the conjunctiva. Our approach has been to first model the illumination/reflection processes that contribute to a scene. We have then used the products of this model to develop some algorithms based on fuzzy logic to reliably detect blood vessel pixels in actual conjunctiva images. Work is progressing in the use of fuzzy logic concepts to track vessels from these detected points so as to provide complete vessel paths and other morphological information.>

The covariance matrices associated with each state of health or disease from a previous study are used as the basis of an image staining display technique for aid in quantitative differential diagnosis. A state of health or disease is chosen by the clinician: this selects the covariance matrix from the data base. A region of interest (ROI) is then scrolled through an abdominal B-scan. For each position of the ROI a point in the four-dimensional feature space is calculated. A natural measure of the distance of this point from the center of mass (multivariate mean) of the disease class is calculated in terms of the covariance matrix of this class; this measure is the Mahalanobis distance. The confidence level for acceptance or rejection of the hypothesized disease class is obtained from the probability distribution of this distance, the T(2) probability law. This confidence level is color coded and used as a color stain that overlays the original scan at that position. The variability of the calculated features is studied as a function of ROI size, or the spatial resolution of the color coded image, and it is found that for an ROI in the neighborhood of 4 cm(2) most of the variability due to the finite number of independent samples (speckles) is averaged out, leaving the "noise floor" associated with inter- and intra-patient variability. ROIs on the order of 1 cm(2) may result with technical advances in B-scan resolution. A small number of points on organ boundaries are entered by the user, to fit with arcs of ellipses to be used to switch between organ (liver and kidney) data bases as the ROI encounters the boundary. By selecting in turn various state-of-health or state-of-disease databases, such images of confidence levels may be used for quantitative differential diagnosis. The method is not limited to ultrasound, being applicable in principle to features obtained from any modality or multimodality combination.

A combined-transform coding (CTC) scheme to reduce the blocking effect of conventional block transform coding and hence to improve the subjective performance is presented. The scheme is described, and its information-theoretic properties are discussed. Computer simulation results for a chest X-ray image are presented. The CTC scheme, the JPEG baseline scheme, and the conventional discrete Walsh-Hadamard transform (DWHT) are compared to demonstrate the performance improvement for the CTC scheme. The advantages of the CTC scheme include no ringing effect as there is no error propagation across the boundary, no additional computation, and distortion always held within a certain level.>

The rate-distortion performance of a class of non-Gaussian source with memory is studied. The source model is generated from a correlated Gaussian source through a memoryless transform, which actually constitutes a special class of the frequently-used Gibbs model in speech and image processing applications. Its entropy and a rate-distortion bound are evaluated to obtain further insights into this class of source models. Several examples are also given to illustrate the usefulness and tightness of the rate-distortion bound.>

The design, implementation, and testing of a computational observer method for objective evaluation of ultrasound images are presented. The method uses digitized ultrasound B-scan images of a test phantom (the contrast-detail phantom), and is able to calculate the detectability of a target (signal) from its background (background noise). A quantitative detectability index, based on the measured signal-to-noise ratio of the image data, that is measured for both the human and the computational observer on the same scale is generated. It is shown that the computational observer (CO) method may be a more useful, objective way of evaluating ulrasound images and imaging systems than methods that rely solely on human observers. It may also be applicable to other types (i.e. other than ultrasound) of imaging systems which produce noisy images. The relevance of the CO method when compared to human observer two-alternative-forced-choice (2AFC) readings of the same data by showing a high correlation between the CO detectability results and those of human observers for the same set of images. The method is (1) quantitative, (2) reproducible, (3) absolute, (4) takes into account, and can calculate the value of TPF and FPF for each target, for the given system, and (5) speeds up the evaluation of an image or imaging system (compared to using human observers), given the right conditions and equipment.

A combined-transform coding (CTC) scheme is proposed to reduce the blocking artifact of conventional block transform coding and hence to improve the subjective performance. The proposed CTC scheme is described and its information-theoretic properties are investigated. Computer simulation results for a class of chest X-ray images are presented. A comparison between the CTC scheme and the conventional discrete cosine transform (DCT) and discrete Walsh-Hadamard transform (DWHT) demonstrates the performance improvement of the proposed scheme. In addition, combined coding can also be used in noiseless coding, yielding a slight improvement in the compression performance if it is used properly.

Classification problems with non-mutually exclusive classes are discussed. A method, new class discovery (NCD), is proposed which can discover potentially unknown or untrained classes. It is formally proved that, in the non-mutually exclusive case, the error probability of the NCD method is less than the conventional classification method using Bayes's rule. Application of the method to real entomological data was successful: a species was discovered that had been independently verified by taxonomists.>

An approach for the application of multivariate pattern recognition techniques to detection of diffuse and focal disease using acoustic data is reviewed. Supervised and unsupervised techniques are implemented to design the best ultrasonic tissue signature for a given task from a set of measurements. The performances of both techniques are evaluated and compared using several methods. However, it is desirable to utilize a technique that quantitatively detects and displays the heterogeneity of an ultrasound image. It is shown that, for a particular task, choosing features with physical significance will make the classification of the data more robust. It is also shown that the success of combining supervised and unsupervised techniques using such features extends beyond discrimination of one class of data from the other and that the approach can be used to grade and the variations in the same tissue type.>

The probabilistic distribution properties of a set of medical images are studied. It is shown that the generalized Gaussian function provides a good approximation to the distribution of antero-posterior chest radiographs. Based on this result and a goodness-of-fit test. a generalized Gaussian autoregressive model (GGAR) is proposed. Its properties and limitations are discussed. It is expected that the GGAR model will be useful in describing the stochastic characteristics of some classes of medical images and can be used in image data compression and other applications.>

Shadows provide information that allows inferences to be made about the three-dimensional nature of objects. Essential to those inferences is the ability to identify segments of the shadow boundary that correspond to various portions of the object that has cast the shadow. Entry-exit vertices, that are extremes of the boundary, and tangent to the projection of the light beams, can be identified as junctions of specific segments of the shadow boundary. The authors restrict the analysis to smooth convex bodies, and prove that points of discontinuity of the derivative on the shadow boundary form another set of vertices. These two types of junctions are discussed extensively, and combined into a labeling scheme that can segment the boundary into logically consistent pieces. The authors discuss the possible ambiguities that may arise during the labeling process due to occlusion of one object by another, and show that there is no ambiguity in the cases of occlusion of shadows and of shadows that fall on other objects.>

A general formula for the slope and mathematical models of digitized straight-line segments are derived and discussed in detail. The models are parametric and the parameters are calculated uniquely and in closed form from the slope without the need for recursion or interaction. The modeling is information-preserving and explicitly uses the slope. The models are in a form convenient for plotting, detecting, and recognizing digitized straight-line segments.>

A new method has been developed for interpreting the shadows of arbitrarily shaped surfaces by segmenting and labeling the shadow boundary. The method is based on the fact that a linear projection of any light ray (the ray is assumed to originate at a single, distant source) across a shadow either enters or exits the shadow at its boundary. Hence, junctures of entry and exit segments form vertices that can be found directly for any given direction of illumination and view. Entry-exit vertices that are extremes of the boundary (which is normal to the axis of light) can be identified as junctures of specific profiles of the shadow-making object. These junctures, in turn identify the segments connected to them. The method assumes successful lower level extraction of shadow boundaries. When one object occludes part of another object's shadow, critical junctures occur, but these sometimes are not entry-exit vertices. These hidden junctures create ambiguities that must be dealt with in the context of neighboring segments. Certain a priori knowledge is helpful in this situation. The method may require knowledge of the surface or the object. The entry-exit method also provides a new link between the tasks of shadow boundary extraction and shape inference in the overall process of shadow interpretation. In conjunction with existing methods for the other tasks, the entry-exit method makes it possible to interpret arbitrarily shaped shadows.