Over the past five years, artificial intelligence (AI) has introduced new models and methods for addressing the challenges associated with the broader adoption of AI models and systems in medicine. This paper reviews recent advances in AI for medical image and video analysis, outlines emerging paradigms, highlights pathways for successful clinical translation, and provides recommendations for future work. Hybrid Convolutional Neural Network (CNN) Transformer architectures now deliver state-of-the-art results in segmentation, classification, reconstruction, synthesis, and registration. Foundation and generative AI models enable the use of transfer learning to smaller datasets with limited ground truth. Federated learning supports privacy-preserving collaboration across institutions. Explainable and trustworthy AI approaches have become essential to foster clinician trust, ensure regulatory compliance, and facilitate ethical deployment. Together, these developments pave the way for integrating AI into radiology, pathology, and wider healthcare workflows.

Line detection is a basic digital image processing operation used by higher-level processing methods. Recently, transformer-based methods for line detection have proven to be more accurate than methods based on CNNs, at the expense of significantly lower inference speeds. As a result, video analysis methods that require low latencies cannot benefit from current transformer-based methods for line detection. In addition, current transformer-based models require pretraining attention mechanisms on large datasets (e.g., COCO or Object360). This paper develops a new transformer-based method that is significantly faster without requiring pretraining the attention mechanism on large datasets. We eliminate the need to pre-train the attention mechanism using a new mechanism, Deformable Line Attention (DLA). We use the term LINEA to refer to our new transformer-based method based on DLA. Extensive experiments show that LINEA is significantly faster and outperforms previous models on sAP in out-of-distribution dataset testing. Code available at https://github.com/SebastianJanampa/LINEA.

Line segment detection is a fundamental low-level task in computer vision, and improvements in this task can impact more advanced methods that depend on it. Most new methods developed for line segment detection are based on Convolutional Neural Networks (CNNs). Our paper seeks to address challenges that prevent the wider adoption of transformer-based methods for line segment detection. More specifically, we introduce a new model called Deformable Transformer-based Line Segment Detection (DT-LSD) that supports cross-scale interactions and can be trained quickly. This work proposes a novel Deformable Transformer-based Line Segment Detector (DT-LSD) that addresses LETR's drawbacks. For faster training, we introduce Line Contrastive DeNoising (LCDN), a technique that stabilizes the one-to-one matching process and speeds up training by 34x. We show that DT-LSD is faster and more accurate than its predecessor transformer-based model (LETR) and outperforms all CNN-based models in terms of accuracy. In the Wireframe dataset, DT-LSD achieves 71.7 for$sAP^{10}$and 73.9 for$sAP^{15}$; while 33.2 for$sAP^{10}$and 35.1 for$sAP^{15}$in the YorkUrban dataset. Code available at: https://github.com/SebastianJanampa/DT-LSD.

The majority of biomedical studies use limited datasets that may not generalize over large heterogeneous datasets that have been collected over several decades. The current paper develops and validates several multimodal models that can predict 1-year mortality based on a massive clinical dataset. Our focus on predicting 1-year mortality can provide a sense of urgency to the patients. Using the largest dataset of its kind, the paper considers the development and validation of multimodal models based on 25,137,015 videos associated with 699,822 echocardiography studies from 316,125 patients, and 2,922,990 8-lead electrocardiogram (ECG) traces from 631,353 patients. Our models allow us to assess the contribution of individual factors and modalities to the overall risk. Our approach allows us to develop extremely low-parameter models that use optimized feature selection based on feature importance. Based on available clinical information, we construct a family of models that are made available in the DISIML package. Overall, performance ranges from an AUC of 0.72 with just ten parameters to an AUC of 0.89 with under 105 k for the full multimodal model. The proposed approach represents a modular neural network framework that can provide insights into global risk trends and guide therapies for reducing mortality risk.

The papers in this special section focus on large-scale medical imaging and video analytics for clinical decision support systems. Biomedical images and videos are ubiquitous and overwhelming in volume, amounting to a database that can be measured in zettabytes.With increased access to open image and video datasets and the recent development of effective image and video analysis systems, there is a unique opportunity for the development of artificial intelligence (AI) systems that can be trained and tested on large-scale biomedical image and video databases. The special issue summarizes emerging methods associated with the development of computer-aided diagnostic systems. More specifically, the special issue discusses the development of methods for dealing with small or large or creating new datasets, biomedical image segmentation, and image classification. The development of new dataset methods allows us to develop methods for specific diseases, employ meta-learning for training on small datasets, or develop methods for reducing larger video datasets. Biomedical image segmentation is a primary focus of the special issue.

Unsupervised latent variable models-blind source separation (BSS) especially-enjoy a strong reputation for their interpretability. But they seldom combine the rich diversity of information available in multiple datasets, even though multidatasets yield insightful joint solutions otherwise unavailable in isolation. We present a direct, principled approach to multidataset combination that takes advantage of multidimensional subspace structures. In turn, we extend BSS models to capture the underlying modes of shared and unique variability across and within datasets. Our approach leverages joint information from heterogeneous datasets in a flexible and synergistic fashion. We call this method multidataset independent subspace analysis (MISA). Methodological innovations exploiting the Kotz distribution for subspace modeling, in conjunction with a novel combinatorial optimization for evasion of local minima, enable MISA to produce a robust generalization of independent component analysis (ICA), independent vector analysis (IVA), and independent subspace analysis (ISA) in a single unified model. We highlight the utility of MISA for multimodal information fusion, including sample-poor regimes ( N = 600 ) and low signal-to-noise ratio, promoting novel applications in both unimodal and multimodal brain imaging data.

The paper presents the results from a multi-year effort to develop and validate image processing methods for selecting the best physical models based on solar image observations. The approach consists of selecting the physical models based on their agreement with coronal holes extracted from the images. Ultimately, the goal is to use physical models to predict geomagnetic storms. We decompose the problem into three subproblems: (i) coronal hole segmentation based on physical constraints, (ii) matching clusters of coronal holes between different maps, and (iii) physical map classification. For segmenting coronal holes, we develop a multi-modal method that uses segmentation maps from three different methods to initialize a level-set method that evolves the initial coronal hole segmentation to the magnetic boundary. Then, we introduce a new method based on Linear Programming for matching clusters of coronal holes. The final matching is then performed using Random Forests. The methods were carefully validated using consensus maps derived from multiple readers, manual clustering, manual map classification, and method validation for 50 maps. The proposed multi-modal segmentation method significantly outperformed SegNet, U-net, Henney-Harvey, and FCN by providing accurate boundary detection. Overall, the method gave a 95.5% map classification accuracy.

Precision medicine promises better healthcare delivery by improving clinical practice. Using evidence-based substratification of patients, the objective is to achieve better prognosis, diagnosis, and treatment that will transform existing clinical pathways toward optimizing care for the specific needs of each patient. The wealth of today's healthcare data, often characterized as big data, provides invaluable resources toward new knowledge discovery that has the potential to advance precision medicine. The latter requires interdisciplinary efforts that will capitalize the information, know-how, and medical data of newly formed groups fusing different backgrounds and expertise. The objective of this paper is to provide insights with respect to the state-of-the-art research in precision medicine. More specifically, our goal is to highlight the fundamental challenges in emerging fields of radiomics and radiogenomics by reviewing the case studies of Cancer and Alzheimer's disease, describe the computational challenges from a big data analytics perspective, and discuss standardization and open data initiatives that will facilitate the adoption of precision medicine methods and practices.

The Discrete Periodic Radon Transform (DPRT) has many important applications in reconstructing images from their projections and has recently been used in fast and scalable architectures for computing 2D convolutions. Unfortunately, the direct computation of the DPRT involves O(N3) additions and memory accesses that can be very costly in single-core architectures. The current paper presents new and efficient algorithms for computing the DPRT and its inverse on multi-core CPUs and GPUs. The results are compared against specialized hardware implementations (FPGAs/ASICs). The results provide significant evidence of the success of the new algorithms. On an 8-core CPU (Intel Xeon), with support for two threads per core, FastDirDPRT and FastDirInvDPRT achieve a speedup of approximately 10× (up to 12.83×) over the single-core CPU implementation. On a 2048-core GPU (GTX 980), FastRayDPRT and FastRayInvDPRT achieve speedups in the range of 526 (for 127 × 127) to 873 (for 1021 × 1021), which approximate ideal speedups of what can be achieved. The DPRT can be computed exactly and in real-time (30 frames per second) for 1471 × 1471 images using FastRayDPRT on the GPU. Furthermore, the GPU algorithms approximate the performance of an efficient FPGA implementation using 2N parallel cores at 100MHz.

The wider adoption of mobile Health video communication systems in standard clinical practice requires real-time control to provide for adequate levels of clinical video quality to support reliable diagnosis. The latter can only be achieved with real-time adaptation to time-varying wireless networks' state to guarantee clinically acceptable performance throughout the streaming session, while conforming to device capabilities for supporting real-time encoding. We propose an adaptive video encoding framework based on multi-objective optimization that jointly maximizes the encoded video's quality and encoding rate (in frames per second) while minimizing bitrate demands. For this purpose, we construct a dense encoding space and use linear regression to estimate forward prediction models for quality, bitrate, and computational complexity. The prediction models are then used in an adaptive control framework that can fine-tune video encoding based on real-time constraints. We validate the system using a leave-one-out algorithm applied to ten ultrasound videos of the common carotid artery. The prediction models can estimate structural similarity quality with a median accuracy error of less than 1%, bitrate demands with deviation error of 10% or less, and encoding frame rate within a 6% margin. Real-time adaptation at a group of pictures level is demonstrated using the high efficiency video coding standard. The effectiveness of the proposed framework compared to static, nonadaptive approaches is demonstrated for different modes of operation, achieving significant quality gains, bitrate demands reductions, and performance improvements, in real-life scenarios imposing time-varying constraints. Our approach is generic and should be applicable to other medical video modalities with different applications.

Through this study we are presenting the initial steps towards a real time motion analysis system to predict the stability of carotid bifurcation plaques. The analysis is performed on B-mode video loops. Loops are analyzed in order to follow systole and diastole sections of the cardiac cycle and trace the motion of plaques during these periods. We had created a system that applies Farnebacks optical flow estimation method in order to estimate the flow between consecutive frames or frames at a predefined interval. Over each pair of video frames we measure velocities, orientation and magnitude of movement. The goal is to identify if a plaque has movement spread to different angles or at nearby angles. This can help us identify discordant or concordant movement. In order to verify our system we had created a set of simulated videos that have structures moving in a similar way as done in a cardiac cycle and videos that move and appear as an atherosclerotic artery. Following these tests the system has been tested and results are presented on two carotid plaques videos classified visually as having concordant and discordant plaque movement.

We present a framework for adaptive video encoding based on video content. The basic idea is to analyze the video to determine camera activity (tracking, stationary, or zooming) and then associate each activity with adaptive video quality constraints. We demonstrate our approach on the UT LIVE video quality assessment database that effective camera activity detection and classification is possible based on the motion vectors and the number of prediction units (PU) extracted using x265 HEVC encoding standard. In our results, by applying leave-one-out validation, we get an 79% correct classification rate using kNN binary classifier for the video segments.

In this paper, we summarize some of the lessons learned from the Advancing Out-of-School Learning in Mathematics and Engineering (AOLME) project. The AOLME project uses an integrated curriculum that relies on the use of basic concepts from middle-school mathematics to teach the foundations of image and video representations. The middle-school students, mostly from underrepresented groups, learn how to program their own video representations using Python libraries running on the Raspberry Pi. Overall, we have found that the students enjoy participating in the project.

The manuscript describes fast and scalable architectures and associated algorithms for computing convolutions and cross-correlations. The basic idea is to map 2D convolutions and cross-correlations to a collection of 1D convolutions and cross-correlations in the transform domain. This is accomplished through the use of the discrete periodic radon transform for general kernels and the use of singular value decomposition -LU decompositions for low-rank kernels. The approach uses scalable architectures that can be fitted into modern FPGA and Zynq-SOC devices. Based on different types of available resources, for P × P blocks, 2D convolutions and cross-correlations can be computed in just O(P) clock cycles up to O(P2) clock cycles. Thus, there is a trade-off between performance and required numbers and types of resources. We provide implementations of the proposed architectures using modern programmable devices (Virtex-7 and Zynq-SOC). Based on the amounts and types of required resources, we show that the proposed approaches significantly outperform current methods.

The discrete periodic radon transform (DPRT) has extensively been used in applications that involve image reconstructions from projections. Beyond classic applications, the DPRT can also be used to compute fast convolutions that avoids the use of floating-point arithmetic associated with the use of the fast Fourier transform. Unfortunately, the use of the DPRT has been limited by the need to compute a large number of additions and the need for a large number of memory accesses. This paper introduces a fast and scalable approach for computing the forward and inverse DPRT that is based on the use of: a parallel array of fixed-point adder trees; circular shift registers to remove the need for accessing external memory components when selecting the input data for the adder trees; an image block-based approach to DPRT computation that can fit the proposed architecture to available resources; and fast transpositions that are computed in one or a few clock cycles that do not depend on the size of the input image. As a result, for an N × N image (N prime), the proposed approach can compute up to N(2) additions per clock cycle. Compared with the previous approaches, the scalable approach provides the fastest known implementations for different amounts of computational resources. For example, for a 251×251 image, for approximately 25% fewer flip-flops than required for a systolic implementation, we have that the scalable DPRT is computed 36 times faster. For the fastest case, we introduce optimized just 2N + ⌈log(2) N⌉ + 1 and 2N + 3 ⌈log(2) N⌉ + B + 2 cycles, architectures that can compute the DPRT and its inverse in respectively, where B is the number of bits used to represent each input pixel. On the other hand, the scalable DPRT approach requires more 1-b additions than for the systolic implementation and provides a tradeoff between speed and additional 1-b additions. All of the proposed DPRT architectures were implemented in VHSIC Hardware Description Language (VHDL) and validated using an Field-Programmable Gate Array (FPGA) implementation.

Field-programmable wiring systems refer to methods and hardware that can maintain the interconnection of components of different types. Generally, field-programmable wiring systems support the use of multidomain fabrics that can be used to route analog, power, digital signals, optical, microwave signals, etc. This paper reviews fundamental concepts associated with the practical implementation of field-programmable wiring systems. The paper also provides different implementation examples and discusses a list of challenges and recommendations for future work in this area.

Decision tree classification (DTC) is a widely used technique in data mining algorithms known for its high accuracy in forecasting. As technology has progressed and available storage capacity in modern computers increased, the amount of data available to be processed has also increased substantially, resulting in much slower induction and classification times. Many parallel implementations of DTC algorithms have already addressed the issues of reliability and accuracy in the induction process. In the classification process, larger amounts of data require proportionately more execution time, thus hindering the performance of legacy systems. We have devised a pipelined architecture for the implementation of axis parallel binary DTC that dramatically improves the execution time of the algorithm while consuming minimal resources in terms of area. Scalability is achieved when connected to a high-speed communication unit capable of performing data transfers at a rate similar to that of the DTC engine. We propose a hardware accelerated solution composed of parallel processing nodes capable of independently processing data from a streaming source. Each engine processes the data in a pipelined fashion to use resources more efficiently and increase the achievable throughput. The results show that this system is 3.5 times faster than the existing hardware implementation of classification.

The paper presents the development of a computer-aided diagnostic (CAD) system for the early detection of endometrial cancer. The proposed CAD system supports reproducibility through texture feature standardization, standardized multifeature selection, and provides physicians with comparative distributions of the extracted texture features. The CAD system was validated using 516 regions of interest (ROIs) extracted from 52 subjects. The ROIs were equally distributed among normal and abnormal cases. To support reproducibility, the RGB images were first gamma corrected and then converted into HSV and YCrCb. From each channel of the gamma-corrected YCrCb, HSV, and RGB color systems, we extracted the following texture features: 1) statistical features (SFs), 2) spatial gray-level dependence matrices (SGLDM), and 3) gray-level difference statistics (GLDS). The texture features were then used as inputs with support vector machines (SVMs) and the probabilistic neural network (PNN) classifiers. After accounting for multiple comparisons, texture features extracted from abnormal ROIs were found to be significantly different than texture features extracted from normal ROIs. Compared to texture features extracted from normal ROIs, abnormal ROIs were characterized by lower image intensity, while variance, entropy, and contrast gave higher values. In terms of ROI classification, the best results were achieved by using SF and GLDS features with an SVM classifier. For this combination, the proposed CAD system achieved an 81% correct classification rate.

The recent emergence of the high-efficiency video coding (HEVC) standard promises to deliver significant bitrate savings over current and prior video compression standards, while also supporting higher resolutions that can meet the clinical acquisition spatiotemporal settings. The effective application of HEVC to medical ultrasound necessitates a careful evaluation of strict clinical criteria that guarantee that clinical quality will not be sacrificed in the compression process. Furthermore, the potential use of despeckle filtering prior to compression provides for the possibility of significant additional bitrate savings that have not been previously considered. This paper provides a thorough comparison of the use of MPEG-2, H.263, MPEG-4, H.264/AVC, and HEVC for compressing atherosclerotic plaque ultrasound videos. For the comparisons, we use both subjective and objective criteria based on plaque structure and motion. For comparable clinical video quality, experimental evaluation on ten videos demonstrates that HEVC reduces bitrate requirements by as much as 33.2% compared to H.264/AVC and up to 71% compared to MPEG-2. The use of despeckle filtering prior to compression is also investigated as a method that can reduce bitrate requirements through the removal of higher frequency components without sacrificing clinical quality. Based on the use of three despeckle filtering methods with both H.264/AVC and HEVC, we find that prior filtering can yield additional significant bitrate savings. The best performing despeckle filter (DsFlsmv) achieves bitrate savings of 43.6% and 39.2% compared to standard nonfiltered HEVC and H.264/AVC encoding, respectively.

There is strong interest in the development of dynamically reconfigurable systems that can meet real-time constraints on energy, performance, and accuracy. The generation of real-time constraints will significantly expand the applicability of dynamically reconfigurable systems to new domains, such as digital video processing. We develop a dynamically reconfigurable 2D FIR filtering system that can meet real-time constraints in energy, performance, and accuracy (EPA). The real-time constraints are automatically generated based on user input, image types associated with video communications, and video content. We first generate a set of Pareto-optimal realizations, described by their EPA values and associated 2D FIR hardware description bitstreams. Dynamic management is then achieved by selecting Pareto-optimal realizations that meet the automatically generated time-varying EPA constraints. We validate our approach using three different 2D Gaussian filters. Filter realizations are evaluated in terms of the required energy per frame, accuracy of the resulting image, and performance in frames per second. We demonstrate dynamic EPA management by applying a Difference of Gaussians (DOG) filter to standard video sequences. For video frame sizes that are equal to or larger than the VGA resolution, compared to a static implementation, our dynamic system provides significant reduction in the total energy consumption (>30%).

The Discrete Periodic Radon Transform (DPRT) has many important applications in image processing that are associated with reconstructing objects from projections (e.g., computed tomography [1]) or image restoration (e.g., [2]). Thus, there is strong interest in the development of fast algorithms and architectures for computing the DPRT. This paper introduces a scalable hardware architecture and associated algorithm for computing the DPRT for prime-sized images. For square images of size N × N, N prime, the DPRT requires N2(N - 1) additions for calculating image projections along a minimal number of prime directions. The proposed approach can compute the DPRT in [N/2h] N + 2N + h clock cycles, h = 1, ..., [log2N], where h is a scaling factor that is used to control the required hardware resources that are needed to implement the fast DPRT. Compared to previous approaches, a fundamental contribution of the proposed architecture is that it allows effective implementations based on different constraints on the resources.

Pathologies that occur on or near the fovea, such as clinically significant macular edema (CSME), represent high risk for vision loss. The presence of exudates, lipid residues of serous leakage from damaged capillaries, has been associated with CSME, in particular if they are located one optic disc-diameter away from the fovea. In this paper, we present an automatic system to detect exudates in the macula. Our approach uses optimal thresholding of instantaneous amplitude (IA) components that are extracted from multiple frequency scales to generate candidate exudate regions. For each candidate region, we extract color, shape, and texture features that are used for classification. Classification is performed using partial least squares (PLS). We tested the performance of the system on two different databases of 652 and 400 images. The system achieved an area under the receiver operator characteristic curve (AUC) of 0.96 for the combination of both databases and an AUC of 0.97 for each of them when they were evaluated independently.

Quantitative color tissue analysis in endoscopy examinations requires color standardization procedures to be applied, so as to enable compatibility among computer aided diagnosis application from different endoscopy labs. The objective of this study was to examine the usefulness of different color correction algorithms (thus facilitating color standardization), evaluated on four different endoscopy cameras. The following five color correction algorithms were investigated: two gamma correction based algorithms (the classical and a modified one), and three (2nd, 3rd, and 4thorder) polynomial based correction algorithms. The above algorithms were applied to four different endoscopy cameras: (a) Circon, (b) Karl-Stortz, (c) Olympus, and (d) Snowden-Pencer. The color correction algorithms and the endoscopic cameras evaluation, was carried out using the testing color palette (24 colors of known digital values) provided by the Edmund Industrial Optics Company. In summary, we have that: (a) the modified gamma correction algorithm gave significantly smaller mean square error compared to the other four algorithms, and (b) the smallest mean square error was obtained for the Circon camera. Future work will focus on evaluating the proposed color correction algorithm in different endoscopy clinics and compare their tissue characterization results.

Breasts are composed of a mixture of fibrous and glandular tissue as well as adipose tissue and breast density describes the prevalence of fibroglandular tissue as it appears on a mammogram. Over the past few years, evaluation and reporting of breast density as it appears on mammograms has received a lot of attention because it impacts one's risk of developing breast cancer but also the capability of detecting breast cancer on mammograms. In addition, mammography fails in the identification of breast cancer in almost half of the women with dense breasts. Different image analysis methods have been investigated for automatic breast density classification. The presented method investigates the use of AmplitudeModulation Frequency-Modulation (AM-FM) multi-scale feature sets for characterization of breast density as the first step in the development of a density specific Computer Aided Detection System. AM-FM decompositions use different scales and bandpass filters to extract the instantaneous frequencies (IF), instantaneous amplitude (IA) and instantaneous phase (IP) components from an image. Normalized histograms of the maximum IA across all frequencies and scales are used to model the different breast density classes. Classification of a new mammogram into one of the breast density classes is achieved using the k-nearest neighbor method with k =5 and the euclidean distance metric. The method is evaluated on the Medical Image Analysis Society (MIAS) mammographic database and the results are presented. The presented method allows breast density classification accuracy reaching over 84%. Future work will involve a new AM-FM methodology approach based on adaptive filterbank design and performance index decision.

We introduce a dynamically reconfigurable framework for implementing single-pixel operations. The system relies on a multiobjective optimization scheme that generates Pareto-optimal realizations in the power/energy-performance-accuracy (PPA/EPA) spaces. The Pareto-optimal realizations and their PPA/EPA values are stored in DDR-SDRAM and can be chosen dynamically to meet time-varying constraints. Results are shown in terms of power, accuracy (peak signal-to-noise ratio) of the resulting image, and performance in frames per second. Dynamic PPA/EPA management is implemented using dynamic partial reconfiguration and dynamic frequency control.

In this study, we describe an effective video communication framework for the wireless transmission of H.264/AVC medical ultrasound video over mobile WiMAX networks. Medical ultrasound video is encoded using diagnostically-driven, error resilient encoding, where quantization levels are varied as a function of the diagnostic significance of each image region. We demonstrate how our proposed system allows for the transmission of high-resolution clinical video that is encoded at the clinical acquisition resolution and can then be decoded with low-delay. To validate performance, we perform OPNET simulations of mobile WiMAX Medium Access Control (MAC) and Physical (PHY) layers characteristics that include service prioritization classes, different modulation and coding schemes, fading channels conditions, and mobility. We encode the medical ultrasound videos at the 4CIF (704 × 576) resolution that can accommodate clinical acquisition that is typically performed at lower resolutions. Video quality assessment is based on both clinical (subjective) and objective evaluations.

The use of multiscale AM-FM analysis systems has been recently demonstrated in a variety of applications in medical image analysis. In all of these applications, a fixed filter-bank is used as a preprocessing step for estimating different AM-FM components from different scales. In this paper, for the first time, we introduce the use of an adaptive, multiscale AM-FM approach that searches for the optimal filter-bank specification for use in image classification. We demonstrate an example application in hysteroscopy imaging, for identification of gynaecological cancer, where the optimal filter-bank turns out to be circularly symmetric.

In minimally invasive surgery, the surgeon and the assistant rely on a single laparoscopic video view for performing different clinical roles. The assistant is tasked with manipulating the camera view so as to maintain a global, panoramic view of the operation. The surgeon needs to remain focused on the operation, requiring a detailed close-up view. We use the term role-specific video imaging to describe the need to provide separate views for the assistant and the surgeon. In this paper we introduce role-specific video imaging for laparosopic surgery. The system is designed to be configurable in the sense that imaging parameters and algorithms can be adjusted in real-time so as to meet the specific needs that arise. The system was evaluated on 4 cases by two surgeons on a Linux-based 3.2.0 Kernel, with 4GB RAM, and Intel 3.4GHz I7 (2nd generation) microprocessor. Clinical evaluation of the different configuration modes has helped us determine that high-quality role-specific imaging can be achieved for zooming factors that are larger or equal to 2×2 with bilinear interpolation, while maintaining 30 frame per seconds for the panoramic and close-up views. In future work, in order to minimize interaction with the surgical team, the system will be upgraded to incorporate tracking of the operating instrument during surgery.

Video loops of B-mode ultrasound images of 35 carotid bifurcation plaques were obtained (4 symptomatic and 31 asymptomatic) from patients with carotid bifurcation atherosclerosis. Video loops were classified visually as showing concordant (n=22) or discordant motion (n=13). Concordant plaques were characterized by uniform orientation of motion throughout the cardiac cycle. Discordant plaques exhibited significant spread in motion orientation at different parts of the cardiac cycle, especially at systole. We developed a real-time motion analysis system that applies Farneback's method to estimate velocities between consecutive video frames. For our purposes, we allow a 100msec time interval between the video frames used in the analysis. This approach allows us to analyze significant motions associated with a larger time interval. Over each video frame, we measure the spread of the motion orientation around the dominant orientation. For each video, we look at the spreads of the motion orientations for different motion magnitudes. Using these motion-spread measurements, we can quantify discordant movement. The sum of maximum fan widths for the median pixel motions 5 to 3 (SMFW5to3) had a median value of 100 degrees and interquartile range (IQR) of (80, 110) degrees for the concordant plaques and 270, (230, 430) for the discordant plaques (P < 0.001). Thus, we have a new tool to differentiate between concordant and discordant plaques.

We present a dynamic framework for 2D complex filter implementation that is based on a multi-objective optimization scheme that generates Pareto-optimal realizations from the Energy-Performance-Accuracy (EPA) space. The EPA space is created by evaluating the 2D complex filter realizations in terms of their required energy, accuracy, and performance. Dynamic EPA management, carried out via Dynamic Partial Reconfiguration (DPR) and Dynamic Frequency Control, then consists on selecting Pareto-optimal realizations that meet time-varying EPA requirements. We demonstrate dynamic EPA management by applying a complex filter to a standard video sequence.

We propose a dynamically reconfigurable DCT architecture system that can be used to optimize performance objectives while meeting real-time constraints on power, image quality, and bitrate. The proposed system can be dynamically reconfigured between 4 different modes: (i) minimum power mode, (ii) minimum bitrate mode, (iii) maximum image quality mode, and (iv) typical mode. The proposed system relies on the use of efficient DCT implementations that are parameterized by the word-length of the DCT transform coefficients and the use of different quantization quality factors. Optimal DCT architectures and quality factors are pre-computed on a training dataset. The proposed system is validated on the LIVE database using leave-one-out. From the results, it is clear that real-time constraints can be successfully met for the majority of the test images while optimizing for the 4 modes of operation.

In this paper we present a robust and improved system for diabetic retinopathy (DR) screening. The goal of the system is to automatically screen out digital fundus photographs of diabetic patients who do not present signs of DR. This work is motivated by the large amount of diabetics in the world who do not receive their recommended eye exams, leading to widespread blindness as a complication of diabetes. The system is based on multiscale amplitude-modulation frequency-modulation (AM-FM) methods for feature extraction, and uses supervised and unsupervised methods to produce its final output, namely, a normal or abnormal grade. The most time-consuming processing routines of the system are implemented in C using a compute unified device architecture (CUDA) to produce results in real-time. The system was tested using 776 images from 388 patients (one macula-centered image from each eye). During the training phase of the system, the data was divided in 70% for training and 30% for testing. The system was tested using 20 random training/testing distributions, obtaining an average sensitivity of 89% and specificity of 59%. Analysis of sight-threatening conditions resulted in a sensitivity of 98% for these types of cases.

This paper presents new methods for phase and magnitude interpolation and demonstrates their usefulness in reconstructing images from a limited number of frequency samples. A collection of multiscale frequency domain sampling geometries are developed based on the partition of the spectrum into low-, medium-, and high-frequency blocks. A nonstationary statistical approach is introduced that is based on adaptively selecting the best stochastic model in each frequency block. To develop effective models, the magnitude spectrum is preprocessed using a logarithmic transformation. Phase interpolation requires preprocessing by an appropriate phase unwrapping method. The new stochastic interpolation method is compared against cubic spline, bilinear, and nearest neighbor interpolation methods. Image reconstruction results are presented for sampling rates that retain 6.01% to 28.91% of the 2-D fast Fourier transform (FFT) samples. Image interpolation methods are compared based on the peak signal-to-noise ratio and the mean structural similarity index for satellite images of rural, natural, and urban images. The results indicate that the stochastic (Kriging) interpolation approach provides the best rural image reconstructions using just 6.01% of the 2-D FFT samples. Bilinear interpolation also gave excellent reconstructions for natural and urban images. For natural and urban images, stochastic interpolation gave the best magnitude-only interpolation results.

Carotid plaques have been associated with ipsilateral neurological symptoms. High-resolution ultrasound can provide information not only on the degree of carotid artery stenosis but also on the characteristics of the arterial wall including the size and consistency of atherosclerotic plaques. The aim of this study is to determine whether the addition of ultrasonic plaque texture features to clinical features in patients with asymptomatic internal carotid artery stenosis (ACS) improves the ability to identify plaques that will produce stroke. 1121 patients with ACS have been scanned with ultrasound and followed for a mean of 4 years. It is shown that the combination of texture features based on second-order statistics spatial gray level dependence matrices (SGLDM) and clinical factors improves stroke prediction (by correctly predicting 89 out of the 108 cases that were symptomatic). Here, the best classification results of 77 ±1.8% were obtained from the use of the SGLDM texture features with support vector machine classifiers. The combination of morphological features with clinical features gave slightly worse classification results of 76 ±2.6% . These findings need to be further validated in additional prospective studies.

The optic disk (OD) center and margin are typically requisite landmarks in establishing a frame of reference for classifying retinal and optic nerve pathology. Reliable and efficient OD localization and segmentation are important tasks in automatic eye disease screening. This paper presents a new, fast, and fully automatic OD localization and segmentation algorithm developed for retinal disease screening. First, OD location candidates are identified using template matching. The template is designed to adapt to different image resolutions. Then, vessel characteristics (patterns) on the OD are used to determine OD location. Initialized by the detected OD center and estimated OD radius, a fast, hybrid level-set model, which combines region and local gradient information, is applied to the segmentation of the disk boundary. Morphological filtering is used to remove blood vessels and bright regions other than the OD that affect segmentation in the peripapillary region. Optimization of the model parameters and their effect on the model performance are considered. Evaluation was based on 1200 images from the publicly available MESSIDOR database. The OD location methodology succeeded in 1189 out of 1200 images (99% success). The average mean absolute distance between the segmented boundary and the reference standard is 10% of the estimated OD radius for all image sizes. Its efficiency, robustness, and accuracy make the OD localization and segmentation scheme described herein suitable for automatic retinal disease screening in a variety of clinical settings.

Digital video processing requires significant hardware resources to achieve acceptable performance. Digital video processing based on dynamic partial reconfiguration (DPR) allows the designers to control resources based on energy, performance, and accuracy considerations. In this paper, we present a dynamically reconfigurable implementation of a 2D FIR filter where the number of coefficients and coefficients values can be varied to control energy, performance, and precision requirements. We also present a high-performance GPU implementation to help understand the trade-offs between these two technologies. Results using a standard example of 2D Difference of Gaussians (DOG) filter indicate that the DPR implementation can deliver real-time performance with energy per frame consumption that is an order of magnitude less than the GPU. On the other hand, at significantly higher energy consumption levels, the GPU implementation can deliver very high performance.

We present an improved and optimized formulation for the estimation of the multiscale amplitude-modulation frequency-modulation (AM-FM) estimates when (i) non-separable filters are used and (ii) the variable spacing, local linear phase method is used. Also, we introduce the use of multiscale directional filterbanks for the feature extraction of images. Recently, AM-FM methods have shown promising results in a variety of medical image analysis applications. The 2D optimized AM-FM demodulation described here presents advantages for feature extraction at different frequency scales and orientations that can be used to detect different patterns, directions, or structures in an image. We test the new formulation using a Gaussian amplitude-modulated Quadratic frequency-modulated synthetic image and natural images. The results show that the optimized estimation produces better results, up to 4.9 times for the IF estimation and in 3 orders of magnitude for the IA estimation, for noise-free signals compared to the state-of-the-art methods.

The intima-media thickness (IMT) of the common carotid artery (CCA) is widely used as an early indicator of cardiovascular disease (CVD). Clinically, there is strong interest in identifying how the composition and texture of the media layer (ML) can be associated with the risk of stroke. In this study, we use 2-D amplitude-modulation frequency-modulation (AM-FM) analysis of the intima-media complex (IMC), the ML, and intima layer (IL) of the CCA to detect texture changes as a function of age and sex. The study was performed on 100 ultrasound images acquired from asymptomatic subjects at risk of atherosclerosis. To investigate texture variations associated with age, we separated them into three age groups: 1) patients younger than 50; 2) patients aged between 50 and 60 years old; and 3) patients over 60 years old. We also separated the patients by sex. The IMC, ML, and IL were segmented manually by a neurovascular expert and also by a snake-based segmentation system. To reject strong edge artifacts, we prefilter with an AM-FM filterbank that is centered along the horizontal frequency axis (parallel to the long axis of the IMC, ML, and IL), while removing the low-pass filter estimates and frequency bands with large, vertical frequency components. To investigate significant texture changes, we extract the instantaneous amplitude (IA) and the magnitude of the instantaneous frequency (IF) over each layer component, for low-, medium-, and high-frequency AM-FM components. We detected significant texture differences between the higher risk age group of >60 years versus the lower risk age group of <50 and the 50-60 group. In particular, between the <50 and >60 groups, we found significant differences in the medium-scale IA extracted from the IMC. Between the >60 and the 50-60 groups, we found significant texture changes in the low-scale IA and high-scale IF magnitude extracted from the IMC, and the low-scale IA extracted from the IL. Also, we noted that the IA for the ML showed significant differences between males and females for all age groups. The AM--FM features provide complimentary information to classical texture analysis features like the gray-scale median, contrast, and coarseness. These findings provide evidence that AM--FM texture features can be associated with the progression of cardiovascular risk for disease and the risk of stroke with age. However, a larger scale study is needed to establish the application in clinical practice.

This study introduces the use of multiscale amplitude modulation-frequency modulation (AM-FM) texture analysis of multiple sclerosis (MS) using magnetic resonance (MR) images from brain. Clinically, there is interest in identifying potential associations between lesion texture and disease progression, and in relating texture features with relevant clinical indexes, such as the expanded disability status scale (EDSS). This longitudinal study explores the application of 2-D AM-FM analysis of brain white matter MS lesions to quantify and monitor disease load. To this end, MS lesions and normal-appearing white matter (NAWM) from MS patients, as well as normal white matter (NWM) from healthy volunteers, were segmented on transverse T2-weighted images obtained from serial brain MR imaging (MRI) scans (0 and 6-12 months). The instantaneous amplitude (IA), the magnitude of the instantaneous frequency (IF), and the IF angle were extracted from each segmented region at different scales. The findings suggest that AM-FM characteristics succeed in differentiating 1) between NWM and lesions; 2) between NAWM and lesions; and 3) between NWM and NAWM. A support vector machine (SVM) classifier succeeded in differentiating between patients that, two years after the initial MRI scan, acquired an EDSS ≤ 2 from those with EDSS > 2 (correct classification rate = 86%). The best classification results were obtained from including the combination of the low-scale IA and IF magnitude with the medium-scale IA. The AM-FM features provide complementary information to classical texture analysis features like the gray-scale median, contrast, and coarseness. The findings of this study provide evidence that AM-FM features may have a potential role as surrogate markers of lesion load in MS.

We propose a unifying framework for efficient encoding, transmission, and quality assessment of atherosclerotic plaque ultrasound video. The approach is based on a spatially varying encoding scheme, where video-slice quantization parameters are varied as a function of diagnostic significance. Video slices are automatically set based on a segmentation algorithm. They are then encoded using a modified version of H.264/AVC flexible macroblock ordering (FMO) technique that allows variable quality slice encoding and redundant slices (RSs) for resilience over error-prone transmission channels. We evaluate our scheme on a representative collection of ten ultrasound videos of the carotid artery for packet loss rates up to 30%. Extensive simulations incorporating three FMO encoding methods, different quantization parameters, and different packet loss scenarios are investigated. Quality assessment is based on a new clinical rating system that provides independent evaluations of the different parts of the video (subjective). We also use objective video-quality assessment metrics and estimate their correlation to the clinical quality assessment of plaque type. We find that some objective quality assessment measures computed over the plaque video slices gave very good correlations to mean opinion scores (MOSs). Here, MOSs were computed using two medical experts. Experimental results show that the proposed method achieves enhanced performance in noisy environments, while at the same time achieving significant bandwidth demands reductions, providing transmission over 3G (and beyond) wireless networks.

The 20 papers in this special issue were originally presented in the International Special Topic Conference on Information Technology in Biomedicine, held in October 2009, in Larnaka, Cyprus.

We develop new multiscale amplitude-modulation frequency-modulation (AM-FM) demodulation methods for image processing. The approach is based on three basic ideas: (i) AM-FM demodulation using a new multiscale filterbank, (ii) new, accurate methods for instantaneous frequency (IF) estimation, and (iii) multiscale least squares AM-FM reconstructions. In particular, we introduce a variable-spacing local linear phase (VS-LLP) method for improved instantaneous frequency (IF) estimation and compare it to an extended quasilocal method and the quasi-eigen function approximation (QEA). It turns out that the new VS-LLP method is a generalization of the QEA method where we choose the best integer spacing between the samples to adapt as a function of frequency. We also introduce a new quasi-local method (QLM) for IF and IA estimation and discuss some of its advantages and limitations. The new IF estimation methods lead to significantly improved estimates. We present different multiscale decompositions to show that the proposed methods can be used to reconstruct and analyze general images.

Noninvasive ultrasound imaging of carotid plaques allows for the development of plaque-image analysis methods associated with the risk of stroke. This paper presents several plaque-image analysis methods that have been developed over the past years. The paper begins with a review of clinical methods for visual classification that have led to standardized methods for image acquisition, describes methods for image segmentation and denoising, and provides an overview of the several texture-feature extraction and classification methods that have been applied. We provide a summary of emerging trends in 3-D imaging methods and plaque-motion analysis. Finally, we provide a discussion of the emerging trends and future directions in our concluding remarks.

In this paper, we propose the use of multiscale amplitude-modulation-frequency-modulation (AM-FM) methods for discriminating between normal and pathological retinal images. The method presented in this paper is tested using standard images from the early treatment diabetic retinopathy study. We use 120 regions of 40 x 40 pixels containing four types of lesions commonly associated with diabetic retinopathy (DR) and two types of normal retinal regions that were manually selected by a trained analyst. The region types included microaneurysms, exudates, neovascularization on the retina, hemorrhages, normal retinal background, and normal vessels patterns. The cumulative distribution functions of the instantaneous amplitude, the instantaneous frequency magnitude, and the relative instantaneous frequency angle from multiple scales are used as texture feature vectors. We use distance metrics between the extracted feature vectors to measure interstructure similarity. Our results demonstrate a statistical differentiation of normal retinal structures and pathological lesions based on AM-FM features. We further demonstrate our AM-FM methodology by applying it to classification of retinal images from the MESSIDOR database. Overall, the proposed methodology shows significant capability for use in automatic DR screening.

Age-related macular degeneration (AMD) is the most common cause of visual loss in the United States and is a growing public health problem. The presence and severity of AMD in current epidemiological studies is detected by the grading of color stereoscopic fundus photographs. The purpose of this study was to show that a mathematical technique, amplitude-modulation frequency modulation (AM-FM) can be used to generate multi-scale features for classifying pathological structures, such as drusen, on a retinal image. AM-FM features were calculated for N=120 40times40 regions from 5 retinal images presenting with age-related macular degeneration. The results show that with this technique, drusen can be differenced from normal retinal structures by more than three standard deviations using the AM-FM histograms. In addition, by using different color spaces highly accurate classification of structures of the retina is achieved. These results are the first step in the development of an automated AMD grading system.

We describe a dynamically reconfigurable image processing system that reaches real time video processing performances despite reconfiguration time overhead. The system is composed of reconfigurable pixel processing units set to process several pixels in parallel. We present a scheme for optimizing a LUT-based architecture by directly mapping it into the Xilinx FPGA CLB primitives. Internally controlled dynamic partial reconfiguration is to modify the LUT values at run-time without stalling the overall operation. The combination of optimized implementations with CLB primitives and dynamic partial reconfiguration leads to multifunctional, area-efficient, and highperformance realizations of LUT-based pixel processing systems. We present results from a dynamically reconfigurable high-performance LUT-based image/video processing system. Experimental measurements show that the system achieves speeds of 226 Mbps with small resource utilization. The architecture can dynamically reconfigure the image processing operation at each new frame and still reach real-time video processing speeds (640 times 480 graylevel frames). We also evaluate the effect that increasing partial reconfiguration rates have on the system's overall performance.

This paper presents a computer-aided diagnostic (CAD) system for analyzing chest radiographs based on the International Labor Organization (ILO) standards. We introduce an amplitude-modulation frequency-modulation (AM-FM) based methodology by which a computer-based system will extract AM-FM features and detect those with suspected interstitial lung diseases. For classification, we use Partial Least Squares (PLS) using a low number of extracted factors (making the system robust). We consider several different AM-FM classifiers based on extracting features from individual scales as well as a final classifier that combines results from the individuals scales. We validate our methodology on 11 standard images graded according to the ILO standard. For several scales, as well as for the combined classifier that uses information from all scales, we get excellent classification results (area under the receiver operator characteristics curve equal to 1.0) using a limited number of latent PLS factors.

The six papers in this special section focus on the most recent applications of computational intelligent systems in medicine. Some papers accepted for this issue (10 in total) were published earlier by mistake.

The intima-media thickness (IMT) of the common carotid artery (CCA) is widely used as an early indicator of cardiovascular disease (CVD). It was proposed but not thoroughly investigated that the media layer (ML), its composition and texture, may be indicative for identifying the risk of stroke and differentiating between patients of high and low risk. In this study we investigate the usefulness of texture analysis of the ML of the CCA. The study was performed on 100 longitudinal ultrasound images acquired from asymptomatic subjects at risk of atherosclerosis. The images were separated into three different age groups, namely below 50, 50 to 60, and above 60 years old. A total of 61 different texture features were extracted from the intima-media complex (IMC), ML and the intima layer (IL). The IMC and ML were segmented manually by a neurovascular expert and automatically by a snakes segmentation system. It was shown that texture features extracted from the IL, ML and IMC are significantly different (mean, gray scale median (GSM), standard deviation, contrast, difference variance, periodicity) and that some of them can be associated with the increase (difference variance, entropy) or decrease (GSM) of patientpsilas age. It was also shown that the GSM of the ML falls linearly with increasing ML thickness (MLT) and with increasing age. Further research on more subjects is required for estimating other features that may provide information for patients at risk of stroke.

For medical and epidemiologic investigators and caregivers, one powerful functionality yet to be developed is the ability to group retinal images based upon common pathologic appearance. Such a tool would enable advances in evidence-based medicine and would accelerate automated or computer-assisted screening and diagnosis. In this report, we show that current, traditional content based image retrieval methods are insufficient to sort dichotomous images (age-related macular degeneration and Stargardt disease) and then propose novel feature extraction techniques that may improve retrieval performance. Prior to processing of the images, a specialized diffusion method to enhance the contrast, reduce the discontinuity, and eliminate edge artifacts is applied to facilitate segmentation. A robust statistic is applied to find abnormal areas and to differentiate AMD from SD. Two methods of analyzing the subretinal deposits are presented - a granulometry based on area morphology and an AM-FM model. Preliminary data show that the image analysis tools show promise as a useful retrieval tool.

In this paper, we introduce new multiscale AM-FM demodulation algorithms that provide significant improvements in accuracy over previously reported approaches. The improvements are due to the use of new filterbanks based on separable filters supported in just two quadrants. The QEA, robust-QEA and Vakman methods are improved with this new filterbanks. A number of 2-D AM-FM examples are presented, where we observe significant accuracy improvements. For Lena, the mean-square-error for the AM-FM harmonic reconstruction is reduced by 88.31%. Similarly, for a AM-FM synthetic example of sinusoidal phase and Gaussian amplitude, the meansquare-error is reduced by: (i) 70.86% for the reconstruction, (ii) 99.66% for the instantaneous amplitude and (iii) 96.52% for the sinusoidal instantaneous frequency component.

We develop a mathematical framework for quantifying and understanding multidimensional frequency modulations in digital images. We begin with the widely accepted definition of the instantaneous frequency vector (IF) as the gradient of the phase and define the instantaneous frequency gradient tensor (IFGT) as the tensor of component derivatives of the IF vector. Frequency modulation bounds are derived and interpreted in terms of the eigendecomposition of the IFGT. Using the IFGT, we derive the ordinary differential equations (ODEs) that describe image flowlines. We study the diagonalization of the ODEs of multidimensional frequency modulation on the IFGT eigenvector coordinate system and suggest that separable transforms can be computed along these coordinates. We illustrate these new methods of image pattern analysis on textured and fingerprint images. We envision that this work will find value in applications involving the analysis of image textures that are nonstationary yet exhibit local regularity. Examples of such textures abound in nature.

Image registration is a vital step in the processing of multispectral imagery. The accuracy to which imagery collected at multiple wavelengths can be aligned directly affects the resolution of the spectral end products. Automated registration of the multispectral imagery can often be unreliable, particularly between visible and infrared imagery, due to the significant differences in scene reflectance at different wavelengths. This is further complicated by the thermal features that exist at longer wavelengths. We develop new mathematical and computational models for robust image registration. In particular, we develop a frequency-domain model for the mutual-information surface around the optimal parameters and use it to develop a robust gradient ascent algorithm. For a robust performance, we require that the algorithm be initialized close to the optimal registration parameters. As a measure of how close we need to be, we propose the use of the correlation length and provide an efficient algorithm for estimating it. We measure the performance of the proposed algorithm over hundreds of random initializations to demonstrate its robustness on real data. We find that the algorithm should be expected to converge, as long as the registration parameters are initialized to be within the correlation-length distance from the optimum

In the early stages of some retinal diseases, such as glaucoma, loss of retinal activity may be difficult to detect with current clinical instruments. Because current instruments require unattainable levels of patient cooperation, high sensitivity and specificity are difficult to attain. We have devised a new retinal imaging system that detects intrinsic optical signals which reflect functional changes in the retina and that do not require patient cooperation. Measured changes in reflectance in response to the visual stimulus are on the order of 0.1%-1% of the total reflected intensity level, which makes the functional signal difficult to detect by standard methods. The desired functional signal is masked by other physiological signals and by imaging system noise. In this paper, we quantify the limits of independent component analysis (ICA) for detecting the low intensity functional signal and apply ICA to 60 video sequences from experiments using an anesthetized cat whose retina is presented with different patterned stimuli. The results of the analysis show that using ICA, in principle, signal levels of 0.1% can be detected. The study found that in 86% of the animal experiments the patterned stimuli effects on the retina can be detected and extracted.

Multidimensional amplitude-modulation frequency-modulation (AM-FM) models allow us to describe continuous-scale modulations in digital images. AM-FM models have led to a wide range of applications ranging from image and video compression, video image segmentation, to image retrieval in digital libraries. We present new, two-dimensional algorithms that provide significant improvements in both accuracy and speed over previously reported non-parametric approaches. Results are shown for both real and synthetic images.

In this paper, we describe a freehand, three-dimensional ultrasound imaging system. The system uses an electromagnetic position and orientation measurement device to capture two-dimensional ultrasound images at arbitrary planar orientations in space. For robust performance, we use a novel electromagnetic interference detection algorithm that can be used to estimate the probability density function of position and orientation measurement errors. Another important contribution of the proposed system is its ability to reconstruct from multiple standard views. The multi-view reconstruction procedure results in significant reduction in reconstruction error. The system uses object-based 3D volume registration, allowing for arbitrary rigid object movements in inter-view acquisition. The proposed system has been validated on simulated data and a physical, 3D ultrasound calibration phantom. Quantitative experimental results demonstrate the effectiveness of the 3D registration system, and a significant reduction in the mean-squared error via the use of the proposed multi-view reconstruction method.

Most image and video compression algorithms that have been proposed to improve picture quality relative to compression efficiency have either been designed based on objective criteria such as signal-to-noise-ratio (SNR) or have been evaluated, post-design, against competing methods using an objective sample measure. However, existing quantitative design criteria and numerical measurements of image and video quality both fail to adequately capture those attributes deemed important by the human visual system, except, perhaps, at very low error rates. We present a framework for assessing the quality of and determining the efficiency of foveated and compressed images and video streams. Image foveation is a process of nonuniform sampling that accords with the acquisition of visual information at the human retina. Foveated image/video compression algorithms seek to exploit this reduction of sensed information by nonuniformly reducing the resolution of the visual data. We develop unique algorithms for assessing the quality of foveated image/video data using a model of human visual response. We demonstrate these concepts on foveated, compressed video streams using modified (foveated) versions of H.263 that are standard-compliant. We rind that quality vs. compression is enhanced considerably by the foveation approach.

A method for scanning and displaying chest radiographs (X-rays) is presented. The new method treats the scanning as independent of the image display, allowing for maximum information content to be captured during the scanning process, and then for this information to be optimally displayed using a new function for maximizing the contrast variation throughout the image. The quality of the digitized X-ray images was compared against the original X-ray films and was found to be of comparable visualization quality. The rib parenchyma is then segmented by an active shape model.

The objective of this work is to develop a system that will facilitate the automated characterization of ultrasonic imaging carotid plaques for the identification of individuals with asymptomatic carotid stenosis at risk of stroke. A total of 166 images were collected which were classified into: symptomatic because of ipsilateral hemispheric symptoms, or asymptomatic because they were not connected with ipsilateral hemispheric events. Ten different texture feature sets were extracted: first order statistics, spatial gray level dependence matrices, gray level difference statistics, neighbourhood gray tone difference matrix, statistical feature matrix, Laws texture energy measures, fractal dimension texture analysis, Fourier power spectrum and shape parameters. A modular neural network classifier was developed composed of self-organizing map (SOM) classifiers, achieving an overall diagnostic yield of 76.4%. The results of this work show that it is possible to identify a group of patients at risk of stroke based on texture features.

New algorithms for computing the discrete Fourier transform (DFT) spectra along different directions are derived and implemented. For computing the DFT spectrum along any given direction (containing N DFT frequencies), a new algorithm is presented that requires N(N-1) additions and a single 1-D FFT. As expected, for a single direction, the directional FFT algorithm is significantly faster than standard 2-D FFT algorithms that compute the entire spectrum (all results are compared against FFTW and FFTPACK). A scalable extension of the unidirectional algorithm for computing the entire DFT spectrum is also derived and implemented. The three most promising features of the new algorithm are that: (i) computation scales nearly linearly with the number of DFT frequencies computed, (ii) the algorithm uses a reduced number of multiplications (yet uses more additions), and (iii) it is more accurate.

For the first time, we explore the application of active contours in the modulation domain by computing snakes on image modulations. As we demonstrate in the examples, such snakes are able to utilize information inherent in the dominant image modulations to acquire and track visually and semantically meaningful structures within the image. We use nonlinear AM-FM image representations to capture regions that are homogeneous in intensity and in texture. A geometric snake approach utilizing a fuzzy classifier is then applied to the image modulations. The combination of AM-FM analysis and the active contour evolution produces an efficacious image partition. As a preliminary demonstration of this novel approach, we apply the modulation domain snakes to the classical texture segmentation problem.

Previously, fovcated video compression algorithms have been proposed which, in certain applications, deliver high-quality video at reduced bit rates by seeking to match the nonuniform sampling of the human retina. We describe such a framework here where foveated video is created by a nonuniform filtering scheme that increases the compressibility of the video stream. We maximize a new foveal visual quality metric. the foveal signal-to-noise ratio (FSNR) to determine the best compression and rate control parameters for a given target bit rate. Specifically, we establish a new optimal rate control algorithm for maximizing the FSNR using a Lagrange multiplier method defined on a curvilinear coordinate system. For optimal rate control, we also develop a piecewise R-D (rate-distortion)/R-Q (rate-quantization) model. A fast algorithm for searching for an optimal Lagrange multiplier lambda* is subsequently presented. For the new models, we show how the reconstructed video quality is affected, where the FSNR is maximized, and demonstrate the coding performance for H.263,+,++/MPEG-4 video coding. For H.263/MPEG video coding, a suboptimal rate control algorithm is developed for fast, high-performance applications. In the simulations, we compare the reconstructed pictures obtained using optimal rate control methods for foveated and normal video. We show that foveated video coding using the suboptimal rate control algorithm delivers excellent performance under 64 kb/s.

The present a novel class of multidimensional orthogonal FM transforms. The analysis suggests a novel signal-adaptive FM transform possessing interesting energy compaction properties. We show that the proposed signal-adaptive FM transform produces point spectra for multidimensional signals with uniformly distributed samples. This suggests that the proposed transform is suitable for energy compaction and subsequent coding of broadband signals and images that locally exhibit significant level diversity. We illustrate these concepts with simulation experiments.

Research on fingerprint classification has primarily focused on finding improved classifiers, image and feature enhancement, and less on the development of novel fingerprint representations. Using an AM-FM representation for each fingerprint, we obtain significant gains in classification performance as compared to the commonly used National Institute of Standards system, for the same classifier.

This paper describes the application of an amplitude modulation-frequency modulation (AM-FM) image representation in segmenting electron micrographs of skeletal muscle for the recognition of: 1) normal sarcomere ultrastructural pattern and 2) abnormal regions that occur in sarcomeres in various myopathies. A total of 26 electron micrographs from different myopathies were used for this study. It is shown that the AM-FM image representation can identify normal repetitive structures and sarcomeres, with a good degree of accuracy. This system can also detect abnormalities in sarcomeres which alter the normal regular pattern, as seen in muscle pathology, with a recognition accuracy of 75%-84% as compared to a human expert.

We segment the structural units of electron microscope muscle images using a novel AM-FM image representation. This novel AM-FM approach is shown to be effective in describing sarcomeres and mitochondrial regions of the electron microscope muscle images.

COPERM is a novel paradigm for energy compaction and signal compression, whose foundation is a simple but powerful idea: any signal can be transformed to resemble a more desirable signal from a class of "target" signals, by means of a suitable permutation of its samples. The approach is well-suited for transform domain energy compaction prior to transform-domain compression of persistent broadband signals. The associated optimal permutation precoders are surprisingly simple, and the permutation precoding overhead can be made modest-resulting in improved overall rate-distortion performance.

Given a set of target bits, video rate control algorithms that use Lagrange multipliers have been generally known as an optimal solution for maximizing the picture quality in the uniform spatial domain. Even if the SNR (signal-to-noise) of a picture is maximized by the rate control scheme, the visual quality can be enhanced using a suitable algorithm for the human visual system. We establish a new optimal rate control algorithm for maximizing the SNRC (signal-to-noise ratio in curvilinear coordinates) using the Lagrange multiplier. In addition, a target bit allocation technique for foveated video is introduced for simplified rate control over MEPG/H.263 video standards.