A novel super-resolution reconstruction (SRR) framework in magnetic resonance imaging (MRI) is proposed. Its purpose is to produce images of both high resolution and high contrast desirable for image-guided minimally invasive brain surgery. The input data are multiple 2-D multislice inversion recovery MRI scans acquired at orientations with regular angular spacing rotated around a common frequency encoding axis. The output is a 3-D volume of isotropic high resolution. The inversion process resembles a localized projection reconstruction problem. Iterative algorithms for reconstruction are based on the projection onto convex sets (POCS) formalism. Results demonstrate resolution enhancement in simulated phantom studies, and ex vivo and in vivo human brain scans, carried out on clinical scanners. A comparison with previously published SRR methods shows favorable characteristics in the proposed approach.

In this paper, we present a data association algorithm for people tracking in a 3D world using multiple cameras. Our approach expands an independent partitioned particle filter with a data association vector. For the association parameter, we propose a proposal function using likelihood functions based on color and distance. This proposed algorithm solves the data association problem without dramatically increasing the computational complexity even in the case of trajectories that cross.

Most video watermarking algorithms embed the watermark in I-frames, but refrain from embedding in P- and B-frames, which are highly compressed by motion compensation. However, P-frames appear more frequently in the compressed video and their watermarking capacity should be exploited, despite the fact that embedding the watermark in P-frames can increase the video bit rate significantly. This paper gives a detailed overview of a common approach for embedding the watermark in I-frames. This common approach is adopted to use P-frames for video watermarking. We show that by limiting the watermark to nonzero-quantized AC residuals in P-frames, the video bit-rate increase can be held to reasonable values. Since the nonzero-quantized AC residuals in P-frames correspond to nonflat areas that are in motion, temporal and texture masking are exploited at the same time. We also propose embedding the watermark in nonzero quantized AC residuals with spatial masking capacity in I-frames. Since the locations of the nonzero-quantized AC residuals is lost after decoding, we develop a watermark detection algorithm that does not depend on this knowledge. Our video watermark detection algorithm has controllable performance. We demonstrate the robustness of our proposed algorithm to several different attacks.

In this paper, we consider the problem of tracking multiple people in a 3D world domain using a microphone array and multiple cameras. The data fusion is done using a particle filter. To support 3D tracking, we propose a new video data likelihood model using a camera calibration matrix that can be used for a moving camera without continuous camera calibration. Then we apply an independent partition particle filter for multiple objects in order to generate particles efficiently. To detect the current speaker, we use a simple cost function using the generated particles. Finally we implement this tracking algorithm as a real-time system.

The conventional run-level variable length coding (RL-VLC), commonly adopted in block-based image and video compression to code quantized transform coefficients, is not efficient in coding consecutive nonzero coefficients. To overcome the deficiency, hybrid variable length coding (HVLC) is proposed in this paper. HVLC takes advantage of the clustered nature of nonzero transform coefficients in the low-frequency (LF) region and the scattered nature of nonzero transform coefficients in the high-frequency (HF) region by employing two types of VLC schemes. A novel two-dimensional position and one-dimensional amplitude coding scheme is proposed to code the LF coefficients while RL-VLC or an equivalent VLC scheme is retained to code the HF coefficients. Experimental results show that HVLC greatly favors the coding of high-resolution, high-complexity scenes, while it preserves low computational complexity.

In this paper, an example-based image denoising algorithm is introduced. Image denoising is formulated as a regression problem, which is then solved using support vector regression (SVR). Using noisy images as training sets, SVR models are developed. The models can then be used to denoise different images corrupted by random noise at different levels. Initial experiments show that SVR can achieve a higher peak signal-to-noise ratio (PSNR) than the multiple wavelet domain Besov ball projection method on document images.

Motivated by the success of support vector regression (SVR) in blind image deconvolution, we apply SVR to single-frame super-resolution. Initial results show that even when trained on as little as a single image, SVR is able to learn a generally applicable model that can super-resolve dissimilar images.

Our earlier work revealed a connection between blind image deconvolution and principal components analysis (PCA). In this letter, we explicitly formulate multichannel and single-channel blind image deconvolution as a PCA problem. Although PCA is derived from blur models that do not contain additive noise, it can be justified on both theoretical and experimental grounds that the PCA-based restoration algorithm is actually robust to the presence of white noise. The algorithm is applied to the restoration of atmospheric turbulence-degraded imagery and compared to an adaptive Lucy-Richardson maximum-likelihood algorithm on both real and simulated atmospheric turbulence blurred images. It is shown that the PCA-based blind image deconvolution runs faster and is more robust to noise.

As H.264 digital video becomes more prevalent, the need for copyright protection and authentication methods that are appropriate for this standard will emerge. This paper proposes a robust watermarking algorithm for H.264. We employ a human visual model adapted for a 4 times 4 discrete cosine transform block to increase the payload and robustness while limiting visual distortion. A key-dependent algorithm is used to select a subset of the coefficients that have visual watermarking capacity. Furthermore, the watermark is spread over frequencies and within blocks to avoid error pooling. This increases the payload and robustness without noticeably changing the perceptual quality. We embed the watermark in the coded residuals to avoid decompressing the video; however, we detect the watermark from the decoded video sequence in order to make the algorithm robust to intraprediction mode changes. We build a theoretical framework for watermark detection based on a likelihood ratio test. This framework is used to obtain optimal video watermark detection with controllable detection performance. Our simulation results show that we achieve the desired detection performance in Monte Carlo trials. We demonstrate the robustness of our proposed algorithm to several different attacks

In the March 2007 paper named above (pp. 14-23), the authors found the following error. The optimality of the detector is only valid for c = 2, which is a Gaussian distribution. The performance and results of the detector when c ≠ 2 as presented in the paper are still valid. However, in this case the detector is suboptimal. Equations (21)-(26) and (28) should be modified as described and the computational complexity of the detector is the same as the detector developed in reference [6].

This letter introduces a new algorithm for the restoration of a noisy blurred image based on the support vector regression (SVR). Experiments show that the performance of the SVR is very robust in blind image deconvolution where the types of blurs, point spread function (PSF) support, and noise level are all unknown.

Motion is inherent to video signals, and high perceptual quality cannot be achieved without accounting for motion. Although the information inferred from motion vectors in an encoder does not always represent the true motion of objects, in this paper, we show that it still carries useful information for finding an estimate of motion characteristics. We use the number of copy-mode macroblocks to measure motion intensity, and we use motion history to capture the spatial distribution of motion. By this means, we can identify moving areas in video frames. We do not embed watermark bits in moving areas to avoid watermarking artifacts introduced into P- and B-frames from watermark bits embedded in I-frames. Our simulations show that the proposed algorithm reduces undesirable watermarking artifacts in the video when compared to methods that do not exploit motion information.

We present the application of Bambeger pyramids to the removal of speckle noise from SAR images using subband domain shrinking. Bamberger pyramids provide better directional selectivity than the commonly used 2D UDWT allowing better representation of edges and geometrical structures. We show that Bamberger pyramids provide superior speckle removal and edge retention properties when compared to the 2D UDWT, and the commonly used Lee MMSE filter.

This paper describes an algorithm for the restoration of a noisy blurred image based on support vector regression. The blind image deconvolution was formulated as a machine learning problem. From the training set, the mapping between the noisy blurred image and the original image are learned by support vector regression (SVR). With the acquired mapping, the degraded image can be restored. Our approach was experimentally compared with the adaptive Lucy-Richardson maximum likelihood (ML) algorithm. In terms of ISNR (improvement of signal to noise ratio), SVR outperforms ML in blind deblurring tests in which the types of blurs, point spread function (PSF) support, and noise energy are all unknown.

In this paper, we describe an algorithm for identifying a parametrically described blur based on kurtosis minimization. Using different choices for the parameters of the blur, the noisy blurred image is restored using Wiener filter. We use the kurtosis as a measurement of the quality of the restored image. From the set of the candidate deblurred images, the one with the minimum kurtosis is selected. The proposed technique is tested in a simulated experiment on a variety of blurs including atmospheric turbulence blurs, Gaussian blurs, and out-of-focus blurs. The proposed approach is also tested on real blurred images. Moreover, we test the performance when a wrong blur model is given. Our experiments show that the kurtosis minimization measurements match well with methods that maximize PSNR.

This paper presents a novel, low complexity watermarking algorithm for H.264 that exploits the specific features of this new standard. The security of the algorithm is based on the randomness of the watermark location. The coefficient with the embedded watermark within the macroblock is determined by a public key extracted from the macroblock and a secret key possessed by the copyright owner. It is proposed to use the relative change of the DC coefficients of the 4/spl times/4 blocks in a macroblock to generate the public key. The watermark is embedded in the quantized AC coefficients of I frames, and the compressed video bit rate is only increased 0.5% on average. Watermark detection can be done without the original video signal, and the algorithm is appropriate for real-time applications. Simulation results show that watermark embedding preserves the perceptual quality of the video. However, the degradation induced by an adversary's attempt to remove the watermark creates visible artifacts.

We study the application of the Bamberger directional filter bank to the problem of rotation invariant texture classification. We explore the use of purely directional decompositions and the use of polar-separable Bamberger pyramids. We obtain comparable classification performance to Gabor-based methods using a smaller feature set.

Due to constraints both at the sensor and on the ground, dimension reduction is a common preprocessing step performed on many hyperspectral imaging datasets. However, this transformation is not necessarily done with the ultimate data exploitation task in mind-for example, target detection or ground cover classification. Indeed, theoretically speaking it is possible that a lossy operation such as dimension reduction might have a negative impact on detection performance. This notion is investigated experimentally using real-world hyperspectral imaging data. The popular principal components transform [aka. principal components analysis (PCA)] is used to explore the impact that dimension reduction has on adaptive detection of difficult targets in both the reflective and emissive regimes. Using seven state-of-the-art algorithms, it is shown that in many cases PCA can have a minimal impact on the detection statistic value for a target that is spectrally similar to the background against which it is sought.

Adaptive detection has been a staple in areas such as radar and sonar for a number of years. A key tenet of adaptive detectors is their use of a "noise only" covariance estimate from a secondary data set. When samples bearing the target signal are included in this covariance estimate, it becomes contaminated. The impact of covariance contamination is evaluated for three popular adaptive detection statistics. Multiple test cases are presented using real data, not simulations, and it is shown that significant contamination occurs with as few as eight target samples included in a 100 000+ sample covariance estimate. In addition to number of samples, sensitivity of the detection statistic to contamination from targets present also depends on algorithm type. Hyperspectral imaging (HSI) data offer insight into the impact of covariance contamination on adaptive detection and how it is related to signal-to-noise ratio (SNR) and spectral similarity.

This work presents a joint source and channel coding method for transmission of progressively compressed three-dimensional (3-D) models where the bit budget is allocated optimally. The proposed system uses the Compressed Progressive Mesh (CPM) algorithm to produce a hierarchical bitstream representing different levels of detail (LODs). In addition to optimizing the transmitted bitstream with respect to the channel characteristics, we also choose the optimum combination of quantization and tessellation to maximize the expected decoded model quality. That is, given a 3-D mesh, a total bit budget (B) and a channel packet-loss rate (P/sub LR/), we determine the optimum combination of: 1) the geometry coordinates quantizer step size (l-bit); 2) the number of transmitted batches (L); 3) the total number of channel coding bits (C); and 4) the distribution of these channel coding bits among the transmitted levels (C/sub L/). Experimental results show that, with our unequal error protection approach that uses the proposed bit-allocation algorithm, the decoded model quality degrades more gracefully (compared to either no error protection or equal error protection methods) as the packet-loss rate increases.

Based on the observation that a Cauchy density is more accurate in estimating the distribution of the ac coefficients than the traditional Laplacian density, rate and distortion models with improved accuracy are developed. The entropy and distortion models for quantized discrete cosine transform coefficients are justified in a frame bit-allocation application for H.264. Extensive analysis with carefully selected anchor video sequences demonstrates a 0.24-dB average peak signal-to-noise ratio (PSNR) improvement over the JM 8.4 rate control algorithm, and a 0.33-dB average PSNR improvement over the TM5-based bit-allocation algorithm that has recently been proposed for H.264 by Li et al. The analysis also demonstrates 20% and 60% reductions in PSNR variation among the encoded pictures when compared to the JM 8.4 rate control algorithm and the TM5-based bit-allocation algorithm, respectively.

Historically, multiple description coding (MDC) and postprocessing error concealment (ECN) algorithms have evolved separately. In this paper, we propose a coordinated application of multiple description scalar quantizers (MDSQ) and ECN, where the smoothness of the video signal helps to compensate for the loss of descriptions. In particular, we perform a reconstruction that is consistent with the data received at the decoder. When only a single description is available, the video is reconstructed in such a way that: 1) if we were to regenerate two descriptions (from the reconstructed video), one of them would be equivalent to the received description and 2) the reconstructed video is spatiotemporally smooth. Experimental results with several video sequences demonstrated a peak signal-to-noise ratio (PSNR) improvement of 0.9-2.8 dB for intracoded frames. The PSNR improvements for intercoded frames were negligible. However, for both cases, the visual improvements were much more striking than what the PSNR improvement suggested.

Hyperspectral images are used for aerial and space imagery applications, including target detection, tracking, agricultural, and natural resource exploration. Unfortunately, atmospheric scattering, secondary illumination, changing viewing angles, and sensor noise degrade the quality of these images. Improving their resolution has a high payoff, but applying super-resolution techniques separately to every spectral band is problematic for two main reasons. First, the number of spectral bands can be in the hundreds, which increases the computational load excessively. Second, considering the bands separately does not make use of the information that is present across them. Furthermore, separate band super-resolution does not make use of the inherent low dimensionality of the spectral data, which can effectively be used to improve the robustness against noise. In this paper, we introduce a novel super-resolution method for hyperspectral images. An integral part of our work is to model the hyperspectral image acquisition process. We propose a model that enables us to represent the hyperspectral observations from different wavelengths as weighted linear combinations of a small number of basis image planes. Then, a method for applying super resolution to hyperspectral images using this model is presented. The method fuses information from multiple observations and spectral bands to improve spatial resolution and reconstruct the spectrum of the observed scene as a combination of a small number of spectral basis functions.

This paper presents a distributed video streaming framework using unbalanced multiple description coding (MDC) and unequal error protection. In the proposed video streaming framework, two senders simultaneously stream complementary descriptions to a single receiver over different paths. To minimize the overall distortion and exploit the benefits of multipath transport when the characteristics of each path are different, an unbalanced MDC method for wavelet-based coders combined with a TCP-friendly rate allocation algorithm is proposed. The proposed rate allocation algorithm adjusts the transmission rates and the channel coding rates for all senders in a coordinated fashion to minimize the overall distortion. Simulation results show that the proposed unbalanced MDC combined with our rate allocation algorithm achieves about 1-6 dB higher peal signal-to-noise ratio compared to conventional balanced MDC when the available bandwidths along the two paths are different under time-varying network conditions.

Hyperspectral images are used for aerial and space imagery applications including target detection, tracking, agricultural and natural resource exploration. Unfortunately, atmospheric scattering, secondary illumination, changing viewing angles and sensor noise degrade the quality of these images. We introduce a novel superresolution reconstruction method for hyperspectral images. An integral part of our work is to model the hyperspectral image acquisition process. We propose a model that enables us to represent the hyperspectral observations from different wavelengths as weighted linear combinations of a small number of basis image planes. Then a method for applying superresolution to hyperspectral images using this model is presented. The method fuses information from multiple observations and spectral bands to improve spatial resolution and reconstruct the spectrum of the observed scene.

Block matching has been used for motion estimation and motion compensation in MPEG standards for years. While it has an acceptable performance in describing motion between frames, it requires quite a few bits to represent the motion vectors. In certain circumstances, the use of whole frame affine motion models would perform equally well or even better than block matching in terms of motion accuracy, while it results in the coding of only 6 parameters. In this paper, we modify an MPEG-4 codec by adding: (1) 6 affine model parameters to the frame header; and (2) mode selection among INTRA, SKIP, INTER-16/spl times/16, INTER-8/spl times/8, and GLOBAL-AFFINE modes by Lagrange optimal rate-distortion criteria. Simulation results demonstrate 10-20% decrease in bit-rate, compared to the MMS codec for an average coded P-frame with the same reconstruction PSNR.

Gaussian mixture modeling (GMM) has become a popular method for representing the statistical variability of spectrally inhomogeneous hyperspectral imaging (HSI) data. Finite mixture models can be applied to image segmentation, which can be considered a sort of blind classification, which is useful when there is no a priori information available about the materials in the scene. However, the GMM approach can be insufficient for many HSI data sets since the Gaussian has fixed narrow tails, uncharacteristic of operational HSI data. Elliptically contoured distributions (ECDs), specifically the multivariate t, have been shown to better fit HSI data. In this paper we develop mixtures of ECDs for the task of robustly modeling hyperspectral image data. An approach based on the EM algorithm is developed for concurrently estimating all parameters in a t mixture, assuming nothing but that the number of components are known. Results from the automatic segmentation of AVIRIS data illustrate the utility of t mixture models using this EM approach for parameter estimation.

Multiple description coding (MDC) is a source coding technique that exploits path diversity to increase the robustness of transmitting a compressed signal over error-prone channels. However, the application of MDC to video coding is still problematic because a prediction mismatch problem, called drift, may occur at the decoder when one description is lost. In the literature, it has been shown that drift can be effectively prevented by encoding this prediction mismatch and sending it as side information in a three-prediction-loop structure. However, this side information is totally redundant when both descriptions are received. We propose a very simple, but efficient two-stage MDC in the central prediction loop based on the three-prediction-loop structure of A. Reibman et al. (see Proc. IEEE Int. Conf. Image Processing - ICIP'99, p.837-41, 1999). We introduce an additional multiple description scalar quantizer into the central prediction loop. The encoding of the side information is modified in such a way that it can be used to improve the video quality when no drift occurs, but still maintain a drift prevention capability. The simulation results, with different video sequences at various packet loss rates, demonstrate that the proposed two-stage MDC structure indeed enhances the quality when the channel is error-free, yet effectively avoids drift when one description is completely lost.

Considerable attention has been directed to the problem of producing high-resolution video and still images from multiple low-resolution images. This multiframe reconstruction, also known as super-resolution reconstruction, is beginning to be applied to compressed video. Super-resolution techniques that have been designed for raw (i.e., uncompressed) video may not be effective when applied to compressed video because they do not incorporate the compression process into their models. The compression process introduces quantization error, which is the dominant source of error in some cases. In this paper, we propose a stochastic framework where quantization information as well as other statistical information about additive noise and image prior can be utilized effectively.

This paper presents a coordinated multiple-substream unequal error-protection and error-concealment algorithm for SPIHT-coded bitstreams transmitted over lossy channels. In the proposed scheme, we divide the video sequence corresponding to a group of pictures into two subsequences and independently encode each subsequence using a three-dimensional SPIHT algorithm. We use two different partitioning schemes to generate the substreams, each of which offers some advantages under the appropriate channel condition. Each substream is protected by an FEC-based unequal error-protection algorithm, which assigns unequal forward error correction codes to each bit plane. Any information that is lost during the transmission for any substream is estimated at the receiver by using the correlation between the substreams and the smoothness of the video signal. Simulation results show that the proposed multiple-substream UEP algorithm is simple, fast, and robust in hostile network conditions, and that the proposed error-concealment algorithm can achieve 2-3-dB PSNR gain over the case when error concealment is not used at high packet-loss rates.

This paper introduces an improved structure for directional filter banks (DFBs) that preserves the visual information in the subband domain. The new structure achieves this outcome while preserving both the efficient polyphase implementation and the exact reconstruction property. The paper outlines a step-by-step framework in which to examine the DFB, and within this framework discusses how, through the insertion of post-sampling matrices, visual distortions can be removed. In addition to the efficient tree structure, attention is given to the form and design of efficient linear phase filters. Most notably, linear phase IIR prototype filters are presented, together with the design details. These filters can enable the DFB to have more than a three-fold improvement in complexity reduction over quadrature mirror filters (QMFs).

This paper presents a distributed video streaming framework using unbalanced multiple description coding (MDC) and forward error correction. In the proposed video streaming framework, two senders simultaneously stream complementary descriptions to the receiver over different paths. To minimize the overall distortion and exploit the benefits of multi-path transport when the characteristics of each path are different, an unbalanced MDC method for wavelet-based coders combined with a TCP-friendly rate allocation algorithm is proposed. The proposed rate allocation algorithm adjusts the transmission rates and the channel coding rates for all senders in a coordinated fashion to minimize the overall distortion. Simulation results show that the proposed unbalanced MDC combined with our rate allocation algorithm achieves about 1-3 dB higher PSNR compared to conventional balanced MDC when the available bandwidth along each path is different in time-varying network conditions.

Multiple description coding addresses the problem of encoding a source signal into two or more sub-streams. Each sub-stream is assumed to be transmitted over an independent path. However, multiple independent paths may not be available, and for most cases only a single path can be used. The scheduling of these descriptions over a single path is still problematic because of the bursty packet loss behavior. In this paper, we propose a novel packet scheduling method for multiple description coded video transmission over a single path. The proposed method adaptively schedules the descriptions according to channel conditions by jointly considering the bursty packet loss behavior and the transmission delay of the network. The simulation results show that the proposed method can improve the video quality by 1.5 dB on average as compared to the method of sending one packet of each description right after the other. It also outperforms a heuristic interleaving approach by roughly 0.3-0.45 dB, depending on the channel conditions.

Layered (LC) and multiple description coding (MDC) have been proposed as source coding techniques that are robust to channel errors for video transmission. In this paper, we investigate and study the error-resilience capabilities of these two encoding techniques through extensive experimentation, and explore the environments where they perform best. Although some of our conclusions agree with those in the literature, we believe that this paper provides the most comprehensive performance comparison yet between LC and MDC.

An imaging system produces a degraded measurement of a real-valued quantity that varies temporally, spectrally, and spatially. When there are multiple measurements of the same scene, it is possible to combine the nonredundant information in those measurements and produce an improved image that has more information than any of the measurements does alone. This type of reconstruction is known as image fusion. Depending on the diversity of information in the measurements, it is possible to achieve temporal, spectral, spatial, and gray-scale improvements with an image fusion algorithm. In this paper, we are proposing an image fusion algorithm that produces an image of higher spatial and gray-scale information from multiple measurements. The algorithm estimates the spatial and illumination correlation between multiple measurements, and employs a set-theoretic reconstruction technique.

This paper presents a TCP-friendly rate allocation algorithm for multiple description coding combined with path diversity. A 3-D SPIHT-based multiple description coding is used to generate two independent substreams and each substream is protected with an FEC-based unequal error protection algorithm. In the proposed video streaming framework, two senders simultaneously stream complementary substreams to the receiver over different paths. The proposed rate allocation algorithm coordinates distributed video streaming from multiple senders to a single receiver to minimize the overall distortion. Simulation results show that the proposed algorithm increases tolerance to packet loss due to network congestion and achieves higher visual quality compared to non-adaptive methods.

Multiple description coding (MDC) is a source coding technique that exploits path diversity to increase the robustness of transmitting a compressed signal over error-prone channels. However, the applicant of MDC to video coding is still problematic because a prediction mismatch problem, called drift, may occur at the decoder when one description is lost. In this paper, we propose a drift free motion-compensated predictive coding method for multiple description scalar quantizers. The proposed method maintains two more prediction loops at the encoder side to produce all possible predictions. Then, the two descriptions are generated from these two additional prediction loops in such a way that the drift can be prevented when only one description is received. By receiving both descriptions, the decoder can still combine these two descriptions in the central prediction loop to improve the video quality. Experimental results show that the proposed method can effectively prevent drift and improve the quality of single-description reconstruction by 0.2-0.8 dB as compared to the method in [A. Reibman et al., 1999].

Two-dimensional mesh-based models provide a good alternative to motion estimation and compensation. The estimation of the best node-point motion vectors is a challenging task. To this effect, Nakaya and Harashima (1994) proposed a hexagonal matching procedure. Toklu et al. improved the hexagonal search algorithm in terms of both motion-estimation accuracy and computational complexity by employing a hierarchy of regular meshes. Recognizing the limitations of regular meshes, Van Beek et al. (1997) extended Toklu et al.'s (1996) work by utilizing content-based meshes. Here, we provide an alternative hierarchical motion-estimation method with content-based meshes where hierarchical representations are employed for both the images and the irregular meshes in order to provide further improvements in computational complexity as well as motion accuracy. The comparison results are provided with real video sequences.

Methods for estimating motion in video sequences that are based on the optical flow equation (OFE) assume that the scene illumination is uniform and that the imaging optics are ideal. When these assumptions are appropriate, these methods can be very accurate, but when they are not, the accuracy of the motion field drops off accordingly. This paper extends the models upon which the OFE methods are based to include irregular, time-varying illumination models and models for imperfect optics that introduce vignetting, gamma, and geometric warping, such as are likely to be found with inexpensive PC cameras. The resulting optimization framework estimates the motion parameters, illumination parameters, and camera parameters simultaneously. In some cases these models can lead to nonlinear equations which must be solved iteratively; in other cases, the resulting optimization problem is linear. For the former case an efficient, hierarchical, iterative framework is provided that can be used to implement the motion estimator.

Face images that are captured by surveillance cameras usually have a very low resolution, which significantly limits the performance of face recognition systems. In the past, super-resolution techniques have been proposed to increase the resolution by combining information from multiple images. These techniques use super-resolution as a preprocessing step to obtain a high-resolution image that is later passed to a face recognition system. Considering that most state-of-the-art face recognition systems use an initial dimensionality reduction method, we propose to transfer the super-resolution reconstruction from pixel domain to a lower dimensional face space. Such an approach has the advantage of a significant decrease in the computational complexity of the super-resolution reconstruction. The reconstruction algorithm no longer tries to obtain a visually improved high-quality image, but instead constructs the information required by the recognition system directly in the low dimensional domain without any unnecessary overhead. In addition, we show that face-space super-resolution is more robust to registration errors and noise than pixel-domain super-resolution because of the addition of model-based constraints.

This paper presents a new bit-plane-wise unequal error protection algorithm for progressive bitstreams transmitted over lossy networks. The proposed algorithm protects a compressed embedded bitstream generated by a 3-D SPIHT algorithm by assigning an unequal amount of forward error correction (FEC) to each bit-plane. The proposed algorithm reduces the amount of side information needed to send the size of each code to the decoder by limiting the number of quality levels to the number of bit-planes to be sent while providing a graceful degradation of picture quality as packet losses increase. We also apply our proposed algorithm to transmission of JPEG 2000 coded images over the Internet. To get additional error-resilience at high packet loss rates, we extend our algorithm to multiple-substream unequal error protection. Simulation results show that the proposed algorithm is simple, fast and robust in hostile network conditions and, therefore, can provide reasonable picture quality for video applications under varying network conditions.

This paper describes a system for learning a face model that is used for 3D tracking of the human face. The face is modeled as a linear combination of shape basis vectors and action vectors. Shape space models the difference in face shape of different people while action space models the facial expressions. First, real stereo tracking data is used to learn the space of these shape and action vectors using Principal Component Analysis. Then this low-complexity model is used to simultaneously track shape, pose and expression from a monocular image sequence. The main contribution of this paper is in learning shape and action deformation models simultaneously from real data. Results of monocular model-based tracking for subjects not included in the training set show that the model derived from data is robust and generalizes well.

Speech not only conveys the linguistic information, but also characterizes the talker's identity and therefore can be used in personal authentication. While most of the speech information is contained in the acoustic channel, the lip movement during speech production also provides useful information. In this paper we investigate the effectiveness of visual speech features in a speaker veri£cation task. We £rst present the visual front-end of the automatic speechreading system. We then develop a recognition engine to train and recognize sequences of visual parameters. The experimental results based on the XM2VTS database [I] demonstrate that visual information is highly effective in reducing both false acceptance and false rejection rates in speaker veri£cation tasks.

Most commercial digital cameras have three types of color sensors (for red, green, and blue channels) that are placed on a detector surface according to a specific pattern. At the location of each pixel only one color sample is taken, and the values of the other colors must be interpolated using neighboring samples. This color plane interpolation is known as demosaicing; it is one of the important tasks in a digital camera pipeline. If demosaicing is not performed appropriately, images suffer from highly visible color artifacts. In this paper we present a new demosaicing technique that uses inter-channel correlation effectively in an alternating-projections scheme. We have compared this technique with various state-of-the-art demosaicing techniques, and it outperforms all of them, both visually and in terms of mean square error.

It is important to detect shot and sub-shot changes, which are the fundamental units of video sequences, in video indexing techniques. In this paper, a unified method for detecting shot and sub-shot changes is proposed using a background image. There are two steps used to detect shot changes. In the first step, pixel-wise differences using a new criterion are used to screen out less likely scene changes. The second step is based on the percentage of pixels having motion vectors different from the background image, and is applied to the candidate frames from the first step. To detect sub-shot changes, camera motion parameters are estimated from the background images. The experimental results show that the proposed method is robust and accurate in detecting shot and sub-shot changes.

Mainstream automatic speech recognition has focused almost exclusively on the acoustic signal. The performance of these systems degrades considerably in the real world in the presence of noise. On the other hand, most human listeners, both hearing-impaired and normal hearing, make use of visual information to improve speech perception in acoustically hostile environments. Motivated by humans' ability to lipread, the visual component is considered to yield information that is not always present in the acoustic signal and enables improved accuracy over totally acoustic systems, especially in noisy environments. In this paper, we investigate the usefulness of visual information in speech recognition. We first present a method for automatically locating and extracting visual speech features from a talking person in color video sequences. We then develop a recognition engine to train and recognize sequences of visual parameters for the purpose of speech recognition. We particularly explore the impact of various combinations of visual features on the recognition accuracy. We conclude that the inner lip contour features together with the information about the visibility of the tongue and teeth significantly improve the performance over using outer contour only features in both speaker dependent and speaker independent recognition tasks.

This paper presents a new bit-plane-wise unequal error protection algorithm for Internet video applications. The proposed algorithm protects a compressed embedded bitstream generated by a 3-D SPIHT algorithm by assigning unequal forward error correction (FEC) codes to each bit-plane. The proposed algorithm reduces the amount of side information needed to send the size of each code to the decoder by limiting the number of quality levels to the number of bit-planes to be sent while providing a graceful degradation of picture quality as packet losses increase. To get additional error-resilience at high packet loss rates, we extend our algorithm to multiple-substream unequal error protection. Simulation results show that the proposed algorithm is simple, fast and robust in hostile network conditions and therefore can provide reasonable picture quality for Internet video applications under varying network conditions.

Historically, multiple description coding (MDC) and post-processing error concealment (ECN) algorithms for video have evolved separately. Although multiple description coding provides a way to make the video quality degrade gracefully in the presence of packet losses, the resulting quality may be unsatisfactory when only one description is used to reconstruct the signal. This paper proposes a coordinated application of multiple description scalar quantizer (MDSQ) and error concealment, where the smoothness of the video signal helps to compensate for the loss of descriptions. This joint optimization results in significant improvements in reconstructed video quality over independent use of MDSQ.

Face images that are captured by surveillance cameras usually have a very low resolution, which significantly limits the performance of face recognition systems. In the past, super-resolution techniques have been proposed that attempt to increase the resolution by combining information from multiple images. These techniques use super-resolution as a preprocessing system to obtain a high resolution image that can later be passed to a face recognition system. Considering that most state-of-the-art face recognition systems use an initial dimensionality reduction method, we propose embedding the super-resolution algorithm into the face recognition system so that super-resolution is not performed in the pixel domain, but is instead performed in a reduced dimensional domain. The advantage of such an approach is a significant decrease in the computational complexity of the super-resolution algorithm because the algorithm no longer tries to construct a visually improved high quality image, but instead constructs the information required by the recognition algorithm directly in the lower dimensional domain without any unnecessary overhead.

This paper presents a coordinated multiple-substream unequal error protection and error concealment algorithm for SPIHT-coded bitstreams transmitted over lossy channels. In the proposed scheme, we divide the video sequence corresponding to a group of pictures into two sub-sequences in the temporal domain and independently encode each sub-sequence with a 3-D SPIHT algorithm to generate two independent substreams. Each substream is protected by an FEC-based unequal error protection algorithm that assigns unequal forward error correction codes for each substream with bit-plane granularity. The information that is lost during transmission for one substream is estimated at the receiver by using the correlation between the two substreams and the smoothness of the video signal. Simulation results show that the proposed multiple-substream UEP algorithm is simple, fast, and robust in hostile network conditions, and that the proposed error concealment algorithm achieves about 1-3 dB PSNR gain over the case where there is no error concealment at high packet loss rates.

In this paper, we propose a very simple, but efficient two-stage MDC in the central prediction loop based on Reibman's three-prediction-loop structure (Reibman et al. (1999)). In our design, an additional multiple description scalar quantizer (MDSQ) is introduced into the central prediction loop. The encoding of the side information is modified in such a way that it can be used to improve the video quality when no drift occurs, but still maintain drift prevention capability. The simulation results with different video sequences at various packet loss rates demonstrate that the proposed two-stage MDC structure indeed enhances the quality by about 0.25 dB when the channel is error-free, yet effectively avoids drift when one description is completely lost.

Extending automatic speech recognition (ASR) to the visual modality has been shown to increase recognition accuracy greatly and improve system robustness over purely acoustic systems, especially in acoustically hostile environments. An important aspect of designing such systems is how to incorporate the visual component into the acoustic speech recognizer to achieve optimal performance. We investigate methods of integrating the audio and visual modalities within HMM-based classification models. We examine existing integration schemes and propose the use of a coupled hidden Markov model (CHMM) to exploit audio-visual interaction. Our experimental results demonstrate that the CHMM consistently outperforms other integration models for a large range of acoustic noise levels and suggest that it better captures temporal correlations between the two streams of information.

We propose a protocol for efficient streaming of 3-D animations over lossy channels. In order to improve the expected quality on the client's side, we first transmit a crude model of the 3-D mesh, of its texture, and of its immediate evolution. We endow this data with significant error-protection against transmission error. Then we transmit a series of upgrades that refine the accuracy of the model and/or of the animation. These are encoded with lower levels of error protection that is proportional to their impact on the quality of the upgraded animation. We propose the following types of upgrade chunks: selection of a subset of vertices, adjustments of the accuracy of the positions of the selected vertices, connectivity refinements, motion adjustments of the selected subset of vertices, adjustments of the accuracy of the texture coordinates of the selected vertices, and upgrade of the quality of the texture. Finally, given the allocated bit-budget, we determine the optimal number of both source and channel coding bits assigned for each chunk to maximize the animation quality on the client's side. The optimization takes into account both the bandwidth and the error characteristics of the channel.

Multiframe resolution enhancement ("superresolution") methods are becoming widely studied, but only a few procedures have been developed to work with compressed video, despite the fact that compression is a standard component of most image- and video-processing applications. One of these methods uses quantization-bound information to define convex sets and then employs a technique called "projections onto convex sets" (POCS) to estimate the original image. Another uses a discrete cosine transformation (DCT)-domain Bayesian estimator to enhance resolution in the presence of both quantization and additive noise. The latter approach is also capable of incorporating known source statistics and other reconstruction constraints to impose blocking artifact reduction and edge enhancement as part of the solution. We propose a spatial-domain Bayesian estimator that has advantages over both of these approaches.

There are a number of useful methods for creating high-quality video or still images from a lower quality video source. The best of these involve motion compensating a number of video frames to produce the desired video or still. These methods are formulated in the space domain and they require that the input be expressed in that format. More and more frequently, however, video sources are presented in a compressed format, such as MPEG, H.263, or DV. Ironically, there is important information in the compressed domain representation that is lost if the video is first decompressed and then used with a spatial-domain method. In particular, quantization information is lost once the video has been decompressed. Here, we propose a motion-compensated, transform-domain super-resolution procedure for creating high-quality video or still images that directly incorporates the transform-domain quantization information by working with the compressed bit stream. We apply this new formulation to MPEG-compressed video and demonstrate its effectiveness.

A major drawback of block-based still-image or video-compression methods at low rates is the visible block boundaries that are also known as blocking artifacts. Several methods have been proposed in the literature to reduce these artifacts. Most are single image methods, which do not distinguish between video and still images. However, video has a temporal dimension that can lead to better reconstruction if utilized effectively. We show how to combine information from multiple frames to reduce blocking artifacts. We derive constraint sets using motion between neighboring frames and quantization information that is available in the video bit stream. These multiframe constraint sets can be used to reduce blocking artifacts in an alternating-projections scheme. They can also be included in existing set-theoretic algorithms to improve their performance by narrowing down the feasibility set. Experimental results show the effectiveness of using these multiframe constraint sets.

Most commercial digital cameras use color filter arrays to sample red, green, and blue colors according to a specific pattern. At the location of each pixel only one color sample is taken, and the values of the other colors must be interpolated using neighboring samples. This color plane interpolation is known as demosaicing; it is one of the important tasks in a digital camera pipeline. If demosaicing is not performed appropriately, images suffer from highly visible color artifacts. In this paper we present a new demosaicing technique that uses inter-channel correlation effectively in an alternating-projections scheme. We have compared this technique with six state-of-the-art demosaicing techniques, and it outperforms all of them, both visually and in terms of mean square error.

Compressed video sequences are very vulnerable to channel disturbances when they are transmitted through an unreliable medium such as a wireless channel. Transmission errors not only corrupt the current decoded frame, but they may also propagate to succeeding frames. A number of post-processing error concealment (ECN) methods that exploit the spatial and/or temporal redundancy in the video signal have been proposed to combat channel disturbances. Although these approaches can effectively conceal lost or erroneous macroblocks (MBs), all of them only consider spatial and/or temporal correlation in a single frame (the corrupted one), which limits their ability to obtain an optimal recovery. Since the error propagates to the next few motion-compensated frames in the presence of lost MBs in an I or P frame, error concealment should simultaneously minimize the errors not only in the current decoded frame but also in the succeeding B and P frames that depend on the corrupted frame. We propose a novel multiframe recovery principle which analyzes the propagation of a lost MB into succeeding frames. Then, MPEG-compatible spatial and temporal error concealment approaches using this multiframe recovery principle are proposed, where the lost MBs are recovered in such a way that the error propagation is minimized.

Motivated by our wavelet framework for error-control coding, we proceed to develop an important family of wavelet transforms over finite fields. Paraunitary (PU) filter banks that are realizations of orthogonal wavelets by multirate filters are an important subclass of perfect reconstruction (PR) filter banks. A parameterization of PU filter banks that covers all possible PU systems is very desirable in error-control coding because it provides a framework for optimizing the free parameters to maximize coding performance. This paper undertakes the problem of classifying all PU matrices with entries from a polynomial ring, where the coefficients of the polynomials are taken from finite fields. It constructs Householder transformations that are used as elementary operations for the realization of all unitary matrices. Then, it introduces elementary PU building blocks and a factorization technique that is specialized to obtain a complete realization for all PU filter banks over fields of characteristic two. This is proved for the 2 /spl times/ 2 case, and conjectured for the M /spl times/ M case, where M /spl ges/ 3. Using these elementary building blocks, we can construct all PU filter banks over fields of characteristic two. These filter banks can be used to implement transforms which, in turn, provide a powerful new perspective on the problems of constructing and decoding arbitrary-rate error-correcting codes.

The primary goal of this paper is to give examples of the recently developed (finite-field) wavelet coding method by studying the encoder and decoder for some half-rate codes. We propose a decoding methodology based on estimating the polyphase components of the channel error pattern. To demonstrate the striking computational savings of the wavelet coding method over alternatives, we show that bounded-distance decoding of the (24,12,8) Golay code requires only weight computations (or at the worst case, it needs a cyclic lookup table of table size 12). The simplicity and computational savings that finite field wavelets offer for the encoding and decoding of wavelet block codes indicate their powerful capacities for error control coding applications.

A major drawback of block-based still image or video compression methods at low rates are the visible block boundaries that are also known as blocking artifacts. Several methods have been proposed in the literature to reduce these artifacts for video sequences. However, most are simply adaptations of still image blocking artifact reduction methods, which do not exploit temporal information. We propose a novel multi-frame blocking artifact reduction method that incorporates temporal information effectively. This method uses the spatial correlations that exist between the successive frames to define constraint sets at multiple frames and provides a Projections Onto Convex Sets (POCS) solution. The proposed method operates solely on transform domain (DCT) data, and hence provides a solution that is compatible with the observed video. It does not need to make any spatial smoothness assumptions, which are typical with blocking artifact reduction algorithms for still images.

This paper presents feedback design of a desktop visual communications system. The system consists of video coding using a 3-D graphics model and video transmission over the Internet. To reduce the overall video quality degradation in visual communications caused by coding and networking errors, we jointly researched the compression and transmission of video signals. In building the 3-D graphics model-based coding structure, instead of using feedforward based on pixel intensity, we develop a three-level signal representation and an analysis-by-synthesis feedback framework. In prototyping the video over IP in desktop conferencing, we adapt the video encoders coding rates based on feedback of the network states and receivers. We therefore contribute to the analysis, modeling, and transmission of multimedia signals for desktop visual communications.

We present a new fragile digital watermarking technique that can be used as a digital signature for data authentication. The technique presented is blind, efficient, secure and can detect and locate minor tampering applied to the image pixels. The technique is suitable for certain classes of image formats such as gif, tif or bitmap images. The digital signature is inserted by decorrelating the image samples in the spatial domain, then embedding the watermark in the transform domain of the decorrelated image. The mark is embedded using quantization and modulo arithmetic techniques with a small quantization step size. Any tampering with the image will create errors in the embedded bits, which can be easily detected.

Resolution enhancement for video sequences has always been an attractive application in multimedia signal processing. "Superresolution" methods, that combine non-redundant information from a set of low-resolution images, are beginning to be applied to the most popular video compression standard, MPEG. Bayesian approaches, which are very successful for raw video, largely fail for MPEG video, since they do not incorporate the compression process into their models. This compression process introduces quantization noise, which is comparable to the additive noise that is used in the Bayesian models. We present an analytical derivation that combines the quantization and additive noises in a stochastic framework for MPEG-compressed video. This is a general framework in the sense that different video acquisition models, source statistics, implementation techniques can be used with it.

Compressed video sequences are very sensitive to channel disturbances when they are transmitted through an unreliable environment such as a wireless channel. The transmission errors not only corrupt the current decoded frame, but also propagate to the succeeding frames. Many error concealment methods have been proposed to recover the errors by exploiting temporal and spatial correlation. In this paper, we develop a multi-frame boundary matching algorithm (MF-BMA), which utilizes the boundary smoothness property not only in the decoded frame but also in the succeeding frames. The experimental results demonstrate that the proposed algorithm can provide better performance, especially when horizontal edges exist across the lost macroblocks.

We present a new steganographic technique for covert communications. The technique embeds the hidden information in the DFT domain after permuting the image pixels in the spatial domain using a key. The permutation process introduces randomness into the cover image and results in a significant increase in the number of transform coefficients that can be used to transmit the hidden information. The hidden information is embedded using a quantization technique. The perceptual and statistical properties of the cover and the stego images remain similar for small quantization steps. The security of the system is examined against known stego attack.

The transmission of any data is always subject to corruption due to errors, but video transmission, because of its real time nature must deal with these errors without retransmission of the corrupted data. The error can be handled by using forward error correction in the encoder or error concealment techniques in the decoder. The MPEG-2 compliant coder described here uses steganography to transmit data for error correction in conjunction with several error concealment techniques in the decoder. The decoder resynchronizes more quickly with fewer errors than traditional resynchronization techniques. This provides for a much higher quality picture in an error-prone environment while creating an almost imperceptible degradation of the picture in an error free environment.

The number of bits assigned to represent the color intensity at image pixels is usually referred as the bit depth. When the bit depth is not sufficient, images suffer from ridge-like structures known as false contours. Bit-depth limitations become important when low-contrast details are required, as in medical imaging, aerial/satellite photography, and high-quality scanning: applications. In this paper, we investigate a method for increasing bit depth. Specifically, we show that when a sequence of video frames is available, then it is possible to achieve a higher bit depth through a projections onto convex sets (POCS) based reconstruction method.

The transmission of any data is always subject to corruption due to errors, but video transmission, because of its real time nature must deal with these errors without retransmission of the corrupted data. The error can be handled by using forward error correction in the encoder or error concealment techniques in the decoder. This MPEG-2 compliant codec uses data hiding to transmit error correction information and several error concealment techniques in the decoder. The decoder resynchronizes more quickly with fewer errors than traditional resynchronization techniques. This provides for a much higher quality picture in an error-prone environment while creating an almost imperceptible degradation of the picture in an error free environment.

Digital watermarking is the process of secretly embedding a short sequence of information inside a digital source without changing its perceptual quality. We present a new oblivious digital watermarking method for copyright protection of still images. The technique is based on modifying the sign of a subset of low frequency DCT magnitude coefficients. The robustness to a number of standard image processing attacks is demonstrated using the criteria of the latest Stirmark test.

Paraunitary filter banks are multi-rate filter realizations of orthogonal wavelet transforms. They are an important subclass of perfect reconstruction filter banks that provide a new framework for error control coding and decoding. This paper undertakes the problem of classifying all paraunitary matrices with entries from a polynomial ring, where the coefficients of the polynomials are taken from finite fields. It constructs Householder transformations that are used as elementary operations for the realization of all unitary matrices. Then, it introduces elementary paraunitary building blocks and a factorization technique that are specialized to obtain a complete realization for all paraunitary filter banks over fields of characteristic two (this is proved for the 2-band case, and a conjecture is applied for the proof of the M-band case, where M/spl ges/3). Using these elementary building blocks, we can construct all paraunitary filter banks over fields of characteristic two.

Segmentation followed by shape descriptors represents a common and fundamental approach used in many image processing systems. Active contour models have been used for shape description as a promising method. In particular, the deformable template, which is a kind of active contour model, has been used for various shape description problems. But active contour models, including the deformable template model, suffer from some common difficulties. We propose new approaches in which we represent an object using a weighted Gaussian approximation to find the best candidate template and minimize an appropriately designed cost function to deform the template after finding a best fit candidate in the multiscale representation of the image. This framework can be applied to many real-time applications such as object based video coding, and the estimation of facial features in face recognition.

During the transmission of video, errors can cause the loss of single or multiple macroblocks in MPEG-2 video. Although there are many ways to correct these losses, the use of spatial concealment, which relies on data from surrounding macroblocks in the same frame, is an invaluable scheme. This paper describe a spatial concealment scheme based the Hough transform and image segmentation. The image is initially segmented into bright foreground, dark foreground and background. The Hough transform is used to determine the best direction for filtering or interpolation of the foregrounds. The results show a significant improvement over the use of bidirectional interpolation especially in the area of significant edges and texture. It also improves upon other directional schemes by better isolating and preserving edge features.

Digital watermarking is the process of secretly embedding a short sequence of information containing owner identity or copyright information inside a digital signal, without changing its perceptual quality. The information is embedded in a way that is difficult to erase. We present a new oblivious digital watermarking method for copyright protection of still images. The technique is based on modifying the sign of a subset of low frequency, image transform coefficients with high to moderate magnitudes. These transforms include the DCT, DFT and Hadamard transforms. Watermark detection is done in an oblivious way. The robustness to a number of standard image processing attacks is demonstrated using the criteria of the latest StirMark benchmark test. The technique can be used also for other applications such as image fingerprinting.

We propose a hybrid low bit-rate subband video coding scheme utilizing context models, rate-constrained motion estimation and overlapped motion compensation. Emerging efficient context-based entropy coding techniques fit very well into wavelet/subband-based coding systems, making them very attractive candidates for the entropy coding part of a wavelet/subband-based video coding scheme. A multidimensional probability modeling technique along with context-based entropy coding are utilized to exploit redundancies in the subband structure, yielding better compression.

We address a limitation of video compression standards that affects their coding efficiency. The standard methods use one or two Huffman codes for all DCT run-length events. This implicitly assumes that the statistics of DCT run-length events are nearly stationary, but they are not. There is room for significant improvement in coding efficiency by the use of conditional Huffman codes. We propose to condition the Huffman codes for runlength events on the values of previous events in the block, the quantization parameter, and other information available to the decoder. We give a systematic way of designing optimal sets of Huffman codes, to give the maximum gain in coding efficiency for a given number of codes. We achieve significant improvements in coding efficiency over the standard methods with a reasonable number of codes, in low bit rate compression. Our method is non-adaptive and has little effect on the running time of the encoder or decoder.

The use of color information can significantly improve the efficiency and robustness of lip feature extraction capability over purely grayscale-based methods. Edge information provides another useful tool in characterizing lip boundaries. We present a method of integrating both types of information to address the problem of lip feature extraction for the purpose of speechreading. We first examine various color models and view hue as an effective descriptor to characterize the lips due to its invariance to luminance and human skin color, and its discriminative properties. We use prominent red hue as an indicator to locate the position of the lips. Based on the identified lip area, we further refine the interior and exterior lip boundary using both color and spatial edge information, where those two are combined within a Markov random field (MRF) framework. Experimental results are presented to show the effectiveness of this method.

This paper presents an approach for the effective combination of interpolation with binarization of gray level text images to reconstruct a high resolution binary image from a lower resolution gray level one. We study two nonlinear interpolative techniques for text image interpolation. These nonlinear interpolation methods map quantized low dimensional 2 x 2 image blocks to higher dimensional 4 x 4 (possibly binary) blocks using a table lookup operation. The first method performs interpolation of text images using context-based, nonlinear, interpolative, vector quantization (NLIVQ). This system has a simple training procedure and has performance (for gray-level high resolution images) that is comparable to our more sophisticated generalized interpolative VQ (GIVQ) approach, which is the second method. In it, we jointly optimize the quantizer and interpolator to find matched codebooks for the low and high resolution images. Then, to obtain the binary codebook that incorporates binarization with interpolation, we introduce a binary constrained optimization method using GIVQ. In order to incorporate the nearest neighbor constraint on the quantizer while minimizing the distortion in the interpolated image, a deterministic-annealing-based optimization technique is applied. With a few interpolation examples, we demonstrate the superior performance of this method over the NLIVQ method (especially for binary outputs) and other standard techniques e.g., bilinear interpolation and pixel replication.

An extension of conventional block motion compensation (BMC), overlapped block motion compensation (OBMC) has been shown to reduce residual errors and blocking effects in motion-compensated video. However, the overlap creates a noncausal spatial dependence between blocks and complicates motion estimation (ME) for OBMC. Iterative methods have traditionally been employed for overlapped block motion estimation (OBME). For compression, the rate for the motion vector field (MVF) may also be constrained. This work considers several rate-constrained OBME algorithms, both iterative and noniterative. Experiments demonstrate that a simple raster-scan algorithm is effective as a suboptimal, noniterative solution, with comparable or better rate-distortion performance and computational complexity than iterative OBME algorithms. Depending on the application, either this method or a simple block-matching algorithm plus iteration are the most attractive of the tested OBME schemes.

For several years, we have been teaching DSP as a first course in electrical and computer engineering at Georgia Tech. Such a dramatic rearrangement of the introductory material requires a new organization of topics and courses when teaching circuits and systems. In addition, the use of computer-enhanced course materials has a profound impact on the systems courses, which are quite mathematical and abstract in nature. This paper addresses some of the issues encountered when adopting a signal processing first approach.

We develop the theory of the wavelet transform over Galois fields. To avoid the limitations inherent in the number theoretic Fourier transform over finite fields, our wavelet transform relies on a basis decomposition in the time domain rather than in the frequency domain. First, we characterize the infinite dimensional vector spaces for which an orthonormal basis expansion of any sequence in the space can be obtained using a symmetric bilinear form. Then, by employing a symmetric, non-degenerate, canonical bilinear form we derive the necessary and sufficient condition that basis functions over finite fields must satisfy in order to construct an orthogonal wavelet transform. Finally, we give a design methodology to generate the mother wavelet and scaling function over Galois fields by relating the wavelet transform to a two channel paraunitary filter bank.

A new directional filter bank for image analysis and classification is proposed. This paper introduces an improved structure in order to visualize subband outputs of the directional filter banks, while retaining the attractive properties of the original directional filter banks such as 1-D separable filtering, perfect reconstruction, and maximal decimation. Using this structure, any arbitrary 2/sup n/ band directional filter bank can be implemented by cascading simple directional filter bank blocks, unlike the original structure that needs a parallel structure for visualizing subband outputs. Also, in order to have nondistorted phase information in the subbands for visualization, both FIR and IIR filter prototypes that can be implemented efficiently are provided for linear phase filtering. This paper shows the approach proposed here can be applied to image analysis and classification.

Two new fast algorithms for computing the inverse modified discrete cosine transform (IMDCT) as used in the adaptive spectral entropy coder (ASPEC) (Brandenburg et al. 1991) are proposed. A fixed-point error analysis is presented to determine the number of significant bits required for fixed-point implementations.

Noise degrades the performance of any image compression algorithm. However, at very low bit rates, image coders effectively filter noise that may he present in the image, thus, enabling the coder to operate closer to the noise free case. Unfortunately, at these low bit rates the quality of the compressed image is reduced and very distinctive coding artifacts occur. This paper proposes a combined restoration of the compressed image from both the artifacts introduced by the coder along with the additive noise. The proposed approach is applied to images corrupted by data-dependent Poisson noise and to images corrupted by film-grain noise when compressed using a block transform-coder such as JPEG. This approach has proved to be effective in terms of visual quality and peak signal-to-noise ratio (PSNR) when tested on simulated and real images.

In this correspondence, a fast motion estimation algorithm, based on the successive elimination algorithm (SEA) of Li and Salari (1995), is studied. This fast motion estimation algorithm finds the same displacement vectors as the exhaustive search algorithm with a reduced computational load. A modified fast motion estimation algorithm introducing negligible distortion into a transform coder, but providing for a further computational load reduction, is developed. Implementation issues are also discussed and compared. Results show that the number of searching operations can be reduced dramatically with the help of fast motion estimation algorithms.

A fast N-step search algorithm for rate-constrained motion estimation is presented. The motion vectors are selected from a search window based on a rate-distortion criterion by successively eliminating the search positions at each step. The performance of the proposed algorithm is identical to the performance of the conventional rate-constrained N-step search algorithm, with considerable reduction in computation. Computational savings increase in parallel with the increase in the rate constraint and the number of steps.

We discuss the problem of reconstruction of a high resolution image from a lower resolution image by a jointly optimum interpolative vector quantization method. The interpolative vector quantizer maps quantized low dimensional 2/spl times/2 image blocks to higher dimensional 4/spl times/4 blocks by a table lookup method. As a special case of generalized vector quantization (GVQ), a jointly optimal quantizer and interpolator (GIVQ) is introduced to find the corresponding code books for the low and high resolution image. In order to incorporate the nearest neighborhood constraint on the quantizer and also to obtain the desired distortion in the interpolated image, a deterministic annealing based optimization technique has been applied. With a small interpolation example, we demonstrate the superior performance of this method over nonlinear interpolative vector quantization (NLIVQ), in which the interpolator is optimized for a given input quantizer.

This paper discusses the application of hidden Markov models (HMMs) to solve the translational and rotational invariant automatic target recognition (TRIATR) problem associated with SAR imagery. This approach is based on a cascade of these stages: preprocessing, feature extraction and selection, and classification. Preprocessing and feature extraction and selection involve successive applications of extraction operations from measurements of the Radon transform of target chips. The features which are invariant to changes in rotation, position and shifts, although not to changes in scale are optimized through the use of feature selection techniques. The classification stage successively takes as its inputs the multidimensional multiple observation sequences, parameterizes them statistically using continuous density models to capture target and background appearance variability, and thus results in the TRIATR-HMMs. Experimental results have demonstrated that the recognition rate is as high as 99% over both the training set and the testing set.

Noise degrades the performance of any image compression algorithm. This paper studies the effect of noise on lossy image compression. The effect of Gaussian, Poisson, and film-grain noise on compression is studied. To reduce the effect of the noise on compression, the distortion is measured with respect to the original image not to the input of the coder. Results of noisy source coding are then used to design the optimal coder. In the minimum-mean-square-error (MMSE) sense, this is equivalent to an MMSE estimator followed by an MMSE coder. The coders for the Poisson noise and the film-grain noise cases are derived and their performance is studied. The effect of this preprocessing step is studied using standard coders, e.g., JPEG, also. As is demonstrated, higher quality is achieved at lower bit rates.

A fast exhaustive search algorithm for rate-constrained motion estimation is presented. The motion vectors are selected from a search window based on a rate-distortion criterion by successively eliminating the search positions depending on the rate constraint. The estimation performance of the proposed algorithm is identical to the performance of the rate-constrained full search algorithm, with considerable reduction in computation. Simulation results indicate that the number of matching calculations decreases as the constraint on the rate increases.

In this paper, we investigate the MPEG-2 temporal scalability syntax and introduce a new approach to temporally scalable coding. Temporal scalability is provided by employing various nonlinear prediction and demultiplexing schemes. A nonlinear deinterlacing algorithm is presented and the related issues on interlaced, progressive and mixed mode video processing are addressed. In addition to the considered scalability techniques, a lookahead quantization scheme is presented for P- and B-type picture coding, which improves the coding performance by selective combination of the DCT domain scalar quantization and entropy-constrained vector quantization. Remarkable performance improvement over the simulcast coding is achieved.

A perceptual video distortion measure system based on the human vision model is presented. This system is an extension of the still perceptual distortion measure system. One advantage of the distortion measure is that the distortion can be weighted for frames in the vicinity of scene cuts. Our video distortion measure also requires less computation compared to other approaches. The perceptual distortion measure has wide use in video processing and applications, such as in the selection of the quantization matrices, the selection of the mquant parameter, and as a criterion for mode decision in MPEG encoders. Simulation results are presented.

A computationally efficient exhaustive search algorithm for rate-constrained motion estimation is presented. The motion vectors are selected from a search window based on a rate-distortion criterion by successively eliminating the search positions depending on the rate constraint. The performance of the proposed algorithm is identical to the performance of the rate-constrained full search algorithm, with considerable reduction in computation. The number of matching calculations decreases as the constraint imposed on the rate increases.

In this paper the two-dimensional and three-dimensional video perceptual distortion measure systems based on the human vision model are presented. These systems are extensions of the still perceptual distortion measure system. One advantage of the two-dimensional video distortion measure is that the distortion can be weighted for frames in the vicinity of scene cuts. The two-dimensional video distortion measure also requires less computation compared to the three-dimensional video distortion measure. Simulation results are presented.

For many video compression systems, overlapped block motion compensation (OBMC) provides an effective extension of traditional block MC (BMC). However, because OBMC causes noncausal spatial interaction between blocks, iterative methods have typically been used for motion estimation (ME) for OBMC. For compression purposes, the rate for the motion vector field may also be constrained. This paper considers several rate-constrained ME methods for OBMC. Experiments demonstrate that a simple raster-scan algorithm is effective as a suboptimal non-iterative solution, with better rate-distortion performance and equal or lower computational complexity than several iterative methods.

One emerging application which exploits the correlation between audio and video is speech driven facial animation. The goal of speech driven facial animation is to synthesize realistic video sequences from acoustic speech. Much of the previous research has implemented this audio to visual conversion strategy with existing techniques such as vector quantization and neural networks. We examine how this conversion process can be accomplished with hidden Markov models.

This article presents a new lossless compression algorithm for computer animation image sequences. The algorithm uses transformation information available in the animation script and floating point depth and object number information at each pixel to perform highly accurate motion prediction with vary low computation. The geometric data (i.e., the depth and object number) can either be computed during the original rendering process and stored with the image or computed on the fly during compression and decompression. In the former case the stored geometric data are very efficientlycomporessed using motion prediction and a new technique called direction coding, typically to 1 to 2 bits per pixel. The geometric data are also useful in z-buffer image compsiting and this new compression algorthm offers a very low storage overhead method for saving the information needed for this comoositing. The overall compression ratio of the new algorithm, including the geometic data overhead, in compared to conventional spatial linear prediction compression and block-matching motion. The algorithm improves on a previous motion prediction algorithm by incorporating block predictor switching and color ratio predition. The combination of thes techniques gives compression ratios 30% better than those reported previously.

The evaluation of medical image compression algorithms differs from the evaluation of compression algorithms of other classes of images. It is more concerned with preserving the diagnostic information in the image. Moreover, cardiac SPECT images are usually corrupted by Poisson noise which severely degrades the performance of any compression algorithm. This paper presents a region-based approach to lossy compression of noisy cardiac medical images. First, noisy source coding theory is applied to this problem and the noise is filtered optimally. Afterwards, region-based quantization is applied to the image. The region of the heart and the organs are not quantized in order to preserve the diagnostic information in these images.

We propose to use video-codec framework for the compression of volumetric medical images by treating interslice variations as motion of the tissues. As compression is necessary for telemedicine and archive, such an approach needs only slight modifications of existing codecs and is able to re-use most of the existing software and hardware. This is our main motivation and we seek modifications of the MPEG codec. An obvious modification is to replace block matching (BM) by more sophisticated methods. Specifically, we use the affine transformation (AT) on triangular patches. We implemented and compared our scheme with a simplified MPEG-1 encoder under identical parameters on a set of MRI images of a human hand. The blockiness is reduced dramatically and structures are better preserved. The images also appear less granular and show improvements of several dB in SNR. We expect further improvements when more features of MPEG codec are incorporated in our ongoing work.

A multidimensional probabilistic modeling technique is proposed and its applications to video coding is presented. The probability distribution of two adjacent pixels in subband signals are modeled by bivariate generalized Gaussian distributions (GGD). Quantizer design, bit allocation and entropy coding is based on this probability distribution function (PDF) model. We show that use of bivariate generalized Gaussian PDF models yields a significant improvement in rate-distortion performance over univariate GGD models.

A multiscale blind image restoration scheme using a wavelet decomposition is presented. The proposed algorithm uses a wavelet decomposition to decompose the degradation model g=Df+w into smaller subband degradation models. Although the subband degradation models are in fact coupled together, our procedure decouples them by estimating the coupling error e/sub s/(D,f) and subtracting the estimate from the subband degraded image g/sub s/. The expectation maximization (EM) algorithm is then applied to each subband signal to estimate simultaneously the subband blur operator D/sub s/ and the original undegraded subband image f/sub s/. Due to the well-posed condition of the subband blur operator, common ringing artifact is suppressed and regularization which trades fidelity for stability is not necessary.

Noise degrades the performance of any image compression algorithm. This paper studies the effect of film-grain noise on lossy image compression. Since the goal of lossy image compression algorithms is to achieve the best fidelity for a given bit rate, the distortion is measured with respect to the original image not with respect to the input to the coder. The results of noisy source coding are used to develop this coder. The minimum-mean-squared-error (MMSE) coder involves MMSE restoration of the noisy image. This paper presents an MMSE image restoration algorithm based on modeling the image as a Markov random field. The performance of this preprocessing step is also studied when using JPEG.

In this paper, a video quality measure based on block-based features which has a high correlation with subjective test experiments is presented first. Four block-based features (the average of DFT differences, the standard deviation of DFT differences, the mean absolute deviation of wepstrum differences, and the variance of UVW differences) are extracted from the input and output video sequences and fed into a four-layer backpropagation neural network that has been trained by subjective testing. These features were selected out of a much larger set of candidate features, based on their combined ability to predict observer assessments of image quality. The quality measure is then introduced into the design of an MPEG encoder which is based on the nonconvex quality and bitcount functions at the macroblock level. The joint rate, quality, motion estimation and motion compensation algorithm is discussed. The proposed quality measure scheme has a higher, and also a more consistent, quality measure than the MPEG Test Model 5 scheme. The objective quality, as measured by the signal-to-noise ratio is also improved, and the variations in the signal-to-noise ratio and bit-rate from frame to frame are reduced. These results have also been confirmed by subsequent subjective test experiments.

In transform image coding, the histograms of transform coefficients can be approximately modeled by generalized Gaussian (GG) random variables. However, the GG models may not fit the DC distribution. One approach uses DPCM for the DC data, which greatly complicates bit allocation; another assumes a single Gaussian (SG) model, which may be a poor model. As an alternative, this paper proposes a finite Gaussian mixture (GM) model for the DC data. The GM approach does not require tweaking of the DPCM quantizer stepsize and can allocate bits optimally between the DC and AC data; it is also more flexible than the SG model. Experimentally, the GM method matched DPCM at medium rates and gave 1-5 dB higher PSNR at low and high rates. The GM method also matched the performance of the SG model and gave 0.5-2 dB higher PSNR when the SG assumption failed.

A nonlinear method for image interpolation is presented based on spatial domain directional interpolation. Existing directional interpolation algorithms, which only consider edge regions, are extended to the whole image by interpolating in multiple directions. The interpolated values along various directions are combined using directional weights, which depend on the variation in that direction. The interpolation value for each direction is assigned based on the magnitude of its directional derivative.

We present an video objective quality measure which has good correlation with subjective tests. We then introduce the objective measure into the design of an MPEG encoder. The new MPEG encoder extracts four features (bit rate, a feature that measures blockiness, one that measures false edges, and one that measures blurred edges) from the input and output video sequences and feeds those features into a four-layered back-propagation neural network which has been trained by subjective testing. Then the system uses a simple feedback technique to adjust the GOP (group of pictures) bit-rate to achieve a constant subjective quality output video sequence.

An iterative post-processing algorithm for reducing blocking and ringing in block transform-coded images is presented. It smooths the decompressed image and uses convex projections theory to preserve the image's edges and bit-stream. The algorithm exploits the coding scheme's block structure and ringing behavior. In experiments it outperforms an existing method and gives significant subjective and objective improvement over the decompressed image. Sufficient conditions for algorithm convergence are given.

We present a video quality measure based on block-based features. Four block-based features (an average of FFT differences, a standard deviation of FFT differences, mean absolute deviation of wepstrum differences, and the variance of UVW differences) are extracted from the input and output video sequences and fed into a four-layer backpropagation neural network which has been trained by subjective testing. We then introduce the quality measure into the design of an MPEG encoder to maximize the quality measure. A MPEG bit allocation algorithm is developed which is based on the quality measure that considers the bit-count and quality at the macroblock level. A simple relationship between one macroblock's quality and overall quality is also addressed. The simulation on this optimization scheme has a higher quality measure when compared to the Test Model 5.

Hidden Markov modeling has proven extremely useful for statistical analysis of speech signals. There are, however, inherent problems in two dimensional extensions to HMMs, one of which is the exponential complexity associated with fully 2-D HMMs. We propose a new 2-D HMM-like structure obtained by embedding states within regions of a deformable template structure. With this state-embedded deformable template (SEDT), each region of a deformable template has an underlying observation probability distribution. This structure allows for computation of the P[image/template]. The template that maximizes this probability provides an optimal segmentation of the image. This segmentation capability is demonstrated in facial analysis applications.

Image block transform techniques usually introduce several types of spatial periodic distortion which are mostly noticeable at low bit rates. One way to reduce these artifacts to obtain an acceptable visual quality level is to postfilter the decoded images using nonlinear space-variant adaptive filters derived from the structural relationships and residual spectral information provided by the discrete-time Fourier transform (DTFT) of block transforms such as the discrete cosine transform (DCT) and the lapped orthogonal transform (LOT). A method for analyzing and filtering the DCT blocking noise and the LOT ringing noise for moderate and highly compressed images is described and several test cases are presented. A generalized Fourier analysis of the block transform distortion as seen in the frequency domain is discussed in conjunction with an outline of a separable adaptive postfiltering algorithm for decoded image enhancement.>

A hierarchical coder (HC) is constructed from a number of simple coders for image blocks arranged in order of increasing bit rate and reproduction quality. This paper discusses several aspects of hierarchical coders, including: a general description of the structure of a HC, its advantages and important design considerations, a discussion of how threshold selection can affect the performance of the coder through a simple example, and an introduction of a simple procedure for selecting optimal (in the rate distortion sense) decision thresholds which control the levels of the hierarchy.>

This paper presents a multi-frame pel-recursive two-dimensional motion estimation scheme for video image interpolation. The proposed approach uses more information from the image sequences than previous approaches, and gives a closed-form solution to update motion vectors among frame pairs simultaneously. It is suitable for object motion with acceleration since it places no constraints on the motion vectors among consecutive frames.>

An adaptive video coder is presented. The coder is based on motion compensation and subband coding. The probability distributions of the subband signals are modeled by generalized Gaussian distributions. Quantizer design, bit allocation, and entropy coding are then based on these distribution models. The video coder adapts to changes in signal characteristics by updating the distribution models and redesigning the quantizers and entropy codes.>

This paper presents a new lossless compression algorithm for computer animation image sequences.The algorithm uses transformation information available in the animation script and floating point depth and object number information stored at each pixel to perform highly accurate motion prediction with very low computation.The geometric data, i.e., the depth and object number, is very efficiently compressed using motion prediction and a new technique called direction coding, typically to 1 to 2 bits per pixel.The geometric data is also useful in z-buffer image compositing and this new compression algorithm offers a very low storage overhead method for saving the information needed for z-buffer image compositing.The overall compression ratio of the new algorithm, including the geometric data overhead, is compared to conventional spatial linear prediction compression and is shown to be consistently better, by a factor of 1.4 or more, even with large frame-to-frame motion.

A software procedure is presented for fully automated detection of brain contours from single-echo 3-D MRI data, developed initially for scans with coronal orientation. The procedure detects structures in a head data volume in a hierarchical fashion. Automatic detection starts with a histogram-based thresholding step, whenever necessary preceded by an image intensity correction procedure. This step is followed by a morphological procedure which refines the binary threshold mask images. Anatomical knowledge, essential for the discrimination between desired and undesired structures, is implemented in this step through a sequence of conventional and novel morphological operations, using 2-D and 3-D operations. A final step of the procedure performs overlap tests on candidate brain regions of interest in neighboring slice images to propagate coherent 2-D brain masks through the third dimension. Results are presented for test runs of the procedure on 23 coronal whole-brain data sets, and one sagittal whole-brain data set. Finally, the potential of the technique for generalization to other problems is discussed, as well as limitations of the technique.

A strategy for a knowledge-based system to detect the interior and exterior boundaries of the left ventricle from time-varying cross-sectional images obtained by ECG-gated magnetic resonance imaging (MRI) is discussed. The system uses both fuzzy set theory and Dempster and Shafer theory to manage the knowledge and to control the flow of system information for more efficient use of limited computational resources and memory space. The key to the approach is that it performs edge detection on images through integration and unification of knowledge and information from edge candidates on all the slices and phases of the acquired cardiac MRI dataset. The analysis system does not base decisions on individual measurements, but on consensus opinions by combining many knowledge sources, some of which may not be completely accurate.

A pel-recursive two-dimensional motion estimation algorithm using three or more image frames is presented. It uses more information from the image sequences and gives a closed-form solution to update motion vectors among frame pairs simultaneously. Compared to other multiframe-based algorithms, the proposed algorithm is in a more general form. It is suitable for object motion with acceleration since it places no constraints on the displacement vectors among consecutive frames.>

Two-source decomposition image coders consist of coding algorithms in which a boundary-based interpolation is subtracted from each block of the image to obtain a residual signal that is encoded and transmitted. These coders effectively reduce the objectionable blocking distortion that often accompanies block-by-block coding strategies and, for this reason, are becoming an attractive alternative to traditional transform coders in low bit-rate applications. The authors enhance the appeal of this approach by introducing a three-source decomposition scheme that increases the fidelity of the boundary response or, equivalently, decreases the informational content of the residual. This decomposition permits the development of an interpolative coder capable of producing perceptually good quality images at low bit rates without the disturbing blocking effect.>

A new formulation of the system equations that describe the behavior of the one-dimensional M-band maximally decimated analysis/synthesis filter bank system is presented. The development incorporates the effects of the shift-varying nature of the upsampling and downsampling operations. By explicitly including the filter regions of support, one is able to derive a more general expression for the relationships that must exist between the analysis and synthesis filters in order to cancel all interband aliasing. From this expression one can find (1) the M/sup M-1/ distinct sets of analysis and synthesis filters from which an M-band alias-free filter bank can be constructed, and (2) the constraints on how these filters, once designed, can be shifted without destroying their aliasing cancellation properties.>

Constraints provide an important means of incorporating a priori information into the image restoration process. However, much of the information available for constructing constraints is of a tentative nature. If the validity of this tentative information can be assessed before it is incorporated into the solution, helpful constraints can be retained while harmful ones can be discarded. Cross validation is introduced as a technique for assessing the validity of such constraint sets. Because the full cross validation procedure is computationally burdensome, a modification is suggested that allows a more feasible implementation without substantially sacrificing the performance of the full procedure. Experimental results demonstrate the excellent performance of both the full and modified procedures.

The point spread function (PSF) of a blurred image is often unknown a priori; the blur must first be identified from the degraded image data before restoring the image. Generalized cross-validation (GCV) is introduced to address the blur identification problem. The GCV criterion identifies model parameters for the blur, the image, and the regularization parameter, providing all the information necessary to restore the image. Experiments are presented which show that GVC is capable of yielding good identification results. A comparison of the GCV criterion with maximum-likelihood (ML) estimation shows the GCV often outperforms ML in identifying the blur and image model parameters.

The notion of periodically deleting samples from a discrete image without information loss is reviewed. The set of signals that satisfy the deletion theorem is generalized to a class that is closed under convolution and hence any filtered version of a member of the class is also a member. A polyphase type structure is developed that allows FIR filtering of the image with deleted samples. An example is then given which shows that there are savings in computer memory and computation when this method is used.>

A new computational approach is presented for the estimation of displacement fields along contours of moving regions in image sequences. The proposed scheme uses frame differences to determine moving edges, which are then expressed as straight line tokens. Finally, an edge propagation algorithm is used to determine the distance between these moving edge tokens and the frame difference region boundaries to approximate the displacement fields. The performance of the system is evaluated with a real video image sequence.>

A novel model for one-dimensional signal analysis, called a weighted multiresolution process (WiMP), is introduced. The model combines the scale and time-frequency localization properties of the wavelet representation with the self-affine characteristics of signals to be modeled. A WiMP can be viewed as a Markov chain of weighted translations between levels of the wavelet decomposition. Both aperiodic and periodic signals are investigated, and the corresponding decomposition/reconstruction algorithms are presented.>

A method for obtaining optimal estimates of the regularization parameter for iterative image restoration in a weighted Hilbert space is presented. The method is a modified form of the cross-validation technique. It requires no a priori knowledge of either the amount of noise or its distribution. Results are presented which demonstrate the effectiveness of the technique for different weighting matrices. Cross-validation is based on the concept of prediction error. For a fixed value of the regularization parameter, a restored image is determined using all the values from the observed image but one, and the restored image is reblurred in order to predict the blurred and noisy observation that was left out of the restoration. A different restored image is formed for each observation. The regularization parameter which minimizes the mean-square prediction error over all the observations is chosen as the estimated optimal parameter. A simplified version of cross-validation that is computationally manageable is presented.>

A knowledge-based system is described for boundary detection from magnetic resonance image sequences of a beating heart. It is shown that the Dempster/Shafer theory and fuzzy set theory can be used for control of the system as well as for labeling objects in the images. The quality of the images is measured for ordering the images in boundary detection. The performance of the various knowledge sources is evaluated for learning from experience. These approaches provide the system with top-down feedback functions that can be utilized for more efficient use of limited computational resources. The basic structure of the system and the function of each component is described. The performance of the components is presented with some examples.>

The Kohonen self-organizing feature map algorithm is used to design hypercubically structured codebooks for memoryless vector quantization at a bit rate below 0.75 b/pixel for 512*512 monochrome still images. A decimated-search method that searches only a fraction of the codebook during both the training and the encoding processes is introduced. The effectiveness of the design is demonstrated by comparing the codebooks with those designed by the Linde-Buzo-Gray (LBG) algorithm. The coded images are seen to have similar objective and perceptual quality as those encoded by the LBG codebooks at a fraction of the search complexity.>

Because of the presence of noise in blurred images, an image restoration algorithm must constrain the solution to achieve stable restoration results. Such constraints are often introduced by biasing the restoration toward the minimizer of a given functional. However, a proper choice of the degree of bias is critical to the success of this approach. Generally, the appropriate bias cannot be chosen a priori and must be estimated from the blurred and noisy image. Cross-validation is introduced as a method for estimating the optimal degree of bias for a general form of the constraint functional. Results show that this constraint is capable of improving restoration results beyond the capabilities of the traditional Tikhonov constraint.

The dual approach to signal restoration was studied extensively by R.M. Leahy and C.E. Goutis (1986), who did not provide an explicit formula to generate the signal models and their dual problems. The present authors complement that work by providing explicit formulas and correct some of the dual functionals published. They also discuss several easy ways to apply sufficient conditions under which the dual approach is applicable and discuss what would happen to the dual functional if the measurements were not restorable.>

The authors had previously proposed a subband coding method for color images that used vector quantization and AD-PCM (pulse code modulation) (see C.S. Kim et al., Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing, p.753-6, Apr. 1988). High-quality coded images were achieved at bit rates of 0.6 bits/pixel for 512*512 color images. In the present work, the authors discuss an improved system that enables further reduction in the bit rate. The proposed approach is no longer a subband coder in the true sense, but rather a multirate VQ/DPCM (differential pulse code modulation) system. The approach is based on a three-component multirate representation of the image, composed of a chrominance subimage pair, a medium- to low-frequency luminance image, and a high-frequency luminance residual. FSVQ (finite-state VQ) and DPCM are used strategically in coding these components, with noticeable improvement over previous results.>

The authors describe a knowledge-based system for detecting the interior and exterior boundaries of the left ventricle (LV) from time-varying cross-sectional images on the beating heart obtained noninvasively by magnetic resonance imaging (MRI). The goal is a system for automatically classifying and measuring cardiac function through estimates of LV wall thickness, wall motion, etc. This knowledge-based system makes use of the Dempster and Shafer (D/S) theory to manage the knowledge. The theory is also used to control the flow of system information for more efficient use of limited computational resources and memory space.>

Subband coding (SBC) has been shown to be an effective method for coding monochrome images at low bit rates. SBC systems decompose the input image into a small number of decimated, spectrally disjoint subband images which are then coded by a variety of means. Reconstruction is performed by decoding and merging the reinterpolated subband images. This work investigates the application of the subband concept to the efficient coding of color images. The color representations are cast into a subband coding framework to form color image coding systems. Differential PCM and finite-state vector quantization methods are used in coding the subband images. Bit rates below 1 bit/pixel are obtainable for high-quality representations of 256*256 color images.>

Edge detection in sampled images may be viewed as a problem of numerical differentiation. In fact, most point edge operators function by estimating the local gradient or Laplacian. Adopting this view, Torre and Poggio [2] apply regularization techniques to the problem of computing derivatives, and arrive at a class of simple linear estimators involving derivatives of a low-pass Gaussian kernel. In this work, we further develop the approach by examining statistical properties of such estimators, and investigate the effectiveness of various combinations of the partial derivative estimates in detecting blurred steps and lines. We also touch briefly on the problem of sensitivity to various types of edge structures, and develop an isotropic operator with reduced sensitivity to isolated spikes.

In [1] a regularized iterative algorithm was described which has been shown to be very suitable for solving the ill-posed image restoration problem. By incorporating deterministic constraints and adaptivity this very general algorithm is capable of achieving both noise suppression and ringing reduction in the restoration process. It consumes, however, considerable computation to obtain a (visually) stable solution due to the low convergence speed of the algorithm. The purpose of this paper is to investigate the possibilities for speeding up the convergence of this restoration method. To this end we compare the classical steepest descent algorithm (with linear convergence) with a conjugate gradients based method (superlinear convergence) and a new Q-th order converging algorithm. The latter solution method has the highest convergence rate, but is restricted in its application to space-invariant image restoration with a linear constraint. Although the actual convergence speed of the algorithms involved generally depends on the image data to be restored, it will be shown that for real-life images the constrained conjugate gradients algorithm yields a considerable convergence speed improvement.

Burg's maximum entropy method has been used with success in spectral estimation. This paper is an attempt to generalize the maximum entropy method to the deconvolution of positive signals from a finite set of linear measurements. In this paper, we also investigate the existence of a solution to the deconvolution problem using a geometric approach. By formulating the deconvolution problem as an unconstrained optimization problem, our procedure will provide the maximum entropy deconvolved positive signal when it exists. Otherwise, the algorithm diverges.

A method for implementing the indexing step of multidimensional FFTs is presented. The initial arrangement of the multidimensional data bears a heavy effect on the rearranging of the data. The proposed method seeks an appropriate geometry for the data which allows partial decoupling between the different dimensions. The result is a set of linear integer equations which is solved using the Smith normal factorization of the periodicity matrix. The hexagonal indexing algorithm is given as an illustration.

This paper introduces different types of nonstationary constrained iterative image restoration algorithms. The adaptivity of the algorithm is introduced by the constraint operator which incorporates properties of the response of the human visual system. The properties of the visual system are represented by noise masking and visibility functions. A new way of computing the masking function is also introduced. The proposed algorithms are general and can be used for any type of linear constraint and distortion operators. The algorithms can also be used to restore signals different than images, since the constraint operator can be interpreted as adapting to the local signal activity.

This paper introduces a general formulation of constrained iterative restoration algorithms in which deterministic and/or statistical information about the undistorted signal and statistical information about the noise are directly incorporated into the iterative procedure. This a priori information is incorporated into the restoration algorithm by what we call "soft" or statistical constraints. Their effect on the solution depends on the amount of noise on the data; that is, the constraint operator is "turned off" for noiseless data. The development of the new iterative algorithm is based on results from regularization techniques for stabilizing ill-posed problems.

This paper addresses the problem of segmenting an image by texture. Specifically, we are concerned with estimating the trajectory of the boundary between two regions characterized by different two-dimensional autocorrelation functions. An algorithm is developed which is an approximation to a maximum-likelihood estimator based on the assumption of Gaussian random fields with known mean and covariance. Preliminary experimental results are given.

This paper introduces a new iterative image restoration method which is capable of restoring noisy, blurred images by incorporating a priori knowledge about the image and noise statistics into the iterative procedure. The iteration equation consists of a prediction part which is based on a noncausal image model description and an innovation part which is weighted by a gain factor. The gain is computed using a linear MSE optimization procedure and is updated at each step of the iteration. This image restoration scheme can be interpreted as an iterative procedure with a statistical constraint on the image data.

This paper reports on a preliminary study of applying single-channel (scalar) and multichannel (vector) 2-D linear prediction to color image modeling and coding. Also, the novel idea of a multi-input single-output 2-D ADPCM coder is introduced. The results of this study indicate that texture information in multispectral images can be represented by linear prediction coefficients or matrices, where as the prediction error conveys edge-information. Moreover, by using a single-channel edge-information we obtained, from original color images of 24 bits/pixel, reconstructed images of good quality at information rates of 1 bit/pixel or less.

In this paper we present results from subjective tests done on a data base of distorted images where the distortion took the form of additive colored Gaussian noise. The results from these tests give further support for a filter bank model of the human visual system. The results also suggest that coding schemes employing noise-shaping techniques can give better results than ones that do not. An objective quality measure LINLNSNR is developed based on the subjective evaluations which correlates well with human performance.

This paper is concerned with the use of 2-D linear prediction for image segmentation. It begins with a brief summary of the mathematics involved in 2-D linear predictive analysis of arbitrarily-shaped regions. Then, it introduces a 2-D LPC distance measure based on the error residual of 2-D linear prediction. Finally, it describes how the above results can be applied to image segmentation using a simple cluster seeking algorithm. The results indicate that arbitrarily-shaped image regions can be well identified and clustered using as features their 2-D LPC parameters.

Recent work by Mersereau and Speake [1,2] has shown that multidimensional discrete Fourier transforms (DFTs) can be defined for signals defined on any periodic sampling lattice and that they can be evaluated using a generalization of the Cooley-Tukey FFT algorithm. The main purpose of this work was to develop alternative algorithms which were more suitable to highly parallel machine architectures and which required less data handling than the Cooley-Tukey algorithms. Such an algorithm is described here. It makes use of the Smith normal form representation of an integer matrix. As a sidelight to this work a Chinese remainder theorem for lattices has been developed which permits an extension of Good's prime factor algorithm. This is also described.

This paper is a preliminary report on a study of the application of two-dimensional linear prediction in image quantization. The study has focused on three major concerns: implementation of an adaptive linear predictor, adaptive quantization of the prediction error signal, and the adaptive predictive coding of density (logarithm of intensity) images. The results of the study indicate that through the use of adaptive prediction and quantization, a high level of image fidelity can be obtained for both intensity and density images at information rates well below one bit/pixel.

In this paper the problem of contrained deconvolution is related to the minimization of a functional. By applying the conjugate gradient algorithm of optimization theory to this minimization an efficient iterative algorithm for deconvolution is developed. This algorithm is then extended to perform constrained deconvolution and the performance of the new algorithms is compared with other published algorithms.

In this paper two-dimensional fast Fourier transforms (FFT's) are expressed as special cases of a generalization of the one-dimensional Cooley-Tukey algorithm. This generalized algorithm allows the efficient evaluation of discrete Fourier transforms (DFT's) of rectangularly sampled sequences, hexagonally sampled sequences and arbitrary periodically sampled sequences. Significant computational savings can be realized using this generalized algorithm when the periodicity matrix of the sequence is highly composite. Alternate factorizations of the periodicity matrix lead to different FFT algorithms, including the row-column decomposition and the vector-radix algorithm. This paper will present a generalized DFT, derive the general 2-D Cooley-Tukey algorithm and conclude by interpreting several 2-D FFT algorithms in terms of the generalized one.

Two-dimensional signals are usually represented for the purpose of digital processing as rectangularly sampled functions of two orthogonal independent variables. It has been previously noted that sampling a signal on a hexagonal grid can offer substantial savings in digital storage and computation. In this paper these two sampling schemes will be generalized, via a matrix description, to include arbitrarily sampled 2-D signals. Using this matrix description and a generalized discrete Fourier transform, a technique will be presented for interpolating a set of sample points from one sampling grid to an alternative one. Since the analog signal from which the original samples were obtained is frequently unavailable for resampling, the ability to easily convert from one sampling scheme to another can be important in the efficient processing of a particular signal.

This paper presents a comparison of two-dimensional finite area impulse response (FIR) digital filters designed using three popular design methodologies - through the use of windows, through the use of transformations of one-dimensional designs, and through the use of optimal Chebyshev design techniques. In addition the comparison includes filters designed for processing both rectangularly-sampled and hexagonally-sampled data. The filters are compared with respect to ease of design and the efficiency of the resulting implementation.

Seventy-two 2-D recursive and nonrecursive filters of varying order are designed to approximate, in the least-squares sense, four circularly-symmetric unit-pulse responses, corresponding to four lowpass filters. The approximation error obtained for a given number of filter coefficients is then used as the basis for comparing the efficiency of the designed 2-D recursive and nonrecursive filters.

There has recently been considerable interest in iterative deconvolution algorithms. Some of these algorithms seek to improve performance by incorporating á priori knowledge of the components of a convolution. For example, incorporation of both positivity and finite interval constraints tends to automatically introduce an extrapolation of the bandwidth. A question which naturally arises concerns the effect of noise on the performance of such algorithms. In this paper we describe an experimental evaluation of the effect of noise using synthetic data. Results of extensive tests are presented and some suggestions for mitigating the effects of noise are offered.

In a pulsed-doppler radar system the return signal is naturally viewed as a sequence of samples of one or more complex sinusoids plus noise. We have studied several digital processing techniques for estimating a short-time spectral representation of such signals as well as digital techniques for estimating the frequency of a single dominant spectral component. Techniques considered include: (1) Discrete short-time Fourier analysis followed by peak-picking and median smoothing to remove errors. (2) Short-time maximum entropy spectrum estimation incorporating adaptive noise suppression. (3) A new recursive algorithm based on recursive computation of the short-time auto-correlation function. The relative merits of each technique are discussed and examples of performance are given.

An iterative method for the space-domain design of two-dimensional recursive digital filters is presented. The design method minimizes a weighted mean-square-error between the prototype impulse response and the impulse response of the designed filter. A slight modification to the procedure permits the design of filters with a separable denominator in the transfer function.

Two-dimensional windows are generally derived by beginning with a one-dimensional window as a prototype and either rotating that window in the 2-D plane to produce a window with a round region of support or forming the outer product of that window with itself to produce a window with a square region of support. In this paper we present a comparison of results obtained using these two window formulations for 2-D FIR filter design. Design formulas are produced for estimating filter order for a given set of frequency domain specifications when a one-dimensional Kaiser window is used as the prototype window. The results suggest some interesting differences from the one-dimensional case.

Two-dimensional signals are normally sampled and processed as rectangular arrays. For signals which are bandlimited with a circular region of support in the Fourier plane, however, it has been known for some time that a savings in the number of samples required for an exact reconstruction can be realized by sampling the signal on a hexagonal sampling raster. In this presentation we show that a substantial savings in computation can result as well. Included are methods for signal representation, linear system implementation, Fourier transform computation and FIR filter design. Some comparisons between systems defined over rectangular and hexagonal sampling rasters will also be given.

It is shown that a number of well known iterative deconvolution algorithms are closely related members of a general class. This point of view leads to new understanding of previous approaches and to a new algorithm that incorporates both a finite interval constraint and a positivity constraint on the output of the deconvolution process. Examples are given to illustrate performance on gamma-ray spectra.