Accurate and robust feature matching is a critical issue in the preprocessing of multispectral image data sets. Higher order features, such as lines can provide useful matching information but are heavily affected by the unreliable detection of lines. Existing methods typically make the unrealistic assumption that end points of lines can be accurately detected across the reference and test images. To address the unreliable detection of line end points, this paper proposes mapping line intersections and then employing tentatively mapped intersections as “anchor” points to compute line descriptors. The computed line descriptors are utilized to determine whether the two lines forming an intersection are matched with the two lines forming its mapped intersection. This eliminates the reliance on the accurate detection of line end points and results in improved matching accuracy. The proposed method is tested on a large number of multispectral images containing various scenes. Experimental results show that it can effectively deal with the detection inaccuracy of end points for line matching.

Merging short-exposure frames can provide an image with reduced noise in low light conditions. However, how best to align images before merging is an open problem. To improve the performance of alignment, we propose an inertia-sensor aided algorithm for smartphone burst photography, which takes rotation and out-plane relative movement into account. To calculate homography between frames, a three by three rotation matrix is calculated from gyro data recorded by smartphone inertia sensor and three dimensional translation vector are estimated by matched feature points detected from two frames. The rotation matrix and translations are combined to form the initial guess of homography. An unscented Kalman filter is utilized to provide a more accurate homography estimation. We have tested the algorithm on a variety of different scenes with different camera relative motions. We compare the proposed method to benchmark single-image and multi-image denoising methods with favourable results.

This paper proposes a line segment-based image registration method. Edges are detected from images by a modified Canny operator, and line segments are then extracted from these edges. At registration, triplets (quaternions) of line segment correspondences are tentatively formed by applying the distance and orientation constraints, which determine an intermediate transformation. Those triplets (quaternions) of lines resulting in higher similarity metrics are preserved, and their intersections are refined by an iterative process or random sample consensus. The proposed method is tested on indoor and outdoor EO/IR image pairs, and the average registration error is calculated to be compared with existing algorithms. Experimental results show that the proposed registration method can robustly align EO/IR images containing line segments, providing more reliable and accurate registration results on multimodal images.

Scene depth variation is an important factor that leads to spatially-varying camera motion blur. Most of the previous methods require auxiliary cameras or user interaction to make depth-aware deblurring tractable. In this work, we propose to use a noisy/blurred/noisy image sequence and simultaneously recorded inertial measurements to jointly estimate scene depth and remove spatially-varying blur caused by depth variation and camera in-plane motion. The inertial data could provide initialization of camera motion parameters, while the noisy image pair preserve large-scale sharp edges from which a coarse disparity map can be generated. However, this initial estimate is not accurate enough to produce a high-quality clean image. Therefore, we develop an optimization scheme to refine depth, motion parameters and latent image alternately. A Markov Random Field (MRF) framework is formulated to solve for the depth map by exploiting both stereo cues and motion blur cues and the residual Richardson-Lucy algorithm is used to effectively suppress deconvolution ringing artifacts. Experimental results demonstrate that our approach can address both depth estimation as well as image deblurring.

In the case of motion blur due to unknown motion, most of the existing image deblurring algorithms rely on good initial estimate of the kernel or latent image obtained through blind deconvolution and only consider 3-dimensional camera motions. To overcome these problems, Joshi [1] presented a novel blur kernel estimation and image deblurring approach by integrating 6-dimensional inertial sensors with a camera. However, the drift in the estimated camera motion path introduced by inertial measurement noise was not well handled in Joshi's work. In this paper, we propose an alternating optimization scheme to move the drifted camera motion path to the correct position. The camera pose space in the projective motion blur model is replaced by a motion path set to compensate for path drift. Experiments are performed on synthetic and real images to show the effectiveness of our approach.

This paper presents a similarity metric on multimodal images utilizing curves as comparing primitives. Curves are detected from images, and then junctions are detected along curves and used to partition curves into subcurves. A modified Hausdorff distance is applied to determine whether a test subcurve is matched to a reference curve. The similarity metric is defined to be the number of matched curves. The number of overlapped edge pixels between two images is also defined on the basis of matched curves, which does not require accurately localizing edge pixels. The partitioning scheme avoids addresing curve partial matching and allows for test subcurves being matched to a reference curve if they correspond to each other. Experimental results show that the presented similarity metric gives more robust and reliable results, especially under noise.

The Seidel aberration model has proven an invaluable tool in the design of optical systems [1, 2], by providing a model that bridges the gap between first-order optics models and actual system performance. However, it has been largely neglected in the modeling of blur kernels, despite the ability of the model to accurately predict the point-spread function (PSF) of an optical system due to aberration and defocus. As a step towards developing a parameterized, spatially-varying PSF model, we propose a novel mapping to Seidel aberration coefficients from observable spread measures of discretely-defined PSFs. We demonstrate through simulation that this mapping, and the associated estimation algorithm, allow a noisy dataset of 120 PSF observations (consisting of thousands of degrees of freedom) to be unified under a single 4-parameter model.

This paper describes a studentized dynamical system (SDS) for robust target tracking using a subspace representation. Dynamical systems (DS) provide a powerful framework for the probabilistic modeling of temporal sequences. Visual tracking problems are often cast as a sequential inference problem within the DS framework and a compact way to model the observation distributions (i.e., object appearances) is through probabilistic principal component analysis (PPCA). PPCA is a classic Gaussian based subspace representation method and a popular tool for appearance modeling. Although Gaussian density has theoretically nice properties, resulting in models that are always tractable, they are also severely limited in practical settings. One of the central issues in the use of PPCA for target appearance modeling is that it is very sensitive to outliers. The Gaussian density has a very light tail, while real world data with outliers exhibit heavy tails. Recently, more heavy-tailed distributions (e.g., Student's t-distribution) have been introduced to increase the robustness of the original PPCA. We propose to augment the traditional target tracking DS by adding a set of auxiliary latent variables to adjust the shape of the observation distribution. We show that by carefully choosing the probability density of these auxiliary latent variables, a more robust observation distribution can be obtained with tails heavier than Gaussian. Numerical experiments verify that the proposed SDS has a better capability to handle considerable amount of outlier noise and an improved tracking performance over DS with a Gaussian based observation model.

In this paper, an alternative efficient energy functional is proposed for estimating optical flow. The problem of optical flow estimation is posed as one of solving for the minimum configurations of its two components in an L1 total variation framework augmented with a p-harmonic energy functional. A novel smooth term based on a p-harmonic map energy is added as a regularization constraint in the optical flow computation. The p-harmonic smooth term deals with the regularization of the angle of directional data, while preserving the magnitude. The proposed algorithm is based on separating the optical flow vector into its magnitude and angle, and then processing each one of these components iteratively, in a closed-loop manner. Experiments show that the proposed algorithm outperforms a recently published, top performing L1 total variation method, which shows the capability of the new regularization term in improving the optical flow estimation performance.

This paper presents a new general framework for contour tracking based on the synergy of two powerful segmentation tools, namely, spatial temporal conditional random fields (CRFs) and geodesic active contours (GACs). The contours of targets are modeled using a level set representation. The evolution of the level sets toward the target contours is formulated as one of the joint region-based (CRF) and boundary-based (GAC) segmentations under a unified Bayesian framework. A variational inference technique is used to solve this otherwise intractable inference problem, leading to approximate MAP solutions of both the new 3D spatial temporal CRF and the GAC model. The tracking result of the previous frame is used to initialize the curve in the current frame. Typical contour tracking problems are considered and experimental results are given to illustrate the robustness of the method against noise and its accurate performance in moving objects boundary localization.

Principal component analysis (PCA) is a technique that is widely used for applications such as dimensionality reduction, image compression, feature extraction and data visualization. One of the key issues in the use of PCA for modelling is that it is very sensitive to outliers since its formulation is based on Gaussian density model. Lately, more heavy-tailed distribution (i.e., Student's t-distribution) is introduced to increase the robustness of traditional PCA. But the robust version of PCA is expressed as the maximum likelihood solution of a probabilistic latent variable model. This reformulation raises the question of how to determine the optimal number of principal components to be retained. In this paper, we develop a Bayesian model selection approach to estimate the true dimensionality of the data. The proposed algorithm is based on a new Bayesian treatment of robust Student's t-distribution PCA. A simple Expectation-Maximization (EM) solver is introduced to find approximate solutions for the model. Experiments show that the proposed model achieves simultaneous optimal dimensionality selection an.d accurate principal components recovery.

Principal component analysis (PCA) is a technique that is widely used for applications such as dimensionality reduction, image compression, feature extraction and data visualization. One of the key issues in the use of PCA for modelling is that it is very sensitive to outliers since its formulation is based on Gaussian density model. Lately, more heavy-tailed distribution (i.e., Student’s t-distribution) is introduced to increase the robustness of traditional PCA. But the robust version of PCA is expressed as the maximum likelihood solution of a probabilistic latent variable model. This reformulation raises the question of how to determine the optimal number of principal components to be retained. In this paper, we develop a Bayesian model selection approach to estimate the true dimensionality of the data. The proposed algorithm is based on a new Bayesian treatment of robust Student’s t-distribution PCA. A simple Expectation-Maximization (EM) solver is introduced to find approximate solutions for the model. Experiments show that the proposed model achieves simultaneous optimal dimensionality selection and accurate principal components recovery.

Augmenting electro-optical (EO) based target tracking systems with infrared (IR) modality has been shown to be effective in increasing the accuracy rate of the tracking system. A key issue in designing such a multimodal tracking system is how to combine information observed from different sensor types in a systematic way to obtain desirable performance. In this paper, we present an investigation into integrating EO and IR sensors within hidden Markov model (HMM) based frameworks. We propose to use a coupled hidden Markov model (CHMM) to improve upon the existing fusion schemes. Another contribution is that we propose to use a robustt-distribution based subspace representation in the CHMM to model appearance changes of the target. Numerical experiments demonstrate that the proposed CHMM tracking system has improved performance over other integration schemes for situations where the target object is corrupted by noise or occlusion.

This paper proposes an image registration method. Edges are detected from images and partitioned into segments as matching primitives. Then, corners on the edges are detected to guide registration. A similarity metric is proposed based on the number of pairs of matching segments. Corner mappings are sequentially tried along a segment, from which a transformation is obtained. The corner mappings are evaluated by the similarity metric under their resulting transformation. By this means, corner mappings are established by utilizing whole images. Since the sensitivity of transformation parameters to the accuracy of corner mappings, as many corner mappings as possible are used. Experimental results show that the proposed method is robust, especially when there is no integral corresponding edges between two images.

This paper describes a new, robustified Hidden Markov Model for target tracking using a subspace representation. The Hidden Markov Model (HMM) provides a powerful framework for the probabilistic modelling of observations and states. Visual tracking problems are often cast as an inference problem within the HMM framework. Probabilistic Principal Component Analysis (PPCA), a classic subspace representation method, is a popular tool for appearance modelling because it provides a compact representation for high-dimensional data. Previous subspace based tracking algorithms assume the image observations were generated from a Gaussian distribution parameterized by principal components. One drawback of using Gaussian density model is that atypical observations cannot be modelled well. Hence, they are very sensitive to outliers. To address this problem, we propose to augment the HMM by adding a set of latent variables {wi}ti=1 to adjust the shape of the observation distribution. By carefully choosing the distribution of {w i}ti=1, we obtain a more robust observation distribution with heavier tails than a Gaussian. Numerical experiments demonstrate the effectiveness of this new framework in cases where the target objects are corrupted by noise or occlusion.

This paper proposes an edge-based multimodal image registration approach. It aims to address image registration as a whole rather than tackle each of its elements independently. One-pixel-wide curves are firstly extracted from images, and junctions along the curves are detected. Then, each curve is divided into subsegments as matching primitives. A similarity metric based on the number of matched pairs of subsegments is proposed and experimental results show that the presented approach is a robust and effective tool for multimodal image registration.

Multiple description coding (MDC) and multihypothesis motion-compensated prediction (MHMCP) are two promising error resilient video coding techniques. We propose an enhanced error resilient video coding scheme based on multiple state video coding (MSVC). It uses multihypothesis motion prediction in addition to normal motion-compensated video compression steps. Yielded additional motion vectors between frames of two descriptions can help efficiently and quickly estimate lost content in one description when the other one is received correctly. Negligible redundancy is added to originally coded MSVC bitstreams while complicated motion search is avoided at decoder when there is a need to estimate lost content. Simulation results show that the proposed scheme has better error recovery performance than previously used approaches in most situations, especially when multiple and complex motions appear.

The output-queued (OQ) switch is often considered an ideal packet switching architecture for providing quality-of-service guarantees. Unfortunately, the high-speed memory requirements of the OQ switch prevent its use for large-scale devices. A previous result indicates that a crossbar switch fabric combined with lower speed input and output memory and two times speedup can exactly emulate an OQ switch; however, the complexity of the proposed centralized scheduling algorithms prevents scalability. This paper examines switch fabrics with limited memory and their ability to exactly emulate an OQ switch. The switch architecture of interest contains input queueing, fabric queueing, flow-control between the limited fabric buffers and the inputs, and output queueing. We present sufficient conditions that enable this combined input/fabric/output-queued switch with two times speedup to emulate a broad class of scheduling algorithms operating an OQ switch. Novel scheduling algorithms are then presented for the scalable buffered crossbar fabric. It is demonstrated that the addition of a small amount of memory at the crosspoints allows for distributed scheduling and significantly reduces scheduling complexity when compared with the memoryless crossbar fabric. We argue that a buffered crossbar system performing OQ switch emulation is feasible for OQ switch schedulers such as first-in-first-out, strict priority and earliest deadline first, and provides an attractive alternative to both crossbar switch fabrics and to the OQ switch architecture.

In lossy image compression schemes utilizing the discrete cosine transform (DCT), quantization of the DCT coefficients introduces error in the image representation and a loss of signal information. At high compression ratios, this introduced error produces visually undesirable compression artifacts that can dramatically lower the perceived quality of a particular image. This paper provides a spatial domain model of the quantization error based on a statistical noise model of the error introduced when quantizing the DCT coefficients. The resulting theoretically derived spatial domain quantization noise model shows that, in general, the compression noise in the spatial domain is both correlated and spatially varying. This provides some justification for many of the ad hoc artifact removal filters that have been proposed. An accurate description of quantization noise is essential if one hopes to remove, or at least alleviate, the visibility of compression artifacts.

There are numerous methods of post-processing that make use of iterative techniques. Many of these schemes have been demonstrated to be very effective in removing artifacts from compressed video, producing better and better image estimates at each iteration. However, this artifact removal comes at the cost of a large computational burden. This paper introduces two methods for iterative post-processing of compressed video in an efficient manner. One of these methods is applicable to one particular maximum a posteriori scheme. The other method has direct application to other, more general, iterative post-processing schemes that make use of a convex constraint set, which is the set of all images that will recompress to yield the originally-received data. The combination of these two methods produces a post-processing algorithm that has the advantage of many iterative schemes (excellent visual results), while requiring a relatively low amount of computational effort to achieve these results.

A simultaneous multi-frame super-resolution video reconstruction procedure, utilizing spatio-temporal smoothness constraints and motion estimator confidence parameters is proposed. The ill-posed inverse problem of reconstructing super-resolved imagery from the low resolution, degraded observations is formulated as a statistical inference problem and a Bayesian, maximum a-posteriori (MAP) approach is utilized for its approximate solution. The inclusion of motion estimator confidence parameters and temporal constraints result in higher quality super-resolution reconstructions with improved robustness to motion estimation errors.

This paper presents an approach for improving the effective dynamic range of cameras by using multiple photographs of the same scene taken with different exposure times. Using this method enables the photographer to accurately capture scenes that contain a high dynamic range, i.e., scenes that have both very bright and very dark regions. The approach requires an initial calibration, where the camera response function is determined. Once the response function for a camera is known, high dynamic range images can be computed easily. The high dynamic range output image consists of a weighted average of the multiply-exposed input images, and thus contains information captured by each of the input images. From a computational standpoint, the proposed algorithm is very efficient, and requires little processing time to determine a solution.

This paper presents a lossy image compression scheme that employs a generalized rank-ordered prediction filter for pyramidal image coding. The proposed prediction method renders significantly reduced variance of the quantizer input. Consequently, the quality of the decompressed image is much enhanced due to the greatly reduced quantization distortion. Both analytical and simulation results show that the proposed scheme yields high-quality performance.

Fixed length entropy codes (FLC) have been successfully employed as an alternative to the popular variable length codes (VLC) in video codecs. This paper presents a codeword assignment for FLC coded interframe coefficients and motion vectors that minimizes the effects of bit errors on the decoded video sequence. Intensive testing has shown that with the proposed solution, the distortion introduced by transmission errors is coarsely half of that obtained with VLC codes, while maintaining the compression efficiency.

The joint channel-source decoding scheme for using turbo codes to protect compressed image data proposed by Peng, Huang, Costello and Stevenson (see, Proc. 1998 IEEE International Symposium on Circuits and Systems, Monterey, California, 1998) is modified for subband coded images. Two different modifications are presented and compared. The factors affecting the performance of the schemes are studied. Both modified schemes show superiority over a separate decoding system.

This paper is intended to investigate the bit rate allocation between source coding and channel coding rates when turbo codes are used to protect vector quantized images. The experimental results show that an appropriate bit rate allocation can lead to significant gains in the quality of the reconstructed images compared to an arbitrary bit rate allocation scheme.

Maximum a posteriori (MAP) filtering using the Huber-Markov (1981) random field (HMRF) image model has been shown in the past to be an effective method of reducing compression artifacts in images. Unfortunately, this MAP formulation requires iterative techniques for the solution of a constrained optimization problem. In the past, these iterative techniques have been computationally intensive, making the filter infeasible in situations where it is desired to filter images (or video frames) quickly. This paper introduces two methods for reducing the computational requirements of the constrained optimization, as well as theoretical and experimental justifications for using them.

Super-resolution enhancement algorithms are used to estimate a high-resolution video still (HRVS) from several low-resolution frames, provided that objects within the digital image sequence move with subpixel increments. A Bayesian multi_frame enhancement algorithm is presented to compute an HRVS using the spatial information present within each frame as well as the temporal information present due to object motion between frames. However, the required subpixel-resolution motion vectors must be estimated from low-resolution and noisy video frames, resulting in an inaccurate motion field which can adversely impact the quality of the enhanced image. Several subpixel motion estimation techniques are incorporated into the Bayesian multiframe enhancement algorithm to determine their efficacy in the presence of global data transformations between frames (i.e., camera pan, rotation, tilt, and zoom) and independent object motion. Visual and quantitative comparisons of the resulting high-resolution video stills computed from two video frames and the corresponding estimated motion fields show that the eight-parameter projective motion model is appropriate for global scene changes, while block matching and Horn–Schunck optical flow estimation each have their own advantages and disadvantages when used to estimate independent object motion.

Entropy coding is a fundamental stage in all video compression algorithms in terms of compression efficiency and error resilience. Variable-length entropy codes (VLC) are used in current video codecs. Designed to be employed in noiseless applications, these codes are very sensitive to transmission errors. This paper proposes the use of fixed-length entropy codes (FLC) as an alternative to VLC in video compression applications. In noisy transmissions, the FLC-based codec has shown a superior performance compared to VLC-based codecs with synchronization words, while matching its performance in terms of coding efficiency.

Compressed images transmitted over noisy channels are extremely sensitive to bit errors. This necessitates the application of error control channel coding to the compressed representation before transmission. This paper presents an image transmission system which takes advantage of the superior performance of turbo codes, an important new class of parallel concatenated codes. Several aspects of the application of turbo codes to image transmission are studied, including comparison to a previous image transmission system using convolutional codes. Experimental results for several channel signal-to-noise ratios show that, in the same SNR range, turbo codes achieve much better performance with less decoding complexity than convolutional codes and that similar performance can be achieved at much lower channel SNRs. Studies also show that the use of feedback from an outer Reed-Solomon code to aid turbo decoding results in further improvement.

Entropy coding is a fundamental stage in all video compression algorithms. Variable length entropy codes (VLC) are used in current video codecs. Designed to be employed in noiseless applications, these codes are very sensitive to transmission errors. This paper proposes the use of fixed length entropy codes (FLC) as an alternative to VLC in video compression applications. In noisy transmissions the FLC codes have shown a superior performance compared to VLC schemes with synchronization words.

This paper presents a lossy image compression scheme which employs a pyramidal coding technique with segmentation based on the local activity levels. A novel prediction method is introduced, rendering significantly reduced variance of the quantizer input. Consequently, the coding efficiency is much enhanced due to the greatly reduced quantization distortion. Both theoretical analysis and simulation results show that the proposed scheme yields high quality performance. A reconstruction PSNR of 30.5 dB is achieved at a bit rate of 0.181 bpp for a 512×512 image "Lenna" before entropy coding.

Multiframe resolution enhancement methods are used to estimate a high-resolution video still (HRVS) from several low-resolution image sequence frames, provided that objects within the video sequence move with subpixel increments. Estimating accurate subpixel-resolution motion vectors is a challenging, albeit critically important component of super-resolution enhancement algorithms. A Bayesian motion estimation technique is proposed which models the motion field with a discontinuity-preserving prior. The method is related to Horn-Schunck (1981) optical flow estimation, except that the discontinuity-preserving prior can allow abrupt changes within the motion field without the use of line processes. Simulations compare the high-resolution video stills which result from using the subpixel motion vectors calculated by block matching and the proposed Bayesian motion estimation technique.

In image communication systems, subsampling, can be used to decrease the transmission rate of images. One common technique is to subsample the chrominance information. This process, however, introduces visible color artifacts. This paper develops an algorithm for enhancing the subsampled image data, thereby eliminating the artifacts.

There has been a tremendous amount of research in the area of image halftoning, where the goal has been to find the most visually accurate representation given a limited palette of gray levels (often just two, black and white). This paper focuses on the inverse problem, that of finding efficient techniques for reconstructing high-quality continuous-tone images from their halftoned versions. The proposed algorithms are based on a maximum a posteriori (MAP) estimation criteria using a Markov random field (MRF) model for the prior image distribution. Image estimates obtained with the proposed model accurately reconstruct both the smooth regions of the image and the discontinuities along image edges. Algorithms are developed and example gray-level reconstructions are presented generated from both dithered and error-diffused halftone originals. Application of the technique to the problems of rescreening and the processing of halftone images are shown.

The human visual system appears to be capable of temporally integrating information in a video sequence in such a way that the perceived spatial resolution of a sequence appears much higher than the spatial resolution of an individual frame. While the mechanisms in the human visual system that do this are unknown, the effect is not too surprising given that temporally adjacent frames in a video sequence contain slightly different, but unique, information. This paper addresses the use of both the spatial and temporal information present in a short image sequence to create a single high-resolution video frame. A novel observation model based on motion compensated subsampling is proposed for a video sequence. Since the reconstruction problem is ill-posed, Bayesian restoration with a discontinuity-preserving prior image model is used to extract a high-resolution video still given a short low-resolution sequence. Estimates computed from a low-resolution image sequence containing a subpixel camera pan show dramatic visual and quantitative improvements over bilinear, cubic B-spline, and Bayesian single frame interpolations. Visual and quantitative improvements are also shown for an image sequence containing objects moving with independent trajectories. Finally, the video frame extraction algorithm is used for the motion-compensated scan conversion of interlaced video data, with a visual comparison to the resolution enhancement obtained from progressively scanned frames.

The sensitivity of the compressed image representation to bit errors requires application of a channel code before transmission over noisy channels. To prevent the uncontrolled degradation caused by a channel error, an error controlling channel code is applied to the compressed representation before transmission. Many image communication systems have constraints on bandwidth, power and time which prohibit transmission of uncompressed raw image data. Compressed image formats, however, are extremely sensitive to bit errors which can seriously degrade the quality of the image at the receiver. A new list-based iterative trellis decoder is proposed which accepts feedback from a post-processor which can detect channel errors in the reconstructed image. Experimental results are shown which indicate the new decoder provides significant improvement over the standard Viterbi decoder.

The human visual system seems to be capable of temporally integrating information in a video sequence in such a way that the perceived spatial resolution of a sequence appears much higher than the spatial resolution of an individual frame. This paper addresses how to utilize both the spatial and temporal information present in an image sequence to create a high-resolution video still. A novel observation model based on motion compensated subsampling is proposed for a video sequence. Since the reconstruction problem is ill-posed, Bayesian restoration with an edge-preserving prior image model is used to extract a high-resolution video frame from a low-resolution sequence. Estimates computed from an image sequence containing a camera pan show dramatic improvement over bilinear, cubic B-spline, and Bayesian single frame interpolations. Improved definition is also shown for a video sequence containing objects moving with independent trajectories.

This paper presents an image communication system with improved decoding of compressed image information. A convolutional code protects the compressed image information from channel noise while a Reed-Solomon outer code gives additional protection to the critical image header information. A post-processor detects uncorrected channel errors in the reconstructed image and feeds error location information to a list-based iterative trellis decoder. This list-based decoder provides significant improvement in image quality. Experimental results are given for varying channel SNR and for varying bit rate.

To build inexpensive image display hardware, designers severely restrict the number of colors which can be displayed simultaneously on a device. A color image of a natural scene typically has tens or hundreds of thousands of different colors in it. To display such an image on a reduced palette display device requires the image to be mapped from the full resolution color space to the reduced color space allowed by the device. Over the years, many algorithms for both selecting the reduced palette and for mapping the full resolution color space image to the reduced palette have been developed. This mapping is lossy and degrades the original image information. If the reduced color space image is processed using common image-processing algorithms, or if the image is viewed on a device with a higher resolution color space, this degradation is very noticeable. In these cases, much better results can be obtained by first reconstructing a full color image from the reduced palette image. This creates a need for a palette restoration algorithm. This paper develops an algorithm for reconstructing high-color-resolution image data from reduced color palette images. The algorithm is based on stochastic regularization using a non-Gaussian Markov random field model for the image data. This results in a constrained optimization algorithm which is solved using an iterative constrained gradient descent computational algorithm. Results of the proposed palette restoration algorithm have indicated that it is effective for the reconstruction of palettized images. Visual results of several experiments are presented.

Due to limitations in bandwidth or storage space, many applications require compression of digital images. One of the main hindrances to progress in the area of image communications is the large amount of information contained in an image. In this paper, we propose a wavelet-based image compression technique which incorporates some of the properties of the human visual system (HVS). A new wavelet decomposition based on the HVS will be introduced. The wavelet coefficients will be quantized with a vector quantization technique which uses HVS properties to allocate bits to the various subbands. The new HVS-based wavelet decomposition will be compared with the existing separable and nonseparable wavelet decompositions. Compression techniques based on such considerations yield higher quality results compared to standard techniques. Several experimental results will be shown.

The perceived quality of images reconstructed from low bit rate compression is severely degraded by the appearance of transform coding artifacts. This paper proposes a method for producing higher quality reconstructed images based on a stochastic model for the image data. Quantization (scalar or vector) partitions the transform coefficient space and maps all points in a partition cell to a representative reconstruction point, usually taken as the centroid of the cell. The proposed image estimation technique selects the reconstruction point within the quantization partition cell which results in a reconstructed image that best fits a non-Gaussian Markov random field (MRF) image model. This approach results in a convex constrained optimization problem that can be solved iteratively. At each iteration, the gradient projection method is used to update the estimate based on the image model. In the transform domain, the resulting coefficient reconstruction points are projected to the particular quantization partition cells defined by the compressed image. Experimental results will be shown for images compressed using scalar quantization of block DCT and using vector quantization of subband wavelet transform. The proposed image decompression provides a reconstructed image with reduced visibility of transform coding artifacts and superior perceived quality.

Multispectral images consist of multiple channels, each containing data acquired from a different band within the frequency spectrum. Since most objects emit or reflect energy over a large spectral bandwidth, there usually exists a significant correlation between channels. Due to often harsh imaging environments, the acquired data may be degraded by both blur and noise. Simply applying a monochromatic restoration algorithm to each frequency band ignores the cross-channel correlation present within a multispectral image. A Gibbs prior is proposed for multispectral data modeled as a Markov random field, containing both spatial and spectral cliques. Spatial components of the model use a nonlinear operator to preserve discontinuities within each frequency band, while spectral components incorporate nonstationary cross-channel correlations. The multispectral model is used in a Bayesian algorithm for the restoration of color images, in which the resulting nonlinear estimates are shown to be quantitatively and visually superior to linear estimates generated by multichannel Wiener and least squares restoration.

Maximum a posteriori (MAP) estimation using a robust signal model is capable of extracting a signal from additive noise, preserving edges in the signal estimate. A digital network can compute the estimate numerically using the gradient descent algorithm, or an analog network can be designed which provides the estimate as a steady-state output. An ASIC designed to perform robust signal processing has been manufactured, using the Huber-Markov random field (HMRF) signal model. This robust model is dependent upon two tunable parameters, which may be adjusted for a particular signal due to the design of novel analog circuit blocks.>

The application of a monochromatic restoration algorithm to each channel within a multispectral image does not result in an estimate which takes into account cross-channel correlation. A non-Gaussian prior model is proposed for multispectral images, using the Gibbs distribution. The density accounts for both spatial (within channel) and spectral (between channel) information. Spatial components use a nonlinear operator to preserve discontinuities within each channel, while spectral components incorporate cross-channel information in the model. The prior density is used in a maximum a posteriori (MAP) estimation algorithm for the restoration of color images. The resulting nonlinear estimates are shown to be quantitatively superior to linear estimates produced by multichannel Wiener and least squares restoration, which implicitly use Gaussian priors.>

For signals containing discontinuities, the usual assumptions of Gauss-Markov distributed signal sources do not hold. To preserve edges, non-Gaussian prior models have been developed for use in Bayesian restoration. These models are generally dependent upon two parameters, one controlling the size of reconstructed discontinuities, and the other controlling data smoothing. The authors propose a maximum likelihood technique for automatically estimating these parameters, resulting in the optimization of an expression dependent upon the prior model partition function. An exact expression is derived for the 1D signal model partition function, while an approximation is proposed for the 2D image model partition function. Parameters estimated from degraded signals result in high quality restorations.>

Accurate image expansion is important in many areas of image analysis. Common methods of expansion, such as linear and spline techniques, tend to smooth the image data at edge regions. This paper introduces a method for nonlinear image expansion which preserves the discontinuities of the original image, producing an expanded image with improved definition. The maximum a posteriori (MAP) estimation techniques that are proposed for noise-free and noisy images result in the optimization of convex functionals. The expanded images produced from these methods will be shown to be aesthetically and quantitatively superior to images expanded by the standard methods of replication, linear interpolation, and cubic B-spline expansion.

The method of Tikhonov regularization has been widely used to form well-posed inverse problems in low-level vision. The application of this technique usually results in a least squares approximation or a spline fitting of the parameter of interest. This is often adequate for estimating smooth parameter fields. However, when the parameter of interest has discontinuities the estimate formed by this technique will smooth over the discontinuities. Several techniques have been introduced to modify the regularization process to incorporate discontinuities. Many of these approaches however, will themselves be ill-posed or ill-conditioned. This paper presents a technique for incorporating discontinuities into the reconstruction problem while maintaining a well-posed and well-conditioned problem statement. The resulting computational problem is a convex functional minimization problem. This method is compared to previous approaches and examples are presented for the problems of reconstructing curves and surfaces with discontinuities and for estimating image data. Computational issues arising in both analog and digital implementations are also discussed.>

A method for nonlinear image expansion is introduced. The method preserves the discontinuities of the original image, therefore producing an expanded image with improved definition. The maximum a posteriori (MAP) estimation technique that is proposed results in the constrained optimization of a convex functional. The expanded image produced from this method will be shown to be aesthetically and quantitatively superior to images expanded by the standard methods of replication, linear interpolation, and cubic B-spline expansion.>

An algorithm for the reconstruction of visual surfaces from sparse data is proposed. An important aspect of this algorithm is that the surface estimated from the sparse data is approximately invariant with respect to rigid transformation of the surface in 3D space. The algorithm is based on casting the problem as an ill-posed inverse problem that must be stabilized using a priori information related to the image and constraint formation. To form a surface estimate that is approximately invariant with respect to viewpoint, the stabilizing information is based on invariant surface characteristics. With appropriate approximations, this results in a convex functional to minimize, which is then solved using finite element analysis. The relationship of this algorithm to several previously proposed reconstruction algorithms is discussed, and several examples that demonstrate its effectiveness in reconstructing viewpoint-invariant surface estimates are given.>

A new algorithm is described for the reconstruction of visual surfaces from sparse data. An important aspect of this algorithm is that the surface estimated from the sparse data is approximately invariant with respect to rigid transformation of the surface in three-dimensional space. To form a surface estimate which is invariant with respect to viewpoint the stabilizing information is based on invariant surface characteristics. With appropriate approximations this results in a convex functional which must be minimized. The relationship of this algorithm to several previously proposed reconstruction algorithms is discussed, and several examples are given which demonstrate the effectiveness of the proposed algorithm in reconstructing viewpoint invariant surface estimates.>