Scalability is an important requirement for video coding when coded videos stream over dynamic-bandwidth networks. The state-of-the-art scalable video coding schemes adopt layer-based methods upon H.265, represented by the SHVC standard. Compared to layer-based schemes, wavelet-based schemes were suspected less efficient for a long while. We try to improve the compression efficiency of wavelet-based scalable video coding by leveraging the recent progresses of deep learning. First, we propose an entropy coding method, using trained convolutional neural networks (CNNs) for probability estimation, to compress the wavelet-transformed subbands. Second, we design a CNN-based method for inverse temporal wavelet transform. We integrate the two proposed methods into a traditional wavelet-based scalable video coding scheme, named Interframe-EZBC. The two methods together achieve more than 20% bits savings. Then, our scheme outperforms the SHVC reference software by 9.09%, 6.55%, and 8.66% BD-rate reductions in YUV respectively.

Nowadays, the development of video coders is resulting in significantly increased performance. Among all coder categories, fine grain scalable video coders have the chance to show great advantage by freely extracting code-streams with different bitrates, resolutions, and frame rates from one source-coded bit file to satisfy the various requirements of subscribers in a multicast network. Based on the widely known ENH-MC-EZBC coder with motion-compensated temporal filtering (MCTF) and multi-mode motion-compensation, we enhance and revise its motion model to incorporate advanced mechanisms such as motion-vector prediction competition, block merging, and affine motion-compensation. Experimental results reveal that up to 17% of bit savings can be achieved. Moreover, we consider JPEG 2000 as a backend coder option for the MCTF-generated temporal subbands to directly enable multi-stage code-stream extraction for hierarchical network distribution. Experimental results on common test clips are presented and compared against the current Internet standard AVC/H.264 (JM16.1 test model).

Multiple description coding has been shown to provide flexible and distortion-rate optimal video streaming transmission over lossy links. In this paper, we provide an adaptive MD-FEC algorithm for point-to-multipoint video streaming that approximately minimizes the average distortion of the multicast video. Our new algorithm accomplishes this by adapting both the packet lengths and their number to fit heterogeneous link conditions.

Caching has been shown to enhance network performance. In this paper, we study fine-grain scalable video caching. We start from a single cache scenario by providing a solution to the caching allocation problem that optimizes the average expected video quality for the most popular video clips. Actual trace data is applied to verify the performance of our algorithm and compare its backhaul link bandwidth consumption relative to non-scalable video caching. In addition, we extend our analysis to collaborative caching and integrate network coding for further transmission efficiency. Our experimental results demonstrate considerable performance enhancement.

The affine-motion model is able to capture rotation, zooming, and the deformation of moving objects, thereby providing a better motion-compensated prediction. However, it is not widely used due to difficulty in both estimation and efficient coding of its motion parameters. To alleviate this problem, a new control-point representation that favors differential coding is proposed for efficient compression of affine parameters. By exploiting the spatial correlation between adjacent coding blocks, motion vectors at control points can be predicted and thus efficiently coded, leading to overall improved performance. To evaluate the proposed method, four new affine prediction modes are designed and embedded into the high-efficiency video coding test model HM1.0. The encoder adaptively chooses whether to use the new affine mode in an operational rate-distortion optimization. Bitrate savings up to 33.82% in low-delay and 23.90% in random-access test conditions are obtained for low-complexity encoder settings. For high-efficiency settings, bitrate savings up to 14.26% and 4.89% for these two modes are observed.

Higher-order motion models were introduced in video coding a couple of decades ago, but have not been widely used due to both difficulty in parameters estimation and their requirement of more side information. Recently, researchers have put them back into consideration. In this paper, the affine motion model is employed in SKIP and DIRECT modes to produce a better prediction. In affine SKIP/DIRECT, candidate predictors of the motion parameters are derived from the motions of neighboring coded blocks, with the best predictor determined by rate-distortion tradeoff. Extensive experiments have shown the efficiency of these new affine modes. No additional motion estimation is needed, so the proposed method is also quite practical.

Multiple Description with Forward Error Correction (MD-FEC) coding provides the flexibility, easy adaptivity and distortion-rate optimality that are desirable for delivering streaming video in a network environment with time-varying bandwidth fluctuations and random packet losses. In this paper, we consider the issue of how diverse receivers of a video stream should be grouped - where each group receives a MD-FEC coded bitstream optimized for that group - so that the average video distortion is minimized across all receivers. We show that a sequential grouping solution is optimal for linear distortion-rate functions. For non-linear distortion-rate functions, while the optimal grouping structure may not be sequential in general, we observe that the approximation factor attained by the best sequential solution can be characterized in terms of the “degree of convexity” of the distortion-rate function. Numerical experiments with realistic distortion-rate functions reveal that the difference between the globally optimal grouping solution and the best sequential solution, is typically small. We provide a dynamic programming based polynomial-time algorithm to compute the best sequential solution.

In this paper, a new motion modeling method is introduced. We employ the popular quadtree structure to divide an image frame into variable size blocks. Each block is considered as an independent deformable mesh cell, and can be connected to causal neighbor cells to produce partial meshes. The motion model in each mesh cell is adaptively selected by minimizing a Lagrangian cost. We evaluate the proposed method by motion compensated prediction and coding. Preliminary experimental results show its potential advantages.

Multiple description codes have been shown to facilitate heterogeneous multicast using routing with replication. We develop the combination of practical network coding and multiple description coding (introduced by Chou ), but here for video multicast in lossless networks where users get video according to their available resources. We also show that network coding enables the server to provide quality of service that is centrally controlled. The lack of the need to maintain multiple trees (as in routing) helps in ease of construction of the multicast delivery system. The simulation results show that using practical network coding provides significant control over the rate allocation for the resources.

In this paper, we investigate a fine-grain adaptive forward error correction (FGA-FEC) coding scheme for scalable video bitstreams. In our work, both the embedded source bitstream and the error-control codes are granularly adapted at block level in intermediate overlay nodes to satisfy heterogeneous users with both different video frame-rate/spatial resolution/quality preferences and different network connections. The proposed FGA-FEC scheme encodes and adapts the embedded source-coded bitstream in such a way that if part of the video source data is actively dropped, parity bits protecting that piece of data are also removed, yielding an efficient result without any transcoding.

In the above titled paper (ibid., vol. 18, no. 10, pp. 1432-1436, Oct 08), Figs. 3 and 4 were printed incorrectly. The correct figures are presented here.

A highly desirable feature in storage video applications is uniform quality. Variable bit rate (VBR) coding has the potential to produce nearly constant quality throughout an entire movie. This can be defined as a bit allocation problem with a long-term constraint on distortion variation. We consider the optimal bit allocation with multiple constraints including disk capacity and the distortion bounds on the individual frames. We find the theoretical optimality conditions and propose a practical iterative solution based on the Lagrangian methods. While minimizing average distortion and distortion variation cannot be achieved simultaneously for a given bit budget, the proposed algorithms are able to efficiently balance the tradeoff between the two goals. The computational complexity of the exact rate-distortion$(R-D)$functions for real movies is addressed by a statistical$R-D$model proposed in this work. Based on the generic block-based motion compensated transform coding theories, the model is formed by a rate-quantization$(R-Q)$function and the corresponding distortion-quantization$(D-Q)$function. A novel two-pass MPEG-2 VBR encoder based on the proposed algorithms is developed for coding with long-term nearly constant quality. Experimental results are promising and the encoder effectively achieves the fit-to-disc function and at the same time controls objective quality variation. By incorporating basic subjective coding techniques into the encoder, significant visual quality improvement is observed during the subjective tests.

In this paper, we provide some recent extensions to the scalable subband/wavelet video coder MC-EZBC. The enhanced MC-EZBC employs an adaptive motion-compensated temporal filter (MCTF) framework. Directional I-BLOCKs and overlapped block motion compensation (OBMC) further improve MCTF efficiency. A scalable motion vector coder based on CABAC is shown to improve the overall performance at low bitrates/resolution. Frequency rolloff is incorporated to reduce spatial aliasing at low resolution without PSNR loss at full resolution. Experimental results show that the new features significantly improve the performance of MC-EZBC. We provide several comparisons to other recent coders.

The extracted low-resolution video from a motion compensated 3-D subband/wavelet scalable video coder is unnecessarily sharp and sometimes contains significant aliasing, compared to that obtained by the MPEG4 lowpass filter. In this paper, we propose a content adaptive method for aliasing reduction in subband/wavelet scalable image and video coding. We try to make the low-resolution frame (LL subband) visually and energy-wise similar to that of the MPEG4 decimation filter through frequency roll-off. Scaling of the subbands is introduced to make the variances of the subbands comparable in these two cases. Thanks to the embedded properties of the EZBC coder, we can achieve the needed scaling of energies in each subband by subbitplane shift in the extractor and value (coefficient) scaling in the decoder. Two methods are presented which offer substantial peak signal-to-noise ratio (PSNR) gain for lower spatial resolution, as well as substantial reduction in visible aliasing, with little or no reduction in full resolution PSNR.

In this paper, we extend our proposed FGA-FEC coding scheme, a generalized MD-FEC method, to wireless networks. To protect the encoded scalable video bitstream over a lossy channel and facilitate content adaptation at intermediate nodes, we use product codes based on BCH/CRC codes as row codes and RS codes as column codes. We give a fast algorithm to optimize the product codes within several iterations from a near optimal point. Simulations show good performance in both content adaptation and protection.

In this paper, we present two error-concealment algorithms developed for scalable motion-compensated subband/wavelet video coders. These algorithms exploit the properties of motion-compensated temporal filtering to recover lost video data by motion compensation from correctly received previous and future video frames. Our experiments indicate that backward motion-compensated prediction outperforms replacement from neighboring correctly received frames by up to 3 dB in terms of PSNR. In addition, a bidirectional algorithm tops the unidirectional one by up to 1 dB. Also, visual improvements are often higher that PSNR improvements would suggest.

This paper investigates 3-D subband coding via motion-compensated temporal filtering (MCTF) using biorthogonal filters. While nonoverlapping Haar filter-based MCTF has a fixed size group of pictures (GOP) structure, for longer filters we need to use extensions at the GOP ends resulting in coding efficiency loss and/or significant peak signal-to-noise (PSNR) variation. We solve this problem by introducing a "sliding window" approach to MCTF. While we find the longer filters have higher coding gain and significant PSNR improvement at high bit rates, a necessary doubling of the number of motion vectors (MVs) causes a drop in PSNR at lower bit rates. The paper then concentrates on improving the efficiency of both the MV estimation and compression. We employ the MV estimate at higher temporal resolutions as the starting point for local MV refinement at the present temporal resolution thereby reducing the complexity of motion search and obtaining a more uniform motion field. We improve MV coding performance by adapting the context-based binary arithmetic coder (CABAC) from H.264. Instead of encoding MV residuals along a quadtree scanning path, we reduce the MV bit rate by prediction from both neighboring MV blocks and blocks in the previous or the next lower temporal level. Our proposed motion estimation and coding schemes can reduce the MV bit rate by 10%-20% for Haar and 15%-30% for 5/3 MCTF

In the scalable video coder MC-EZBC, the motion-vector (MV) bitstream was not scalable in bitrate or resolution. In this short paper, we enhance MC-EZBC with a new scalable MV coder based on context adaptive binary arithmetic coding. Alphabet general partition of MV symbols is proposed to achieve accuracy or quality scalability of MVs. A selective layered structure is used to reduce the number of MVs transmitted when appropriate, mainly needed for resolution scalability. With these two additions, we have a layered temporal, SNR, and resolution scalability for the MV bitstream. Experimentally, we find that this gives significant visual and objective improvement for low bitrates and/or resolutions with only very slight PSNR loss and unnoticeable visual loss at high bitrates

In this paper, we propose a joint source-network error control coding (JSNC) scheme which efficiently integrates scalable video coding, error control coding and overlay infrastructure to stream video to heterogeneous users. The distributed overlay nodes adapt both the video bitstream and error control coding based on both user requirements and available bandwidth. A novel fine granular adaptive FEC (FGA-FECcheme, a generalization of MD-FEC, is proposed for error recovery during video transmission to heterogeneous users. Encoding once, the FGA-FEC can satisfy multiple heterogeneous users simultaneously without decoding/re-encoding FEC at intermediate nodes.

In subband/wavelet interframe video compression, motion-compensated coders make use of estimated motion paths to compress the data. When these motion paths are not valid, due to either occlusion or difficulty in their estimation, artifacts can sometimes be created in the embedded lower frame rate video. Since these coders are intended for scalable application, it is important that the lower frame rate video be as free of artifacts as possible. The incorporation of I-blocks in the context of hierarchical variable size block matching (HVSBM) allows for inevitable occasional poorly connected motion blocks. Directional spatial interpolation/prediction can then be employed to minimize such a block's energy in a temporal high frame. Results are provided for the MC-EZBC (motion compensated embedded zeroblock coding) coder that has been recently under investigation at MPEG.

Overlay networks offer promising capabilities for video streaming, due to their support for application-layer processing at the overlay forwarding nodes. In this paper we propose a novel overlay multi-hop FEC (OM-FEC) scheme that provides FEC encoding/decoding capabilities at intermediate nodes in an overlay path. Based on the current network conditions, the end-to-end overlay path is partitioned into segments, and appropriate FEC codes are applied over those segments. We evaluate our work in a real-world scenario and illustrate that the proposed OM-FEC can outperform a pure end-to-end strategy by 10-15 dB in terms of video PSNR.

This paper presents an efficient multiplexing and error control system to improve streaming video performance over path aggregates. While providing the application with increased aggregate bandwidth, the scheme reduces performance degradation due to high path latencies and loss rates. The reduction in effective loss and delay is achieved by smart multiplexing and exploiting the high latency paths to the user's advantage. A novel out-of-order transmission algorithm utilizes the higher latency paths to transfer suitable frames from within the transmit buffer. We present an FEC strategy for our scheme that decouples the transmission of error correction frames from the associated data. This provides protection against correlated losses. Our scheme, while not completely optimized, can provide close to optimal performance at a considerably lower complexity. We verify the performance of our scheme using the ns-2 simulator.

In conventional motion-compensated three-dimensional subband/wavelet coding, where the motion compensation is unidirectional, incorrect classification of connected and unconnected pixels caused by incorrect motion vectors (MVs) has resulted in some coding inefficiency and visual artifacts in the embedded low-frame-rate video. In this paper, we introduce bidirectional motion compensated temporal filtering with unconnected pixel detection and I blocks. We also incorporate a recently suggested lifting implementation of the subband/wavelet filter for improved MV accuracy in an MC-EZBC coder. Simulation results compare PSNR performance of this new version of MC-EZBC versus H.26L under the constraint of equal groups of pictures size, and show a general parity with this state-of-the-art nonscalable coder on several test clips.

Three-dimensional (3-D) subband/wavelet coding with motion compensation has been demonstrated to be an efficient technique for video coding applications in some recent research works. When motion compensation is performed with half-pixel accuracy, images need to be interpolated in both temporal subband analysis and synthesis stages. The resulting subband filter banks developed in these former algorithms were not invertible due to image interpolation. In this paper, an invertible temporal analysis/synthesis system with half-pixel-accurate motion compensation is presented. We look at temporal decomposition of image sequences as a kind of down-conversion of the sampling lattices. The earlier motion-compensated (MC) interlaced/progressive scan conversion scheme is extended for temporal subband analysis/synthesis. The proposed subband/wavelet filter banks allow perfect reconstruction of the decomposed video signal while retaining high energy compaction of subband transforms. The invertible filter banks are then utilized in our 3-D subband video coder. This video coding system does not contain the temporal DPCM loop employed in the conventional hybrid coder and the earlier MC 3-D subband coders. The experimental results show a significant PSNR improvement by the proposed method. The generalization of our algorithm for MC temporal filtering at arbitrary subpixel accuracy is also discussed.

In this paper we present a video communication system that integrates end-to-end buffer management and congestion control at the source with the playout adjustment mechanism at the receiver. While each component of the system has been considered independently in the literature, our focus in this work is their integration. The proposed system exploits the fact that when congestion control is implemented at the source, most of the loss occurs at the source and not within the network. Based on this observation, we design the buffer management to trade off random loss for controlled loss of visually less important data. Frame rate is adjusted at the receiver to maximize the visual quality of the displayed video based on the overall loss. We tested our system with both H.26L and a subband/wavelet video coder, and found that it significantly improves the received video quality in both cases.

MC-EZBC is a fully scalable video coder using a motion-compensated 3-D subband/wavelet decomposition. Temporal filtering is performed along estimated motion trajectories to remove temporal redundancy. To get greatly increased motion compensation (MC) accuracy and increased coding efficiency, we use the lifting filter implementation. Additionally, we introduce unconnected blocks and spatial lowpass transition filtering (SLTF). This results in improved lower frame rate data for the scalable MC-EZBC coder. Finally, an optimized quantization strategy is introduced to compensate for variations in synthesis filtering for connected and unconnected pixels.

In this paper, we present a method of creating domain-based multiple descriptions of images and video. These descriptions are created by partitioning the transform domain of the signal into sets whose points are maximally separated from each other. This property enables simple error concealment methods to produce good estimates of lost signal samples. We present the approach in the context of Internet transmission of subband/wavelet-coded images and scalable motion compensated three-dimensional (3D) subband/wavelet-coded video, but applications are not limited to these scenarios. The results indicate that the proposed methods offer improvements over similar competing methods by up to 1 dB for images, and several decibels for video. Visual quality is also improved.

We study the problem of dividing the /spl Zopf//sup 2/ lattice into partitions so that minimal intra-partition distance between the points is maximized. We show that this problem is analogous to the problem of sphere packing. An upper bound on the achievable intra-partition distances for a given number of partitions follows naturally from this observation, since the optimal sphere packing in two dimensions is achieved by the hexagonal lattice. Specific instances of this problem, when the number of partitions is 2/sup m/, were treated in trellis-coded modulation (TCM) code design by Ungerboeck (1982) and others. It is seen that methods previously used for set partitioning in TCM code design are asymptotically suboptimal as the number of partitions increases. We propose an algorithm for solving the /spl Zopf//sup 2/ lattice partitioning problem for an arbitrary number of partitions.

In this paper we present a method for concatenated multiple description (MD) coding of frame-rate scalable video. The proposed method combines domain-based MD coding and FEC-based MD coding. We find that the combined system benefits from both of its components and is significantly better than either of them at higher packet loss rates.

In this paper, we present several techniques to improve the efficiency of motion-compensated 3-D subband/wavelet video coding for digital cinema (DC). Since DC data coming from scanning photographic film contains film-grain noise, we introduce one level spatial subband decomposition to reduce the noise, and impose motion-compensated temporal filtering (MCTF) only on the spatial LL subbands. Also we make the GOP size adaptive to better handle motion complexity and editing effects. In order to realize approximately constant quality in a full length motion picture, we propose to interleave the subbands of the temporal frames, and encode all the GOPs to the same fractional bit plane using the EZBC coder.

A source coding algorithm is presented which delivers a spatiotemporal scalable encoded bitstream. Motion compensated temporal filtering is combined with a spatial subband/wavelet pyramid to provide an efficient 3-D multiresolution representation. Error-feedback hierarchical coding permits near-optimal results for each subvideo. The proposed ordering of refinement addresses the noncommutative property of motion compensated temporal filtering and spatial subband/wavelet analysis. Finally, adaptive conditional arithmetic coding of quantizer significance maps increases coding efficiency. Experimental results demonstrate a significant improvement in performance over earlier published algorithms. The code-and-refine nature of our hierarchical algorithm makes it possible to use the algorithm as a scalable extension to other source coding algorithms. Finally, the complexity of the algorithm is modest and is well suited to parallel implementation.

Since we need to handle arbitrary shapes, the sample lattices in the region-based discrete wavelet transform (RBDWT) will be more complicated than in the conventional discrete wavelet transform (DWT). But they can not be arbitrary for certain filters, so we first analyze the influence of sample lattices on filter design. Then we discuss how to use them on the RBDWT. We find the sample lattices of odd length biorthogonal filters are very flexible and suitable for the RBDWT. Furthermore, they can simplify odd length filtering, and perfect reconstruction can be realized without the need to add any new sample. When compared to previously reported results there is a significant improvement.

Block based full search motion estimation is an expensive technique both in terms of computation and motion side information. An adaptive method based on vector quantization techniques with a dynamically updated codebook is introduced which provides a reduction of the complexity of the motion estimation process while keeping the side information low.

With fast computation and excellent compression efficiency, two embedded coding techniques, zero-tree/-block coding and context modeling of the subband/wavelet coefficients, have been widely utilized for image coding applications. In this research, we present a new embedded wavelet image coding algorithm with an attempt to combine advantages of these two successful coding schemes. The experimental results show that the proposed algorithm outperforms the respected zero-tree/-block coders, SPIHT and SPECK, in compression efficiency. It is also comparable to the state-of-art JPEG 2000 test coder in PSNR performance while retaining the attractive low-complexity feature of the zeroblock coders.

We consider the problems of motion-compensated frame interpolation (MCFI) and bidirectional prediction in a video coding environment. These applications generally require good motion estimates at the decoder. We use a multiscale optical-flow-based motion estimator that provides smooth, natural motion fields under bit-rate constraints. These motion estimates scale well with change in temporal resolution and provide considerable flexibility in the design and operation of coders and decoders. In the MCFI application, this estimator provides excellent interpolated frames that are superior to those of conventional motion estimators, both visually and in terms of peak signal-to-noise ratio (PSNR). We also consider the effect of occlusions in the bidirectional prediction application and introduce a dense label field that complements our motion estimator. This label field enables us to adaptively weight the forward and backward predictions and gives us substantial visual and PSNR improvements in the covered/uncovered regions of the sequence.

This paper describes a video coding system based on motion-compensated three-dimensional (3-D) subband/wavelet coding (MC-3DSBC), which can overcome the limits of both 3-D SBC and MC prediction-based coding. In this new system, spatio-temporal subbands are generated by MC temporal analysis and a spatial wavelet transform, and then encoded by 3-D subband-finite state scalar quantization (3DSB-FSSQ). The rate allocation from the GOP level to each class of subbands is optimized by utilizing the structural property of MC-3DSBC that additive superposition approximately holds for both rate and distortion. The proposed video coding system is applied to several test video clips. Its performance exceeds that of both a known MPEG-1 implementation and a similar subband MC predictive coder while maintaining modest computational complexity and memory size.

Robust transmission of images is achieved by using fixed and variable-length coding together without much loss in compression efficiency. The probability distribution function of a DCT coefficient can be divided into two regions using a threshold, so that one portion contains roughly equiprobable transform coefficients. While fixed-length coding, which is a powerful solution to the synchronization problem, is used in this inner equiprobable region without sacrificing compression, the outer (saturating) region is reserved for variable-length codes. The proposed image coder first encodes the bit allocated DCT coefficients using a fixed-rate quantizer bank, then the saturated values for these coefficients are encoded using an entropy constrained scalar quantizer, followed by an arithmetic encoder. Our simulations show that the proposed encoder is appropriate for applications in which an acceptable quality must always be maintained in any channel condition.

This paper describes an object-based video coding system with new ideas in both the motion analysis and source encoding procedures. The moving objects in a video are extracted by means of a joint motion estimation and segmentation algorithm based on the Markov random field (MRF) model. The two important features of the presented technique are the temporal linking of the objects, and the guidance of the motion segmentation with spatial color information. This facilitates several aspects of an object-based coder. First, a new temporal updating scheme greatly reduces the bit rate to code the object boundaries without resorting to crude lossy approximations. Next, the uncovered regions can be extracted and encoded in an efficient manner by observing their revealed contents. The objects are classified adaptively as P objects or I objects and encoded accordingly. Subband/wavelet coding is applied in encoding the object interiors. Simulations at very low bit rates yielded comparable performance in terms of reconstructed PSNR to the H.263 coder. The object-based coder produced visually more pleasing video with less blurriness and devoid of block artifacts, thus confirming the advantages of object-based coding at very low bit-rates.

This paper presents a new three-dimensional (3-D) Markov model for motion vector fields. The three dimensions consist of the two space dimensions plus a scale dimension. We use a compound signal model to handle motion discontinuity in this 3-D Markov random field (MRF). For motion estimation, we use an extended Kalman filter as a pel-recursive estimator. Since a single observation can be sensitive to local image characteristics, especially when the model is not accurate, we employ windowed multiple observations at each pixel to increase accuracy. These multiple observations employ different weighting values for each observation, since the uncertainty in each observation is different. Finally, we compare this 3-D model with earlier proposed one-dimensional (1-D) (coarse-to-fine scale) and two-dimensional (2D) spatial compound models, in terms of motion estimation performance on a synthetic and a real image sequence.

We present a motion-compensated interpolation scheme that allows recovery of skipped frames at the receiver. The interpolation is performed based on an object-based interpretation of the video. The main advantage of the scheme is that the decoded motion and segmentation information is used without refinement. This is attributable to the fact that the object-based representation is true to the "real" motion. We present simulation results in a very low bit-rate coding environment, where multiple frames need to be skipped. The interpolated frames show good reconstruction quality, whereas standard block-based methods lead to unacceptable artifacts.

We present a novel object-based spatiotemporal (3-D) subband/wavelet coding scheme for very low bit-rate video compression. First, the image sequence is segmented into objects based on motion and color. The objects are then independently decomposed by subband/wavelet analysis, both in the temporal and spatial domain. The temporal analysis is greatly facilitated over existing methods by the object-based representation. Each object's subband/wavelet coefficients are then quantized according to a bit allocation scheme based on the generalised BFOS algorithm. Simulations at very low bit-rates indicate competitive performance to classical motion-compensated predictive (MCP) coding schemes, while allowing the flexibility of frame-rate scalability.

We propose a novel method for designing linear pre/post filters for transform and subband/wavelet coders. We use a linear gain-plus-additive noise model to describe the effects of a class of point distribution function-optimized quantizers. Using this quantizer model, we provide an explicit expression for the power spectral density of the reconstruction error in an improved transform coding system that includes linear pre/post filters. This expression allows us to design pre/post filters that minimize the overall mean square error at a given target rate, by improving the frequency domain behavior of the overall system. For a 1-D Gauss-Markov source, we show that pre/post filters help DCT-based coders to match the performance of subband coders. We also describe an a posteriori method of designing postfilters for a JPEG coder, that uses the actual image data and maximizes the PSNR performance of the system. The postfilter coefficients are designed at the transmitter, and are sent to the receiver as part of the coded data.

We use the linear gain-plus-additive noise model for the class of PDF-optimized scalar quantizers to develop a frequency domain analysis of transform/subband coding systems. We derive a novel and explicit expression for the PSD of the reconstruction error, and show that this PSD has signal, alias and noise components. We employ this expression to compare different block transforms and subband filters with the rate-distortion optimal source coder, for a 1-D Gauss-Markov source. This paper quantifies, via the 'out-of-band' noise energy in the reconstruction error, the performance gap between DCT-based coders and coders that use subband/wavelet filters.

This paper presents three-dimensional (spatio-temporal) Kalman filters for video as the extension of the two-dimensional (2-D) reduced update Kalman filter (RUKF) approach for images. We start out with three-dimensional (3-D) RUKF, a shift-invariant recursive estimator with efficiency advantages over the 3-D Wiener filter. Then, we turn to the motion-compensated extension MC-RUKF, which gives improved performance when coupled with a motion estimator. Since motion compensation sometimes fails, causing severe fluctuations in temporal correlation, we then present multimodel MC-RUKF, to adapt to variation in temporal and spatial correlation, by detecting the local image model out of a class, and using it in MC-RUKF. Finally, we introduce a novel multiscale model detection algorithm for use in high noise environments.

Both the approximate reduced updated Kalman filter (RUKF) and reduced order model Kalman filter (ROMKP) have been used in image processing for some time. This correspondence shows that the ROMKF can be interpreted as a special case of the approximate RUKF and in so doing, provides a way to estimate a needed error variance input to ROMKF.

We present a systematic approach to forward-motion-compensated predictive video coding. The first step is the definition of a flexible model that compactly represents motion fields. The inhomogeneity and spatial coherence properties of motion fields are captured using linear multiscale models. One possible design is based on linear finite elements and yields a multiscale extension of the triangle motion compensation (TMC) method. The second step is the choice of a computational technique that identifies the coefficients of the linear model. We study a modified optical flow technique and minimize a cost function closely related to Horn and Schunck's (1981) criterion. The cost function balances accuracy and complexity of the motion compensated predictor and is viewed as a measure of goodness of the motion field. It determines not only the coefficients of the model, but also the quantization method. We formulate the estimation and quantization problems jointly as a discrete optimization problem and solve it using a fast multiscale relaxation algorithm. A hierarchical extension of the algorithm allows proper handling of large displacements. Simulations on a variety of video sequences have produced improvements over TMC and over the half-pel-accuracy, full-search block matching algorithm, in excess of 0.5 dB in average. The results are visually superior as well. In particular, the reconstructed video is entirely free of blocking artifacts.

Motion estimation is an important component of video coding systems because it enables us to exploit the temporal redundancy in the sequence. The popular block-matching algorithms (BMAs) produce unnatural, piecewise constant motion fields that do not correspond to "true" motion. In contrast, our focus here is on high-quality motion estimates that produce a video representation that is less dependent on the specific frame-rate or resolution. To this end, we present an iterated registration algorithm that extends previous work on multiscale motion models and gradient-based estimation for coding applications. We obtain improved motion estimates and higher overall coding performance. Promising applications are found in temporally-scalable video coding with motion-compensated frame interpolation at the decoder. We obtain excellent interpolation performance and video quality; in contrast, BMA leads to annoying artifacts near moving image edges.

We consider the adaptive restoration of inhomogeneous textured images, where the individual regions are modeled using a Wold-like decomposition. A generalized Wiener filter is developed to accommodate mixed spectra, and unsupervised restoration is achieved by using the expectation-maximization (EM) algorithm to estimate the degradation parameters. This algorithm yields superior results when compared with supervised Wiener filtering using autoregressive (AR) image models.

A new image-coding algorithm that exploits the relationship between various subbands of an image is presented in this correspondence. Using local variance for state selection and uniform threshold quantizers on subbands, we have achieved a reduction in quantization noise compared to other published techniques for Lena at various bit rates.

This paper presents a maximum-likelihood solution to the general problem of fitting a parametric model to observations from a single realization of a two-dimensional (2-D) homogeneous random field with mixed spectral distribution. On the basis of a 2-D Wold-like decomposition, the field is represented as a sum of mutually orthogonal components of three types: purely indeterministic, harmonic, and evanescent. The suggested algorithm involves a two-stage procedure. In the first stage, we obtain a suboptimal initial estimate for the parameters of the spectral support of the evanescent and harmonic components. In the second stage, we refine these initial estimates by iterative maximization of the conditional likelihood of the observed data, which is expressed as a function of only the parameters of the spectral supports of the evanescent and harmonic components. The solution for the unknown spectral supports of the harmonic and evanescent components reduces the problem of solving for the other unknown parameters of the field to a linear least squares. The Cramer-Rao lower bound on the accuracy of jointly estimating the parameters of the different components is derived, and it is shown that the bounds on the purely indeterministic and deterministic components are decoupled. Numerical evaluation of the bounds provides some insight into the effects of various parameters on the achievable estimation accuracy. The performance of the maximum-likelihood algorithm is illustrated by Monte Carlo simulations and is compared with the Cramer-Rao bound.

Motion estimation is an important part of most video coding schemes because it enables us to exploit the high degree of temporal redundancy present. Though block matching algorithms (BMA) yield coarse and piecewise-constant fields, they are very popular due to their simplicity and low bit overhead. In this paper, we propose to use a more advanced gradient-based technique to overcome the disadvantages of BMA. A dense motion field is estimated and compressed using a hierarchical finite element (HFE) representation, leading to an efficient, highly parallel, iterative, multiresolution optimization algorithm. The scheme also uses multiresolution measurements and a coarse-to-fine strategy to estimate large displacements. At comparable bit rates, the motion fields are much smoother and more natural than those produced by BMA. Coding gains of about 0.6 dB were obtained on Claire. More importantly, substantial visual improvements were obtained, mainly due to improved performance near the edges.

Multiresolution, or scalable, video coding has emerged to be of great interest in the compression community. It has been recognized that many applications, such as, image database browsing, desktop computing, terrestrial broadcasting, and others, can benefit from multiresolution viewing of embedded data sets. In this paper we present a new three-dimensional subband coder which provides six different spatio-temporal resolutions of a video signal. Motion-compensated temporal filtering (MCTF) is used for temporal analysis. The encoding algorithm efficiently compresses the data from the six resolutions into a single bit-stream while removing data which is redundant across different resolutions. Our algorithm successfully merges spatial and temporal scalability without imposing restrictions on which of the six video resolutions can be decoded, while allowing optimal encoding of the video at each spatial and temporal resolution.

In the original paper by Sharifi and Leon-Garcia (see ibid., vol.5, p. 52-56, Feb. 1995) the authors propose a method to estimate the shape parameter for a generalized Gaussian probability density function to be fitted to the subbands of a video signal. Their method, which uses certain moments of the subband histogram, has in fact already been presented some years ago by Mallat (1989). The authors reply that they thank Murching and Woods for bringing to their attention a previously published work, which has developed the method reported in their paper. They regret that their search of the previous related literature did not reveal the paper by Mallat. They note, however, that the effectiveness of the method in comparison to statistical goodness-of-fit tests, was shown in their work and not in Mallat's.

Multiresolution transmission of compressed video is performed using subbands. Multistage quantization is necessary for efficient embedded video transmission on rate constrained channels. This paper investigates the performance of a number of multistage scalar quantization techniques: concatenated coding, conditional entropy coding and conditional quantization. Intraframe as well as interframe coding approaches are investigated with these multistage quantization techniques on high definition and standard definition source material.>

Abstruct-We present a solution to the problem of modeling, parameter estimation, and synthesis of natural textures.The texture field is assumed to be a realization of a regular homogeneous random field, which can have a mixed spectral distribution.On the basis of a 2-D Wold-like decomposition, the field is represented as a sum of a purely indeterministic component, a harmonic component, and a countable number of evanescent fields.We present a maximum-likelihood solution to the joint parameter estimation problem of these components from a single observed realization of the texture field.The proposed solution is a twostage algorithm.In the first stage, we obtain an estimate for the number of harmonic and evanescent components in the field, and a suboptimal initial estiqate for the parameters of their spectral supports.In the second stage, we refine these initial estimates by iterative maximization of the likelihood function of the observed data.By introducing appropriate parameter transformations the highly nonlinear least-squares problem that results from the maximization of the likelihood function, is transformed into a separable least-squares problem.In this new problem, the solution for the unknown spectral supports of the harmonic and evanescent components reduces the problem of solving for the transformed parameters of the field to linear least squares.Solution of the transformation equations then provides a complete solution of the field-model parameter estimation problem.The Wold-based model and the resulting analysis and synthesis algorithms are seen applicable to a wide variety of texture types found in natural images.The support or shape of the analyzed texture patch may be arbitrary.Our model is very efficient in terms of the number of parameters required to represent and faithfully reconstruct the original texture.

A new algorithm is developed for the vector quantization of motion vectors. This algorithm, called motion vector quantization (MVQ), simultaneously estimates and vector quantizes the motion vectors by reinterpreting the block matching algorithm as a type of vector quantization. An iterative design algorithm, based on this concept, is developed. In addition to reducing rate for fixed length encoding, the algorithm also reduces the computation considerably. We include coding simulation results on the Flower Garden sequence.

Conventional coding of digital color images is done in a luminance-chrominance domain, and the chrominance components are subsampled in a fixed ratio with respect to the luminance. We propose an adaptive method of subsampling the chrominance components that improves the quality vs. sample rate trade-off. In applications such as compositing color graphics images, the advantage of using the adaptive method as against the conventional method is demonstrated.>

Several methods to perform multiresolution transmission of HDTV through subband coding are compared. The first is simple intraframe encoding of the spatial subbands of the video. The authors have used the subband decomposition for both hierarchical estimation of the motion as well as encoding of the compensated error function. They have used block matching for motion estimation on both 2D lowpass and bandpass (QMF) pyramids. They found that the motion estimation on the lowpass QMF pyramid performs better than the motion estimation on the bandpass QMF pyramid. They achieved approximately the same quality reconstructed video with either backward or forward motion compensated schemes. For multiresolution transmission, by simulating a test sequence, they have obtained a PSNR of 35 dB at the bit rate of 0.44 bits per pixel with adaptive PCM and at a bit rate of 0.53 bits pet pixel with PCM. Such a result would require 0.75 bits per pixel for the intraframe scheme. The complexity of the motion compensated multiresolution coding scheme is comparable to that of a single resolution method.>

When faced with a large support point spread function (PSF), the iterative expectation-maximization (EM) algorithm, which is often used for PSF identification, is very sensitive to the initial PSF estimate. To deal with this problem, the authors propose to do EM image identification and restoration in the subband domain. After the image is first divided into subbands, the EM algorithm is applied to each subband separately. Since the PSF can be taken to have smaller support in each subband, these subbands should be less of a problem with the EM model identification. They also introduce an adaptive subband EM method for use in the upper frequency subbands.

We consider the adaptive restoration of inhomogeneous textured images degraded by linear blur and additive white Gaussian noise. The method consists of segmenting the image into individual homogeneous textures and restoring each texture separately. The individual textures are assumed to be realizations of 2-D Wold-decomposition based regular, homogeneous random fields which may possess deterministic components. The conventional Wiener filter assumes that the spectral distribution of the signal is absolutely continuous and, therefore, cannot be directly used to restore the individual textures. A generalized Wiener filter accommodates the unified texture model and is shown to yield minimum mean-squared error estimates for fields with discontinuous spectral distributions. Texture discrimination is performed by obtaining maximum a posteriori estimates for the label field using simulated annealing. The performance of our segmentation algorithm is investigated in the presence of noise.

New methods for the subsampling of color images are presented. A block-based orthogonalizing transform is used to represent the RGB image in an efficient domain. This transformation offers substantial advantages when preceded by a segmentation of the image based on chrominance content. An adaptive scheme, based on a threshold test on the local variance, is proposed for subsampling the resulting chrominance-related components. The proposed subsampling scheme outperforms conventional methods based on transforms such as the YCRCB, in terms of an improved quality versus sample-rate (or bit-rate) trade-off. It is possible to achieve 4:4:4 quality at near 4:2:0 rates. A block-based strategy to code the resulting color image components is also presented, which is based on the JPEG Baseline method.

A maximum-likelihood (ML) estimation procedure for the parameters of homogeneous random fields is presented. On the basis of a 2D Wold-like decomposition, the random field is decomposed into a sum of mutually orthogonal components of three types: purely indeterministic, harmonic, and generalized evanescent. Because, in general, a homogeneous random field is characterized by a mixed spectral distribution, parameter estimation techniques which are solely based on spectral density estimators are not adequate. An algorithm which results in a complete parameter estimation for the purely indeterministic and harmonic components of the homogeneous random field and in estimates for the evanescent component frequencies is presented. The new ML method produces highly accurate estimates of these parameters.>

When faced with a large-sized point spread function, the expectation-maximization (EM) algorithm is very sensitive to local minima. To deal with this problem, it is proposed that EM image identification and restoration be done in the subband domain. After the image is first divided into subbands, then the EM algorithm is applied to each subband separately. In each subband the point spread function can be modeled by a reduced number of parameters and the image model can be better represented also. An adaptive subband EM method for quantization of the upper frequency subbands is introduced.>

Recently, Uz et al. (ibid., vol.1., no.1, p.86-99, 1991) analyzed the propagation of quantization noise in a pyramid (with feedback) and subband decomposition schemes. In this study each band was independently quantized by a scalar quantizer of equal step size. The resulting reconstruction error spectrum indicated that in both the pyramid (without feedback) and subband coding schemes, noise was building up in lower frequencies. The commenters show that a quantizer assignment method, using mean-square-error (MSE) optimal bit allocation, avoids the problem. The authors argue that the problem is more involved than just MSE optimal bit allocation, and that compatible coding was the focus, as well as guaranteed quality, effects of numerical computations, and perceptual effects. The commenters further support their argument.>

The authors compare the subband compression capabilities of eight filter sets (consisting of linear-phase quadrature mirror filters (QMFs), perfect reconstruction filters, and nonlinear phase wavelets) at different bit rates, using-a filter-based bit allocation procedure. Using DPCM and PCM in HDTV subband coding, it is found that QMFs have an edge over the rest.

A parallel simulated annealing algorithm is presented for image restoration using a compound Gauss-Markov field model for the image. Results are provided for its implementation on a distributed array processor (DAP510) which is a single instruction multiple data (SIMD) machine with 1024(32*32) mesh-connected processor elements, and a clock rate of 10 MHz. The total time required for restoring a monochrome blurred and noisy image with continuous range of intensities is reduced to about 10 minutes as compared to 20 hours for its sequential implementation (VAX11/785). Both the maximum a posteriori (MAP) and minimum mean square error (MMSE) estimates of the original image are obtained. The parallel estimates are shown, as well as the sequential estimate and the classical Wiener filter estimate.>

An entropy-constrained algorithm for a type of finite-state vector quantizer called sliding-block predictive vector quantization (SBPVQ) is presented. This coding algorithm searches small codebooks to achieve high performance at low coding rates. Results from training the coder on a set of monochrome images and then testing it on an image outside the training set are presented. Also, three parallel entropy-constrained SBPVQ coders are used to code 24-b/pixel color images. A total rate of less than 0.5 b/pixel is achieved with good performance by using this vector predictor variant.>

Recently, a stochastic relaxation technique called simulated annealing has been developed to search for a globally optimal solution in image estimation and restoration problems. The convergence of simulated annealing has been proved only for random fields with a compact range space. Because of this, images were modeled as random fields with bounded discrete or continuous values. However, in most image processing problems, it is more natural to model the image as a random field with values in a noncompact space, e.g. conditional Gaussian models. The proof of convergence of the stochastic relaxation method is extended to a class of compound Gauss-Markov random fields. Simulation results are provided to show the power of these methods.>

The authors compare various options for combining linear prediction with vector quantization for purposes of image coding. A previously developed algorithm called sliding block predictive vector quantization (SBPVQ) is extended with multiple-level quantizers (i.e. n>2), and is compared to vector-predictive quantization (VPQ). The particular version considered here is termed sliding block vector predictive quantization (SBVPQ) and uses a (M,L) tree search on the encoded blocks. Arithmetic coding is used to transmit the SBPVQ and SBVPQ encoded data at rates closer to their output entropy than rates obtained with fixed-rate coding.>

The authors propose a compound random field for texture discrimination called the doubly stochastic Gaussian (DSG) random field, to reduce isolated errors. Two major advantages of the DSG model are that it is easy to extract the features (the autoregressive parameters) and the a priori information can be incorporated into the model through the probability function of the lower level field. Experimental results on synthetic and natural images are presented. The results are quite good for the cases of both supervised and unsupervised models obtained from the simulated annealing algorithm and the HCF (highest confidence first) algorithm.>

Concerns developing algorithms for obtaining the maximum a posteriori probability (MAP) estimate from blurred and noisy images modeled as compound Gauss-Markov random fields. These models consist of several image submodels having different characteristics along with a structure model, a 2D Markov chain, which governs transitions between these image submodels. Compound random field models are attractive for image estimation because the resulting estimates do not suffer the over-smoothing of edges that occurs when one employs linear shift-invariant (LSI) models.>

A decision-directed filtering algorithm has been developed for model-based segmentation and restoration of images degraded by a class of space-variant blurs. It is assumed that the space-variant blur can be represented by a collection of L distinct point-spread functions, where L is a predetermined integer, so that at each pixel one of the functions will be more or less matched to the observed data. A multiple-model Kalman filtering procedure with online model detection based on maximum a posteriori probability decision was used to restore the image. The result of the decision process constitutes a model-based segmentation of the degraded image into regions of spatially invariant blurs. There are several applications of the proposed algorithm. Among these, automatic blur detection, i.e. segmentation of a partially blurred image into regions of blur and no-blur, is demonstrated as an example.>

A novel two-dimensional subband coding technique is presented that can be applied to images as well as speech. A frequency-band decomposition of the image is carried out by means of 2D separable quadrature mirror filters, which split the image spectrum into 16 equal-rate subbands. These 16 parallel subband signals are regarded as a 16-dimensional vector source and coded as such using vector quantization. In the asymptotic case of high bit rates, a theoretical analysis yields that a lower bound to the gain is attainable by choosing this approach over scalar quantization of each subband with an optimal bit allocation. It is shown that vector quantization in this scheme has several advantages over coding the subbands separately. Experimental results are given, and it is shown the scheme has a performance that is comparable to that of more complex coding techniques.>

The improvements in VLSI technology over the past few years have made high-speed digital filtering commonplace. This paper presents a video rate architecture for a two-dimensional (2-D) filter which can be implemented using wafer scale integration. The filter is made up of 1-D row operators. These row operators in turn are realized as a block-processed system composed of three basic cells: multipliers, adders, and registers. The row operators are combined recursively to form a 2-D fully recursive filter. A practical application of image enhancement with this filter is presented.

This paper presents parallel realizations of the two-dimensional reduced update Kalman filter. It is shown that sufficiently spaced pixels on certain diagonals may be predicted and updated in parallel. This parallelism is then exploited on a conventional shared memory parallel processor and timings are reported. Predicted performance is provided for area-array SIMD type parallel processors, e.g. CLIP4 and CLIP7.

The two-dimensional (2-D) doubly stochastic Gaussian (DSG) model was introduced by one of the authors to provide a complete model for spatial filters which adapt to the local structure in an image signal. Here we present the optimal estimator and 2-D fixed-lag smoother for this DSG model extending earlier work of Ackerson and Fu. As the optimal estimator has an exponentially growing state space, we investigate suboptimal estimators using both a tree and a decision-directed method. Experimental results are presented.

In this paper a parallel scheme is presented for the identification of two-dimensional noncausal image blurs. It is shown that the blur identification problem can be specified as a parallel set of one-dimensional complex autoregressive moving average (ARMA) identification problems. By expressing the ARMA models as equivalent infinite-order auto-regressive (AR) models, an entirely linear estimation procedure can be followed yielding minimum phase solutions. Further, it will be shown that under certain conditions (blur symmetry) it is possible to reconstruct a useful noncausal set of MA (blur) parameters from the identified minimum-phase set. Several identification results are given as examples.

This paper presents an extension of sub-band coding to two-dimensions with particular application to images. We employ a 16 band decomposition using a tree structure of separable quadrature mirror filters and decimators. The sub-bands are encoded using DPCM with bits allocated to approximately minimize the mean-square error. A block-adaptive variant is also presented. A limited SNR comparison shows adaptive sub-band coding to outperform the adaptive discrete cosine transform and two types of vector quantizers.

An optimal statistical parameter estimation technique is presented for the identification of unknown image and blur-model parameters. Maximum likelihood estimates of the unknown parameters are derived both in the absence and in the presence of observation noise. The proposed algorithms are able to locate the zeros of the observed image spectrum on the entire Z1-Z2plane and therefore, unlike previous algorithms, are not restricted to the Fourier domain. Images restored with the proposed algorithms are shown as examples.

Recursive spatial domain filtering is often used in image restoration. The techniques range from deterministic inverse filter algorithms [1] to stochastic Kalman filters [2,3]. However, all the techniques have in common the problem of choosing the appropriate boundary values. It is the purpose of this paper to demonstrate the importance of the boundary values in image restoration and to show how we can improve the transient response of the steady-state Kalman filters in [2] and [3] with better choices for the boundary values.

In general, images are inhomogeneous and no single model can accurately represent all the N×N data points of an image. Thus the linear space-invariant (LSI) filter can not produce the best estimates, especially at the lower SNR's. In fact, LSI filters tend to smooth the edges excessively when estimating undistorted images corrupted by additive white Gaussian noise. If we transform the original image space to a more appropriate space, and then process images in the new space, we may obtain better visual quality and lower numeric error also. Investigating such a transformation is the main concept of this paper.

The purpose of this paper is to present new image coding schemes based on a predictive vector quantization (PVQ) approach. The predictive part of the encoder is used to partially remove redundancy, and the VQ part further removes the residual redundancy and selects good quantization levels for the global waveform. Two implementations of this coding approach have been devised, namely, sliding block PVQ and block tree PVQ. Simulations on real images show significant improvement over the conventional DPCM and tree codes using these new techniques. The strong robustness property of these coding schemes is also experimentally demonstrated.

This paper compares four image filtering algorithms on common data sets for various signal to noise ratios and white Gaussian noise. The algorithms are the median filter, the Wallis filter, the reduced update Kalman filter, and a multiple model, decision-directed filter. This comparison produces some surprising results from both a subjective (visual error) and a mean square error (MSE) viewpoint.

In a recent paper [1] a model for radar images was derived and a method for mean-square error (MSE) filtering of noisy radar images was presented. The purpose of this correspondence is to point out that the filter in [1] is not optimum in the MSE sense and to show that the ultimate filter scheme is based on a very restricted model for radar images.

The near merging of paths in tree searching is explored. This near merging can degrade coding performance when a suboptimal search algorithm is used. The problem is first identified and then a feasible solution is presented. Some examples from image source coding using the(M, L)algorithm are given.

In this paper we present results from our work on adaptive prediction dpcm for grey level images using the doubly stochastic Gaussian image model. This model consists of a lower level level 2-D Markov chain which takes on L different values and correspondingly L different sets of predictor parameters for an upper level conditionally Gaussian field. A simple gradient based algorithm which uses fuzzy decision theory is used to identify the lower level chain from observations on the upper level field, i.e. the actual image. The upper level field is encoded by 2-D DPCM using spatially varying predictors as determined by the lower level chain. For our simulations L was chosen to be 5, with 4 models representing edges at 0, 45, 90, and 135 degrees and the fifth model representing the non-edge regions. Both fixed and adaptive (Jayant type) quantizers were used. Greater compression is achieved by subsampling in the non-edge regions of the image and then interpolating at the decoder.

We present a multiple model image restoration technique with on-line edge detection over noisy and blurred images. Four edge models corresponding to major correlation directions and an isotropic model are used to represent the image. Unlike previous results, a decision-directed approach is used to adaptively estimate the edge orientation which then defines the appropriate deconvolution model. The overall effect is space-variant deconvolution implemented by efficient reduced update filters. Processed images are shown as examples.

We present a recursive estimation algorithm for autoregressive moving average (ARMA) random field models. The estimator is an extension to ARMA models of the efficient reduced update Kalman filter (RUKF). We also discuss the identification of the ARMA model parameters from noise-free image data. The ARMA estimator is run on several sets of random field data as well as on real images. The experimental results are compared to those achievable using an AR model and RUKF on the same data fields. We find no significant improvement for comparable model orders.

This paper presents an extension of the spectral factorization based optimization method to approximate both magnitude and phase specifications for 2-D, recursive, direct-form, nonsymmetric half-plane filters. Examples are presented of filters approximating ideal linear phase lowpass and fan filters as well as causal (in the nonsymmetric half-plane sense) Wiener filters which have non-linear phase characteristics.

This paper compares the steady-state reduced update Kalman filter to the unrealizable Wiener filter for the two-dimensional LMMSE estimation of imaqes and random fields. The comparison is composed of three parts: experimental MSE performance, subjective quality of the estimates, and computational complexity. The performance comparison is conducted on both real and synthetic image data. The Wiener filters are designed using both estimated power density spectra and the AR models necessary for the Kalman filter. These AR models are determined using 2-D linear prediction techniques on real image data. The computational comparison considers both multiplies and adds as well as amount and type of required memory.

In this paper, we demonstrate the application of the reduced update Kalman filter in the enhancement of two-dimensional images using a composite model description of the image. Typically, for the purpose of simulation, five models corresponding to four predominant correlation directions (at angles of 0°, 45°, 90°, 135° to the horizontal) and one isotropic model, are considered. These models are then used to synthesize a filtering algorithm that estimates the image with near minimum mean square error. The results show considerable improvement in the visual quality compared with linear constant coefficient Kalman filtering.

In the above-named work, the two-dimensional (2-D) reduced update Kalman filter was introduced as an optimal approximation to the 2-D Kalman filter for scalar observations obtained from a raster scan. The reduced update filter consists of two parts: a prediction part and a reduced update part, i.e., an update of

In this paper a new design algorithm for two-dimensional (2-D) recursive digital filters is presented, with emphasis on the general class of half-plane filters. A recently developed 2-D spectral factorization procedure and a nonlinear optimization algorithm are incorporated to iteratively converge to a stable, (locally) optimum filter. Details of the computations required in the implementation of the design procedure are presented, in addition to an example of its application.

In this paper we demonstrate a new application of two-dimensional (2-D) signal processing. The 2-D data consists of frequency-time plots or spectrograms derived from non-stationary one-dimensional data. We show that 2-D processing can offer significant advantages over the standard spectral estimation procedures used to obtain spectrograms. These advantages mainly occur because the frequency data is correlated in time, a fact ignored in conventional spectrogram procedures. Recursive fan type filters are designed by the homomorphic method and applied to the spectrogram data. Significant improvement is obtained for a simulation consisting of chirp type signals in white noise.

A new facsimile source coding algorithm, called twodimensional (2-D) delta-mod (Δmod), is presented. The algorithm is based on conventional run length coding. Additional compression is achieved by vertically delta-modulating the horizontal run lengths. The delta-modulation introduces a distortion into the image which can be traded-off versus the rate of transmission. Cumulative error is prevented, as in conventional Δ mod, by including the (run length) quantizer in the feedback loop. Some parametric results are presented

The Kalman filtering method is extended to two dimensions. The resulting computational load is found to be excessive. Two new approximations are then introduced. One, called the strip processor, updates a line segment at a time; the other, called the reduced update Kalman filter, is a scalar processor. The reduced update Kalman filter is shown to be optimum in that it minimizes the post update mean-square error (mse) under the constraint of updating only the nearby previously processed neighbors. The resulting filter is a general two-dimensional recursive filter.

A constructive proof is given for the existence and uniqueness of a two-dimensional discrete Markov random field which agrees with correlation values in a nearest neighbor array. The corresponding spectrum is the two-dimensional maximum entropy (ME) spectrum whose form was discovered by Burg. An iterative algorithm is developed for computing an approximation to this Markov spectrum for a regularly spaced array. The algorithm approximates the desired Markov correlation function by a truncated convolution power series (CPS) in an operatorh. The algorithm's performance is demonstrated on both simulated data and real noise data. The Markov spectral estimate can offer higher resolution than previously proposed spectral estimates.

A definition of discrete Markovian random fields is formulated analogously to a definition for the continuous case given by Lévy. This definition in the homogeneous Gaussian case leads to a difference equation that sets forth the state of the field in terms of its values on a band of minimum widthP, wherePis the order of the process. The state of the field at position(i,j)is given by the set of values of the nearest neighbors within distancePof the point(i,j). Conversely, given a difference equation satisfying certain conditions relating to stability, there corresponds a homogeneous discrete Markov random field. This theory is applied to the problem of obtaining spectral estimates of a two-dimensional field, given observation over a limited aperture.