Recent developments in intra-prediction show the advantage of increasing the number of prediction directions. This trend can be observed in most previous standards, and should remain in the future. However, increasing the number of intra-prediction directions incurs a non-negligible side-information bit-rate. To reduce this side-information bit-rate, we propose a method to adaptively decide whether to use a larger or smaller set of candidate intra-prediction directions by simply evaluating the maximum gradient magnitude, theoretically motivated by the prediction inaccuracy model. In this scenario, we can achieve most of the gain from using the larger intra-prediction direction set while only requiring a small amount of side-information. This method yields a significant BD-rate reduction on multiple resolutions of images using fixed block-size when implemented in a simplified HEVC-based encoder.

In many video coding systems, motion compensation is used to reduce temporal correlation. Motion-compensated residuals are encoded with transforms. In this paper, we develop transforms for motion-compensated residuals based on prediction inaccuracy modeling. Specifically, we observe that motion-compensation is very effective in smooth regions and still regions. In regions where strong motions occur, the accuracy of motion-compensation is sensitive to the accuracy of motion vectors and local changes of image data. From this observation, we propose a model for the motion-compensated residuals. Based on this model, we relate the residual covariance function to the gradient function of the reference frame. The KLT of the covariance function is proposed. Experimental results show that a significantly smaller number of transform coefficients are needed in preserving the same amount of energy with the proposed transform.

Recent studies have shown that there may be a significant improvement in the compression of a signal when multiple-transforms are used to exploit local characteristics [1]. By choosing an appropriate transform from a set of transforms, we can have a significant encoding improvement. In this paper, our objective is to find which transform is optimal for each block of a given signal, and to determine how many coefficients we should allocate in each block, from an energy compaction perspective, as defined in [2]. Two algorithms were proposed in [2] to solve this problem. The first algorithm, called Algorithm A, is an iterative method, that converges to a locally optimal solution. The second algorithm, called Algorithm B, first considers an alternate metric based on rate distortion and uses convex optimization techniques.We propose two new methods to solve this problem [3]. The first method, called Algorithm C, uses thresholding to solve the rate-distortion optimization problem. We threshold the magnitude-squared of the transform coecients by the rate-distortion parameter for all transforms, and choose the best transform. The second method, called Algorithm D, uses dynamic programming to find the best transform and coefficient allocation for the current block based on that of all the previous blocks. This method solves the problem of finding the best transform and coecient allocation in terms of energy compaction. In comparing these methods, Algorithm A is the fastest, although only locally optimal. Algorithm C is faster than Algorithm B. Both Algorithm B and Algorithm C solve only the rate-distortion problem. Algorithm D solves the original energy compaction problem, but is the slowest among the four methods.

In intra video coding, intra frames are predicted with intra prediction and the prediction residual signal is encoded. In many transform-based video coding systems, intra prediction residuals are encoded with transforms. For example, the Discrete Cosine Transform (DCT) and the Asymmetric Discrete Sine Transform (ADST) are used for intra prediction residuals in many coding systems. In the recent work, a set of transforms based on prediction inaccuracy modeling (PIM) has been proposed. These transforms are developed based on the observation that much of the residual non-stationarity is due to the use of an inaccurate prediction parameter. These transforms are shown to be effective for non-stationarity that arises in directional intra prediction residuals. In this paper, we implement the transforms based on prediction inaccuracy modeling on the H.264 intra coding system. The proposed transform is used in hybrid with the ADST. We compare the performance of the hybrid transform with the ADST and show that a significant bit-rate reduction is obtained with the proposed transform.

In intra video coding and image coding, the directional intra prediction is used to reduce spatial redundancy. Intra prediction residuals are encoded with transforms. In this paper, we develop transforms for directional intra prediction residuals. In particular, we observe that the directional intra prediction is most effective in smooth regions and edges with a particular direction. In the ideal case, edges can be predicted fairly accurately with an accurate prediction direction. In practice, an accurate prediction direction is hard to obtain. Based on the inaccuracy of prediction direction that arises in the design of many practical video coding systems, we can estimate the residual covariance and propose a class of transforms based on the estimated covariance function. The proposed method is evaluated by the energy compaction property. The experimental results show that, with the proposed method, the same amount of energy in directional intra prediction residuals can be preserved with a significantly smaller number of transform coefficients.

In intra video coding and image coding, the directional intra prediction is used to reduce spatial redundancy. Intra prediction residuals are encoded with transforms. In this paper, we develop transforms for directional intra prediction residuals. In particular, we observe that the directional intra prediction is most effective in smooth regions and edges with a particular direction. In the ideal case, edges can be predicted fairly accurately with an accurate prediction direction. In practice, an accurate prediction direction is hard to obtain. Based on the inaccuracy of prediction direction that arises in the design of many practical video coding systems, we can estimate the residual covariance and propose a class of transforms based on the estimated covariance function. The proposed method is evaluated by the energy compaction property. The experimental results show that, with the proposed method, the same amount of energy in directional intra prediction residuals can be preserved with a significantly smaller number of transform coefficients.

In video coding, motion compensation prediction residuals are typically compressed by applying two-dimensional transforms such as the two-dimensional discrete cosine transform (2D-DCT). Using direction-adaptive one-dimensional discrete cosine transforms (1D-DCTs) can provide significant additional bitrate savings. However, this requires optimization over all available transforms to minimize the overall bitrate, which can be expensive in terms of time and computation. We examine the use of only the horizontal and vertical 1D-DCTs in addition to the 2D-DCT for coding motion compensation residuals. By reducing the number of available transforms, the amount of required computation decreases significantly, with a potential cost in performance. We perform experiments using a modified H.264/AVC codec to compare the performance of using different sets of available transforms. The results indicate that for typical applications of video coding, most of the performance benefit from using directional 1D-DCTs can be retained by keeping only the horizontal and vertical 1D-DCTs.

With a proper transform, an image or motion-compensated residual can be represented quite accurately with a small fraction of the transform coefficients. This is referred to as the energy compaction property. When multiple block transforms are used, selecting the best transform for each block that leads to the best energy compaction is difficult. In this paper, we develop two algorithms to solve this problem. The first algorithm, which is computationally simple, leads to a locally optimal solution. The second algorithm, which is more computationally intensive, gives a globally optimal solution. We discuss the algorithms and their performances. Two-dimensional discrete cosine transform (2-D DCT) and direction-adaptive one-dimensional discrete cosine transforms (1-D DCTs) are used to evaluate the performance of our algorithms. Results obtained are consistent with those from previous research.

Transforms used in image coding are also commonly used to compress prediction residuals in video coding. Prediction residuals have different spatial characteristics from images, and it is useful to develop transforms that are adapted to prediction residuals. In this paper, we explore the differences between the characteristics of images and motion compensated prediction residuals by analyzing their local anisotropic characteristics and develop transforms adapted to the local anisotropic characteristics of these residuals. The analysis indicates that many regions of motion compensated prediction residuals have 1-D anisotropic characteristics and we propose to use 1-D directional transforms for these regions. We present experimental results with one example set of such transforms within the H.264/AVC codec and the results indicate that the proposed transforms can improve the compression efficiency of motion compensated prediction residuals over conventional transforms.

Typically the same transforms, such as the 2-D Discrete Cosine Transform (DCT), are used to compress both images in image compression and prediction residuals in video compression. However, these two signals have different spatial characteristics. In, we analyzed the difference between these two signals and proposed 1-D directional transforms for prediction residuals. In this paper, we provide further experimental results using these transforms in the H.264/AVC codec and present other related information which can provide insights in understanding the use of these transforms in video coding applications.

The Discrete-Cosine-Transform (DCT) is the most widely used transform in image and video compression. Its use in image compression is often justified by the notion that it is the statistically optimal transform for first-order Markov signals, which have been used to model images. In standard video codecs, the motion-compensation residual (MC-residual) is also compressed with the DCT. The MC-residual may, however, possess different characteristics from an image. Hence, the question that arises is if other transforms can be developed that can perform better on the MC-residual than the DCT. Inspired by recent research on direction-adaptive image transforms, we provide an adaptive auto-covariance characterization for the MC-residual that shows some statistical differences between the MC-residual and the image. Based on this characterization, we propose a set of block transforms. Experimental results indicate that these transforms can improve the compression efficiency of the MC-residual.

Various directional transforms have been developed recently to improve image compression. In video compression, however, prediction residuals of image intensities, such as the motion compensation residual or the resolution enhancement residual, are transformed. The applicability of the directional transforms on prediction residuals have not been carefully investigated. In this paper, we briefly discuss differing characteristics of prediction residuals and images, and propose directional transforms specifically designed for prediction residuals. We compare these transforms with the directional transforms proposed for images using prediction residuals. The results of the comparison indicate that our proposed directional transforms can provide better compression of prediction residuals than the directional transforms proposed for images.

Multiple description (MD) video coding can be used to reduce the detrimental effects caused by transmission over lossy packet networks. Each approach to MD coding consists of a tradeoff between compression efficiency and error resilience. How effectively each method achieves this tradeoff depends on the network conditions as well as on the characteristics of the video itself. This paper proposes an adaptive MD coding approach which adjusts to these conditions through the use of adaptive MD mode selection. The encoder in this system is able to accurately estimate the expected end-to-end distortion, accounting for both coding and packet-loss-induced distortions, as well as for the bursty nature of channel losses and the effective use of multiple transmission paths. With this model of end-to-end expected distortion, the encoder selects between MD coding modes in a rate-distortion optimized manner to most effectively trade-off compression efficiency for error resilience. We show how this approach adapts to the local characteristics of the video as well as to current network conditions and demonstrate the resulting gains in performance.

In conventional bit-rate control, the buffer level is controlled by adapting the quantization step size with a fixed frame rate and spatial resolution. We consider a multidimensional (M-D) bit-rate control where the frame rate, spatial resolution and quantization step size are jointly adapted for buffer control. We introduce a fundamental framework to formalize the description of the M-D buffer-constrained allocation problem. Given a set of operating points on a M-D grid to code a nonstationary source in a buffer-constrained environment, we formulate the optimal solution. The formulation allows a skipped frame to be reconstructed from one coded frame using any temporal interpolation method and is shown to be a generalization of formulations considered in the literature. In the case of intraframe coding, a dynamic programming algorithm is introduced to find the optimal solution. The algorithm allows one to compare operational rate-distortion bounds of the M-D and conventional approaches. We also discuss how a solution can be obtained for the case of interframe coding using the optimal dynamic programming algorithm for intraframe coding by making an independent allocation approximation. We illustrate that the M-D approach can provide bit-rate reductions over 50%. We also show that the M-D approach with limited-lookahead provides a slightly suboptimal solution that consistently outperforms the conventional approach with full-lookahead.

Transform/subband representations form an important element of many signal processing algorithms and applications. Until recently, representations have typically been designed for signals with convenient supports, e.g. 2-D signals with rectangular supports. However, a number of applications require representations for signals with arbitrary (non-rectangular) regions of support. We present a novel algorithm for creating critically sampled perfect reconstruction wavelet representations for signals defined over arbitrary supports. The proposed algorithm selects a subset of vectors from a convenient superset basis which under appropriate conditions provides a basis over the given arbitrary support. The algorithm can be interpreted as solving a corresponding sampling problem.

In a typical MC-DCT encoding scheme, a large portion of the bit rate is used to encode the location and amplitude information of the nonzero quantized DCT coefficients. Therefore efficient encoding of the DCT coefficients is extremely important. We describe the joint position-dependent encoding (PDE) approach to encode the DCT coefficients. Joint PDE exploits the variations in statistical properties of the runlengths and amplitudes as a function of position by introducing a set of 2-D codebooks in which each quantized DCT coefficient is assigned to one codebook in the set based on its location. Utilizing an MPEG-2 codec, we compare the bit rates using the joint PDE variable length codes (VLCs) with the bit rates produced by the MPEG-2 VLCs. We also examine how performance is affected by the number of codebooks.

Transform/subband representations form a basic building block for many signal processing algorithms and applications. Most of the effort has focused on developing representations for infinite-length signals, with simple extensions to finite-length 1-D and rectangular support 2-D signals. However, many signals may have arbitrary length or arbitrarily shaped (AS) regions of support (ROS). We present a novel framework for creating critically sampled perfect reconstruction transform/subband representations for AS signals. Our method selects an appropriate subset of vectors from an (easily obtained) basis for a larger (superset) signal space, in order to form a basis for the AS signal. In particular, we have developed a number of promising wavelet representations for arbitrary-length l-D signals and AS 2-D/M-D signals that provide high performance with low complexity.

Acoustic signal representations used in current audio coding algorithms can be improved by the incorporation of biorthogonality into Malvar's extended lapped transform (ELT). Biorthogonality allows more flexibility in the design of the analysis and synthesis windows by increasing the number of degrees of freedom. This paper examines this increase for two special cases and demonstrates the importance of the additional flexibility to the proper implementation of psychoacoustic modeling, a feature central to all modern audio compression schemes.

In this paper, we describe a generalized dual excitation (GDE) speech model that is more accurate in its characterization than the dual excitation (DE) model in that it takes into account pitch variations. This model, together with an analysis window whose length varies adaptively according to the changing characteristics of speech, forms the backbone of a new speech enhancement system. Informal comparisons of the GDE system with the traditional systems have shown a clear preference for the former.

Efficiently representing the interior of an arbitrarily-shaped region is an important problem in many applications, including object- or region-based image/video compression. This paper focuses on representing the interior as a linear combination of vectors defined on a superset basis. Two seemingly different, though highly related, problem formulations are given and a number of approaches that result are discussed and analyzed. A geometric interpretation of the problem is given to provide insight into the approaches and examine their properties. The incorporation of quantization is also briefly discussed. In addition, the chosen set of vectors corresponds to a special/structured class of overcomplete representations with O(N log/sub 2/ N)-type processing, low memory requirements, and other important properties.

The US HDTV process has fostered substantial research and development activity over the last several years. The Advisory Committee on Advanced Television Service (ACATS) was formed to advise the FCC on the technology and systems suitable for delivery of high definition service over terrestrial broadcast channels. Four digital HDTV systems where tested at the Advanced Television Testing Center. All the systems gave excellent performance, but the results were inconclusive and a plan for a second round of tests was prepared. As each of the four systems where being readied for retest. The proponents of the four individual digital HDTV proposals worked together to define a single HDTV system which incorporated the best technology from the individual systems. The consortium of companies, called the Grand Alliance (GA), announced a combined system and submitted it to ACATS for consideration. After ACATS certification, the GA began construction of a prototype system to submit for laboratory testing at the end of 1994. This paper describes the video compression subsystem and the hardware prototype. The preprocessing, motion estimation, quantization, and rate control subsystems are described. The system uses bidirectional motion compensation, discrete cosine transform, quantization and Huffman coding. The resulting bitstream is input into a transport system which uses fixed length packets. The multiplex transport stream is input into the 8-VSB transmission system. Finally, the specifics of the hardware implementation are described and some simulation results are presented.>

In typical MC-DCT compression algorithms, up to 90% of the available bit rate is used to encode the location and amplitude information of the nonzero quantized DCT coefficients. Therefore, efficient encoding of the location and amplitude information is extremely important for high quality video compression. In this paper we describe the position-dependent encoding (PDE) approach for encoding the DCT coefficients. This method attempts to exploit the inherent differences in statistical properties of the run-lengths and amplitudes as a function of position. This paper includes preliminary comparisons between the PDE scheme and conventional coding approaches. The PDE approach can be applied in any transform/subband filtering scheme for image or video compression.>

Multicarrier modulation (MCM) has traditionally been used on severely distorted channels for which there is a reverse path to the transmitter. A potential area of application of multicarrier modulation is for the terrestrial broadcasting of high rate digital television, including high-definition television (HDTV). We show that certain multipath conditions can substantially degrade the performance of MCM, even when combined with coding. If certain conditions are met, however, multipath contributes positively to the performance of the MCM receiver.>

Broadcasting High Definition Television (HDTV) requires the transmission of an enormous amount of information within a highly restricted bandwidth. Adhering to the transmission constraints, channel errors are inevitable. This paper proposes error concealment techniques to remove subjective effects of transmission errors. Error concealment techniques for several data parameters transmitted for the compressed representation of HDTV are considered. Specifically, we address errors in motion vectors and DCT coefficients. The concealment techniques estimate the true value of the corrupted parameters by exploiting the spatial and temporal correlation within the image sequence. In general, the error concealment techniques are found to be extremely successful in removing degradations from the decoded image.

The authors propose a scheme for interpolation of spatially subsampled images. The theoretical formulation is based on the fundamental assumption that luminance remains constant along the generalized path motion of a pixel in the image plane. This constant luminance model (CLM) of the intensity function in turn leads to a constraint equation, the solution to which is used to compute the unknown intensities. Due to the generalized nature of the algorithm, the most highly visible artifact associated with previously developed methods is significantly suppressed, thereby resulting in noticeably enhanced reconstructed images.>

The authors present an algorithm for designing an optimal FIR (finite impulse response) filter with real coefficients that best approximates an arbitrary complex-valued frequency response. This algorithm is theoretically guaranteed to converge with each iteration requiring O(N/sup 2/) computations, where N represents the filter length. The properties of the optimal filter are derived and presented, and particular attention is paid to the case where the desired filter has arbitrary constant group delay.>

The speech spectrogram is a two-dimensional time-frequency display of a one-dimensional signal. The wideband spectrogram and the narrowband spectrogram are deficient either in frequency or in time resolution. The authors present a method for combining the two spectrograms by evaluating the geometric mean of their corresponding pixel values. The combined spectrogram appears to preserve the visual features associated with high resolution in both frequency and time.>

A 6.4 kb/s improved multiband excitation (IMBE) speech coder is presented. This speech coder combines high speech quality with a robustness to channel impairments which is necessary for successful operation in a mobile communication environment. MOS (mean opinion score) results for the IMBE speech coder are compared against those of four 6.4-kb/s CELP (code excited linear prediction) based speech coders which were tested as part of the INMARSAT-M voice codec evaluation. The IMBE system yielded the best performance of the systems tested. It received an MOS score of 3.4 at both 0% and 1% bit error rate. The test results show that the IMBE system is a viable alternative to CELP based speech coders.>

A method that reduces transmission channel noise in a receiver compatible manner is developed. In this method, the television signal is processed on a frame-by-frame basis, dividing each frame into low- and high-spatial-frequency components. The low-frequency component of each frame is subsampled, quantized, and transmitted digitally on a side channel. The high-frequency components is adaptively modulated and added back to a compressed version of the low-frequency component to produce a receiver-compatible signal to be transmitted on the normal analog channel. An advanced receiver can use the information in the digital side channel along with the receiver analog signal to produce a picture devoid of most channel degradations. A standard receiver can still receive an acceptable picture, although with slight contrast. It is shown how the parameters of the system can be adjusted to favor either compatibility with existing receivers or the noise reduction capabilities of advanced receivers.>

A real-time, single digital signal processing (DSP) chip implementation of a 2.4-, 4.8-, and 8.0-kb/s improved multiband excitation (IMBE) vocoder is presented. The IMBE vocoder is based on the MBE speech model, and it is shown to generate high-quality speech under both clean and noisy conditions. In addition, the IMBE vocoder is well suited for real-time implementation since it does not require excessive computation or storage. Full-duplex operation is demonstrated using a single AT&T WE DSP 32. Aspects of the hardware architecture, algorithm implementation, and system performance are addressed.>

The authors present a novel algorithm for spatial interpolation of 2-to-1 interlaced television pictures. The basic problem that this algorithm addresses is reconstruction of a frame from one of the fields. The algorithm is based on two spatially varying image models. It can be applied to color as well as black-and-white pictures. Experimental results demonstrate that the resulting frames contain less spatial aliasing and are sharper than frames generated with commonly used approaches involving zero- or first-order interpolation.>

A speech model, referred to as the multiband excitation (MBE) speech model, has been shown to be capable of synthesizing speech without the artifacts common to model-based speech systems and has been used to develop a 4.8 kb/s speech coder. This system was developed using several new approaches to quantize the MBE model parameters. These techniques were designed to utilize additional redundancy amongst these parameters, thereby permitting more efficient quantization. The result of information listening tests indicate that this system can achieve high quality for both clean and noisy speech as the MBE speech is extremely robust to the presence of background noise in speech.>

The authors present a novel frequency-sampling design method for two-dimensional FIR filters with nonuniformly spaced samples. By imposing some mild constraints on sample location in the 2-D frequency plane, the method always provides a unique design solution. Important characteristics of the method are design flexibility through use of nonuniform samples and computational efficiency.>

In many military environments, such as fighter jet cockpits, the increasing use of digital communication systems has created a need for robust vocoders and speech recognition systems. However, the high level of ambient noise in such environments makes vocoders less intelligible and makes reliable speech recognition more difficult. One method of enhancing the noise-corrupted speech is adaptive noise cancellation. In previous research, this method was tested in a simulated cockpit environment, yielding impressive results. However, in new simulations, reflecting more realistic conditions, adaptive noise cancellation has been less successful. Spectral analysis of the data showed that the diffuseness of the ambient noise fieid, along With the microphone characteristics, has a significant effect on the performance of adaptive noise cancellation.

A 9.6 kbps speech coding system based on a new speech model is presented. In this model, the short-time spectrum of speech is modeled as the product of an excitation spectrum and a spectral envelope. The spectral envelope is some smoothed version of the speech spectrum and the excitation spectrum is represented by a fundamental frequency, a voiced/unvoiced (V/UV) decision for each harmonic of the fundamental, and the phase of each harmonic declared voiced. In speech analysis, the model parameters are estimated by explicit comparison between the original speech spectrum and the synthetic speech spectrum. Preliminary results indicate that high quality reproduction can be obtained with this speech coding system for both clean and noisy speech without the "buzziness" and severe degradation in noise typically associated with vocoder speech.

There has been considerable recent interest on the problem of enhancing degraded speech. This interest is motivated by several factors including a broad set of important applications and the apparent lack of robustness in recent speech compression and recognition systems. One objective of this paper is to provide an overview of various techniques that have been proposed for enhancement of speech. Another objective is to suggest some directions for future research in the speech enhancement problem.

In this paper, we present new results on the reconstruction of two-dimensional signals from zero crossing or threshold crossing information. These results follow from our previous work on the reconstruction of two-dimensional signals from one bit of Fourier transform phase. Experimental results illustrating image reconstruction from threshold crossings are included.

This paper discusses the resolution property of the Improved Maximum Likelihood Method (IMLM), which was recently proposed as a high resolution multi-dimensional spectral estimation algorithm. For a uniformly sampled two-dimensional array, the IMLM is compared to the Bartlett Method, the Maximum Likelihood Method (MLM) and the Maximum Entropy Method (MEM).

A new model-based speech analysis/synthesis system is presented in this paper. In this model, the short-time spectrum of speech is modeled as the product of an excitation spectrum and a spectral envelope. The spectral envelope is some smoothed version of the speech spectrum and the excitation spectrum is represented by the pitch period and a voiced/unvoiced (V/UV) decision for each harmonic. In speech analysis, the model parameters are estimated by explicit comparison between the original speech spectrum and the synthetic speech spectrum. Preliminary results indicate that speech and noisy speech synthesized based on this model do not have the "buzziness" typically associated with vocoder speech and is essentially the same as the original speech or the noisy speech in both intelligibility and quality. Potential applications of this new model and its parameter estimation include high quality speech analysis/synthesis, time scale modification of speech and noisy speech, and pitch detection.

Signals derived from video systems which generate motion picture scenes possess a great deal of temporal redundancy. The temporal correlation of adjacent frames is often completely determined by the displacement field, which over the course of time traces out the motion trajectories of moving objects within the scene. It is possible to exploit the available redundancy in these signals for the purpose of noise reduction by compensating for the presence of motion. We present an adaptive filtering technique which implicitly compensates for motion within a scene by performing 1-D signal estimation along a set of hypothesized motion directions. This technique adapts the estimator parameters to the local image characteristics along a set of specified directions in the 3-D signal space. Experiments with real life scenes have shown that this method achieves substantial noise reduction in many cases, while preserving the edge contrast of the desired signal.

In this paper, we present a one-dimensional (1-D) approach to the problem of image restoration. Our approach involves a cascade of four 1-D adaptive filters oriented in the four major correlation directions of the image, with each filter treating the image as a 1-D signal. The objective of our 1-D approach is to improve the performance of the more general two-dimensional (2-D) approach. This differs considerably from previous 1-D approaches, the objectives of which have typically been to approximate a more general 2-D approach for computational reasons and not to improve its performance. To illustrate this point, our approach is applied to an existing 2-D image restoration algorithm. Experiments with images at low SNRs (signal to noise ratios) show that the 1-D approach performs better than the 2-D approach for the specific image restoration algorithm. Our 1-D approach preserves edges while removing noise in all regions of the image including the edge regions.

In this paper, we present new results on the reconstruction of signals from one bit of Fourier transform phase, defined as the sign of the real part of the Fourier transform. Specifically, we develop a new theoretical result which shows that most two-dimensional signals can in fact be reconstructed to within a scale factor from only one bit of FF phase. Experimental results showing images reconstructed from one bit of FT phase are also presented.

Existence of an exact relationship between the maximum likehood method (MLM) and autoregressive (AR) signal modeling in multi-dimensional (m-D) power spectral estimation is shown. For one-dimensional (1-D) uniformly sampled autocorrelation functions (ACF), Burg has shown a relationship between the maximum entropy method (MEM) and MLM spectral estimates. In this note we show a similar relationship between the MLM and AR spectral estimates for m-D (or 1-D) signals sampled non-uniformly or uniformly.

In this paper, speech synthesis directly from the processed Short-Time Fourier Transform Magnitude (STFTM) using the LSEE-MSTFTM algorithm [6,7] is compared to more conventional algorithms for several speech processing applications. For the applications considered, the most improvement occurs for time-scale modification of multiple speaker speech and noisy speech since these input signals are not well modeled by the analysis/synthesis system used for comparison. However, for the applications of speech synthesis from speech model parameters, time-scale modification of clean speech, speech enhancement by spectral subtraction, and helium speech enhancement, significant improvement is not gained by using the LSEE-MSTFTM algorithm. Significantly better results are not obtained since a good STFT phase estimate is available and employed in the conventional approaches to these applications.

In this paper we discuss some preliminary results on using Widrow's Adaptive Noise Cancelling (ANC) algorithm to reduce the background noise present in a fighter pilot's speech. With a dominant noise source present and with the pilot wearing an oxygen facemask, we demonstrate that good (>10 dB) cancellation of the additive noise and little speech distortion can be achieved by having the reference microphone attached to the outside of the facemask and by updating the filter coefficients only during silence intervals.

This paper discusses the use of spatial linear prediction to detect region borders in aerial photographs and thus segment the image into regions of differing natural terrain. The algorithm is motivated by a significance test which under certain assumptions and approximations can be expressed as the weighted sum of error residuals of two-dimensional (2-D) linear prediction. Examples are presented to illustrate the performance of the algorithm.

The performance of a pulse code modulation (PCM) speech coding system that incorporates quantization noise reduction is evaluated by a speech quality test. The results of the test indicate that quantization noise reduction improves the performance of a PCM system by 1 to 2 bits/ sample over a wide range of bit rates.

In this paper, we present an algorithm to estimate a signal from its modified short-time Fourier transform (STFT). This algorithm is computationally simple and is obtained by minimizing the mean squared error between the STFT of the estimated signal and the modified STFT. Using this algorithm, we also develop an iterative algorithm to estimate a signal from its modified STFT magnitude. The iterative algorithm is shown to decrease, in each iteration, the mean squared error between the STFT magnitude of the estimated signal and the modified STFT magnitude. The major computation involved in the iterative algorithm is the discrete Fourier transform (DFT) computation, and the algorithm appears to be real-time implementable with current hardware technology. The algorithm developed in this paper has been applied to the time-scale modification of speech. The resulting system generates very high-quality speech, and appears to be better in performance than any existing method.

In this paper, we present a new method for two-dimensional spectral estimation. This method is based on the extension of the relationship that exists between the maximum likelihood method and the maximum entropy method for one-dimensional signals to two-dimensional signals. This method has a computational requirement similar to that of the maximum likelihood method, but has a resolution property which is considerably better than that of the maximum likelihood method. Examples are shown to illustrate the performance of the new algorithm.

We have previously established a number of conditions under which a real signal can be uniquely reconstructed from its STFT magnitude. For the STFT magnitude to be a practical signal representation, we need robust reconstruction algorithms. In this paper, we discuss such a class of algorithms within the framework of sequential extrapolation techniques. Such techniques reconstruct the short-time sections of a signal in an order determined by their positions on the time axis. We find that compared to direct reconstruction, the robust algorithms are less sensitive to roundoff errors. To further test the robustness of these algorithms, we applied them to STFT magnitudes which were purposely modified for accomplishing signal processing tasks such as noise reduction and time-scale modification of speech.

In some important image coding techniques, such as transform coding, an image is first divided into subimages, and then each subimage is coded independently. The segmentation procedure has significant advantages, but when used in a low bit rate scheme, an undesirable side effect can occur. Specifically, when the image is reconstructed at the decoder, a "blocking effect" can develop due to discontinuities between the subimages. Two methods are developed to reduce the blocking effect. The performance of these methods when applied to a Discrete Cosine Transform image coder is discussed.

The effects of noise in the given phase on signal reconstruction from the Fourier transform phase are studied. Specifically, the effects of different methods of sampling the degraded phase, of the number of non-zero points in the sequence, and of the noise level on the sequence reconstruction are examined. A sampling method is developed to significantly reduce the error in the reconstructed sequence, and the error is found to increase as the number of non-zero points in the sequence increases and as the noise level increases. In addition, an averaging technique is developed which reduces the effects of noise when the continuous phase function is known. Finally, as an illustration of how the results in this paper may be applied in practice, Fourier transform signal coding is considered. Coding only the Fourier transform phase and reconstructing the signal from the coded phase is found to be considerably less efficient (i.e. a higher bit rate is required for the same mean square error) than reconstructing from both the coded phase and magnitude.

In this paper we summarize the results of a number of experiments that have been performed to study the properties of two-dimensional maximum entropy power spectrum estimates. The results presented include studies on the resolution differences for real and complex data, resolution properties of the spectral estimates, and the determination of the relative power of the sinusoids from the spectral estimates. The results also include the effects of signal to noise ratio, size of the known autocorrelation region, data length and initial phase of a sinusoid, on the spectral estimates.

This paper presents various conditions that are sufficient for reconstructing a discrete-time signal from samples of its short-time Fourier transform magnitude. For applications such as speech processing, these conditions place very mild restrictions on the signal as well as the analysis window of the transform. Examples of such reconstruction for speech signals are included in the paper.

A new iterative algorithm for the maximum entropy power spectrum estimation is presented in this paper. The algorithm which is applicable to two-dimensional signals as well as one-dimensional signals, utilizes the computational efficiency of the Fast Fourier Transform (FFT) algorithm and has been empirically observed to solve the maximum entropy power spectrum estimation problem. Examples are shown to illustrate the performance of the new algorithm.

Major artifacts of the spectral subtraction signal restoration technique are the presence of harmonic patterns and blurring effects for images, and the presence of short tone bursts of varying frequency for speech signals. In this paper, we discuss several techniques developed to reduce these artifacts. Examples are shown to illustrate the performance of these techniques.

In this paper, we develop an image enhancement algorithm that modifies the local luminance mean of an image and controls the local constrast as a function of the local luminance mean of the image. The algorithm first separates an image into its lows (low-pass filtered form) and highs (high-pass filtered form) components. The lows component then controls the amplitude of the highs component to increase the local contrast. The lows component is then subjected to a non-linearity to modify the local luminance mean of the image and is combined with the processed highs component. The performance of this algorithm when applied to enhance typical undegraded images, images with large shaded areas, and also images degraded by cloud cover will be illustrated by way of examples.

In this paper, we develop a set of conditions under which a sequence is uniquely specified by the phase or samples of the phase of its Fourier transform. These conditions are applicable to mixed-phase one-dimensional and multi-dimensional sequences. Under the specified conditions, we also present several algorithms which may be used to reconstruct a sequence from its phase.

A new image restoration system that is applicable to the problem of restoring an image degraded by additive noise is presented in this paper. The system is developed by attempting to estimate more accurately the frequency response of typical image restoration filters available in the literature. The resulting system combined with its short space implementation is computationally simple, appears to compare quite well in performance with other restoration techniques, and can be easily extended to account for other types of degradations such as bluring and additive noise, multiplicative noise, etc. Some examples are given to illustrate the performance of the new image restoration system.

In this paper, we develop a new homomorphic deconvolution system that is useful when the signal to be deconvolved is a convolution of two components one of which is a train of pulses and about the other we have some a priori information on its approximate number of poles and zeros. The system that we develop is essentially the same as the logarithmic homomorphic deconvolution system (1,2,3) except that the logarithmic and exponential operations are replaced with ( )γand ( )1/γoperations. By theoretical and/or empirical results, we illustrate that under appropriate conditions, the new system with the proper choice of γ can potentially yield a better result than the logarithmic homomorphic deconvolution system and that the results obtained by the new system with the value of γ close to zero are essentially the same as those obtained by the logarithmic homomorphic deconvolution system.

For most signal models of interest, Maximum Likelihood (ML) parameter estimation in the presence of noise is a difficult, non-linear problem. A new iterative algorithm has been developed for ML estimation, however, which effectively decouples the uncertainty in the signal and parameter values, thus simplifying the calculation required. It can be shown that the likelihood function increases on each iteration of the algorithm. When applied to a particular pole-zero (ARMA) signal model, each pass consists of a linear smoothing filter followed by solving a set of linear equations for both the pole and zero polynomial coefficients.

It is generally known that preservation of phase information in images is important as evidenced by the fact that modifying an image such that phase is preserved but the magnitude of all the spectral components is set to unity generally preserves many of the features of the original image. A similar experiment has been carried out with speech, showing that preservation of only the long-time phase retains many important features of the original speech. An interpretation of these results, and their potential application to blind deconvolution, is suggested.

Application of a Maximum A Posteriori (MAP) estimation procedure in estimating the LPC coefficients from speech waveforms degraded by additive random noise generally leads to solving a set of non-linear equations which are computationally undesirable. However, an attempt to approximate the true MAP estimation procedure leads to two iterative methods that require solving only sets of linear equations. These two methods have been applied to real speech data degraded by additive white Gaussian noise, and in this paper some preliminary results are discussed.