Media flows coexist with TCP-based data traffic on the Internet and are required to be TCP-friendly. The TCP protocol slowly increases its sending rate until episodes of congestion occur, and then it quickly reduces its rate to remove congestion. However, media flows can be sensitive to even brief episodes of congestion. In this paper, we are interested in protecting media flows from TCP-induced congestion while maintaining their TCP friendliness. In particular, we consider media flows carried over the TCP-Friendly Rate Control (TFRC) protocol and we design algorithms that dynamically adapt the level of forward error correction (FEC) based on the congestion state of the network. To this end, first, we investigate the loss and delay characteristics of TFRC flows in several TCP-induced congestion scenarios, and we develop novel predictors of loss events based on packet delay information. Second, we use these predictors to dynamically adapt the level of FEC protection based on the predicted level of congestion. We show that this technique can significantly improve the overhead versus reliability trade-off compared to fixed FEC. Third, we select the FEC and original media packets within each FEC block, in a rate-distortion optimized way, and we show that this technique significantly improves media quality.

In this paper, we propose a method for peer assisted on demand delivery of scalable video using an optimal, distributed caching algorithm for cache constrained peers. We assume that there are one or more content delivery servers that are provisioned to support delivery of the base layer for all of the peers, and the enhancement layer is only supported for a fraction of peers. It has been shown that in such an application, peer assisted content distribution improves the delivery of the content. In this paper, we use peer assisted delivery of the enhancement layer and show that in a cache constrained video-on-demand environment, availability of the cached content becomes a bottleneck. By using peer arrival rates and supply-demand analysis of the peers, we propose better caching policies at each peer. One proposed policy (referred to as "static optimal") optimizes the utility of the currently available caching opportunities. The other proposed policy (referred to as "dynamic programming" solution) on the other hand performs utility optimization by further considering the future caching opportunities during the streaming operation. Experimental results show that the proposed methods allow more peers to access the enhancement layer; hence, more peers experience improved video quality compared to traditional caching methods.

In this paper, we consider a hybrid P2P video on-demand architecture that utilizes both the server and the peer resources for efficient transmission of popular videos. In our system architecture, each peer dedicates some cache space to store a particular segment of a video file as well as some of its upload bandwidth to serve the cached segment to other peers. Peers join the system and issue a streaming request to a control server. Control server directs the peers to streaming servers or to other peers who have the desired video segments. Control server also decides which peer should cache which video segment. Our main contribution in this paper is to determine the proper caching strategies at peers such that we minimize the average load on the streaming servers.

Modern video compression includes a mature set of codecs, tools, and techniques that correspond to a variety of coding efficiencies and royalty costs in video delivery. In early work we showed that there can be significant benefits to employing a royalty-aware encoder that jointly optimizes rate, distortion, and royalty cost by choosing among several codecs having different royalty costs [1]. In this paper we assume the existence of such an encoder and concentrate on how royalty costs should be assigned to benefit a system of users, video service providers, and tool/codec owners. Our work operates on the intersection of rate-distortion principles with basic concepts from economics and tries to address issues that are rapidly becoming very relevant in media delivery. Using a simple model that captures user preferences of video quality levels (given costs associated with these levels), we formulate the assignment of royalty costs as an optimization problem and solve it for optimal costs for several scenarios. The main trade-offs that govern our results are the cost of bandwidth and previously unachievable quality levels (if any) that a state-of-the-art codec gives access to. We show that content-based cost assignment not only increases user utilization but also the royalty earnings for tool owners compared to content-unaware pricing policies.

Today's packet networks including the Internet offer an intrinsic diversity for media distribution in terms of available network paths and servers or information sources. Novel communication infrastructures such as ad hoc or wireless mesh networks use network diversity to extend their reach at low cost. Diversity can bring interesting benefits in supporting resource greedy applications such as media streaming services, by aggregation of bandwidth and computing resources. Typically, overlay network architectures compensate for lack of quality-of-service guarantees in the network by introducing redundancy in the media delivery system through network diversity. They can support efficient multimedia services when routing, coding, and scheduling algorithms are able to adapt to both the media information and the dynamic network status. This paper presents an overview of the distributed streaming solutions that profit from network diversity in order to improve the quality of multimedia applications. We discuss the coding techniques used for adaptive and flexible media streaming with network diversity. We describe the problem of media streaming with path diversity and focus on routing, path computation, and packet scheduling problems in multipath networks. Then, the advantages of server or source peer diversity in collaborative streaming solutions are discussed. Lastly, we present an overview of wireless mesh networks and focus on the typical constraints imposed by these novel communication models on media streaming with network diversity.

A video compression standard incorporates many tools and technologies which must be licensed by systems that deploy the standard. The licensing determines the royalty costs that must be paid to the holders of intellectual property on the respective tools. With current abundance of well understood and effective video compression tools, one can imagine the formation of cross cutting tool libraries with tools drawn from different video compression standards. This allows dynamic selection from a large pool of tools, having potentially overlapping functionality, when encoding individual video sequences. In this paper we examine the royalty cost aspect of the scenario where video is encoded using a library of royalty bearing tools by considering encoding that jointly optimizes rate, distortion, and royalty cost. We provide a system that optimizes video delivery under various licensing conditions imposed on tool intellectual property. We present an example of royalty based encoding (using assumed royalty costs) to show the merit of the proposed framework.

We present a novel client-driven multi-view video streaming system that allows a user watch 3-D video interactively with significantly reduced bandwidth requirements by transmitting a small number of views selected according to his/her head position. The proposed scheme can be used to efficiently stream a dense set of multi-view sequences (light-fields) or wider baseline multi-view sequences together with depth information. The user's head position is tracked and predicted into the future to select the views that best match the user's current viewing angle dynamically. Prediction of future head positions is needed so that views matching the predicted head positions can be requested from the server ahead of time in order to account for delays due to network transport and stream switching. Highly compressed, lower quality versions of some other views are also requested in order to provide protection against having to display the wrong view when the current user viewpoint differs from the predicted viewpoint. The proposed system makes use of multi-view coding (MVC) and scalable video coding (SVC) concepts together to obtain improved compress ion efficiency while providing flexibility in bandwidth allocation to the selected views. Rate-distortion performance of the proposed system is demonstrated under different experimental conditions.

We present a cross-layer optimized video rate adaptation and user scheduling scheme for multi-user wireless video streaming aiming for maximum quality of service (QoS) for each user,, maximum system video throughput, and QoS fairness among users. These objectives are jointly optimized using a multi-objective optimization (MOO) framework that aims to serve the user with the least remaining playback time, highest delivered video seconds per transmission slot and maximum video quality. Experiments with the IS-856 (1timesEV-DO) standard numerology and ITU pedestrian A and vehicular B environments show significant improvements over the state-of- the-art wireless schedulers in terms of user QoS, QoS fairness, and the system throughput.

We present a survey of transport methods for 3-D video ranging from early analog 3DTV systems to most recent digital technologies that show promise in designing 3DTV systems of tomorrow. Potential digital transport architectures for 3DTV include the DVB architecture for broadcast and the Internet Protocol (IP) architecture for wired or wireless streaming. There are different multiview representation/compression methods for delivering the 3-D experience, which provide a tradeoff between compression efficiency, random access to views, and ease of rate adaptation, including the "video-plus-depth" compressed representation and various multiview video coding (MVC) options. Commercial activities using these representations in broadcast and IP streaming have emerged, and successful transport of such data has been reported. Motivated by the growing impact of the Internet protocol based media transport technologies, we focus on the ubiquitous Internet as the network infrastructure of choice for future 3DTV systems. Current research issues in unicast and multicast mode multiview video streaming include network protocols such as DCCP and peer-to-peer protocols, effective congestion control, packet loss protection and concealment, video rate adaptation, and network/service scalability. Examples of end-to-end systems for multiview video streaming have been provided.

We present a novel client-driven multiview video streaming system that allows a user to watch 3D video interactively with significantly reduced bandwidth requirements by transmitting a small number of views selected according to his/her head position. The user's head position is tracked and predicted into the future to select the views that best match the user's current viewing angle dynamically. Prediction of future head positions is needed so that views matching the predicted head positions can be prefetched in order to account for delays due to network transport and stream switching. The system allocates more bandwidth to the selected views in order to render the current viewing angle. Highly compressed, lower quality versions of some other views are also prefetched for concealment if the current user viewpoint differs from the predicted viewpoint. An objective measure based on the abruptness of the head movements and delays in the system is introduced to determine the number of additional lower quality views to be prefetched. The proposed system makes use of multiview coding (MVC) and scalable video coding (SVC) concepts together to obtain improved compression efficiency while providing flexibility in bandwidth allocation to the selected views. Rate-distortion performance of the proposed system is demonstrated under different experimental conditions.

We propose a new pre-roll delay-distortion optimization (DDO) framework that allows determination of the minimum pre-roll delay and distortion while ensuring continuous playback for on-demand content-adaptive video streaming over limited bitrate networks. The input video is first divided into temporal segments, which are assigned a relevance weight and a maximum distortion level, called relevance-distortion policy, which may be specified by the user. The system then encodes the input video according to the specified relevance-distortion policy, whereby the optimal spatial and temporal resolutions and quantization parameters, also called encoding parameters, are selected for each temporal segment. The optimal encoding parameters are computed using a novel, multi-objective optimization formulation, where a relevance weighted distortion measure and pre-roll delay are jointly minimized under maximum allowable buffer size, continuous playback, and maximum allowable distortion constraints. The performance of the system has been demonstrated for on-demand streaming of soccer videos with substantial improvement in the weighted distortion without any increase in pre-roll delay over a very low-bitrate network using AVC/H.264 encoding

Transmission of time-critical video traffic over the networks with large end-to-end delays poses two main difficulties: slow adaptation to changing network conditions and inability of timely recovery from lost packets. Previously, we proposed a proxy-based solution that enabled us to cope with these problems. Our Internet experiments with a single proxy system showed that the QoS delivered by interactive video applications could be greatly improved by the Intermediate-Proxy approach. In multi-proxy systems, however, the availability of a large number of proxies throughout the Internet will bring the proxy selection problem to the fore. This is an important problem since the benefits of the intermediate-proxy approach can be best exploited with a proper proxy selection. In this study, our goal is to model the dynamics involved in the networks with proxies, and investigate mathematical and practical proxy selection methods.

In wireless networks, transmission of constant-quality, high bitrate video is a challenging task due to channel capacity and buffer limitations. Content adaptive rate control, is used as a solution to this problem. Instead of transmitting all of the video content at low quality, the most important content can be transmitted at high quality while still preserving an acceptable quality for the remaining segments. Furthermore, the rate control strategy inside the individual temporal segments plays a key role for the network performance and viewing quality. Although constant quality video encoding inside the temporal segments is preferable for the best viewing experience, it causes more network packet losses due to adverse bitrate fluctuations in the video stream. In cases when the network is too much loaded, it may be better to employ constant bitrate encoding for network friendliness. In this paper, a performance analysis of constant bitrate and constant peak signal-to-noise ratio encoding for content adaptive rate controlled video streaming over wireless networks is presented. Experimental results obtained using AVC/H.264 encoding in a CDMA/HDR multi-user environment with cross-layer optimized scheduling show performance comparisons of CBR and CPSNR encoding.

A new peer-to-peer architecture for multipoint video conferencing that targets end points with low bandwidth network connections (single video in and out) is presented. It enables end points to create a multipoint conference without any additional networking and computing resources than what is needed for a point-to-point conference. The new architecture is based on layered video coding with two layers at the end points. It allows each conference participant to see any other participant at any given time under all multipoint configurations of any number of users, with a caveat that some participants may have to receive only the base layer video. Layered encoding techniques usable within this architecture are described. A protocol for implementation of the new approach has been developed and simulated. Its performance is analyzed.

Efficient peer-to-peer (P2P) video streaming is a challenging task due to time-varying nature of both the number of available peers and network/channel conditions. To this effect, we propose a receiver driven P2P streaming system which utilizes a flexible scalable multiple description coding method, where the number of base and enhancement descriptions, and the rate and redundancy level of each description can be adapted on the fly. The optimization of the parameters of the proposed MDC scheme according to network conditions is discussed within the context of the proposed adaptive P2P streaming framework, where the number and quality of available streaming peers/paths are a priori unknown and vary in time. Experimental results, by means of NS-2 network simulation of a P2P video streaming system, show that adaptation of the number, type, and rate of descriptions and the redundancy level of each description according to network conditions yields significantly superior performance when compared to MDC schemes using a fixed number of descriptions/layers with fixed rate and redundancy level.

In mobile video transmission systems, the initial delay for pre-fetching video at the client buffer needs to be short due to buffer limitations and application-layer user convenience. Therefore, an effective cross-layer wireless design is required that considers both physical and application layer aspects of such a system. We present a cross-layer optimized multi-user video adaptation and scheduling scheme for wireless video communication, where quality-of-service (QoS) fairness among users is provided while maximizing user convenience and video throughput. Application and physical layer aspects are jointly optimized using a multi-objective optimization (MOO) framework that tries to schedule the user with the least remaining playback time and the highest video throughput (delivered video seconds per transmission slot) with maximum video quality. Experiments with the IS-856 (1xEV-DO) standard and ITU pedestrian A and vehicular B environments show the improvements over today's schedulers in terms of QoS fairness and user utility.

Today, stereoscopic and multi-view video are among the popular research areas in the multimedia world. In this study, we have designed and built a platform consisting of stereo-view capturing, real-time transmission and display. At the display stage, end users view video in 3D by using polarized glasses. Multi-view video is compressed in an efficient way by using multi-view video coding techniques and streamed using standard real-time transport protocols. The entire system is built by modifying available open source systems whenever possible. Receiver can view the content of the video built from multiple channels as mono or stereo depending on its display and bandwidth capabilities

The Datagram Congestion Control Protocol (DCCP) is designed to be a replacement for the User Datagram Protocol (UDP) that is used for media delivery, with Congestion Control Identifier 3 (CCID3) tailored for video streaming applications. In this paper, we investigate the methods of streaming video, which is coded with the Scalable Video Coding (SVC) extension of H.264/AVC, over the Transmission Control Protocol (TCP) and DCCP (using CCID3), and try to come up with a solution such that the quality of the received video is maximized while both the number and the length of buffering events are minimized.

In wireless packet transmission systems, it is crucial to provide fairness in service while maximizing user utility and channel throughput. This is possible via intelligent allocation of system resources. For real time video applications, a pre-roll time for pre-fetching data at the client buffer is needed in order to compensate for channel variations that cause client buffer under/overflows, hence facilitating continuous playout of the video. In this paper, a novel multiple objective optimized (MOO) opportunistic multiple access scheme for optimal scheduling of users in a 1xEV-DO (IS-856) system is presented. At each time slot, the user that experiences the best compromise between the least buffer occupancy and the best channel condition is served. Experiments conducted in ITU Pedestrian A and Vehicular B environments show that our algorithm treats each user fairly while its channel throughput performance is very close to the ideal case, in which the user with the best channel characteristics is always served with no buffer constraints.

Multiple description video coding mitigates the effects of packet losses introduced by congestion and/or bit errors. In this paper, we propose a novel multiple description video coding technique, based on fully scalable wavelet video coding, which allows post encoding adaptation of the number of descriptions, the redundancy level of each description, and bitrate of each description by manipulation of the encoded bitstream. We demonstrate that the proposed method provides excellent coding efficiency, outperforming most other multiple description methods proposed so far. We also provide experimental results to show that varying the number of descriptions according to network conditions is superior to using a fixed number of descriptions, by means of NS-2 network simulation of a peer-to-peer video streaming system.

Wireless mesh systems offer several advantages for emerging high-bandwidth networks because of their cooperative routing capabilities. In this study, we consider the potential benefits of using wireless meshes within residential networks for applications that require high bandwidth and low latency. In particular, we consider high-bitrate video transmission inside a mesh-networked home. To quantify our findings, we present experimental results obtained from a high-resolution video streaming application. Our experiments involve single-hop, multi-hop, single-path and multi-path transmission methods, and two types of video coding techniques, namely single description coding and multiple description coding. In the light of our results, we discuss the pros and cons of each streaming method, and motivate a set of research problems.

We present a new channel adaptive stream switching solution for variable bitrate video transmission, where the receiver buffer status is used for making switching decisions and instantaneous transmission rate is determined by means of explicit channel feedback or TCP friendly rate control. Receiver buffer status is used in selecting from a set of pre-encoded bitstreams avoiding buffer underflows and overflows. A pre-roll delay to buffer data at the receiving side is necessary in order to compensate for variations in the channel throughput and the encoding bitrate. For each stream, content dependent video coding parameters are chosen by means of dynamic programming such that the maximum overall video quality and minimum pre-roll delay are achieved for a finite number of transmission rates. Experimental results show that buffer violations are successfully avoided using the proposed stream switching framework as opposed to a regular non-adaptive streaming case.

We propose an adaptive motion-compensated temporal filtering (MCTF) structure to provide efficient temporal scalability within the H.264/AVC video compression standard. MCTF has traditionally been considered within fully scalable wavelet video coders. However, motion-compensated simple 5/3 lifted temporal wavelet filtering suffers at scene changes, as well as occlusion regions. We note that the bi-directional motion compensation mode in the H.264 standard is best equipped with the state of the art adaptive features such as adaptive block size, mode switching between forward, backward and bidirectional prediction and in-loop deblocking filter. Hence, we propose a GOP structure to implement block-based adaptive MCTF within the H.264 syntax using stored B-pictures, similar to the motion-compensated 5/3 wavelet filtering. We provide experimental results to compare the results of our proposed codec with those of other scalable wavelet video coders which use MCTF. It is also possible to employ the proposed adaptive MCTF structure within fully scalable wavelet video codecs.

A novel dynamic programming based technique for optimal selection of input video format and compression rate for video streaming based on "relevancy" of the content and user context is presented. The technique uses context dependent content analysis to divide the input video into temporal segments. User selected relevance levels (weights) are refined by using audio information and assigned to these segments. The weights are used in formulating a constrained optimization problem, which is solved using dynamic programming. The technique minimizes a weighted distortion measure and the initial waiting time for continuous playback under maximum acceptable distortion constraints. Spatial resolution, frame rate and average audio volume of the temporal segments and the DCT quantization parameters are used as optimization variables.

In this paper, we describe the implementation of a system for reliable recording and on-demand indexed playback of multimedia conferences using the services provided by RTP (Real-time Transport Protcol) and RTCP (Real-Time Control Protocol) over an intranet. Implementation issues are underlined regarding the file structure used to support random access during playback, the indexing techniques and the database searching capabilities.

The explosive growth of the Internet and the intranets have attracted a great deal of attention to the implementation and performance of networked multimedia services, which involve the transport of real-time multimedia data streams over non-guaranteed quality of service (QoS) networks based on the Internet Protocol (IP). In this paper, the author presents an overview of the existing architectural elements supporting real-time data transmission over the Internet. Effective implementations of such systems require a thorough understanding of both the network protocols and the coding systems used for compressing the signals to be transmitted in real-time. The paper includes a section discussing the issues to be considered in designing signal compression applications suitable for network use.

We describe the implementation of a system for reliable recording and on-demand playback of multimedia conferences using the services provided by the RTP/RTCP protocols over an intranet. Implementation issues are underlined regarding the file indexing strategy to support random access during playback, the scheduling of packets to reduce transmission delay jitter, replacement of lost packets within the media stream and recorder allocation and deployment.

Discusses coding standards for still images and motion video. We first briefly discuss standards already in use, including: Group 3 and Group 4 for bilevel fax images; JPEG for still color images; and H.261, H.263, MPEG-1, and MPEG-2 for motion video. We then cover newly emerging standards such as JBIG1 and JBIG2 for bilevel fax images, JPEG-2000 for still color images, and H.263+ and MPEG-4 for motion video. Finally, we describe some directions beyond the standards such as hybrid coding of graphics/photo images, MPEG-7 for multimedia metadata, and possible new technologies.

Transmission of compressed video over packet networks with nonreliable transport benefits when packet loss resilience is incorporated into the coding. One promising approach to packet loss resilience, particularly for transmission over networks offering dual priorities such as ATM networks, is based on layered coding which uses at least two bitstreams to encode video. The base-layer bitstream, which can be decoded independently to produce a lower quality picture, is transmitted over a high priority channel. The enhancement-layer bitstream(s) contain less information, so that packet losses are more easily tolerated. The MPEG-2 standard provides four methods to produce a layered video bitstream: data partitioning, signal-to-noise ratio scalability, spatial scalability, and temporal scalability. Each was included in the standard in part for motivations other than loss resilience. This paper compares the performance of these techniques (excluding temporal scalability) under various loss rates using realistic length material and discusses their relative merits. Nonlayered MPEG-2 coding gives generally unacceptable video quality for packet loss ratios of 10/sup -3/ for small packet sizes. Better performance can be obtained using layered coding and dual-priority transmission. With data partitioning, cell loss ratios of 10/sup -4/ in the low-priority layer are definitely acceptable, while for SNR scalable encoding, cell loss ratios of 10/sup -3/ are generally invisible. Spatial scalable encoding can provide even better visual quality under packet losses; however, it has a high implementation complexity.

Recent developments in DSP hardware has made it possible to use almost any pulse shape for digital data trans- mission. This resulted in a search for algorithms capable of constructing pulse shapes matching to the properties of a given channel. The projection onto convex sets technique is suitable for the solution of this problem particularly because of its ffexibility in modeling a variety of constraints. The use of the technique for the optimal pulse shape design problem is demonstrated through a detailed example from power line communications. I. INTRODUCTION INCE Nyquist's invention of the intersymbol interference (ISI), the optimal pulse shaping problem for digital data transmission over channels with various properties has been investigated by a number of researchers (1)-(7). The main differences between various digital Nyquist pulse design al- gorithms published in the literature are the type of the filter (FIR or IIR) they choose to generate the pulse shape and the formulation and solution of the optimization problem. Some of these approaches use general optimization techniques and they can handle additional constraints if needed. However, addition of each new constraint requires reformulation of the problem and the solution algorithm. The projections onto convex sets (POCS) technique is an optimization method introduced in the 1960's (8). A number of successful applications of the POCS technique in the signal processing area have been reported in the recent years (9)-(ll). The POCS technique is attractive for optimum pulse shape design particularly because of its modular way of formulating problems with many constraints. In this technique, the addition of new constraints or removal of some of the existing ones does not require reformulation. It is also possible to handle certain inconsistent requirements in the POCS formulation (12). If there are nonintersecting convex sets, it is possible to find the vector which is as close as possible to one set of constraints, while satisfying the rest.

This paper describes a method for coding arbitrarily shaped image segments. The method uses an iterative technique based on the theory of successive projection onto convex sets to determine the best transform coefficients. It uses block transforms with frequency domain region-zeroing and space domain region-enforcing operations for effective coding of image segments of arbitrary shape. A major strength of this method is that it can be implemented in real-time using existing codec hardware at an insignificant additional cost.>

The increasing availability of high speed local area networks (LANs) and the recent developments in image coding hardware that follow the international standardizations have made multipoint desktop video teleconferencing feasible using mostly off-the-shelf hardware and software. We have implemented a multipoint video teleconferencing system using workstations and personal computers (PCs) on a fiber distributed data interface (FDDI) LAN. The system is based on motion JPEG image coding and uses commercially available hardware and software wherever possible. In this paper, we outline our design experiences.

Many scattered estimation problems in systems science and signal processing have been approached in set theoretic terms over the past three decades. A single formal framework is presented to synthesize these various approaches, and the fundamental philosophy, goals, and analytical techniques of set theoretic estimation are discussed.>

Performance evaluation criteria are presented to provide a useful tool for comparing current digital signal processing (DSP)-based parallel machines used in image processing (IP) applications. Additionally, these criteria can be considered as a step toward a unified computation model for IP which may influence the design of future parallel architectures using distributed frame buffers. As an example, it is shown how to determine the criteria for a parallel machine with a mesh connected architecture and 2-D interleaving scheme. Benchmarks are derived as a function of key architectural parameters (e.g. the number of processors) and image parameters. The use of the criteria in determining task performance is explained.>

Recent development in digital signal processing (DSP) hardware has made it possible to use almost any pulse shape for digital data transmission, resulting in a search for design algorithms capable of constructing pulse shapes matching the properties of a given channel. The technique of projection onto convex sets is particularly suitable for the solution of this problem because of its flexibility in modeling a variety of constraints. The use of the technique for the optimal pulse design problem is demonstrated through by an example from power line communications.>

With the availability of a 144 Kbits/sec user's digital access to the integrated service digital network (ISDN), transmission of video data over the telephone network will become a reality. However, the required compression is extremely high and improving the visual quality of the decoded image data continues to be a challenge. The authors present a new image coding algorithm which includes a technique for segmenting image sequences and a modified adaptive 3-D arithmetic coder (MAAC). With these enhancements, it will be possible to transmit an image sequence at 30 frames per second over a digital telephone line.>

A new segmentation method based on the properties of the human visual system is described in this paper. The segmentation method is part of a new second generation image coder. In addition, one of the properties is that it is not necessary to transmit (or store) the visual residual for use in reconstructing the received signal. It is assumed that the characteristics of this visual residual are known at the receiver and can be used in the reconstruction process.

The minimum mean square error estimate for the deconvolution problem of a Gaussian signal in Gaussian noise is shown to be feasible in the sense of being inside closed convex sets defined by the noise statistics. It is pointed out that there is some a priori knowledge which is not satisfied by the Wiener solution but the set formed by this information is not convex.

Iterative tomographic reconstruction methods have been developed which can enforce various constraints on the reconstructed image An important step in these methods is the reprojection of the reconstructed image. A common disadvantage of these methods is they fail to converge due to the accumulation of the reprojection errors. In this paper the errors generated by such reprojection schemes are investigated and bounds for these errors are derived. The approach in deriving the error bounds is probabilistic and the resulting bounds are much smaller than the previously derived maximum bounds.

Tomographic image reconstruction using limited projection data requires incorporation of all available a priori information in the reconstruction process. This necessitates the use of transformations between projection and image spaces. The consistency of these transformations is important in elevating the confidence in the solution. It is also observed that better reconstructions can be obtained through consistent transformations.

One of the characteristics of the interframe difference pictures for video teleconferencing types of image sequences is large zero areas and impulse-like non-zero regions. It is possible to use this property to improve ordinary transform coders. In this paper, the method of projection onto convex sets is used to obtain an improved transform coder for impulse-like pictures in general and difference pictures in particular. Although the amount of improvement depends on many factors, including the percentage of motion, it is found significant for many cases.

The fuzzy set deconvolution technique provides increased flexibility in incorporating a priori information. This method makes it possible to model partially defined information as well as exact knowledge and it has generated successful results when the conventional techniques have failed.

The problem of obtaining the optimal initial conditions and pulse values is discussed. If the pulse positions are available then these optimal values can be obtained by minimizing the mean square error between the original signal and the reconstructed signal at the receiver. The result is obtained by solving a large but simple system of linear equations. The optimal values improve the signal-to-noise ratio 3-5 dB when compared with a multipulse algorithm which operates on data at the transmitter.

The method of projection onto convex sets is extended to the problem of deconvolution in the presence of noise. A collection of convex sets is defined by using properties of the noise and of the ideal signal. The ideal signal is then a member of the intersection of these convex sets. A solution to the deconvolution problem is to choose a member of this intersection. Such a member is obtained by successive projections onto convex sets. If the intersection is small, then any member of the intersection should be a good estimate. One and two dimensional results have shown that this method is effective in producing estimates that are superior to conventional methods when sufficient a priori knowledge is available to define the convex sets.

A relationship is shown between the usual formulation of constrained iterative restoration and restoration by successive projections onto convex sets. When used with a reasonable convergence criterion, this relationship characterizes the restoration as the point closest to the initial estimate which satisfies the convergence criterion. Experimental work is presented to demonstrate the effects of the choice of the initial estimate.