Multiple description coding (MDC) is an effective means to combat bursty packet losses in the Internet and wireless networks. MDC is especially promising for video applications where retransmission is unacceptable or infeasible. When combined with multiple path transport (MPT), MDC enables traffic dispersion and hence reduces network congestion. This work describes principles in designing MD video coders employing temporal prediction and presents several predictor structures that differ in their tradeoffs between mismatch-induced distortion and coding efficiency. The paper also discusses example video communication systems integrating MDC and MPT.

Multiple description motion compensation (MDMC) is a multiple description video coding scheme that has shown good error resilience performance. MDMC enables one to vary coding parameters according to the desired trade-off between coding efficiency and error resilience. To fully utilize this advantage, one needs to establish a set of models, relating the rate, encoder distortion, and the end-to-end distortion after transmission, with the encoder parameters and channel parameters. Using these models, one can find the optimal coding parameters for given channel parameters and rate (or distortion) constraints. In this paper, we formulate and validate the rate and encoder distortion models.

Multiple description motion compensation (MDMC) is a multiple description video coding scheme that has shown good error resilience performance. MDMC enables one to vary coding parameters according to the desired trade-off between coding efficiency and error resilience. To fully utilize this advantage, one needs to establish a set of models, relating the rate, encoder distortion, and the end-to-end distortion after transmission, with the encoder parameters and channel parameters. Using these models, one can find the optimal coding parameters for given channel parameters and rate (or distortion) constraints. In this paper, we formulate and validate the rate and encoder distortion models.

Enabling video transport over ad hoc networks is more challenging than over other wireless networks. The wireless links in an ad hoc network are highly error prone and can go down frequently because of node mobility, interference, channel fading, and the lack of infrastructure. However, the mesh topology of ad hoc networks implies that it is possible to establish multiple paths between a source and a destination. Indeed, multipath transport provides an extra degree of freedom in designing error resilient video coding and transport schemes. In this paper, we propose to combine multistream coding with multipath transport, to show that, in addition to traditional error control techniques, path diversity provides an effective means to combat transmission error in ad hoc networks. The schemes that we have examined are: 1) feedback based reference picture selection; 2) layered coding with selective automatic repeat request; and 3) multiple description motion compensation coding. All these techniques are based on the motion compensated prediction technique found in modern video coding standards. We studied the performance of these three schemes via extensive simulations using both Markov channel models and OPNET Modeler. To further validate the viability and performance advantages of these schemes, we implemented an ad hoc multiple path video streaming testbed using notebook computers and IEEE 802.11b cards. The results show that great improvement in video quality can be achieved over the standard schemes with limited additional cost. Each of these three video coding/transport techniques is best suited for a particular environment, depending on the availability of a feedback channel, the end-to-end delay constraint, and the error characteristics of the paths.

Multiple description motion compensation (MDMC) is a multiple description coding scheme which has shown good error resilience performance. This paper investigates the behavior of an MDMC video codec in a packet loss environment. A model for the decoder distortion induced by transmission errors is developed and their accuracy is validated by simulation studies. The model shows the necessity of coding mismatch signals in an MDMC codec. Also, a new mismatch signal coding method and the corresponding decoding strategy are presented which improves upon the original MDMC scheme in a packet lossy environment.

The multihypothesis motion-compensated prediction (MHMCP) approach has shown significant gain in terms of coding efficiency both in theory and practice. However, the fact that it also enhances the error resilience of a codec has not been fully realized. This paper analyzes the error resilience gain of MHMCP. Models for the decoder distortion induced by transmission errors and the rate-distortion function of the encoder in a codec using MHMCP are presented and their accuracy is validated by simulation studies. Using these models, we show how to design the multihypothesis predictor to jointly consider the coding gain and the error resilience gain for given channel error characteristics.

Typical video applications may need a higher bandwidth and/or higher reliability connection than that provided by a single link in current or emerging wireless networks. We propose to employ path diversity to provide higher bandwidth and more robust end-to-end connections than that affordable by a single path. Under this transport environment, two viable strategies for video coding are multiple description coding (MDC) and layered coding (LC). MDC is more effective when the underlying application has a very stringent delay constraint and the round trip time on each path is relatively long. LC can be a good alternative when limited retransmission of the base layer is acceptable and when it is feasible to apply unequal error protection over different paths. The paper describes the general issues involved in integrating MDC/LC with multiple path transport, and compares the performances of MDC and LC, under different path conditions.

A new approach for multiple description video coding is proposed. It employs a second-order predictor for motion-compensation, which predicts a current frame from two previously coded frames. The coder generates two descriptions, containing the coded even and odd frames, respectively. When only a single description (say, that containing even frames) is received, the decoder can only use previous even frames for prediction. The mismatch between the predicted frames at the encoder and decoder is explicitly coded to avoid error propagation in the ideal MD channels, where a description is either received intact or lost completely. By using the second-order predictor and coding the mismatch signal, one can also suppress error propagation in packet lossy networks where packets in either description can be lost. The predictor and the mismatch signal quantizer can be varied to achieve a wide range of tradeoffs between coding efficiency and error resilience.

Enabling video transmission over ad-hoc networks is more challenging than over conventional mobile networks because a connection path in an ad-hoc network is highly error-prone and the path can go down frequently. On the other hand, it is possible to establish multiple paths between a source and a destination, which provides an extra degree of freedom in coding algorithm design. This paper presents a feedback-based reference picture selection scheme for video transmission over ad-hoc networks. Encoded video streams are transmitted over multiple paths and the reference frames for motion compensated prediction are selected according to the feedback information about the paths' condition. Simulations under the two paths scenario have shown significant improvement over two standard techniques, layered coding and video redundancy coding, which do not use feedback. A novel statistical model for the ad-hoc multi-path environment is also proposed and used in our simulation of transmission loss.

A new approach for multiple description video coding using motion compensated prediction is proposed. The temporal predictor is designed to provide a desired trade-off between prediction gain and mismatch between the predictions obtained at the encoder and at the decoder when only one description is available. The mismatch signal is explicitly coded to avoid error propagation. The system can achieve a wide range of redundancy, by varying the predictor and the quantizer for the mismatch signal.

The increase in the bandwidth of the wireless channels and the computing power of the mobile devices makes it possible to offer video service for wireless networks in the near future. In an ad-hoc network, strong error protection is required because of the lack of a fixed infrastructure. On the other hand, the mesh structure of an ad-hoc network implies that there may be multiple paths existing between a source and destination, which can be used to enhance video transmissions. We propose a simple but robust scheme for reliable transmission of video in bandwidth limited ad-hoc networks. In our scheme, a video stream is layer coded. The base layer (BL) packets and the enhancement layer (EL) packets are transmitted separately on two disjoint paths using multipath transport (MPT). BL packets are protected by automatic repeat request (ARQ), and a lost BL packet is retransmitted through the path where EL packets are transmitted. An EL packet has lower priority than a retransmitted BL packet and may be dropped at the sender when congestion occurs. Simulation results show that this scheme can guarantee a graceful video quality in adverse channel conditions. It is effective for video transmission over the high loss environment found in ad-hoc networks.

This paper presents novel algorithms that perform motion estimation for video processing and compression. We observe that "smoothness" is a very important and intuitive property in the estimation of motion fields. It is pointed out that most current motion estimation techniques implement smoothness as a constraint, differing only in terms of the specific type of smoothness demanded from video data. This paper views smoothness as a property that is determined by the underlying video data rather than a predetermined and specific property that is imposed on video data. Instead of forcing the available video data to conform to an abstract smoothness model, we try to select the "smoothest" motion field that conforms to the available data. We propose fast and efficient techniques that determine a set of possible motion fields and that select the smoothest field from this set. Issues like quantization and embedded video compression (via embedded motion fields) are discussed.