Efficient streaming of video over wireless networks requires real-time assessment of distortion due to packet loss, especially because predictive coding at the encoder can cause inter-frame propagation of errors and impact the overall quality of the transmitted video. This paper presents an algorithm to evaluate the expected receiver distortion on the source side by utilizing encoder information, transmission channel characteristics and error concealment. Specifically, distinct video transmission units, Group of Blocks (GOBs), are iteratively built at the source by taking into account macroblock coding modes and motion-compensated error concealment for three different combinations of packet loss. Distortion of these units is then calculated using the structural similarity (SSIM) metric and they are stochastically combined to derive the overall expected distortion. The proposed model provides a more accurate estimate of the distortion that closely models quality as perceived through the human visual system. When incorporated into a content-aware utility function, preliminary experimental results show improved packet ordering & scheduling efficiency and overall video signal at the receiver.

This paper presents an improved technique for estimating the end-to-end distortion, which includes both a quantization error after encoding and a random transmission error, after transmission in video communication systems. The proposed technique mainly differs from most existing techniques in that it takes into account filtering operations, e.g., interpolation in subpixel motion compensation, as introduced in advanced video codecs. The distortion estimation for pixels or subpixels under filtering operations requires the computation of the second moment of a weighted sum of random variables. In this paper, we prove a proposition for calculating the second moment of a weighted sum of correlated random variables without requiring knowledge of their probability distribution. Then, we apply the proposition to extend our previous error-resilient algorithm for prediction mode decision without significantly increasing complexity. Experimental results using an H.264/AVC codec show that our new algorithm provides an improvement in both rate-distortion performance and subjective quality over existing algorithms. Our algorithm can also be applied in the upcoming high-efficiency video coding standard, where additional filtering techniques are under consideration.

Demand for multimedia services, such as video streaming over wireless networks, has grown dramatically in recent years. The downlink transmission of multiple video sequences to multiple users over a shared resource-limited wireless channel, however, is a daunting task. Among the many challenges in this area are the time-varying channel conditions, limited available resources, such as bandwidth and power, and the different transmission requirements of different video content. This work takes into account the time-varying nature of the wireless channels, as well as the importance of individual video packets, to develop a cross-layer resource allocation and packet scheduling scheme for multiuser video streaming over lossy wireless packet access networks. Assuming that accurate channel feedback is not available at the scheduler, random channel losses combined with complex error concealment at the receiver make it impossible for the scheduler to determine the actual distortion of the sequence at the receiver. Therefore, the objective of the optimization is to minimize the expected distortion of the received sequence, where the expectation is calculated at the scheduler with respect to the packet loss probability in the channel. The expected distortion is used to order the packets in the transmission queue of each user, and then gradients of the expected distortion are used to efficiently allocate resources across users. Simulations show that the proposed scheme performs significantly better than a conventional content-independent scheme for video transmission.

Wireless video transmission is prone to unpredictable degradations due to time-varying channel conditions. Such degradations are difficult to overcome using conventional video coding techniques. Scalable video coding offers a flexible bitstream that can be dynamically adapted to fit the prevailing channel conditions. Within a scalable video coding framework, we develop simple packet prioritization strategies, which, when combined with a reasonable error concealment scheme and a content-aware resource allocation technique, provide for robust video transmission over time-varying channels. The packet prioritization as well as the calculation of the content-aware scheduling metric can be performed offline and signaled to the wireless scheduler.

Video streaming applications have gained in popularity in recent years. The quality of service offered by such applications is limited by the available transmission rates as well as time-varying conditions, such as, channel fading and network congestion, which lead to packet losses. Scalable video coding techniques that allow for the flexible adaptation of temporal resolution as well as quality of an encoded bitstream can be immensely useful in developing video streaming applications that can adapt to time-varying network and channel conditions. Scalable coding techniques, however, are generally designed to offer progressive refinement, which introduces dependencies between encoded video packets. Therefore, when determining a packet scheduling technique for scalable coded video, the possibility of random packet losses, which might affect the decodability of subsequent packets, must be taken into account. In this paper, we take into account the available transmission rate, possibly time-varying channel conditions, and the possibility of random packet losses, to design a scheduling technique for video packets in a scalable bit-stream. Since the optimal solution to the scheduling problem requires an exhaustive, and therefore, intractable computation, we propose a greedy algorithm that will schedule the optimal packet for transmission at a given transmission opportunity based on the encoded content and the available channel state information. Simulation results show significant gains in performance when the proposed technique is compared to content and channel independent packet scheduling techniques.

The emergence of 3G and 4G wireless networks brings with it the possibility of streaming high quality video content on-demand to mobile users. Wireless video applications require appropriate scheduling techniques that make use of the specific characteristics of video content, as well as the well known gains from multiuser diversity. While fast and frequent channel feedback is available in the new generation of wireless networks, the channel estimates cannot be perfect, and channel losses should be taken into account in the packet scheduling and resource allocation. The proposed scheme is formulated as a joint optimization over the resource allocation and channel loss protection, in order to minimize the distortion of the received video sequences. The distortion is a function of the packets deliberately dropped at the transmission queue due to congestion, as well as of random channel losses. The scheme makes use of a packet prioritization strategy that orders video packets based on their contribution to reducing the expected distortion of the received video sequence. Simulation results show that the proposed technique significantly outperforms content-independent packet scheduling schemes.

A cross-layer packet scheduling scheme that streams pre-encoded video over wireless downlink packet access networks to multiple users is presented. The scheme can be used with the emerging wireless standards such as HSDPA and IEEE 802.16. A gradient based scheduling scheme is used in which user data rates are dynamically adjusted based on channel quality as well as the gradients of a utility function. The user utilities are designed as a function of the distortion of the received video. This enables distortion-aware packet scheduling both within and across multiple users. The utility takes into account decoder error concealment, an important component in deciding the received quality of the video. We consider both simple and complex error concealment techniques. Simulation results show that the gradient based scheduling framework combined with the content-aware utility functions provides a viable method for downlink packet scheduling as it can significantly outperform current content-independent techniques. Further tests determine the sensitivity of the system to the initial video encoding schemes, as well as to non-real-time packet ordering techniques.

Abstract As wireless technology evolves towards its fourth generation (4G) of development, the prospect of offering multimedia services such as on‐demand video streaming and video conferencing to wireless mobile clients becomes increasingly more viable. The eventual success of such applications depends on the efficient management of the limited system resources while taking into account the time‐varying wireless channel conditions as well as the varying multimedia source content. In this paper, we review some of the recent advances in cross‐layer design schemes, which aim at providing significant gains in performance for video streaming systems through content‐aware resource allocation. Advances in both, real‐time video streaming, where the video is encoded and transmitted in real‐time, as well as, on‐demand video streaming, where the video is pre‐encoded in a media server, are considered. Copyright © 2007 John Wiley & Sons, Ltd.

The goal of video summarization is to generate a shorter video sequence of a lengthy original sequence using only the key frames of the original sequence. We consider a video summarization scheme that generates a video summary that can be transmitted over an unreliable network such as the Internet with minimum distortion of the original video. We consider two methods of distortion measurement in our optimization scheme, and we apply the methods to a scenario in which feedback is available with the possibility of retransmitting lost packets. Simulation results showing the effectiveness of using the proposed schemes are presented

The use of wireless sensor networks for target tracking is an active area of research. Imaging sensors that obtain video-rate images of a scene can have a significant impact in such networks, as they can measure vital information on the identity, position, and velocity of moving targets. Since wireless networks must operate under stringent energy constraints, it is important to identify the optimal set of imagers to be used in a tracking scenario such that the network lifetime is maximized. We formulate this problem as one of maximizing the information utility gained from a set of sensors subject to a constraint on the average energy consumption in the network. We use an unscented Kalman filter framework to solve the tracking and data fusion problem with multiple imaging sensors in a computationally efficient manner, and use a lookahead algorithm to optimize the sensor selection based on the predicted trajectory of the target. Simulation results show the effectiveness of this method of sensor selection.