Despite the promising performance of video compressive sensing (CS) in compressing volumetric visual data from resource-constrained sensors in emerging internet of video things (IoVT) applications, achieving high-quality video CS reconstruction remains challenging. The most representative video CS reconstruction approaches include multi-hypothesis (MH) prediction and deep neural network-based schemes. However, in almost all existing methods, motion-induced irregular deformations are neglected, leading to suboptimal reconstruction quality. In this paper, these irregular deformations are addressed by devising irregular-shaped hypotheses into the geometrical hypothesis set. Relying on the geometrical hypothesis set, a novel and efficient geometrical MH prediction approach is proposed for video CS reconstruction. The proposed approach simultaneously considers both the computational cost and the optimality of the hypothesis set. Extensive experiments and comparisons with recent state-of-theart methods on various video contents and resolutions demonstrate that the proposed approach not only achieves remarkable average reconstruction quality—with overall average improvements of 2.95 dB, 6.91 dB, and 7.56 dB in PSNR and 0.125, 0.051, and 0.155 in SSIM over standard CIF-sized test video sequences, real-world capsule endoscopy videos, and realworld surveillance videos, respectively—but also effectively suppresses severe and undesirable fluctuations in the quality of reconstructed key and non-key frames. Furthermore, the approach preserves the structural integrity and texture details of video frames while maintaining reasonable computational complexity and runtime. These high-quality reconstructed frames with well-preserved structures and textures contribute to more reliable diagnosis and decision-making, thereby enabling optimal intelligent services in various IoVT applications.

InIn HTTP Adaptive Streaming (HAS), each video is divided into smaller segments, and each segment is encoded at multiple pre-defined bitrates to construct abitrate ladder. To optimize bitrate ladders, per-title encoding approaches encode each segment at various bitrates and resolutions to determine the convex hull. From the convex hull, an optimized bitrate ladder is constructed, resulting in an increased Quality of Experience (QoE) for end-users. With the ever-increasing efficiency of deep learning-based video enhancement approaches, they are more and more employed at the client-side to increase the QoE, specifically when GPU capabilities are available. Therefore, scalable approaches are needed to support end-user devices with both CPU and GPU capabilities (denoted as CPU-only and GPU-available end-users, respectively) as a new dimension of a bitrate ladder. To address this need, we proposeDeepStream, ascalable content-awareper-title encoding approach to support both CPU-only and GPU-available end-users. (i) To supportbackward compatibility,DeepStreamconstructs a bitrate ladder based on any existing per-title encoding approach. Therefore, the video content will be provided for legacy end-user devices with CPU-only capabilities as a base layer (BL). (ii) For high-end end-user devices with GPU capabilities, an enhancement layer (EL) is added on top of the base layer comprising lightweight video super-resolution deep neural networks (DNNs) for each bitrate-resolution pair of the bitrate ladder. A content-aware video super-resolution approach leads to higher video quality, however, at the cost of bitrate overhead. To reduce the bitrate overhead for streaming content-aware video super-resolution DNNs,DeepCABAC, context-adaptive binary arithmetic coding for DNN compression, is used. Furthermore, the similarity among (i) segments within a scene and (ii) frames within a segment are used to reduce the training costs of DNNs. Experimental results show bitrate savings of 34% and 36% to maintain the same PSNR and VMAF, respectively, for GPU-available end-users, while the CPU-only users get the desired video content as usual.

To the distributed video coding approach, which focuses on fully or partially shifting the computational complexity from video encoder to the decoder, the simplicity and highly compact sampling in emerging compressive sensing appear to be very efficient tools. So, the distributed compressive video sensing (DCVS) has attracted much attention in the video coding community by applying constant high measurement rate (MR) for key frames sampling and constant low MR for non-key frames sampling. According to the use of constant and different MRs for the key and non-key frames sampling, severe and undesirable fluctuations in quality of reconstructed video frames is a common unresolved shortcoming in the DCVS, which negatively affects the users’ visual experience. To suppress sharp and undesirable quality fluctuations, group of picture (GOP)-level non-uniform MR allocation models are proposed in this article for the key and non-key frames at the encoder of the DCVS. This enhances visual quality without incurring noticeable computational cost to the encoder. A new multi-step reconstruction scheme is also proposed at the decoder exploiting spatial-temporal information in the reconstruction process with a tolerable computational complexity and remarkable reconstruction performance. It compensates for quality degradations, which may be caused by non-uniform MR allocation, to successive GOPs to reach high average quality and temporal smoothness of quality at the same time. Extensive experiments on different video sequences show that not only desirable high average reconstruction quality is maintained, but severe and undesirable quality fluctuations are also well suppressed. Hence, the users’ perceived quality is highly promoted, while the compression ratio does not exceed a certain target by restricting average MR to reach the target MR in the long term.

With the ubiquity of video streaming, optimizing the delivery of video content while reducing energy consumption has become increasingly critical. Traditional adaptive streaming relies on a fixed set of bitrate-resolution pairs, known as bitrate ladders, for encoding. However, this "one-size-fits-all" approach is suboptimal for diverse video content. As a result, per-title encoding approaches dynamically select the bitrate ladder for each content. In this paper, we address the pressing issue of increasing energy consumption in video streaming by introducing GreenRes, a novel approach that goes beyond the traditional selection of quality-centric resolutions. Instead, GreenRes considers both video quality and energy consumption to construct an optimal bitrate ladder tailored to the unique characteristics of each video content. To achieve this, GreenRes, similar to per-title encoding, encodes each video content at various resolutions, each with a set of bitrates. It then sets a maximum acceptable quality drop threshold and selects resolutions that maintain video quality above this threshold while minimizing energy consumption. Our experimental results demonstrate an average reduction in energy consumption of 30.82%, while ensuring a maximum quality drop of only 0.53 Video Multimethod Assessment Fusion (VMAF) points.

Video streaming and its applications are growing rapidly, making video optimization a primary target for content providers looking to enhance their services. Enhancing the quality of videos requires the adjustment of different encoding parameters such as bitrate, resolution, and frame rate. To avoid brute force approaches for predicting optimal encoding parameters, video complexity features are typically extracted and utilized. To predict optimal encoding parameters effectively, content providers traditionally use unsupervised feature extraction methods, such as ITU-T’s Spatial Information (SI) and Temporal Information (TI) to represent the spatial and temporal complexity of video sequences. Recently, Video Complexity Analyzer (VCA) was introduced to extract DCT-based features to represent the complexity of a video sequence (or parts thereof). These unsupervised features, however, cannot accurately predict video encoding parameters. To address this issue, this paper introduces a novel supervised feature extraction method named DeepVCA, which extracts the spatial and temporal complexity of video sequences using deep neural networks. In this approach, the encoding bits required to encode each frame in intra-mode and inter-mode are used as labels for spatial and temporal complexity, respectively. Initially, we benchmark various deep neural network structures to predict spatial complexity. We then leverage the similarity of features used to predict the spatial complexity of the current frame and its previous frame to rapidly predict temporal complexity. This approach is particularly useful as the temporal complexity may depend not only on the differences between two consecutive frames but also on their spatial complexity. Our proposed approach demonstrates significant improvement over unsupervised methods, especially for temporal complexity. As an example application, we verify the effectiveness of these features in predicting the encoding bitrate and encoding time of video sequences, which are crucial tasks in video streaming. The source code and dataset is available at https://github.com/cd-athena/ DeepVCA.

Adaptive live video streaming applications utilize a predefined collection of bitrate-resolution pairs, known as abitrate ladder, for simplicity and efficiency, eliminating the need for additional run-time to determine the optimal pairs during the live streaming session. These applications do not incorporate two-pass encoding methods due to increased latency. However, an optimized bitrate ladder could result in lower storage and delivery costs and improvedQuality of Experience(QoE). This paper presents a Just Noticeable Difference (JND)-aware constrained Variable Bitrate (cVBR) Two-pass Per-title encoding Scheme (JTPS) designed specifically for live video streaming. JTPS predicts a content- and JND-aware bitrate ladder using low-complexity features based onDiscrete Cosine Transform(DCT) energy and optimizes the constant rate factor (CRF) for each representation using random forest-based models. The effectiveness of JTPS is demonstrated using the open source video encoder x265, with an average bitrate reduction of 18.80% and 32.59% for the same PSNR and VMAF, respectively, compared to the standardHTTP Live Streaming(HLS) bitrate ladder using Constant Bitrate (CBR) encoding. The implementation of JTPS also resulted in a 68.96% reduction in storage space and an 18.58% reduction in encoding time for a JND of six VMAF points.

Recent years have witnessed video streaming demands evolve into one of the most popular Internet applications. With the ever-increasing personalized demands for highdefinition and low-latency video streaming services, networkassisted video streaming schemes employing modern networking paradigms have become a promising complementary solution in the HTTP Adaptive Streaming (HAS) context. The emergence of such techniques addresses long-standing challenges of enhancing users’ Quality of Experience (QoE), end-to-end (E2E) latency, as well as network utilization. However, designing a cost-effective, scalable, and flexible network-assisted video streaming architecture that supports the aforementioned requirements for live streaming services is still an open challenge. This article leverages novel networking paradigms, i.e., edge computing and Network Function Virtualization (NFV), and promising video solutions, i.e., HAS, Video Super-Resolution (SR), and Distributed Video Transcoding (TR), to introduce A Latency-and cost-aware hybrId P2P-CDN framework for liVe video strEaming (ALIVE). We first introduce the ALIVE multi-layer architecture and design an action tree that considers all feasible resources (i.e., storage, computation, and bandwidth) provided by peers, edge, and CDN servers for serving peer requests with acceptable latency and quality. We then formulate the problem as a Mixed Integer Linear Programming (MILP) optimization model executed at the edge of the network. To alleviate the optimization model’s high time complexity, we propose a lightweight heuristic, namely, Greedy-Based Algorithm (GBA). Finally, we (i) design and instantiate a large-scale cloud-based testbed including 350 HAS players, (ii) deploy ALIVE on it, and (iii) conduct a series of experiments to evaluate the performance of ALIVE in various scenarios. Experimental results indicate that ALIVE (i) improves the users’ QoE by at least 22%, (ii) decreases incurred cost of the streaming service provider by at least 34%, (iii) shortens clients’ serving latency by at least 40%, (iv) enhances edge server energy consumption by at least 31%, and (v) reduces backhaul bandwidth usage by at least 24% compared to baseline approaches.

In this paper, we propose a novel attack model called the Got Plaintext Attack (GPA), where the attacker only requires one plaintext and the ciphertext image set stored in the cloud to attack the content of the ciphertext image. Using this model, we examine the security of the Improved Redundant Space Transfer (IRST) encryption method. To this end, we define an ordered characteristic matrix based on the properties of the three keys used in IRST. By comparing the histogram distance of the ordered characteristic matrix, we are able to obtain a plain-ciphertext pair. Furthermore, by leveraging the invariant properties of the ordered characteristic matrix of image blocks in the plain-ciphertext pair, we estimate the block permutation Π2and the bit-plane permutation sequence Π1. Our experiments show that the accuracy of estimating Π2is higher than 70% for block sizes of 3x3 pixels or larger. Despite a 40% accuracy in estimating Π1, the content information of the ciphertext image can still be exposed.

This paper presents a method for evaluating the quality of images altered by Gaussian blur. The method is based on the observation of bokeh mode images where the region of interest (foreground) is sharp, while the remaining parts (background) are intentionally blurred to enhance the perceptual quality of the image. The blurriness of the background increases attention towards the foreground part of the image. The proposed quality metric is obtained by combining the attention factor and the sharpness of the region of interest. The accuracy, in terms of Spearman’s-rank-order correlation-coefficient (SROCC), for popular and publicly available databases such as LIVE, VCL, TID2008, CSIQ, and TID2013, is 0.963, 0.925, 0.900, 0.930, and 0.930, respectively. The proposed method achieves high and consistent Spearman’s rank-order correlation coefficient (SROCC) values compared to the majority of state-of-the-art algorithms. Furthermore, in terms of speed, the proposed method surpasses other state-of-the-art methods. The MATLAB code of the proposed metric is publicly available at https://drive.google.com/drive/folders/1SRmUp0N157Ati9l3kV13uoCxw5PhMgQn?usp=sharing .

Abstract Although subjective test is the most accurate image/video quality assessment tool, it is extremely time demanding. In the past two decades, a variety of objective quality measuring tools, such as SSIM, IW-SSIM, SPSIM, FSIM, etc., have been devised, that well correlate with the subjective tests results. However, the main problem with these methods is that they do not discriminate the measured quality well enough, especially at high quality range. In this article we show how the accuracy/precision of these Image Quality Assessment (IQA) meters can be increased by mapping them into a Logistic Function (LF). The precisions are tested over a variety of image/video databases. Our experimental tests indicate while the used high-quality images can be discriminated by 23% resolution on the MOS subjective scores, discrimination resolution by the widely used IQAs are only 2%, but their mapped IQAs to Logistic Function at this quality range can be improved to 9 − 17%, depending on the characteristics of the LF function. Moreover, their precision at low to mid quality range can also be improved. At this quality range, while the discrimination resolution of MOS of the tested images is 23.2%, those of raw IQAs is nearly 8.9%, but discrimination of their adapted logistic functions can be very close to that of MOS. Moreover, with the used image databases the Pearson Linear Correlation Coefficient (PLCC) of MOS with the logistic function can be improved by 2 − 20% as well.

The human visual system response to picture quality degradation due to packet loss is very different from the responses of objective quality measures. While video quality due to packet loss may be impaired by at most for one Group of Pictures (GOP), its subjective quality degradation may last for several GOPs. This has a great impact on resource allocation strategies, which normally make decisions on instantaneous conditions of multiplexing buffer. This is because, when the perceptual impact of degraded video quality is much longer than its objective degradation period, any assigned resources to the degraded flow is wasted. This paper, through both simulations and analysis shows that, during resource allocation, if the quality of a video stream is significantly degraded, it is better to penalize this degraded flow from getting its full bandwidth share and instead assign the remaining share to other flows preventing them from undergoing quality degradation.

With the ever-increasing demands for high-definition and low-latency video streaming applications, network-assisted video streaming schemes have become a promising complementary solution in the HTTP Adaptive Streaming (HAS) context to improve users’ Quality of Experience (QoE) as well as network utilization. Edge computing is considered one of the leading networking paradigms for designing such systems by providing video processing and caching close to the end-users. Despite the wide usage of this technology, designing network-assisted HAS architectures that support low-latency and high-quality video streaming, including edge collaboration is still a challenge. To address these issues, this article leverages the Software-Defined Networking (SDN), Network Function Virtualization (NFV), and edge computing paradigms to proposeAcollaboRative edge-Assisted framewoRk for HTTPAdaptive video sTreaming (ARARAT). Aiming at minimizing HAS clients’ serving time and network cost, besides considering available resources and all possible serving actions, we design a multi-layer architecture and formulate the problem as a centralized optimization model executed by the SDN controller. However, to cope with the high time complexity of the centralized model, we introduce three heuristic approaches that produce near-optimal solutions through efficient collaboration between the SDN controller and edge servers. Finally, we implement theARARATframework, conduct our experiments on a large-scale cloud-based testbed including 250 HAS players, and compare its effectiveness with state-of-the-art systems within comprehensive scenarios. The experimental results illustrate that the proposedARARATmethods (${i}$) improve users’ QoE by at least 47%, (ii) decrease the streaming cost, including bandwidth and computational costs, by at least 47%, and (iii) enhance network utilization, by at least 48% compared to state-of-the-art approaches.

In HTTP Adaptive Streaming (HAS), videos are encoded at multiple bitrates and spatial resolutions ( i.e. , representations ) to adapt to the heterogeneity of network conditions, device attributes, and end-user preferences. Encoding the same video segment at multiple representations increases costs for content providers. State-of-the-art multi-encoding schemes improve the encoding process by utilizing encoder analysis information from already encoded representation(s) to reduce the encoding time of the remaining representations. These schemes typically use the highest bitrate representation as the reference to accelerate the encoding of the remaining representations. Nowadays, most streaming services utilize cloud-based encoding techniques, enabling a fully parallel encoding process to reduce the overall encoding time. The highest bitrate representation has the highest encoding time than the other representations. Thus, utilizing it as the reference encoding is unfavorable in a parallel encoding setup as the overall encoding time is bound by its encoding time. This paper provides a comprehensive study of various multi-rate and multi-encoding schemes in both serial and parallel encoding scenarios. Furthermore, it introduces novel heuristics to limit the Rate Distortion Optimization (RDO) process across various representations. Based on these heuristics, three multi-encoding schemes are proposed, which rely on encoder analysis sharing across different representations: (i) optimized for the highest compression efficiency , (ii) optimized for the best compression efficiency-encoding time savings trade-off , and (iii) optimized for the best encoding time savings . Experimental results demonstrate that the proposed multi-encoding schemes (i) , (ii) , and (iii) reduce the overall serial encoding time by 34.71%, 45.27%, and 68.76% with a 2.3%, 3.1%, and 4.5% bitrate increase to maintain the same VMAF, respectively compared to stand-alone encodings. The overall parallel encoding time is reduced by 22.03%, 20.72%, and 76.82% compared to stand-alone encodings for schemes (i) , (ii) , and (iii) , respectively.

Ultra High Definition Television (UHDTV) offers a better immersive audiovisual experience than HDTV by improving the aesthetic sense of the content [1]. How-ever, it may lead to an increase of both encoding time complexity and compression artifacts at lower bitrates. To address this challenge, a low-latency pre-processing algorithm named COntent-aware frame Dropping Algorithm (CODA) is proposed to predict the optimized framerate per video segment in streaming scenarios. The optimized framerate$(\hat{f})$for every video segment at each target bitrate is modelled as an exponential decay (increasing) function whose decay rate is directly proportional to the temporal characteristics$(h)$[2] [3] of the video and the target bitrate$(b)$, and inversely proportional to the spatial characteristics$(E)$of the video. The encoding is carried out with the predicted framerate, saving encoding time and improving visual quality at lower bitrates. At the decoder side, the video is upscaled in the temporal domain to the original framerate$(f_{max})$for display.

Current per-title encoding schemes encode the same video content at various bitrates and spatial resolutions to find an optimized bitrate ladder for each video content in Video on Demand (VoD) applications. However, in live streaming applications, a bitrate ladder with fixed bitrate-resolution pairs is used to avoid the additional latency caused to find optimum bitrate-resolution pairs for every video content. This paper introduces an online per-title encoding scheme (OPTE) for live video streaming applications. In this scheme, each target bitrate’s optimal resolution is predicted from any pre-defined set of resolutions using Discrete Cosine Transform (DCT)-energy-based low-complexity spatial and temporal features for each video segment. Experimental results show that, on average, OPTE yields bitrate savings of 20.45% and 28.45% to maintain the same PSNR and VMAF, respectively, compared to a fixed bitrate ladder scheme (as adopted in current live streaming deployments) without any noticeable additional latency in streaming.

With the emerging demands of high-definition and low-latency video streams, HTTP Adaptive Streaming (HAS) is considered the principal video delivery technology over the Internet. Network-assisted video streaming schemes, which employ modern networking paradigms, e.g., Software-Defined Networking (SDN), Network Function Virtualization (NFV), and edge computing, have been introduced as promising complementary solutions in the HAS context to improve users’ Quality of Experience (QoE) as well as network utilization. However, the existing network-assisted HAS schemes have not fully used edge collaboration techniques and SDN capabilities for achieving the aforementioned aims. To bridge this gap, this paper introduces a coLlaborative Edge- and SDN-Assisted framework for HTTP aDaptive vidEo stReaming (LEADER). In LEADER, the SDN controller collects various information items and runs a central optimization model that minimizes the HAS clients’ serving time, subject to the network’s and edge servers’ resource constraints. Due to the NP-completeness and impractical overheads of the central optimization model, we propose an online distributed lightweight heuristic approach consisting of two phases that runs on the SDN controller and edge servers, respectively. We implement the proposed framework, conduct our experiments on a large-scale testbed including 250 HAS players, and compare its effectiveness with other strategies. The experimental results demonstrate that LEADER outperforms baseline schemes in terms of both users’ QoE and network utilization, by at least 22% and 13%, respectively.

In two-pass encoding, also known as multi-pass encoding, the input video content is analyzed in the first-pass to help the second-pass encoding utilize better encoding decisions and improve overall compression efficiency. In live streaming applications, a single-pass encoding scheme is mainly used to avoid the additional first-pass encoding run-time to analyze the complexity of every video content. This paper introduces an Efficient low-latency Two-Pass encoding Scheme (ETPS) for live video streaming applications. In this scheme, Discrete Cosine Transform (DCT)-energy-based low-complexity spatial and temporal features for every video segment are extracted in the first-pass to predict each target bitrate’s optimal constant rate factor (CRF) for the second-pass constrained variable bitrate (cVBR) encoding. Experimental results show that, on average, ETPS compared to a traditional two-pass average bitrate encoding scheme yields encoding time savings of 43.78% without any noticeable drop in compression efficiency. Additionally, compared to a single-pass constant bitrate (CBR) encoding, it yields bitrate savings of 10.89% and 8.60% to maintain the same PSNR and VMAF, respectively.

In live streaming applications, a fixed set of bitrate-resolution pairs (known as bitrate ladder) is used for simplicity and efficiency to avoid the additional encoding run-time required to find optimum resolution-bitrate pairs for every video content. However, an optimized bitrate ladder may result in (i) decreased storage or delivery costs or/and (ii) increased Quality of Experience (QoE). This paper introduces a perceptually-aware per-title encoding (PPTE) scheme for video streaming applications. In this scheme, optimized bitrate-resolution pairs are predicted online based on Just Noticeable Difference (JND) in quality perception to avoid adding perceptually similar representations in the bitrate ladder. To this end, Discrete Cosine Transform (DCT)-energy-based low-complexity spatial and temporal features for each video segment are used. Experimental results show that, on average, PPTE yields bitrate savings of 16.47% and 27.02% to maintain the same PSNR and VMAF, respectively, compared to the reference HTTP Live Streaming (HLS) bitrate ladder without any noticeable additional latency in streaming accompanied by a 30.69% cumulative decrease in storage space for various representations.

The increasing demand for high-quality and low-cost video streaming services calls for the prediction of video encoding complexity. The prior prediction of video encoding complexity including encoding time and bitrate predictions are used to allocate resources and set optimized parameters for video encoding effectively. In this paper, a light-weight video encoding complexity prediction (VECP) scheme that predicts the encoding bitrate and the encoding time of video with high accuracy is proposed. Firstly, low-complexity Discrete Cosine Transform (DCT)-energy-based features, namely spatial complexity, temporal complexity, and brightness of videos are extracted, which can efficiently represent the encoding complexity of videos. The latent vectors are also extracted from a Convolutional Neural Network (CNN) with MobileNet as the backend to obtain additional features from representative frames of each video to assist the prediction process. The extreme gradient boosting (XGBoost) regression algorithm is deployed to predict video encoding complexity using the extracted features. The experimental results demonstrate that VECP predicts the encoding bitrate with an error percentage of up to 3.47% and encoding time with an error percentage of up to 2.89%, but with a significantly low overall latency of 3.5 milliseconds per frame which makes it suitable for both Video on Demand (VoD) and live streaming applications.

This paper introduces an algorithm to improve the security, efficiency, and embedding capacity of reversible data hiding in encrypted images (RDH-EI). It is based on classification encryption difference and adaptive fixed-length coding. Firstly, the prediction error image is obtained, the difference with a bin value greater than the encryption threshold in the difference histogram is found, and it is further modified to obtain the embedding threshold range. Then, under the condition of ensuring that the difference inside and outside the embedding threshold range is not confused, the difference within the threshold is only scrambled, and the difference outside the threshold is scrambled and mod encrypted. After obtaining the encrypted image, an adaptive difference fixed-length coding method is proposed to encode and compress the differences within the threshold. The secret data is embedded in the multiple most significant bits of the encoded difference. Experimental results show that the embedding capacity of the proposed algorithm is improved compared with the state-of-the-art algorithm.

This paper provides an overview of the open Video Complexity Dataset (VCD) which comprises 500 Ultra High Definition (UHD) resolution test video sequences. These sequences are provided at 24 frames per second (fps) and stored online in losslessly encoded 8-bit 4:2:0 format. In this paper, all sequences are characterized by spatial and temporal complexities, rate-distortion complexity, and encoding complexity with the x264 AVC/H.264 and x265 HEVC/H.265 video encoders. The dataset is tailor-made for cutting-edge multimedia applications such as video streaming, two-pass encoding, per-title encoding, scene-cut detection, etc. Evaluations show that the dataset includes diversity in video complexities. Hence, using this dataset is recommended for training and testing video coding applications. All data have been made publicly available as part of the dataset, which can be used for various applications.

For online analysis of the video content complexity in live streaming applications, selecting low-complexity features is critical to ensure low-latency video streaming without disruptions. To this light, for each video (segment), two features, i.e., the average texture energy and the average gradient of the texture energy, are determined. A DCT-based energy function is introduced to determine the block-wise texture of each frame. The spatial and temporal features of the video (segment) are derived from this DCT-based energy function. The Video Complexity Analyzer (VCA) project aims to provide an efficient spatial and temporal complexity analysis of each video (segment) which can be used in various applications to find the optimal encoding decisions. VCA leverages some of the x86 Single Instruction Multiple Data (SIMD) optimizations for Intel CPUs and multi-threading optimizations to achieve increased performance. VCA is an open-source library published under the GNU GPLv3 license.

In live streaming applications, a fixed set of bitrate-resolution pairs (known as bitrate ladder) is generally used to avoid additional pre-processing run-time to analyze the complexity of every video content and determine the optimized bitrate ladder. Furthermore, live encoders use the fastest available preset for encoding to ensure the minimum possible latency in streaming. For live encoders, it is expected that the encoding speed is equal to the video framerate. An optimized encoding preset may result in (i) increased Quality of Experience (QoE) and (ii) improved CPU utilization while encoding. In this light, this paper introduces a Content-Adaptive encoder Preset prediction Scheme (CAPS) for adaptive live video streaming applications. In this scheme, the encoder preset is determined using Discrete Cosine Transform (DCT)-energy-based low-complexity spatial and temporal features for every video segment, the number of CPU threads allocated for each encoding instance, and the target encoding speed. Experimental results show that ChPS yields an overall quality improvement of 0.83 dB PSNR and 3.81 VMAF with the same bitrate, compared to the fastest preset encoding of the HTTP Live Streaming (HLS) bitrate ladder using $\times265$ HEVC open-source encoder. This is achieved by maintaining the desired encoding speed and reducing CPU idle time.

Recently, a larger interest in the different plenoptic formats, including digital holograms, has emerged. Aside from other challenges that several steps of the holographic pipeline, from digital acquisition to display, have to face, visual quality assessment of compressed holograms is particularly demanding due to the distinct nature of this 3D image modality when compared to regular 2D imaging. There are few studies on holographic data quality assessment, particularly with respect to perceptual effects of lossy compression. This work aims to study the quality evaluation of digital hologram reconstructions, presented on regular 2D displays, in the presence of compression distortions. As there is no established or generally agreed compression methodology for digital holograms compression on the hologram plane with available implementations, a set of state-of-the-art compression codecs, namely HEVC, AV1, and JPEG2000, were used for compression of the digital holograms on the object plane. Both computer generated and optically generated holograms were considered. Two subjective tests were conducted to evaluate distortions caused by compression. The first subjective test was conducted on the reconstructed amplitude images of central views, while the second test was conducted on pseudo-videos generated from the reconstructed amplitudes of different views. The subjective quality assessment was based on mean opinion scores. A selection of objective quality metrics was evaluated, and their correlations with mean opinion scores were computed. The VIFp metrics appeared to have the highest correlation.

In light field image compression, facilitating random access to individual views plays a significant role in decoding views quickly, reducing memory footprint, and decreasing the bandwidth requirement for transmission. Highly efficient light field image compression methods mainly use inter view prediction. Therefore, they typically do not provide random access to individual views. On the other hand, methods that provide full random access usually reduce compression efficiency. To address this trade-off, a light field image encoding method that favors random access is proposed in this paper. Light field image views are grouped into independent (3× 3) views, which are called Macro View Images (MVIs) . To encode MVIs, the central view is used as a reference to compress its adjacent neighboring views using a hierarchical reference structure. To encode the central view of each MVI, the most central view along with the center of a maximum of three MVIs, are used as reference images for the disparity estimation. In addition, the proposed method allows the use of parallel processing to reduce the maximum encoding/decoding time-complexity in multi-core processors. Tile partitioning can also be used to randomly access different regions of the light field images. The simulation results show that the proposed method outperforms other state-of-the-art methods in terms of compression efficiency while providing random access to both views and regions of interest.

Light field imaging enables some post-processing capabilities like refocusing, changing view perspective, and depth estimation. As light field images are represented by multiple views, they contain a huge amount of data that makes compression inevitable. Although there are some proposals to efficiently compress light field images, their main focus is on encoding efficiency. However, some important functionalities such as viewpoint and quality scalabil- ities, random access, and uniform quality distribution have not been addressed adequately. In this paper, an efficient light field image compression method based on a deep neural network is proposed, which classifies multiple views into various layers. In each layer, the target view is synthesized from the available views of previously encoded/decoded layers using a deep neural network. This synthesized view is then used as a virtual reference for the target view inter-coding. In this way, random access to an arbitrary view is provided. Moreover, uniform quality distribution among multiple views is addressed. In higher bitrates where random access to an arbitrary view is more crucial, the required bitrate to access the requested view is minimized.

Video delivery over the Internet has been becoming a commodity in recent years, owing to the widespread use of Dynamic Adaptive Streaming over HTTP (DASH). The DASH specification defines a hierarchical data model for Media Presentation Descriptions (MPDs) in terms of segments. This paper focuses on segmenting video into multiple shots for encoding in Video on Demand (VoD) HTTP Adaptive Streaming (HAS) applications. Therefore, we propose a novel Discrete Cosine Transform (DCT) feature-based shot detection and successive elimination algorithm for shot detection and compare it against the default shot detection algorithm of the x265 implementation of the High Efficiency Video Coding (HEVC) standard. Our experimental results demonstrate that our proposed feature-based pre-processor has a recall rate of 25% and an F-measure of 20% greater than the benchmark algorithm for shot detection.

Current per-title encoding schemes encode the same video content (or snippets/subsets thereof) at various bitrates and spatial resolutions to find an optimal bitrate ladder for each video content. Compared to traditional approaches, in which a predefined, content-agnostic ("fit-to-all") encoding ladder is applied to all video contents, per-title encoding can result in (i) a significant decrease of storage and delivery costs and (ii) an increase in the Quality of Experience (QoE). In the current per-title encoding schemes, the bitrate ladder is optimized using only spatial resolutions, while we argue that with the emergence of high framerate videos, this principle can be extended to temporal resolutions as well. In this paper, we improve the per-title encoding for each content using spatio-temporal resolutions. Experimental results show that our proposed approach doubles the performance of bitrate saving by considering both temporal and spatial resolutions compared to considering only spatial resolutions.

Recent studies have revealed that network-assisted techniques, by providing a comprehensive view of the network, improve HTTP Adaptive Streaming (HAS) system performance significantly. This paper leverages the capability of Software-Defined Networking, Network Function Virtualization, and edge computing to introduce a CDN-Aware QoE Optimization in SDN-Assisted Adaptive Video Streaming (CSDN) framework. We employ virtualized edge entities to collect various information items and run an optimization model with a new server/segment selection approach in a time-slotted fashion to serve the clients’ requests by selecting optimal cache servers. In case of a cache miss, a client’s request is served by an optimal replacement quality from a cache server, by a quality transcoded from an optimal replacement quality at the edge, or by the originally requested quality from the origin server. Comprehensive experiments conducted on a large-scale testbed demonstrate that CSDN outperforms other approaches in terms of the users’ QoE and network utilization.

High Efficiency Video Coding (HEVC) improves the encoding efficiency by utilizing sophisticated tools such as flexible Coding Tree Units (CTUs) partitioning. The Coding Units (CUs) can be split recursively into four equally sized CUs ranging from 64×64 to 8×8 pixels. At each depth level (or CU size), intra prediction via exhaustive mode search was exploited in HEVC to improve the encoding efficiency and result in a very high encoding time complexity. This paper proposes an Intra CU Depth Prediction (INCEPT) algorithm, which limits Rate-Distortion Optimization (RDO) for each CTU in HEVC by utilizing the spatial correlation with the neighboring CTUs, which is computed using a DCT energy-based feature. Thus, INCEPT reduces the number of candidate CU sizes required to be considered for each CTU in HEVC intra coding. Experimental results show that the INCEPT algorithm achieves a better trade-off between the encoding efficiency and encoding time saving (i.e., BDR/∆T) than the benchmark algorithms. While BDR/∆T is 12.35% and 9.03% for the benchmark algorithms, it is 5.49% for the proposed algorithm. As a result, INCEPT achieves a 23.34% reduction in encoding time on average while incurring only a 1.67% increase in bitrate than the original coding in the x265 HEVC open-source encoder.

Recently, HTTP Adaptive Streaming (HAS) has become the dominant video delivery technology over the Internet. In HAS, clients have full control over the media streaming and adaptation processes. Lack of coordination among the clients and lack of awareness of the network conditions may lead to sub-optimal user experience and resource utilization in a pure client-based HAS adaptation scheme. Software Defined Networking (SDN) has recently been considered to enhance the video streaming process. In this paper, we leverage the capability of SDN and Network Function Virtualization (NFV) to introduce an edge- and SDN-assisted video streaming framework called ES-HAS. We employ virtualized edge components to collect HAS clients' requests and retrieve networking information in a time-slotted manner. These components then perform an optimization model in a time-slotted manner to efficiently serve clients' requests by selecting an optimal cache server (with the shortest fetch time). In case of a cache miss, a client's request is served (i) by an optimal replacement quality (only better quality levels with minimum deviation) from a cache server, or (ii) by the original requested quality level from the origin server. This approach is validated through experiments on a large-scale testbed, and the performance of our framework is compared to pure client-based strategies and the SABR system [12]. Although SABR and ES-HAS show (almost) identical performance in the number of quality switches, ES-HAS outperforms SABR in terms of playback bitrate and the number of stalls by at least 70% and 40%, respectively.

Since video accounts for the majority of today's internet traffic, the popularity of HTTP Adaptive Streaming (HAS) is increasing steadily. In HAS, each video is encoded at multiple bitrates and spatial resolutions (i.e., representations) to adapt to a heterogeneity of network conditions, device characteristics, and end-user preferences. Most of the streaming services utilize cloud-based encoding techniques which enable a fully parallel encoding process to speed up the encoding and consequently to reduce the overall time complexity. State-of-the-art approaches further improve the encoding process by utilizing encoder analysis information from already encoded representation(s) to improve the encoding time complexity of the remaining representations. In this paper, we investigate various multi-encoding algorithms (i.e., multi-rate and multi-resolution) and propose novel multi-encoding algorithms for large-scale HTTP Adaptive Streaming deployments. Experimental results demonstrate that the proposed multi-encoding algorithm optimized for the highest compression efficiency reduces the overall encoding time by 39% with a 1.5% bitrate increase compared to stand-alone encodings. Its optimized version for the highest time savings reduces the overall encoding time by 50% with a 2.6% bitrate increase compared to standalone encodings.

In per-title encoding, to optimize a bitrate ladder over spatial resolution, each video segment is downscaled to a set of spatial resolutions, and they are all encoded at a given set of bitrates. To find the highest quality resolution for each bitrate, the low-resolution encoded videos are upscaled to the original resolution, and a convex hull is formed based on the scaled qualities. Deep learning-based video super-resolution (VSR) approaches show a significant gain over traditional upscaling approaches, and they are becoming more and more efficient over time. This paper improves the per-title encoding over the upscaling methods by using deep neural network-based VSR algorithms. Utilizing a VSR algorithm by improving the quality of low-resolution encodings can improve the convex hull. As a result, it will lead to an improved bitrate ladder. To avoid bandwidth wastage at perceptually lossless bitrates, a maximum threshold for the quality is set, and encodings beyond it are eliminated from the bitrate ladder. Similarly, a minimum threshold is set to avoid low-quality video delivery. The encodings between the maximum and minimum thresholds are selected based on one Just Noticeable Difference. Our experimental results show that the proposed per-title encoding results in a 24% bitrate reduction and 53% storage reduction compared to the state-of-the-art method.

Due to the growing importance of optimizing the quality and efficiency of video streaming delivery, accurate assessment of user-perceived video quality becomes increasingly important. However, due to the wide range of viewing distances encountered in real-world viewing settings, the perceived video quality can vary significantly in everyday viewing situations. In this paper, we investigate and quantify the influence of viewing distance on perceived video quality. A subjective experiment was conducted with full HD sequences at three different fixed viewing distances, with each video sequence being encoded at three different quality levels. Our study results confirm that the viewing distance has a significant influence on the quality assessment. In particular, they show that an increased viewing distance generally leads to increased perceived video quality, especially at low media encoding quality levels. In this context, we also provide an estimation of potential bitrate savings that knowledge of actual viewing distance would enable in practice. Since current objective video quality metrics do not systematically take into account viewing distance, we also analyze and quantify the influence of viewing distance on the correlation between objective and subjective metrics. Our results confirm the need for distance-aware objective metrics when the accurate prediction of perceived video quality in real-world environments is required.

Adaptive HTTP streaming provides multiple representations of the same content at different bit-rates and resolutions and allows the client to request segments from the available representations in a dynamic, adaptive way depending on its context. The growing number of representations in adaptive HTTP streaming makes encoding of one video segment at different representations a challenging task in terms of encoding time-complexity. In this paper, information of both highest and lowest quality representations are used to limit Rate Distortion Optimization (RDO) process for each Coding Unit Tree (CTU) in High Efficiency Video Coding. Our proposed method first encodes the highest quality representation and consequently uses its information to encode the lowest quality representation. Thereafter, information from both the highest and the lowest quality representations are used to predict features of intermediate quality representations. In particular, the block structure and the selected reference frame of both highest and lowest quality representations are used to predict and shorten the RDO process of each CTU for intermediate quality representations. Our proposed method introduces a delay of two CTUs if parallel encoding is used. Experimental results show significant reduction in time-complexity over the reference software (38%) and the state-of-the-art (10%) while quality degradation is negligible.

Recently, there has been a growing interest in the various formats of plenoptic information, including digital holograms. However, the reduced number of studies on holographic data quality assessment results in a difficult analysis of the effects caused by any type of post-processing procedures, like lossy compression. In this work, several state of the art compression codecs, namely HEVC, AV1, and JPEG2000, are applied to the digital holographic data complex representation on the reconstruction plane. Both computer generated and experimental holograms are considered. The perceptual quality of the amplitude is evaluated to measure the quality degradation at four different bit rates. The performance of a selection of quality metrics is also tested.

HTTP Adaptive Streaming (HAS) is the most common approach for delivering video content over the Internet. The requirement to encode the same content at different quality levels (i.e., representations) in HAS is a challenging problem for content providers. Fast multirate encoding approaches try to accelerate this process by reusing information from previously encoded representations. In this paper, we propose to use convolutional neural networks (CNNs) to speed up the encoding of multiple representations with a specific focus on parallel encoding. In parallel encoding, the overall time-complexity is limited to the maximum time-complexity of one of the representations that are encoded in parallel. Therefore, instead of reducing the time-complexity for all representations, the highest time-complexities are reduced. Experimental results show that FaME-ML achieves significant time-complexity savings in parallel encoding scenarios (41% in average) with a slight increase in bitrate and quality degradation compared to the HEVC reference software.

This article investigates the relative performance of multiple data embedding into H.264 compressed video under two schemes of closed-loop and open-loop methods. In closed-loop, a part of an encoder is modified to embed data within the encoding loop during re-compression, while in the open-loop scenario, a part of decoder is modified to embed data out of decoding loop. It is shown, while for the first instance of embedding, both methods behave almost similarly for all picture types, for later instances of embedding, open-loop outperforms the closed-loop method. Moreover, the required time for embedding and extracting processes of open-loop method is only 1-1.5% of that of the closed-loop counterpart. In both methods, the quality of watermarked video and the data hiding capacity are controlled by the position of last non-zero (LNZ) coefficient in the H.264 zigzag scanning order. However, for B-pictures, the side-effect of embedding distortion is very limited (in the order of 0.002 in terms of SSIM), but for I- and P-pictures it can be significant. Picture degradations in I- and P-frames are alleviated by confining data embedding only into the last block of macroblocks of I-frames and the last P-frame in the GOP, respectively. Finally, it is shown that while with CAVLC type entropy coder, the number of increased bits due to data embedding can vary from 5%-65% of the metadata volume (depending on quality degradation) this value with CABAC coder is only less than 20% of that under CAVLC.

In highly-interactive video streaming applications such as video conferencing, tele-presence, or tele-operation, retransmission is typically not used, due to the tight deadline of the application. In such cases, the lost or erroneous data must be concealed. While various error concealment techniques exist, there is no defined rule to compare their perceived quality. In this paper, the performance of 16 existing image and video quality metrics (PSNR, SSIM, VQM, etc.) evaluating errorconcealed video quality is studied. The encoded video is subjected to packet loss and the loss is concealed using various error concealment techniques. We show that the subjective quality of the video cannot be necessarily predicted from the visual quality of the error-concealed frame alone. We then apply the metrics to the error-concealed images/videos and evaluate their success in predicting the scores reported by human subjects. The errorconcealed videos are judged by image quality metrics applied on the lossy frame, or by video quality metrics applied on the video clip containing that lossy frame; this way, the impact of error propagation is also considered by the objective metrics. The measurement and comparison of the results show that, mostly though not always, measuring the objective quality of the video is a better way to judge the error concealment performance. Moreover, our experiments show that when the objective quality metrics are used for the assessment of the performance of an error concealment technique, they do not behave as they would for general quality assessment. In fact, some newly developed metrics show the correct decision only about 60% of the time, leading to an unacceptable error rate of as much as 40%. Our analysis shows which specific quality metrics are relatively more suitable for error-concealed videos.

One challenge in video transmission is to deal with packet loss. Since the compressed video streams are sensitive to data loss, the error resiliency of the encoded video becomes important. When video data is lost and retransmission is not possible, the missed data should be concealed. But loss concealment causes distortion in the lossy frame which also propagates into the next frames even if their data are received correctly. One promising solution to mitigate this error propagation is intra coding. There are three approaches for intra coding: intra coding of a number of blocks selected randomly or regularly, intra coding of some specific blocks selected by an appropriate cost function, or intra coding of a whole frame. But Intra coding reduces the compression ratio; therefore, there exists a trade-off between bitrate and error resiliency achieved by intra coding. In this paper, we study and show the best strategy for getting the best rate-distortion performance. Considering the error propagation, an objective function is formulated, and with some approximations, this objective function is simplified and solved. The solution demonstrates that periodical I-frame coding is preferred over coding only a number of blocks as intra mode in P-frames. Through examination of various test sequences, it is shown that the best intra frame period depends on the coding bitrate as well as the packet loss rate. We then propose a scheme to estimate this period from curve fitting of the experimental results, and show that our proposed scheme outperforms other methods of intra coding especially for higher loss rates and coding bitrates.

Pseudo-sequence based light field compression methods are a highly efficient solution to compress light field images. They use state-of-the-art video encoders like HEVC to encode the image views. HEVC exploits Coding Tree Unit (CTU) structure which is flexible and highly efficient but it is computationally demanding. Each CTU is examined in various depths, prediction and transformation modes to find an optimal coding structure. Efficiently predicting depth of the coding units can reduce complexity significantly. In this paper, a new depth decision method is introduced which exploits the minimum and maximum of previously encoded co-located coding units in spatially closer reference images. Minimum and maximum depths of these co-located CTUs are computed for each coding unit and are used to limit the depth of the current coding unit. Experimental results show up to 55% and 85% encoding time reduction with serial and parallel processing respectively, at negligible degradations.

In light field compression, besides coding efficiency, providing random access to individual views is also a very significant factor. Highly efficient compression methods usually lack random access. Similarly, random access methods usually reduce the compression efficiency. To address this trade-off, a light field image encoding method is proposed in this paper which favors random access. In the proposed scheme 15×15 view images are divided into 25 independent 3×3 view images which are called Macro View Image (MVI). To encode MVIs, the central view image is used to compress its immediate neighboring view images using a hierarchical reference structure. To encode the central view of each MVI, the most central view image, along with the center of at most three MVIs, are used as the reference images for the disparity estimation. In addition, the proposed method enables the use of parallel computation to improve encoding/decoding time complexity. To reduce memory footprint in case a Region of Interest (ROI) is required, HEVC tile partitioning is used.

Light field imaging is a promising technology for 3D computational photography. As Light Field images are represented for multiple views, their subjective evaluation is a very demanding task. Hence, identifying reliable objective quality assessment methodologies plays a very important role. In this paper six objective quality metrics; PSNR-Y, PSNR-YUV, SSIM-Y, MSSSIM-Y, FSIM-Y and HDRVDP2-Y are assessed for five state-of-the-art codecs at various bit-rates. Moreover, the metrics are computed in the linear, perceptually uniform and perceptual quantizer spaces. The results are compared against those of a subjective study and is concluded that the average FSIM-Y is the most reliable metric. The paper also introduces maps of the objective metrics to evaluate the quality dispersion among the different light field image views.

Vehicular ad‐hoc networks (VANETs) despite their potential benefits, especially for intelligent transportation and safety systems face some important challenges. The main challenges are caused by high mobility of vehicles and dynamic environment. In addition, VANET channel is wireless and is extremely error prone. To overcome these problems, several routing strategies have been proposed in order to find paths with high reliability and low delay. One of these strategies is the opportunistic routing (OR) paradigm where its performance can be significantly affected by the method applied for prioritised transmission to the nodes within the relays set. The OR protocol proposed in this study is based on the Dempster–Shafer evidence theory. In this method, the source vehicle employs the packet advancement, vehicle density, and packet delivery probability as parameters for determining the appropriate vehicle as to the next hop. The number of vehicles participating in forwarding the packets is optimised using trust‐based calculations. In addition, the source vehicle can schedule a set of relay nodes based on their degree of trust. Highway environment simulations suggest that compared to other published methods in the literature, the proposed method can improve routing performance in terms of all quality‐of‐service metrics.

An objective image quality assessment tool is proposed to measure image enhancement quality with emphasis on contrast. In the proposed tool, which is based on maximizing contrast with minimum artefact (MCMA), local and global properties of an image are measured through pixel‐wise and histogram‐wise features, respectively. To this aim, three sub‐measures are introduced, each of which able to detect one contrast‐related quality aspect: (i) low dynamic range of image; (ii) histogram shape preservation during image enhancement process; and (iii) local pixel diversity. These sub‐measures are combined through a subjective test to adapt them to the mean opinion scores (MOSs) of a diverse set of training contrast‐enhanced images. A regression algorithm performs the adaptation by fitting the three sub‐measures to the MOS values and finding an optimal linear combination by maximizing the Pearson correlation. In order to evaluate the performance of the MCMA algorithm, another independent, subsequent, subjective test was performed on a set of images enhanced by various known contrast enhancement algorithms to obtain MOS values and to compare them with the output of the proposed MCMA method. The experimental results show that MCMA has the highest correlation to the MOS when compared to the existing tested contrast measurement tools.

In this study, a block‐based estimation of noise due to blurriness distortion is proposed based on auto‐regressive (AR) modelling. In the proposed method; a de‐correlated, low‐energy version of the blurred image is auto regressively modelled. To this end, AR parameters are estimated using the Yule–Walker equations. As these equations include auto‐correlation function (ACF) coefficients, ACF estimation is also required. The Yule–Walker equations are solved making use of Durbin–Levinson algorithm. Finally, noise energy is mathematically defined and computed for each block. Since blurriness is a signal‐dependent image distortion, estimating and describing its characteristics via a noise like that of the AR model input, is significant. In fact, extracting features of such ‘noise’ can lead to the design and development of a new method of image quality metrics. Inspired by the ‘stem cells’ concept in medical science that is convertible to other cell types, the AR model input is called ‘stem noise’. To visualise contribution of the ‘Stem Noise’ in the reconstruction of blurriness image distortion, a map called stem noise energy map is created. It is shown that the characteristics of the estimated noise energy are well correlated with the human subjective scores.

Motion vectors (MVs) are the most common temporal descriptors in video analysis, indexing and retrieval applications. However, video indexing and analysis based on MVs do not perform well for videos at different dimension ratios (DRs) or even various resolutions. As a result, video indexing and analysis which are based on identifying similar video face with many difficulties at different DRs or resolutions by MVs. In this study, a two‐stage algorithm is introduced to make MV descriptors robust against variations first in DR and then at resolution. In the experiments performed on motion vector histograms, the proposed method improves the performance on identifying similar videos at various spatial specifications by up to 73%. Moreover, in the video retrieval experiments, the proposed modified MV outperforms original MV feature vector. This is an indication of improvement in differentiation of similar and dissimilar videos by the proposed temporal feature vector.

Rate control in scalable video coding (SVC) is a very challenging problem because of the inter-layer prediction structure, which makes developing an efficient rate-control algorithm complex and difficult. Little prior work is available for joint temporal-quality (T-Q) scalability considering the rate-distortion (R-D) dependence among the temporal and quality layers. However, most of the rate-control algorithms in SVC suffer from high computational complexity, growing significantly with the number of layers. In this paper, a single-pass joint temporal-quality rate-control algorithm is presented for H.264/SVC. In this algorithm, by analyzing the R-D dependence of joint T-Q scalability, Cauchy distribution-based rate-quantization, and distortion-quantization models, a set of empirical values are first derived to estimate the initial values of the R-D model parameters for the joint temporal and quality layers. Then, a novel prediction mechanism to update these model parameters is proposed to allocate the bit budgets efficiently among the temporal and quality layers, and hence to improve the performance of the proposed algorithm. Experimental results show that the proposed algorithm achieves better coding efficiency with low computational complexity compared with two other benchmark rate-control algorithms.

The number of cloud gaming (CG) users is constantly growing. The idea in cloud gaming is to render the game events on a cloud server and stream the resulted scenes as a video sequence to players. CG requires a high bandwidth in order to run appropriately and create a good quality of experience for players. In order to reduce the high required bandwidth, video should be compressed without any negative impact on user's quality of experience (QoE). Thus CG providers, researchers who develop new compression methods for CG, and those who are improving network protocols for CG require to evaluate user experience using subjective methods. Over the years, many researches have investigated the subjective quality of video, but all of them have one of the following two main drawbacks, which makes them unsuitable for game videos. The subjective quality assessment methods which are designed for short duration video sequences suffer from Forgiveness and Recency effects. On the other hand, the methods which are designed for long duration video sequences usually use some sort of a handset device for rating scores, and hence cannot be used for most games where both hands are busy while playing. In this paper, a novel subjective test apparatus for assessment of game videos is proposed, where players give their opinion scores using a foot pedal while playing the game. Evaluation results indicate that the proposed scheme is more accurate and less distractive than existing methods.

Deployment of small cells is considered as one of the most promising solutions for increasing the capacity and coverage of the wireless networks. The networks consisting of different base station levels are called heterogeneous cellular networks (HCN). In such networks, cell range expansion is applied for offloading more users from the macro base station (MBS) to the femtocell access point (FAP). In this paper, we use the structure of HCNs for transmission of scalable video coding (SVC). For two-layer spatial scalable video, due to the umbrella coverage of a macrocell, MBS is suitable for transmission of the base layer content to the users within its coverage. Moreover, the enhancement layer is transmitted via the FAP because of its high data rate. Using the tools from stochastic geometry, we derive the rate distributions of the users receiving the base layer. Moreover, we quantify two performance metrics, which are the standard-definition (SD) outage probability and the high-definition (HD) probability. We prove that offloading more users by the cell range expansion and resource partitioning improve the HD probability.

In order to compensate inefficiency of traditional macrocell base stations (MBS), femtocell base stations (FBS) are deployed in cell areas. This deployment can enhance the Quality of Service (QoS) for the users which have difficulty communicating with the MBS. However, the presence of FBSs causes interference for MBS that should be controlled. Guaranteed QoS such as delay-bounds is required in real-time video applications. Moreover, the rapid growth of mobile-video traffic and time varying nature of wireless channels make it difficult to guarantee stringent delay constraints. However, providing delay-bounds based on effective capacity looks more appropriate for unreliable channels. This paper proposes a power allocation scheme based on Nash Bargaining Solution(NBS). NBS is a cooperative solution for Nash bargaining competitive game, which leads to a fair resource allocation and also satisfies the Pareto efficiency. We derive NBS in a tractable closed form formula by taking into account the delay-bounds and interference constraints. Analyzing the simulation results demonstrates that our proposed solution reaches a high level of fairness among users.

In this study, a method for enhancing low‐contrast images is proposed. This method, called Gaussian mixture model‐based contrast enhancement (GMMCE), brings into play the Gaussian mixture modelling of histograms to model the content of the images. On the basis of the fact that each homogeneous area in natural images has a Gaussian‐shaped histogram, it decomposes the narrow histogram of low‐contrast images into a set of scaled and shifted Gaussians. The individual histograms are then stretched by increasing their variance parameters, and are diffused on the entire histogram by scattering their mean parameters, to build a broad version of the histogram. The number of Gaussians as well as their parameters are optimised to set up a Gaussian mixture modelling with lowest approximation error and highest similarity to the original histogram. Compared with the existing histogram‐based methods, the experimental results show that the quality of GMMCE enhanced pictures are mostly consistent and outperform other benchmark methods. Additionally, the computational complexity analysis shows that GMMCE is a low‐complexity method.

Abstract This article presents an information hiding algorithm for H.264/AVC video stream. It utilizes position of the last nonzero level of quantized discrete cosine transform block to embed information bits. Because only the high‐frequency levels are changed, it can guarantee a high peak signal‐to‐noise ratio and slight increase in bit rate after the watermark embedding. The extraction process is not complex; thus, the proposed technique is an excellent solution for real‐time applications such as broadcasting. Experimental results on several test sequences demonstrate that the proposed approach can realize blind extraction with real‐time performance; it also provides very high capacity, low distortion and increase in bit rate by about 0.5%. Copyright © 2015 John Wiley & Sons, Ltd.

This paper presents two edge detectors and a preprocessing algorithm for histogram based data hiding schemes. The proposed technique takes advantage of the edge detectors to segment the image into plain and textured areas. The plain areas are selected for embedding where a histogram based scheme is used to embed information in these areas. Since in the plain areas, pixel intensities are close to each other, with the proposed preprocessing algorithm the number of shifted pixels for the same amount of hidden data capacity are decreased which results in a better transparency. To validate the efficiency of the technique two of the best known histogram based schemes which use prediction and interpolation are implemented and results are compared with and without the preprocessing scheme. The experimental results show that the histogram based methods with preprocessing algorithm, under the same capacity, have better transparency than the schemes without preprocessing. This scheme improves capacity, even by 200%, at equal distortion, or about 4 dB improvement in PSNR, at the same hiding capacity.

This study presents a satellite image contrast enhancement technique based on the discrete cosine transform (DCT) pyramid and singular value decomposition (SVD), in contrast to the methods based on wavelet decomposition and SVD which could fail to produce satisfactory results for some low‐contrast images. With the proposed method, an input image is decomposed into a low sub‐band image and reversed L‐shape blocks containing the high‐frequency coefficients of the DCT pyramid. The singular value matrix of the equalised low sub‐band image is then estimated from the combination between the singular matrix of the low sub‐band image and the singular matrix of its global histogram equalisation. The qualitative and quantitative performances of the proposed technique are compared with those of conventional image equalisation such as general histogram equalisation and local histogram equalisation, as well as some state‐of‐the‐art techniques such as singular value equalisation technique. Moreover, the proposed technique is contrasted against the technique based on the discrete wavelet transform (DWT) and SVD (DWT–SVD) as well as the technique based on DCT–SVD. The experimental results show that the proposed method outperforms both conventional and the state‐of‐the‐art techniques.

This study investigates the use of structural similarity index (SSIM) on the minimised side effect to image watermarking. For the fast implementation and more compatibility with the standard discrete cosine transform (DCT)‐based codecs, watermark insertion is carried out on the DCT coefficients and hence an SSIM model for DCT‐based watermarking is developed. For faster implementation, the SSIM index is maximised over independent 4 × 4 non‐overlapped blocks, but the disparity between the adjacent blocks reduces the overall image quality. This problem is resolved through optimisation of overlapped blocks, but, the higher image quality is achieved at a cost of high computational complexity. To reduce the computational complexity while preserving the good quality, optimisation of semi‐overlapped blocks is introduced. The authors show that while SSIM‐based optimisation over overlapped blocks has as high as 64 times the complexity of the 4 × 4 non‐overlapped method, with semi‐overlapped optimisation the high quality of overlapped method is preserved only at a cost of less than 8 times the non‐overlapped method.

The distribution of copyrighted scalable video content to differing digital devices requires protection during rendering and transmission. In this paper, we propose a complete security system for H.264/scalable video coding (SVC) video codec and present a solution for the bit-rate and format compliance problems by careful selection of entropy coder syntax elements (bin-strings) for selective encryption (SE), and the problem of managing multiple layer encryption keys for scalable video distribution. A standard key management protocol, multimedia Internet keying protocol, is implemented for the hierarchical key generation mechanism, in which a subscriber has only one encryption key to unlock all scalable layers that have been subscribed to. The evaluation demonstrates the resulting video quality degradation arising from SE for manyCIFand 4CIFtest video sequences, without there being any impact upon the bit-rate or format compliancy, and with small computational delay. The security and statistical analysis performed further verify the effectiveness of the proposed security system for H.264/SVC. The proposed system is highly suitable for video distribution to users who have subscribed to a varying degree of video quality on devices with medium to high computational resources.

Prior research on wireless multicast video streaming has addressed the risk of feedback implosion when providing adaptive forward error correction (FEC). This approach has not been adopted by providers, who may either stick to unicast streaming or use a sufficient level of application layer FEC to avoid the need for adaptation. Instead in this paper, an adaptive unicast rateless channel coding scheme is also run in multicast mode. This is accomplished simply by turning off requests for FEC repair data. The paper demonstrates the concept and provides an indication of the operating conditions for such a joint multicast/unicast service in terms of data rates and suitable video content type.

The distribution of copyrighted scalable video content to differing digital devices should be protected during rendering and transmission. The proposed scheme is applied to H.264 Scalable Video Coding (SVC) CABAC bin-strings in a compression-friendly and decoder format compliant manner. It achieves this by careful selection of the entropy coder syntax elements for selective encryption (SE) with respect to SVC. Tests show that: decoding delay is small, replacement and key substitution attacks are fruitless; there is no increase in bitrate; and the stream remains format compliant. The proposed SE scheme is extremely suitable for video distribution to users who have subscribed to differing video qualities on medium- to high-computationally capable digital devices.

Multi-layered scalable bit-stream distribution requires protection for the rendering and transmission of its individual layers. This paper presents a sufficient encryption (SE) scheme for SVC layers which maintains the compression efficiency and the format compliancy of scalable bit-streams, without compromising the security. The purpose of SE is achieved by applying the partial encryption on carefully selected codewords and bin-strings of the CAVLC and CABAC of H.264/SVC respectively. The performance of the scheme is tested on various resolution sequences, which demonstrate the advantages of the scheme when compared to alternative techniques. These advantages include: minimal computational delay by encrypting partial data; no bit-rate escalation by keeping the compression ratio unchanged; and, format compliancy of the bit-stream at the decoder. A comparative security analysis of the scheme confirms that it is suitable for commercial, real-time applications. The minimal increase in processing requirements ensures that the scheme is highly suitable for video distribution to users who have subscribed to differing video qualities on end systems ranging from small handheld devices to those with high computational capability.

As intelligent content management of IPTV moves popular material nearer to the end-user, application-layer channel coding schemes, involving retransmission of extra redundant data, become attractive. Application-layer, adaptive rateless channel coding is exploited in this paper's scheme to reconstruct streamed video across an IEEE 802.16e (mobile WiMAX) channel. The paper concentrates on the trade-offs in implementing the scheme, showing that exact calculation of the redundant data has the potential to reduce the forward error correction bit-rate overhead. To reduce delay an appropriate compression rate should also be selected.

In transmitting video over low bandwidth networks a cyclic intra-refresh (IR) line is preferable to periodic intra frames as an error mitigation technique. This paper shows that an IR line introduces non-uniform error sensitivity between regions within a picture that are identifiable by the position of the IR line. An error sensitivity aware slicing scheme is proposed to enable the use of unequal error protection techniques in order to offer better protection to the more important packets. Experimental results justify the validity of the proposed priority based protection scheme.

When transmitting video over low bandwidth networks a cyclic intra-refresh (IR) line is preferable to periodic intra frames as an error mitigation technique. This paper shows that IR lines introduce non-uniform error sensitivity among regions within a picture that are identifiable by the position of the IR line. To exploit this characteristic, a packet prioritization scheme is proposed in which priorities are assigned on a packet-level basis. Thus, higher priority packets can be given better protection. The proposed scheme is demonstrated for prioritized access to a congested IEEE 802.11e network. Experimental results indicate that, when compared with the classic non-prioritized packet access scheme, the proposed scheme can achieve quality gains of up to 3.6 dB for a 10% data loss rate.

Abstract — Developing applications, especially real-time ones, for wireless vehicular ad hoc networks (VANETs) requires a reasonable assurance of the likely performance of the network, at the least in terms of packet loss ratios and end-to-end delay. Because wireless propagation strongly influences performance, especially in an urban environment, this paper improves on simpler propagation models for simulations by augmenting ray-tracing derived models of propagation. In the non-line-of-sight component: the propagation distance is more closely calculated according to the reflection distance; the effect of roadside obstacles is included; and for modeling of fast fading a phase factor is introduced, all without necessarily overly increasing computational load. In the line-of-sight component, as well as roadside obstacle modeling: single and double reflections from roadside buildings are added to the standard two-ray ground-propagation model; the distribution of vehicles within a street segment is used to model the ground reflection ray more closely; and the reflection coefficient is also adjusted accordingly to account for reflections from vehicles. The results have been compared with widely-used measurement studies of city streets in the literature, which have confirmed the overall advantage of the improvements, especially in the case of the non-line-of-sight component. A simulation case study shows that in general optimistic performance predictions of packet loss occur with the two-ray ground propagation model when indiscriminately applied. The paper, therefore, represents a way forward for VANET wireless channel modeling in simulations. Index Terms—simulation, urban environment, vehicle-to-vehicle communication, wireless propagation G I.

Mapping an H.264/SVC (Scalable Video Coding) bitstream to the IEEE 802.11e video service class may result in the decoder discarding an excessive number of packets. This is because of dependencies between the discarded packets and those dropped in the service queue. Instead, the paper proposes mapping SVC quality layers across multiple service classes, thereby reducing the impact of traffic congestion upon video quality. Video quality in the sensor network scenario tested was almost stabilized, despite up to 10% packet drops through traffic congestion. An added advantage of the mapping is that the end-to-end delay of the base layer packets was considerably reduced compared to the standard IEEE 802.11e mapping.

In the modern era of Internet, many user-end applications require the estimation of quality of images directly from the bitstreams, as the original image may not be available. This is a challenging issue. In this paper, we propose a novel approach of no-reference (NR) objective quality assessment of wavelet coded images. The proposed method is based on estimation of the contribution of each received bit in an embedded bitstream in the reduction of mean-square error (MSE), assuming initial image with all zero pixel values. Mathematical analysis is carried out to estimate the quantization errors in terms of MSE reduction, in wavelet-based SPIHT coding algorithm. The simulation results demonstrate the effectiveness of the proposed method.

Data partitioning is a way of separating out data from a compressed bitstream according to its importance in reconstructing a video stream. This paper notices that this procedure also results in relatively smaller packets for more important data if the quantization parameter (QP) is set accordingly. Raptor channel coding is then applied and the quality of the video is improved by distributing a significant proportion of intra-coded macroblocks within predictively coded frames. When this scheme is applied to an IEEE 802.16e (mobile WiMAX) system, according to frame size a number of trade-offs arise in respect to balancing the number of packet drops, corrupted packets through channel conditions, overall video quality, and data latency.

This paper proposes a method to efficiently find motion vector predictions for zonal search motion re-estimation in fast video transcoders. The motion information extracted from the incoming video stream is processed to generate accurate motion vector predictions for transcoding with reduced complexity. Our results demonstrate that motion vector predictions computed by the proposed method outperform those generated by the highly efficient EPZS (Enhanced Predictive Zonal Search) algorithm in H.264/AVC transcoders. The computational complexity is reduced up to 59.6% at negligible cost in R-D performance. The proposed method can be useful in multimedia systems and applications using any type of transcoder, such as transrating and/or spatial resolution downsizing.

Broadband wireless delivery of IPTV is under active investigation, though this environment is challenging. This paper examines the threat from slow and fast fading, traffic congestion, and channel packet drops. The proposed response is a combination of: rateless channel coding, which is adaptively applied; data-partitioned source coding to exploit prioritized packetization; and redundant slice provision, which is the focus of the evaluation in this paper. When fast and slow fading occur, this paper shows that redundant slices are certainly necessary but this provision is more effective for medium-quality video than it is for high-quality video.

This paper demonstrates a robust layered video scheme, based on data-partitioning and intended for IPTV streaming over wireless broadband. Equal error protection through rateless coding is applied, whereby higher-priority data partitioned packets are protected by appropriate selection of quantization parameter and picture slicing, so as to regulate packet size. In the mobile WiMAX channel investigated, both packet drops from congestion and adverse channel conditions are shown to be affected by packet size. The main proposal is for adaptive rateless coding in which additional redundant data are retransmitted to heal corrupted packets. However, though these packets are always repairable, delay increases as the percentage of corrupted packets increases, affecting the design parameters. Intra-refresh macro blocks are added to prevent the objective video quality falling below an acceptable level. Picture slicing further reduces the packet size to increase robustness in the face of measurement noise during channel estimation and the effects of slow and fast fading.

The conventional network layers are mainly designed for fixed networks and are not suitable to cope the challenges offered by the modern wireless networks. Moreover, certain functionalities can be implemented simultaneously in more than one layer. For example, unequal error protection (UEP) can be performed at the application layer, using forward error correction (FEC), but it is bandwidth inefficient. On the other hand, UEP can also be performed at the physical layer using hierarchical modulation to protect higher priority bits more strongly without any additional bandwidth, but at the cost of increased errors in the lower priority bits. The authors propose a cross-layer UEP approach to enhance reliability of scalable video by combining adaptive FEC at the application layer and hierarchical quadrature amplitude modulation (HQAM) at the physical layer. The idea is to slightly reduce the FEC protection of higher priority bits and increase the FEC protection of low-priority bits in such a fashion that the overall data rate at the application layer remains unchanged. The higher priority bits are additionally protected using HQAM at the physical layer. Simulation results show significant improvements in performance when the proposed scheme is used.

This study seeks to establish under what conditions (mobility, network size, wireless channel) multi-source video streaming is feasible across a wireless vehicular ad hoc network (VANET). Overlay networks with multiple sources have proven to be robust, distributed solutions to multimedia transport, including streaming. To achieve video streaming over a VANET overlay, this study introduces a spatial partition of a video stream based on flexible macroblock ordering. Tests show this can achieve a gain of over 5 dB in video quality (PSNR) depending on video content and packet loss rates. However, routing of streamed services over multiple hops and multiple paths may lead to significant packet losses, resulting in unacceptable quality of service. The paper examines the impact of differing traffic densities and road layouts upon an overlay network's performance. The work modelled the emerging IEEE 802.11p for wireless VANETs. The research demonstrates that the vehicles' mobility pattern and their drivers' behaviour need to be carefully modelled to determine signal reception. The study also considers the impact of the wireless channel, which also should be more realistically modelled.

This paper describes a method of approximating a shape contour with a polygon. The polygon vertices are extracted from the curvature extremes, through a scale-space description of the contour, via linear diffusion. These vertices are located on the contour points where the sharper changes of the contour directions occur. Using a proper strategy, a set of extremes that result in a given approximation level is chosen. By adding new vertices, the approximation level can be improved, and a scalable representation of the contour is identified. This method results in an approximation that discriminates local from global geometric features and provides a good visual representation of the original contour. This polygonal approximation method is used for scalable encoding of the shape contours. In this regard, an encoding technique suitable for scalable polygonal approximation has been developed. We show that encoding the approximated polygons result in a good relation between the distortion and the bitrate. Finally, we show that in addition to coding this method can be efficiently used for shape comparison and shape retrieval.

The increasing use of multiparty Web conferencing applications demands for suitable multicast protocols. Limited bandwidth of typical Internet user requires the underlying multicast routing to be efficient. In this paper we present HOMA; an application layer multicast protocol which has been tailored for small scale multiparty videoconferencing applications. Considering the requirements of such applications, HOMA uses a heuristic routing to construct efficient multicast trees on the application layer. We have evaluated the performance of the proposed routing algorithm through simulations. Experimental results indicate that the proposed algorithm performs better than the related algorithms in terms of rejection rate, while satisfying QoS constraints of conferencing applications such as end-to-end delay and bandwidth.

In this paper, a motion compensated temporal filtering (MCTF) framework based on the discrete cosine transform (DCT) is proposed and named MCDCT-TF. It is able to employ filters of any length with particular emphasis on 5/3 DCT and 7/4 DCT. The proposed MCDCT-TF and the two-dimensional (2-D) DCT decimation technique are incorporated into H.264/AVC to provide spatio-temporal scalability. Compared with the current MCTF schemes such as Haar, and 5/3 wavelet filters, simulation results show that the proposed MCDCT-TF utilizing longer tap DCT filters achieves a significant improvement in coding gain. Moreover, the performance gap between the presented scalable video coding and the single layer H.264/AVC is small. However, the presented spatio-temporal coding scheme outperforms the spatio-temporal supported in JSVM.

In this paper we introduce a highly efficient reversible data hiding technique. It is based on dividing the image into tiles and shifting the histograms of each image tile between its minimum and maximum frequency. Data are then inserted at the pixel level with the largest frequency to maximize data hiding capacity. It exploits the special properties of medical images, where the histogram of their non-overlapping image tiles mostly peak around some gray values and the rest of the spectrum is mainly empty. The zeros (or minima) and peaks (maxima) of the histograms of the image tiles are then relocated to embed the data. The grey values of some pixels are therefore modified. High capacity, high fidelity, reversibility and multiple data insertions are the key requirements of data hiding in medical images. We show how histograms of image tiles of medical images can be exploited to achieve these requirements. Compared with data hiding method in the whole image, our scheme can result in 30%-200% capacity improvement with still better image quality, depending on the medical image content.

This paper describes a no-reference temporal qualitymetric to model the impact of frame freezing impairments on perceived video quality. The proposed metric shows a high correlation with subjective data and is equally applicable to any type of objective video quality assessment model: full-reference, reduced-reference or no-reference. This temporal metric is a parameter of the video quality assessment algorithmin ITU-T Recommendation J.247 Annex C for the objective perceptual quality measurement of multimedia video.

This paper demonstrates that growth codes, based on Raptor channel coding, allow incremental protection of H.264 video codec data-partitioned network adaption layer units. When combined with increased protection of video reference frames, in an ADSL erasure channel up to 10 dB in video quality (PSNR) can be gained through this scheme compared to equal error protection with rateless codes. Equivalent gains occur in a wireless channel from combining data-partitioning with error protection. The bitrate overhead from data-partitioning is also shown to be less than from other H.264 error resilient tools.

P2P TV is gradually emerging as a potential alternative to well known client-server applications such as IPTV, VoD and other real-time TV services. Several P2P platforms such as Zattoo, Joost, Sopcast, and PPlive deliver streams using the user terminals as information relays, an approach considered to be more scalable, resilient and economical than the conventional approaches used by cable and network operators. In this paper we offer a different perspective on P2P TV, unveiling the issues that it causes to the network. Through an experimental-based assessment of Zattoo, our study unveils strengths (e.g. good resilience to end-to-end delay and jitter) and shortcomings (e.g. poor load balancing at network level) and yields recommendations for future P2P IPTV systems.

IPTV video services are under development for broadband networks, with final hop delivery across a wireless link. Within these networks, dedicated subchannels transport different types of traffic (video, voice, data...). This paper proposes fuzzy logic control (FLC) to allocate video streams within an IPTV subchannel subject to fluctuating available bandwidth. The method employs spatial and temporal complexity metrics to judge the appropriate share of available bandwidth, taking account of genre and dynamic coding rate fluctuations. So that the video sources can adjust their rates to the likely available bandwidth across the wireless link, reliable prediction of available bandwidth over time is also proposed. In tests, the FLC, which serves to merge the spatial and temporal metrics into a single control output, results in an overall gain in video quality over static equal allocation of bandwidth and also compensates streams that would otherwise suffer from an inadequate or alternatively an unnecessarily large bandwidth share.

This paper introduces P2P communication across a VANET and seeks to establish under what conditions (mobility, network size, wireless channel) data streaming is feasible. Routing of streamed services over multiple hops and multiple paths may result in significant packet losses, resulting in unacceptable quality of service. This paper examines the impact of differing traffic densities and different road layouts upon a P2P overlay network. The work modeled the emerging IEEE 802.11p for vehicular networks. It is shown that the mobility pattern of the vehicles involved needs to be closely modeled to determine signal reception patterns, as does the wireless channel environment, to avoid over optimistic assessment of communication within a VANET.

Peer-to-Peer streaming has been increasingly deployed recently. This comes out from its ability to convey the stream over the IP network to a large number of end-users (or peers). However, due to the heterogeneous nature among the peers, some of them will not be capable to relay or upload the original stream because of bandwidth limitations. Different internet connections these days can be initiated from different devices such as 3G mobile phones or WiFi-connected PDAs. Most of the existing P2P streaming systems are based on video coding techniques which cannot cope with this level of heterogeneity at network and terminal level. Layered video coding techniques are being introduced in simple streaming scenarios, due to their ability to deliver streams at different scales (temporal, spatial and SNR). This eases transmission in case of limited bandwidth as the devices can pick and decode the minimum bit rate base layer. Layered coding is preferred over single-layer coding for its flexibility to be transmitted over heterogeneous networks. In this paper we take a step further and analyze layered video in the context of P2P. We study such an approach in combination with simple cross-layer optimization techniques, comparing the resulting performance with a state-of-the-art P2P TV platform. We identify considerable benefits in terms latency, jitter, throughput, and packet loss.

Because of restricted mobility patterns, multi-hop routing, and wireless channel conditions, streaming video over a vehicular ad hoc network (VANET) is a daunting task. This paper demonstrates that H.264 codec flexible macroblock ordering (FMO) with receiver error concealment is capable of streaming good-quality video across a VANET. FMO is shown to be superior to other error resilience techniques but attention should be given to packet size, choice of protocol and the rate the video is released into the network if there are multiple destinations. Settings close to those of the emerging IEEE 802.11p standard for VANETs were emulated for a Manhattan grid mobility model.

For wireless video, the present paper demonstrates that combining adaptive source and channel coding can provide a robust service over a wide range of transmission channel conditions. Various feedback-controlled adaptive coding solutions are considered for orthogonal frequency-division multiplexing-based systems. At the receiver, the reliabilities of subchannels (SCHs) are estimated and decisions are fed back to the transmitter to adapt modulation modes and channel code-rates to provide reliable transmission and yet avoid over-protection. The resulting available transmission capacity (which can vary widely with channel conditions) can inform an online collaborating source coder which itself exploits the available feedback to reduce error propagation. To ensure further improvement for fading channels, an efficient low-delay retransmission arrangement is also considered in which only the contents of unreliable SCHs within an erroneous packet are retransmitted. Simulation results demonstrate that such collaborative adaptive source and channel coding can provide efficient use of a continuously varying wireless channel.

The anticipated growth of IPTV makes selection of suitable congestion controllers for video-stream traffic of vital concern. Measurements of packet dispersion at the receiver provide a graded way of estimating congestion, which is particularly suited to video as it does not rely on packet loss. A closed-loop congestion controller, which dynamically adapts the bitstream output of a transcoder or video encoder to a rate less likely to lead to packet loss, is presented. The video congestion controller is based on fuzzy logic with packet dispersion and its rate of change forming the inputs. Compared with TCP emulators such as TCP-friendly rate control (TFRC) and rate adaptation protocol (RAP), which rely on packet loss for real-time congestion control, the fuzzy-logic trained system's sending rate is significantly smoother when multiple video-bearing sources share a tight link. Using a packet dispersion method similarly results in a fairer allocation of bandwidth than TFRC and RAP. These gains for video traffic are primarily because of better estimation of network congestion through packet dispersion but also result from accurate interpretation by the fuzzy-logic controller.

Intelligent congestion control is vital for encoded video streaming of a clip or film, as network traffic volatility and the associated uncertainties require constant adjustment of the bit rate. Existing solutions, including the standard transmission control protocol (TCP) friendly rate control equation-based congestion controller, are prone to fluctuations in their sending rate and may respond only when packet loss has already occurred. This is a major problem, because both fluctuations and packet loss affect the end-user's perception of the delivered video. A type-1 (T1) fuzzy logic congestion controller (FLC) can operate at video display rates and can reduce packet loss and rate fluctuations, despite uncertainties in measurements of delay arising from congestion and network traffic volatility. However, a T1 FLC employing precise T1 fuzzy sets cannot fully cope with the uncertainties associated with such dynamic network environments. A type-2 FLC using type-2 fuzzy sets can handle such uncertainties to produce improved performance. This paper proposes an interval type-2 FLC that achieves a superior delivered video quality compared with existing traditional controllers and a T1 FLC. To show the response in different network scenarios, tests demonstrate the response both in the presence of typical Internet cross-traffic as well as when other video streams occupy a bottleneck on an All-Internet protocol (IP) network. As All-IP networks are intended for multimedia traffic, it is important to develop a form of congestion control that can transfer to them from the mixed traffic environment of the Internet. It was found that the proposed type-2 FLC, although it is specifically designed for Internet conditions, can also successfully react to the network conditions of an All-IP network. When the control inputs were subject to noise, the type-2 FLC resulted in an order of magnitude performance improvement in comparison with the T1 FLC. The type-2 FLC also showed reduced packet loss when compared with the other controllers, again resulting in superior delivered video quality. When judged by established criteria, such as TCP-friendliness and delayed feedback, fuzzy logic congestion control offers a flexible solution to network bottlenecks. These findings offer the type-2 FLC as a way forward for congestion control of video streaming across packet-switched IP networks.

Motion-compensated temporal filtering (MCTF) based lifting implementations of various discrete wavelet transforms have recently gained a lot of interest due to their good performance in energy compaction and their ability to provide various scalability features. Although all the existing MCTF schemes are based on the wavelet transform, in this paper, we propose a temporal filter framework based on the discrete cosine transform (DCT) which is an extension of our motion compensated DCT temporal filter (MCDCT-TF). In the current work, in addition to the two-band and three-band temporal decomposition structures employed in the MCDCT-TF technique, a longer tap filter (5/3 DCT) is utilized to improve the compression gain further. Simulation results show that a three-dimensional hybrid 3D subband/DCT codec with longer tap DCT filters yields a significant improvement over our earlier 3/2 MCDCT-TF, Haar, and 5/3 wavelet filters.

Transmission of video over error prone and still bandwidth limited wireless channels demand high compression efficiency and resilience to packet losses and errors. Scalable or layered video coding applied to highly compression efficient codecs is an ideal solution to the problem. However, scalability reduces compression efficiency of the coders. In this paper we show how compression efficiency of two-layer SNR scalable video coders can be retained via joint base-enhancement layer optimization. Simulation results show that joint base-enhancement layer optimization significantly outperforms separate optimization of the layers, and it closely follows the compression performance of the single-layer optimized codec.

In this paper, we examine the presence of forward and backward temporal masking in video when the stimulus is a burst of frozen frames and the mask is a scene change. We assess the impact of frame freezing impairment on the perceived video quality using a variety of source content and freezing events of different durations placed at different locations in the video. Our experimental results indicate the existence of an asymmetrical temporal masking effect with a more significant backward masking than forward masking. However the overall impact of temporal masking on visual quality was measured to be relatively weak. Implications for video coding are also discussed.

A measurement study is conducted of video streaming across a testbed with routers typical of those found at bottlenecks on the wired Internet. During ‘bursty’ traffic packet loss is not always fairly distributed between background flows and a video stream. The paper shows that packet loss indications may be unreliable whereas packet-by-packet delay, an alternative metric, has the ability to closely track queue length, responding to available bandwidth in a timely manner.

This paper introduces block-based rateless coding for video streaming over a wireless interconnect, in which after a packet erasure additional coded blocks are generated and piggy-backed onto outgoing packets. The advantage is shown in comparison to default Bluetooth FEC schemes, as, through the block-based rateless scheme, transmission energy consumption is reduced by a factor of up to 1.8, depending on Rayleigh channel bad state durations. In poorer channel conditions, the rateless scheme improves delivered video quality by as much as 10 dB relative to a Bluetooth v. 2.1 EDR mode. Decode complexity for rateless Raptor codes is linear in block size.

This paper introduces a block-based form of rateless channel coding that minimizes energy consumption by reducing the overhead and latency from channel coding. Consequently, results from a Bluetooth (IEEE 802.15.1) wearable wireless network for augmented reality (AR) show a consistent reduction in energy consumption compared to Bluetooth FEC schemes. AR relies on video transmission across a wearable network. Block-based rateless coding resulted in around 5 dB improvement compared to block-based FEC in video quality at a head-mounted display in the face of worsening channel conditions. System modeling took into account cross-traffic resulting from biosensors that moderate the AR display according to the cognitive load of the wearer.

This paper presents a reduced-reference objective picture quality measurement tool of compressed video. We have used a discriminative analysis of harmonic strength computed from edge-detected pictures to create harmonics gain and loss information that could be associated with the picture. The harmonics gain/loss are derived through the harmonic analysis of the compressed and source pictures to be incorporated in the reduced-reference video quality meter. This information corresponds with the two most prominent compression distortions, namely blockiness and blurriness. We have also studied the impact of motion in a video sequence on these compression distortions and the way they should be weighted and combined to give the best objective quality metric model. The model has been calibrated using several video sequences with dominant blockiness and blurriness. Validation of the model is performed by applying the model to the 50 Hz video sequences of VQEG Test Phase-I. Our results show that the proposed model achieves good correlations with the subjective evaluations of the VQEG datasets and its performance is comparable to those of the full-reference models in the literature.

Congestion control of a variable bit-rate video stream crossing the Internet is crucial to ensuring the quality of the received video. When a fuzzy-logic congestion controller (FLC) changes the sending rate of a video transcoder, it does so without feedback of packet loss, using packet dispersion instead. Compared with the well-known TFRC and RAP controllers, the FLC's sending rate is significantly smoother, allowing it to more closely take up available bandwidth at a bottleneck link. There is an accompanying order of magnitude reduction in packet losses. Due to better utilization of the available bandwidth, video quality is improved over time by several decibels in low-packet-loss conditions. The strength of the FLC solution is demonstrated by the resulting video quality when typical Web traffic forms the background traffic. The FLC avoids any risk of congestion collapse through fairness to coexisting TCP flows and is robust to changes in path delay and router buffer configuration.

In this paper, we propose a new algorithm for occlusion handling for object tracking in the crowded video scenes. The algorithm exploits the properties of undecimated wavelet packet transform (UWPT) coefficients and texture analysis to track arbitrary objects. The algorithm is initialized by the user through specifying a region around the object of interest at the reference frame. Then, coefficients of the UWPT of the region construct a Feature Vector (FV) for every pixel in that region. Optimal search for the best match is then performed by using the generated FVs inside an adaptive search window. Adaptation of the search window is achieved by inter- frame texture analysis to find the direction and speed of the object motion. This temporal texture analysis also assists in tracking of the object under partial or short-term full occlusion. Experimental results show a good performance for occlusion handling for object tracking in crowded scenes, in particular crowds on stairs in airports or train stations.

This paper presents a lossless data hiding method using coefficients of integer wavelet domain. The modification of selected small coefficients of the high frequency subbands are used to embed data. We use the histogram modification to intelligently select the proper coefficients for data hiding. Data embedding is done by processing these selected coefficients. We show that at low payload data hiding our method has comparable PSNR than the best known reversible data hiding techniques, while at higher payloads it has significant superiority on image quality.

Bluetooth's default ARQ scheme is not well suited to real-time video streaming, resulting in missed display and decode deadlines. Therefore, adaptive ARQ combined with a deadline aware buffer is an alternative approach. However, even with the addition of adaptation to picture type content importance, ARQ is not ideal in conditions of deteriorating RF channel. The paper presents fuzzy logic control of ARQ, based on send buffer fullness and the head of line packet's deadline. Tests show an improvement of over 4 dB improvement in video quality compared to a non-fuzzy scheme. The scheme can be applied to any codec with I-, P- and (possibly) B-slices by inspection of packet headers without encoder intervention.

In this work, we demonstrate a systematic way to analyze a binary partition tree representation of natural images for the purposes of archiving and segmentation. Within the tree structure, these problems are transformed into locating prevalent tree branches. With a user interface these points can be found manually by browsing branches. However, tree visualization is difficult due to the high node density. A simpler version of the tree is desired which facilitates subsequent retrieval whilst maintaining as much semantic detail as possible. By studying the evolution of region statistics, our method highlights nodes which represent the boundary between salient detail and provide a set of tree levels from which simplifications and segmentations can be derived. A series of subjective tests are performed to demonstrate the effectiveness of using the simplified trees for object extraction. Segmentation results are compared to ground truths showing semantic content is maintained

Pyramid image representations via tree structures are recognized methods for region-based image analysis. Binary partition trees can be applied which document the merging process with small details found at the bottom levels and larger ones close to the root. Hindsight of the merging process is stored within the tree structure and provides the change histories of an image property from the leaf to the root node. In this work, the change histories are modelled by evolvement functions and their second order statistics are analyzed by using a knee function. Knee values show the reluctancy of each merge. We have systematically formulated these findings to provide a novel framework for binary partition tree analysis, where tree simplification is demonstrated. Based on an evolvement function, for each upward path in a tree, the tree node associated with the first reluctant merge is considered as a pruning candidate. The result is a simplified version providing a reduced solution space and still complying with the definition of a binary tree. The experiments show that image details are preserved whilst the number of nodes is dramatically reduced. An image filtering tool also results which preserves object boundaries and has applications for segmentation.

This paper presents an efficient unequal error protection (UEP) approach for data partitioned video over wireless channels. We consider two methods: 1) forward error correction (FEC) with variable code-rates, and 2) hierarchical quadrature amplitude modulation (HQAM) combined with FEC with a fixed code-rate. In a low-delay application, the variable bit rate ratio between the high-priority and low-priority layers imposes certain constraints on these two methods. These constraints are discussed and solutions are proposed. Simulation results with an H.264 video codec show that combined HQAM and FEC outperforms FEC alone in both Gaussian and fading channels while having advantage over the nonlayered transmission

Tracking objects in motion is often done by imposing the constraints of kinematics and local image properties onto the objects. In this work, we propose a novel tracking algorithm which uses the surrounding information of the object to construct the feature profiles. The object feature profiles are then compared across consecutive frames to locate the targets. The feature profiles possess two important properties, distinctive-ness and coherence, which make them robust to measurement noises, short occlusions and false targets. The matching cost function is formulated under a Bayesian framework that enables the algorithm to capture the properties in the form of probabilities. The algorithm is also self-initializing. The computation of the feature profiles is fast due to their simple definition; and the comparison between two profiles can also be done efficiently.

In this paper, we propose a simple and efficient technique for embedded color image coding. The proposed algorithm exploits inter- and intra-subband correlations of the wavelet transformed luminance (Y) and chrominance (U and V) planes as well as the interdependency among the coefficients of the three-color planes. The coefficients of the three-color planes are linked through the composite spatial orientation trees (CSOT) having root nodes in the Y-plane only. The CSOT is defined for blocks of coefficients rather than a single coefficient. Thus, the proposed algorithm combines the features of both zero-tree and zero-block based algorithm into a single algorithm due to the use of block-tree hierarchical structure. Simulation results show the improved performance of the proposed algorithm over other sate-of-the-art coders such as MPEG-4, color SPIHT (CSPIHT), color SPECK (CPECK), and JPEG-2000.

This paper presents a shape prediction algorithm in a noisy video sequence based on pixel representation in the undecimated wavelet domain. In our algorithm for tracking of user-defined shapes in a noisy sequence of images, the amplitude of coefficients in the best basis tree expansion of the undecimated wavelet packet transform are used as feature vectors (FVs). FVs robustness against noise has been achieved through inherent denoising and edge component separation in the best basis selection algorithm. The algorithm uses these FVs to track the pixels of small square blocks located at the vicinity of the object boundary. Searching for the best-matched block has been performed using conventional block matching algorithm in the wavelet domain. Our experimental results show that the algorithm is robust to noise in case of object's shape translation, rotation and/or scaling and can be used to track both rigid and non-rigid shapes in image sequences

The existing standard video coding schemes support spatial scalability because of its prospective applications. Unfortunately, spatial scalable codecs produce high bit rate overhead as compared to a single layer coder. In this paper, we propose a spatio-temporal scalable video coding system based on motion compensated (MC) three-dimensional subband/discrete cosine transform (3-D SBC/DCT). This coder is proposed as a solution to improve the compression performance of spatial scalability and at the same time to reduce the high bit rate consumption of the base layer. In this system, the temporal subbands are generated by a novel approach called MC DCT temporal filtering (MCDCT-TF). The temporal subbands are further spatially decomposed using two-dimensional (2-D) DCT decimation technique . The generated spatio-temporal subbands are then quantized, entropy encoded, and transmitted along with the motion vectors. Several experiments are carried out on various image sequences to test the performance of the proposed coding scheme. The simulation results show that its performance exceeds that of H263+ single layer, spatial and spatio-temporal scalable coders.

Long Reed-Solomon codes over the prime field GF(216+1) are proposed as a low overhead channel code for reliable transmission of video over noisy and lossy channels. The added redundancy is near optimal from the information theoretic point of view contrary to the conventionally used intra-coding and sync (marker) insertion in video transmission that are not justified theoretically. Compared to known source-channel coding methods, we have achieved the quality of the output of source coder by providing nearly error free transmission. (By nearly error free we mean an arbitrarily small error probability.) The price paid for such remarkable video quality improvement and relatively low complexity is long delay due to long codes. However, the incurred delay is justifiable for Internet low bit video transmission and in high quality video broadcasting systems like streaming MPEG2 video where each encoder and decoder may have three frames delay. The proposed method surpasses the previous works for video transmission over binary symmetric channels, bursty channels, and packet loss channels. Also we propose a nearly error free transmission system for Gaussian channels for completeness of the work. We have also proposed short codes, which with larger overhead provide nearly error free transmission, and still outperform the previous works

Feature vector extraction, based on local image texture, is a primitive algorithm for many other applications, like segmentation, clustering and identification. If these feature vectors are a good match to the human visual system (HVS), we can expect to get the appropriate results by using them. Gabor filters have been used for this purpose successfully. In this paper we introduce a novel refinement, with the use of matching pursuits (MP) to improve the Gabor based texture feature extractor. With this improvement, we show that the separability of different textures increases. Another consideration in this work is computation complexity. Therefore, we limit the basis function set to reduce MP computation time.

We present an objective reduced-reference image quality assessment method based on harmonic gain/loss information through a discriminative analysis of local harmonic strength (LHS). The LHS is computed from the gradient of images, and its value represents a relative degree of the appearance of blockiness on images when it is related to energy gain within an image. Furthermore, comparison between local harmonic strength values from an original, distortion-free image and a degraded, processed, or compressed version of the image shows that the LHS can also be used to indicate other types of degradations, such as blurriness that corresponds with energy loss. Our simulations show that we can develop a single metric based on this gain/loss information and use it to rate the quality of images encoded by various encoders such as DCT-based JPLG, wavelet-based JPLG 2000, or various processed images. We show that our method can overcome some limitations of the traditional PSNR.

In this paper we propose a rate-efficient peer-to-peer architecture for video multicasting over the Internet. The limited capacity of the Internet hosts and the heterogeneous property of their access links are the main challenges of the peer-to-peer video multicasting over the Internet. Although, the rate-optimized overlay tree construction is a NP-hard problem, we propose a number of distributed and efficient protocols for rate-efficient overlay tree construction. Our proposed protocols include efficient join, improvement, overlay tree refinement and an optimum rate allocation protocol. The simulation results show the efficiency of the proposed protocols in rate-efficient overlay tree construction. We show that through combination of the efficient join protocol, tree refinement operations and an optimum rate allocation algorithm we can achieve a suboptimum overlay tree.

In this letter, hierarchical quadrature amplitude modulation (HQAM) is used to provide unequal error protection (UEP) for layered (data partitioned) H.264 coded video. In a conventional HQAM system, the high-priority (HP) and low-priority (LP) capacities have a constant ratio, whereas in H.264 data partitioning, the corresponding parts do not necessarily have this constant ratio. This letter proposes a multilevel HQAM arrangement with adaptive constellation distances that provides a graceful degradation in the quality of the decoded video without requiring feedback from the receiver. The arrangement improves the quality relative to nonhierarchical transmission through poor signal-to-noise ratio (SNR) channels at the price of a modest quality reduction through good SNR channels.

A hybrid video codec, where the intraframe pictures are wavelet-based coded and the interframe pictures are coded with a standard H.263 codec is proposed. We show that intraframe-wavelet coded pictures not only improve the quality of I-pictures but also result in lower distorted prediction pictures that outperforms a pure H.263 video codec.

This paper introduces techniques for segmentation and tracking based on analysis of region derived descriptors. By partitioning the image into a series of homogeneous regions, the problem of object extraction changes from pixel based to database analysis. A region based approach has distinct advantages over pixel. In particular, it has low dimensionality, is resilient to rotation, shear, and photometric changes, preserves boundaries, gives implied segmentation, provides comprehensive description, has reduced drift, and permits detailed analysis. The method is amenable to the introduction of prior knowledge permitting the simplification of segmentation, which is otherwise an ill-posed problem. Objects are defined as collections of contiguous regions with known statistics, allowing their identification by performing correlations in the database. The interframe difference of region based sequences is high for low-motion scenes due to segmentation noise, but performs consistently better for high motion, making it suitable for tracking. When applied to tracking the method proves robust, occlusion insensitive, and unlike the other techniques, it re-establishes lost lock.

This paper proposes a novel multilayer video transcoding approach for multicasting pre-encoded video to heterogeneous end-systems via diverse grouping of networks. Multilayer transcoding is first addressed by means of multiquality or SNR scalability of the MPEG-2 standard. Frequency domain transcoding and drift-compensated transcoding are derived from the closed-loop and multiloop SNR scalabilities, respectively. The proposed transcoding architectures are verified in terms of eliminating picture drift whilst preserving compatibility with the MPEG-2 SNR decoder. Multilayer transcoding is then addressed by means of multiresolution or spatial scalability of the MPEG-2 standard that supports different video formats. The transcoder retains the full resolution of the incoming video stream in its enhancement layer while generating a low spatio-temporal resolution base-layer compatible with the H.263 video format. Hence providing both multilayer transcoding and video format portability. The resultant video layers are shown to be free from drift with PSNR results comparable to those of the respective scalable encoders.

The H.264/AVC video coding standard has been recently presented by the Joint Video Team of ITU-T and MPEG experts. The AVC encoder achieves video bitstreams at the same quality as the previous standard coders, while requiring typically 50% of the bit rate. This paper proposes a Lagrangian optimized rate control algorithm for the H.264/AVC video encoder. It controls the bit rate by adjusting the Lagrangian multiplier for every picture and specifying the quantizer parameter for every macroblock. In the proposed method, the accuracy of desired bit rate is tunable by allocating more searching time to the encoder. Experimental results show that the proposed method gives a quality improvement of about 0.5 dB when compared to the rate control method utilized in the JM7.4 test model, with an acceptable encoder complexity.

The non-linear diffusion of the method of P. Perona and J. Malik (see IEEE Trans. Pattern Anal. and Machine Intelligence, vol.PAMI-12, no.7, p.629-39, 1990; Proc. IEEE Comput. Soc. Workshop on Comput. Vision, p.16-22, 1987) is applied to a contour. Unlike most contour diffusion techniques, the contour is described by the angle variation, and the non-linear diffusion procedure is applied to the contour turning angle. The Perona and Malik model determines how strongly diffusion acts on the original function, and depends on a factor K, estimated automatically. In areas with spatial concentration of strong changes of angle, this factor is also adjusted to reduce the contour small perturbations and noise effects.

Due to the use of variable length code (VLC), a single bit error in the discrete cosine transform (DCT)-based coded bit stream, such as H.263, may propagate up to the end of the group of blocks (GOBs) or slice. In this paper, we propose an iterative error detection and correction algorithm for the slice mode of the H.263 bit stream. The visibly erroneous macroblocks (MBs) in the decoded frames are detected by checking a set of error detection conditions derived from the redundant information (such as neighboring MBs and inner-DCT block similarity measure) inherent within the frame. In each slice, the part of the bit stream before the first erroneous MB is decoded in the conventional manner. The remaining part after this MB is redecoded iteratively, skipping one bit at a time until a decodable subbit stream is found. In case an MB is nondecodable, it is replaced with gray level. Once all the slices are checked, the frame is reconstructed and again checked for erroneous MBs. This process is repeated until the decoder detects no more corrupted MB. The proposed step-by-step decoding technique limits the error into a few MBs only, which can easily be concealed by any error concealment technique. The simulation results demonstrate that our scheme can recover the corrupted frames under the bit error rates up to 1% over binary symmetric channel (BSC), and improve the concealed picture quality by 4-6 dB over the conventional methods.

We introduce the DCT pyramid as a layered video coding technique. For efficiency of coding, we propose an optimum bit allocation method for the layers of the DCT pyramid. The proposed method is based on minimization of the overall reconstructed error with the Lagrangian multiplier and distribution of the bit budget among the layers. Experimental results on the improvement due to optimum bit allocation are also presented.

This paper proposes using a novel region space technique to track sport persons for the purpose of extracting their shirt numbers and use this to provide augmented information to the viewer. The region adjacency graph and picture trees are used to perform a search for an object using prior knowledge from a scene description. Once the candidate object has been extracted the sub-space is examined for alphanumeric characters, which are then characterized by optical character recognition. Rogue candidates may be removed based on the recognition histograms with improved robustness using temporal analysis.

In this paper we propose a mesh-based motion compensation technique that uses three-dimensional overlapped warping prediction. This technique defines a mesh warping motion model that includes temporal motion vectors and spatial overlapped motion compensation. The multiframe weighted prediction is obtained by the spatio-temporal overlapped warping of a quadrilateral wiregrid. This framework combines complex motion description with spatial and temporal multihypothesis motion compensation. In addition, by conveying information from multiple frames, it allows some degree of compensation for occlusion in sub-block areas. Experimental tests show that, three-dimensional overlapped mesh warping with multiple reference frames, can achieve a prediction PSNR gain of more than 1.5 dB, or an equivalent 30% reduction in the mean-square error signal, over the conventional single-frame, nonoverlap technique.

In this paper, the motion parameters of the bidirectionally predicted pictures (B-pictures) of MPEG-1,2 are exploited for concealment of large portions of corrupted anchor pictures (and vice versa) that might arise due to channel errors or packet losses. To further enhance the quality of the concealed pictures, we propose two methods of constraining the motion vectors of the B-pictures that strengthen the tie between them and those of the anchor pictures in the same picture subgroup. In one method, the macroblock decisions on the last B-picture in each subgroup is constrained to be bidirectional if those of the other B-pictures are not, such that the derived motion vectors for the concealment of the anchor picture are always composed from the forward and backward motion vectors of the bidirectional motions. Second, the bidirectional motion vectors of the B-pictures in each subgroup is constrained such that the vectorial sum of their forward and backward motion vectors results in an accurate motion prediction of the anchor picture. The experimental results show that while the composed motion vectors improve the quality of concealment over the conventional methods by more than 3-4 dB, another 2 dB improvement can be achieved by constraining the generation of the bidirectional motion vectors.

In this paper, we introduce a multilayered video coding technique to provide spatial scalability. The proposed scheme generates multilayered bitstreams for scalable multicast video transmission in heterogeneous environments as well as overcomes the major drawback of layered coding which is the increase in the total bit rate. We also minimize the picture drift problem associated with the lower resolution layer by introducing a separate drift compensation layer. Simulation results show that the drift compensation layer improves the lower resolution layer quality without significantly affecting the higher layer coding efficiency. Compared to H263+, although our coder generates more bits at the base layer, but the overall bit rate is less than that of H263+.

This paper proposes an efficient extension of virtual set partitioning in hierarchical trees (VSPIHT) for color image coding. Our new scheme, Color-Virtual-SPIHT (CVSPIHT) generates fully embedded bit stream similar to SPIHT. It combines the zerotrees of three color planes in two steps. First, zerotrees within the same color planes are joined together by VSPIHT, then resulting longer zerotrees of three planes are combined through a novel composite tree. The simulation results show the improved performance of the proposed method compared to a SPIHT based color coding scheme.

This paper presents a novel scheme to improve the robustness of set partitioning in hierarchical tree (SPIHT) based color image coder for transmission over noisy channels. In this scheme, the SPIHT bit streams are re-arranged according to their spatial (square block) representation without loss of coding efficiency. A group of blocks, called slice are transmitted independently. The first erroneous block in the erroneous slice is detected by examining error-checking conditions while decoding the image. If any transmission error is detected, a sequence of bit skipping and redecoding process is performed on the corrupted part of the bit stream until a feasible solution is found. The simulation results show significant quality improvement.

This paper proposes a wavelet based efficient and embedded color image coding technique. It extends the use of virtual set partitioning in hierarchical trees (VSPIHT) for color images. In this technique the coefficients of chrominance planes are linked together through composite trees having roots in the virtual LL band of luminance component. The virtual decomposition is performed in the luminance plane only. The simulation results show the improved performance of the proposed technique compared to the color SPIHT (CSPIHT) as well as the technique which uses virtual decomposition in each of the color planes.

We propose the application of overlapped spatial transformation motion estimation with a search strategy that extends motion vectors to past and future frames in a framework of three-dimensional overlapped warping prediction. This arrangement combines the prediction efficiency of multihypothesis motion compensation in the spatial and temporal domains while allowing some degree of compensation for occlusion that occurs in subblock areas. Experimental tests show that multi-frame reference without overlap can achieve a prediction PSNR improvement of more than 1.5 dB over the conventional single-frame reference non-overlap technique. Considering the three-dimensional overlapped approach with multiple reference frames, a prediction PSNR improvement of up to 2 dB can be obtained. A practical implementation with one past and one future frame can achieve a gain of more than 1.5 dB or an equivalent 30% reduction in the mean-square error signal.

A method of improving the traditional polygonal approximation methods is described. It has properties such as scalability, resilience to noise, controlled levels of approximation and good visual representation. These properties are a direct result of the use of scale space information to derive the vertices positions of the polygonal approximation. The vertices are situated on the contour points where the changes of the contour directions are at their major sharp positions. An encoding method with a scalable polygonal approximation of shapes is defined, which results in a good relation between the distortion and the bit rate.

In this paper, we propose a technique for video segmentation using non-linear diffusion, incorporating the watershed transform, colour similarity merging and motion estimation. The current frame t to be segmented is spatially partitioned into homogenous regions based on the watershed transform with colour similarity merging in the CIELUV colour space. A future frame t+x, is then used as reference to establish non-stationary regions within the current frame. These regions are classified as foreground objects, or as regions of interest.

An advanced image and video segmentation system is proposed. The system builds on existing work, but extends it to achieve efficiency and robustness, which are the two major shortcomings of segmentation methods developed so far. Six different schemes containing several approaches tailored for diverse applications constitute the core of the system. The first two focus on very-low complexity image segmentation addressing real-time applications under specific assumptions. The third scheme is a highly efficient implementation of the powerful nonlinear diffusion model. The other three schemes address the more complex task of physical object segmentation using information about the scene structure or motion. These techniques are based on an extended diffusion model and morphology. The main objective of this work has been to develop a robust and efficient segmentation system for natural video and still images. This goal has been achieved by advancing the state-of-art in terms of pushing forward the frontiers of current methods to meet the challenges of the segmentation task in different situations under reasonable computational cost. Consequently, more efficient methods and novel strategies to issues for which current approaches fail are developed. The performance of the presented segmentation schemes has been assessed by processing several video sequences. Qualitative and quantitative result of this assessment are also reported.

We propose an algorithm that automatically detects clear scene cut locations from an MPEG-1 video bit stream coded with a GOP structure of M=1, without B pictures. The algorithm detects scene cuts at P type pictures by monitoring the percentage of intramacroblocks per P picture. while scene cuts at I pictures are detected by matching the macroblock type of the two P pictures at the GOP boundaries. A "type matching parameter" (TMP) is developed to estimate the matching degree between the macroblock types of two P pictures. It is shown that the method is able to identify the location of scene cuts in P and I pictures with a high success rate.

This paper addresses the problems of transmitting layered coded video services over IP networks. The paper discusses the performance of two well-known technologies (RSVP and RTCP) applied to the transmission of layered coded video. Two simulation models were developed for this purpose. A comparison of the simulation results is presented along with a discussion. It is shown that the model based on RTCP has a better performance in transporting layered video streams, but the RSVP model has a fairer behaviour towards the best effort traffic.

The work carried out by the authors on heterogeneous video transcoding (see Shanableh, T. and Ghanbari, M., IEEE Trans. on Multimedia, vol.2, no.2, p.101-10, 2000) is extended to the DCT domain. The heterogeneous video transcoder was mainly proposed for achieving portability between different encoding formats, namely, from MPEG-1/2 into H.261/3 with lower spatio-temporal resolutions. The motion compensation (MC) loops of the decoder-encoder cascade are now simplified by eliminating the DCT/IDCT pairs whilst performing picture decimation and MC in the DCT domain (MC-DCT). Two transcoding architectures are proposed, in one, both the MC-loops are in the DCT-domain, while in the other, it is shown that by implementing the encoder's MC-loop in the pixel domain higher flexibility and better results are achieved. Methods for accelerating the MC-DCT are then divided into two broad categories of lossless and lossy methods. It is shown that by utilising shared information among successive motion compensated macroblocks, better results over those reported in the literature are achieved.

Video streaming over a hostile environment such as the Internet, where the available bandwidth varies in time, requires the source to adjust its transmission rate dynamically. Moreover, inevitable packet losses cannot be totally avoided and, hence, the video streaming schemes should be complemented with appropriate error concealment techniques. We highlight the vitality of the bi-directionally predicted pictures in the MPEG coded bitstreams to applications of video streaming and error concealment. In particular, we show how the properties and the motion parameters of such pictures can be exploited for efficient video transcoding into lower resolutions and different encoding formats for pre-encoded video streams. At the decoder side, however, we also emphasize on the important role of the bidirectional prediction on interframe error concealment.

This paper introduces the concept of dynamic hierarchical management, enabled by mobile agent (MA) technology. The proposed framework addresses the scalability limitations of the centralised paradigm and the poor flexibility of static hierarchical management architectures to changing networking conditions. The increased adaptability of our framework is enabled by a novel management entity, termed mobile distributed manager (MDM), MDMs, being MAs themselves, can dynamically migrate to an assigned network domain (given that certain requirements are met) and undertake its management responsibility, operating at an intermediary level between the central manager and SNMP agents, localising the associated management traffic. The paper also focuses on the design decisions and implementation experiences of the proposed architecture.

In this paper a low-complexity nonlinear filtering technique to smooth textures preserving object contours is presented. The approach is based on a hybrid combination of both isotropic and anisotropic recursive filtering. Using only intensity information the segment borders obtained by applying nonlinear filtering do not necessarily coincide with physical object contours, especially in the case of textured objects. To segment images into regions with physical meaning additional information extracted from disparity or motion is used to weight the filter coefficients. The presented technique has been successfully tested in the context of object segmentation of natural scenes and object-based disparity estimation for stereoscopic applications.

In this paper, transcoding of pre-encoded MPEG-1,2 video into lower bit rates is realized through altering the coding algorithm into H.261/H.263 standards with lower spatial resolutions. We extract and compose a set of candidate motion vectors, from the incoming bit stream, to comply with the encoding format of the output bit stream. For the spatial resolution reduction, we generate one motion vector out of a set of input motion vectors operating on the higher spatial resolution image. We discuss the impact of motion estimation refinement on the new motion vectors and show that for all cases a simple half-pixel refinement is sufficient for near-optimum results.

Spatial transformations motion estimation techniques have been proposed to address the inability of the conventional block matching to compensate for complex motion. Due to the relative high complexity of these approaches, fast motion search techniques are compulsory for practical implementations. In this paper we propose a hierarchical approach to spatial transformations motion estimation that significantly improves the prediction and complexity figures of the conventional methods. On the one hand we significantly reduce the complexity associated with the search techniques by initialising motion vectors at higher levels leaving only small displacement refinements for the more computational intensive lower levels. On the other hand we use the hierarchical differential motion structure to replace a large displacement fast search with small displacement fast searches at each level to overcome the fast methods inaccuracy with large motion. We demonstrate the validity of the approach both for quadrilateral matching motion estimation and warping motion estimation.

The emerging MPEG-7 standard demands shape description and shape retrieval techniques. Polygonal approximations of the shape contours give attractive solutions in this domain, because of the description simplicity. This paper introduces a shape matching technique based on the turning function comparison of the shape contour polygonal approximations.

A novel interframe error concealment technique is proposed for video packet losses in IP environments. The proposed technique benefits from the bidirectional motion parameters of bidirectionally predicted pictures 'B-pictures' to conceal large portions (up to 100%) of corrupted anchor pictures. The lost anchor picture is interframe concealed by backward tracking its motion to its anchor picture through the intermediate bidirectional motion of B-pictures. Furthermore, we show that by constraining the motion compensation of the B-pictures to strengthen the tie between their bidirectional motion and that of the future anchor pictures, better concealment of the latter is achieved. The paper then elaborates on the impact of the proposed constrained-encoding in terms of encoding efficiency and concealment quality.

Blockiness in the MPEG-2 coded pictures is due to luminance discontinuities across the DCT block boundaries. Using the Sobel operators, these discontinuities can be extracted. After segmentation into smaller blocks of pixels and Fourier transformation into the frequency domain, the blockiness will be translated into harmonics whose amplitude and phase can provide vital information for quantifying the blockiness. The amplitude of the harmonics is proportional to the degree of blockiness, while the phase of the harmonics can be used to verify that the harmonics are not due to contextual details in the picture. By examining both the amplitude and phase information of the harmonics, an accurate blockiness detector that needs no reference pictures has been devised, achieving a high correlation of 0.88 between the model prediction and the subjective data.

New MPEG-4 functionalities require the segmentation of input video into different layers (VOPs). Usually, these layers contain arbitrarily shaped objects representing meaningful content of the video stream. With the introduction of the new functionalities in MPEG4, the need of objective and subjective assessment of segmented image quality has emerged. In this paper we introduce an efficient and reliable metric to evaluate segmentation results by comparing them with a given ground truth. Beyond this application the proposed technique can be used for real-time shape description and retrieval in the context of the emerging MPEG7, as well as in general pattern recognition tasks. Selected results obtained by using this metric within these application areas are reported.

The intrinsic scalability limitations of traditional centralised network management (NM) become significantly more pronounced when transfers of bulk network monitoring data are considered. This fact has encouraged a trend towards distributed management intelligence, which promises more flexible and scalable solutions. However, distributed mobile agent (MA) based NM architectures reported in the literature do not adequately address scalability problems when considering data-intensive NM applications. In this paper, we present three novel applications in which MA are used to perform management data aggregation, acquire SNMP table snapshots and filter SNMP table contents subject to filtering expressions. Both real-time and off-line NM data acquisition is considered. The applications, supported by a lightweight management framework described in previous work, are shown to outperform SNMP-based polling in terms of bandwidth consumption.

Using both the amplitude and phase information of the harmonics generated by block artefacts in MPEG2 coded video, it is possible to estimate the blockiness in MPEG2 coded video accurately without the reference signal, achieving correlation of 0.88.

In video distribution services, various networks of different capacity may be linked together to form the branches of a multicasting tree. In order to efficiently use the available bandwidth of each subnetwork, a video transcoder capable of reducing the bit rate of the incoming bit stream must be used at those nodes whose output links have less bandwidth than the input one. In this paper, we analyze the buffering implications of inserting a video transcoder within the transmission path. For transcoders with either fixed or variable compression ratio, we show that the encoder buffer size can be maintained as if no transcoder existed while the decoder has to modify its own buffer size according to both the bit rate conversion ratio and transcoder buffer size. We derive the conditions that have to be met by both the encoder and transcoder buffers for preventing the decoder buffer from underflowing or overflowing. Furthermore, based on the characteristics of constant bit rate (CBR) MPEG coded video, we show that efficient bit rate control can be implemented in CBR video transcoders such that the picture quality of transcoded sequences is practically the same as those directly encoded from the original sequence at the same reduced bit rates.

Different coding schemes introduce different artifacts to the decoded pictures, making it difficult to design an objective quality model capable of measuring all of them. A feasible approach is to design a picture-quality model for each kind of known distortion, and combine the results from the models according to the perceptual impact of each type of impairment. In this letter, a multi-metric model comprising of a perceptual model and a blockiness detector is proposed, designed for MPEG video. Very high correlation between the objective scores from the model and the subjective assessment results has been achieved.

In this work, transcoding of pre-encoded MPEG-1, 2 video into lower bit rates is realized through altering the coding algorithm into H.261/H.263 standards with lower spatio-temporal resolutions. For this heterogeneous transcoding, we extract and compose a set of candidate motion vectors, from the incoming bit stream, to comply with the encoding format of the output bit stream. For the spatial resolution reduction we generate one motion vector out of a set of input motion vectors operating on the higher spatial resolution image. Finally, for the temporal resolution reduction we compose new motion vectors from the dropped frames motion vectors. Throughout the paper, we discuss the impact of motion estimation refinement on the new motion vectors and show that for all cases a simple half-pixel refinement is sufficient for near-optimum results.

In this paper we present a method for motion segmentation, in which accurate grouping of pixels undergoing the same motion is targeted. In the presented technique true object edges are first obtained by combining anisotropic diffusion of the original image with edge detection and contour reconstruction in the inherent scale-space. Contours are then matched according to the distance given by a metric defined on their polygonal approximations and the shape of the one-dimensional intensity function along the contour. Masks of objects are obtained by merging image areas inside of edges having the same motion. The performance of the presented technique has been evaluated by computer simulations.

The use of mobile agent technology to distribute and delegate management tasks promises to overcome the scalability and flexibility limitations of the centralised network management paradigm. An efficient, lightweight infrastructure based on mobile agents is described, which addresses these issues by distributing management operations amongst managed devices and hence reduces the processing load and bandwidth usage. A mobile agent generator that facilitates the creation of new mobile agents for additional services is also presented. Our infrastructure is shown to outperform SNMP both in terms of the polling response time and bandwidth consumption when considering data intensive operations.

Distributed network management (NM) architectures exploiting the advantages of mobile agents (MA) promise to relieve some of the limitations intrinsic to client-server based centralised NM, such as lack of scalability, interoperability and flexibility. However, the use of MAs alone could impose an unnecessary transmission overhead if mobile code were to be used for trivial tasks. Hence, an infrastructure based on a hybridisation of centralised and distributed approaches is proposed, which maintains the lightweight nature of centralised NM for trivial management tasks, whilst adopting a distributed MA-based scheme as task complexity and suitability changes. Remote processing and compression of NM data issues are investigated in order to reduce the bandwidth usage. Security enhancements are also considered.

A stereo vision based system for composition of natural and computer generated images is presented. The system focuses on the solution of four essential tasks in computer vision: disparity estimation, object segmentation, modeling and tracking. These tasks are performed in direct interaction with each other using novel and available multiview analysis techniques and standard computer graphics algorithms. The system is assessed by processing natural video sequences. Selected results are reported.

We propose a classification method of MPEG coded video based on the image texture and motion levels of the scene content. The texture and motion are divided into three levels each, low, medium and high, resulting into nine scene categories. Since in MPEG-1 (and MPEG-2) encoder the DCT coefficients of the I-frame are affected by the image texture, while the motion affects the macroblocks coding mode of P- and B-frames, hence, an automatic scene classification can be obtained from the coded bit stream based on the scene's texture and motion levels. The method is useful for the content-based video analysis, such as video modeling and MPEG-7.

There is a significant interlayer correlation between the base and enhancement layers of two-layer signal-to-noise-ratio (SNR) scalable coders. By delaying the enhancement layer bitstream, spatial and temporal interlayer correlations can be minimized or even made negative. Such an arrangement improves the statistical multiplexing gain to the extent that, even taking into account the normally higher bit rate of SNR scalable coders, negatively correlated two-layer sources can be more efficiently multiplexed than their single-layer counterparts. Simulation results suggest that decorrelation be applicable to both weakly and strongly aligned sources and by inferences to any randomly aligned sources there between. Greatest "gains" in cell-loss ratio (CLR) are displayed by strong alignments such that for videoconferencing-type pictures, SNR scalable sources were able to raise aggregate bit rates by up to 38% before overall CLR started to fall below a single-layer performance of 10/sup -4/. With correlation dependent on the base/enhancement-layer division of bandwidth, only when the enhancement-layer efficiency is at its worst (close base and enhancement quantizer indexes) is the decorrelation "gain" insufficient to compensate the increase in total SNR scalable coder bit rate.

It is shown how the signal-to-noise (SNR) scaleable coder can benefit from optimizing quantified discrete cosine transform (DCT) coefficients, in a rate-distortion context, in order to reduce bit-rate overheads. The technique is based on adjusting the quantized coefficients rather than dropping them, since the former gives finer control over rate-distortion trade-offs. The widely used Lagrangian optimization technique is then applied to arrive at the optimally adjusted coefficients block. We show that such an optimization is very efficient for the second layer, but has little effect on the base layer.

In this paper, we propose a video model to generate VBR MPEG video traffic based on the scene content description. Long sessions of nonhomogeneous video clips are decomposed into homogeneous video shots. The shots are then classified into different classes in terms of their texture and motion complexity. Each shot class was uniquely described with an autoregressive model. Transitions between the shots and their durations have been analyzed. Unlike many classical video source models, this model may be used to generate traffic of any type of video scenes ranging from a low complexity video conferencing to a highly active sport program. The performance of the model is evaluated by measuring the mean cell delay when the generated video traffic is fed to an ATM multiplex buffer.

We propose a bit rate control algorithm for video transcoding, based on a dynamic buffer strategy. We show that the bit rate reduction ratio used for transcoding CBR bit streams should not follow the constant relation given by the input and output channel capacities but follow a variable function dependent upon the picture type instead. Based on a simulation study, the influence of the dynamic buffer occupancy on the picture quality is discussed and the proposed algorithm is shown to be capable of achieving transcoded picture quality comparable to that of single-stage encoding.

We propose a video model to generate a VBR MPEG video traffic based on the scene description. Long sessions of non-homogeneous video clips are decomposed into homogeneous video shots. The shots are then classified into different classes in terms of their texture and motion complexity. Each shot class can be uniquely described and modelled as a homogeneous video. The model may be used to generate traffic of any type of video scenes ranging from a low complexity video conferencing to a highly active sport program.

Multi viewpoint synthesis from stereoscopic video data is a key technology in most emerging content based multimedia systems. The different techniques presented in the article are based on the analysis of 2D images acquired by two or more cameras. It is shown, how suitable analysis tools for disparity estimation and segmentation can benefit from each other. For the generation of virtual images, two methods with different levels of trade-off between complexity and degree of freedom are described. The first approach is disparity compensated view interpolation, which is capable of generating intermediate views along the interocular axis. The second is a more complex approach, which in a first step generates a 3D model of the object. This model can be rotated to any orientation, the original texture is mapped onto the surface, and arbitrary views can be generated by rendering the obtained surface.

Many of the forthcoming video services and multimedia applications are expected to use preencoded video for storage and transmission. Video transcoding is intended to provide transmission flexibility to preencoded bit streams by dynamically adjusting the bit rate of these bit streams according to new bandwidth constraints that were unknown at the time of encoding. In this paper, we propose a drift-free MPEG-2 video transcoder, working entirely in the frequency domain. The various modes of motion compensation (MC) defined in MPEG-2 are implemented in the discrete cosine transform (DCT) domain at reduced computational complexity. By using approximate matrices to compute the MC-DCT blocks, we show that computational complexity can be reduced by 81% compared with the pixel domain approach. Moreover, by using a Lagrangian rate-distortion optimization for bit reallocation, we show that optimal transcoding of high-quality bit streams can produce better picture quality than that obtained by directly encoding the uncompressed video at the same bit rates using a nonoptimized Test Model 5 (TM5) encoder.

Video transcoding techniques offer the possibility of matching coded video to transmission channels of lower capacity by reducing the bit rate of compressed bit streams. In this paper we propose a new frequency domain video transcoder for bit rate reduction of compressed bit streams. A motion compensation (MC) loop, operating in the frequency domain, is used for drift compensation at reduced computational complexity. We derive approximate matrices for fast computation of the MC blocks in the frequency domain. By using the Lagrangian optimisation in calculating the best quantiser scales for transcoding, we show that transcoded pictures from a high quality bit stream are better than those encoded from original frames at the same reduced bit rates.

SNR scalability, used in two-layer video coding, guarantees good base quality pictures at the expense of increased overall bit-rate. By understanding the inherent inefficiencies of enhancement layer coding we have developed an optimisation method called optimal coefficient adjustment in order to reduce overall bit-rates to levels consistent with single-layer operation.

In this paper, video transcoding is introduced as a mechanism capable of decoupling video encoders from network constraints. A video transcoder is proposed for congestion control of pre-encoded video traffic over asynchronous transfer mode (ATM) networks and to provide flexible transmission in video distribution applications. It is shown that as long as the network delay does not exceed the equivalent multiplex buffer capacity, traffic can be efficiently controlled to ease congestion. It is also shown that video distribution over heterogeneous networks can use transcoders for providing fairness among users of the same transmission tree.

SNR scalability and data partitioning have both been proposed by MPEG-2 as a means of guaranteeing minimum quality in the presence of information loss for such applications as ATM networks. Whilst SNR scalability can guarantee a superior base quality to data partitioning it is disadvantaged by a 5-15% increase in total bit-rate. In this paper we consider two methods that use optimal DCT coefficient adjustment in order to reduce bit-rate overheads. By comparing with single layer and data partitioning, we are able to demonstrate significant bit-rate reductions, to the extent that an optimised SNR scalable coder is able to outperform data partitioning, not just in terms of base picture quality, but also in terms of bandwidth utilisation.

Transcoding is regarded as a down conversion process, where the bit rate of a compressed video bit stream is reduced according to a given constraint. Based on a transcoding architecture, previously developed by the authors, an optimal transcoder in a rate-distortion sense is proposed. It is shown that the basic architecture provides the necessary support for measuring the global transcoding distortion to be minimised under the optimisation algorithm. The experimental results show that, optimally transcoded pictures have a better quality than those encoded from the original video material, using a standard encoder at the same bit rates. This paper shows that compressed video can be further compressed with a good efficiency.

Coding of HDTV and super high definition video requires not only high performance compression but also compatibility and scalability to satisfy transmission channels of various speeds and capacity and receivers working at different resolutions. A compatible and scalable coding scheme has been developed for these video signals. Wavelet decomposition is performed such that the low frequency band is of common intermediate format (CIF) order, which forms a low-resolution core of the full-sized video and is coded by MPEG. The high-frequency wavelet coefficients are coded by the proposed virtual zerotree which encodes the wavelet coefficients more efficiently than the simple zerotree. Also embedded in the scheme is the hierarchical motion compensation that gives high performance with a large capable compensation range, requires minimum computation, and reduces overhead motion information.

It is expected that most of the video services will be based on the MPEG2 standard and many of them using recorded streams. When compressed video is recorded, the characteristics of the channel through which it will be transmitted are assumed to be known beforehand. Therefore a great lack of flexibility arises in transmission of these streams when channels of diverse characteristics are used. If the same video programme is to be simultaneously distributed to several users through channels with different capacities, the service provider needs to keep several copies of that programme, each one encoded according to the corresponding channel characteristics. We show that it is only required to keep one copy of the coded data at its highest possible quality. Transcoding the main stream to lower rates is achieved with minimum delay. Therefore the transcoder is also capable of fast response to network demands to prevent packet loss.

Variable bit rate traffic generated by multimedia and video services may experience cell loss when transmitted over ATM networks. In a video distribution environment, where several links with different capacities are employed, it is likely to find nodes with less capacity than the offered traffic by the distributed bit stream. Either high cell loss rate should be allowed at these nodes or the source has to comply with the higher constraint penalising other users. We present a generic congestion control scheme capable of being used with either type of MPEG-2 video sources, real-time encoders or video servers. The scheme is based on a low delay post-processing mechanism capable of reducing the bit rate of an encoded bit stream without drift. Its application in video distribution and congestion control is presented. The results show good performance in all cases.

A fully compatible and scalable coding scheme for super high definition (SHD) video has been developed. Wavelet decomposition is performed such that the low frequency band is of CIF order, which forms a low-resolution core of the full sized video, and is coded by MPEG. The high frequency wavelet coefficients are coded by the proposed virtual zerotree which encodes the wavelets more efficiently than the simple zerotree. Embedded in the scheme is also the hierarchical motion compensation that gives high performance with a large capable compensation range, requires minimum computation, and reduces overhead motion information. With its scalability, data prioritization and precise bit-rate control, the scheme is suitable to code SHD video in multimedia applications.

A method for preventing accumulation of image artifacts due to cell loss in packet video is presented. At each ATM switching node an auxiliary buffer is used to store the overflow traffic of the main switching buffer. The main buffer is served with an absolute priority over the auxiliary buffer. Decoded pictures are normally reconstructed from the cells of the main buffer. The late cells received from the auxiliary buffer are processed and properly added to the current decoded picture. Postprocessing of these cells for the standard video codecs such as H.261 and MPEG is presented. It is shown that although the picture quality due to cell loss is temporarily degraded, it is immediately brought back to its original quality upon the reception of the late cells, as if no loss has occurred.

A method of coding super high definition (SHD) still images based on vector quantization of wavelet coefficients is proposed. A compression scheme for SHD images should achieve data compression without any visible picture quality deterioration. We demonstrate that the proposed method meets this requirement. In this coding technique, each vector of wavelet coefficients is coded by a series of vectors of decreasing magnitudes, resulting in a successive approximation process. It also exploits the structural similarities among the bands. This provides efficient coding together with the ability to guarantee arbitrary distortion levels for each band, which can be exploited to achieve subjectively optimum performance. Conventional image compression techniques such as transform, sub-band, and vector quantization have already been tested for the coding of SHD images. Simulation results show that the proposed method outperforms the other SHD image coding methods reported in the literature.

The behavior of linear phase wavelet transforms in low bit-rate image coding is investigated. The influence of certain characteristics of these transforms such as regularity, number of vanishing moments, filter length, coding gain, frequency selectivity, and the shape of the wavelets on the coding performance is analyzed. The wavelet transforms performance is assessed based on a first-order Markov source and on the image quality, using subjective tests. More than 20 wavelet transforms of a test image were coded with a product code lattice quantizer with the image quality rated by different viewers. The results show that, as long as the wavelet transforms perform reasonably well, features like regularity and number of vanishing moments do not have any important impact on final image quality. The influence of the coding gain by itself is also small. On the other hand, the shape of the synthesis wavelet, which determines the visibility of coding errors on reconstructed images, is very important. Analysis of the data obtained strongly suggests that the design of good wavelet transforms for low bit-rate image coding should take into account chiefly the shape of the synthesis wavelet and, to a lesser extent, the coding.

A coding method for wavelet coefficients of images using vector quantization, called successive approximation vector quantization (SA-W-VQ) is proposed. In this method, each vector is coded by a series of vectors of decreasing magnitudes until a certain distortion level is reached. The successive approximation using vectors is analyzed, and conditions for convergence are derived. It is shown that lattice codebooks are an efficient tool for meeting these conditions without the need for very large codebooks. Regular lattices offer the extra advantage of fast encoding algorithms. In SA-W-VQ, distortion equalization of the wavelet coefficients can be achieved together with high compression ratio and precise bit-rate control. The performance of SA-W-VQ for still image coding is compared against some of the most successful image coding systems reported in the literature. The comparison shows that SA-W-VQ performs remarkably well at several bit rates and in various test images.

A novel coding method of wavelet coefficients of images using vector quantization, referred to as successive approximation wavelet vector quantization (SA-W-VQ) is proposed. In this method, each vector is coded by a series of vectors of decreasing magnitudes until a certain distortion level is reached. Analysis of the successive approximation using vectors is given, and conditions for convergence are derived. It is shown that lattice codebooks offer an efficient tool to meet these conditions, with the extra advantage of fast encoding algorithms. In SA-W-VQ, distortion equalization of the wavelet coefficients can be achieved together with high compression ratio and precise bit rate control. Simulation results for still image coding show that SA-W-VQ outperforms both the EZW coder and the standard JPEG.

A modified version of the Orwell protocol is presented. While this method preserves all the basic features, it introduces the concept of the surplus cell. These surplus cells enhance the performance of the network for class-2 traffic as well as those of the enhancement data in two-layer coded video by reducing the queuing delays. It is shown that the new method improves the delay characteristics of real-time services under mid-to-high traffic loads.>

Two methods for bit-rate reduction of two-layer video codecs, without impairing their robustness to cell loss are introduced. Both methods employ an H.261 compatible coder at the base layer and an interframe coder at the second layer. In the first method a second H.261 is used to code the residual errors of input pixels. Parameters of this Twin-H.261 codec in terms of effect of motion estimation, resilience to cell loss and forced updating are investigated. The second method employs an interframe coder among the enhancement DCT coefficients (Inter-Enhance) of successive frames of similar frequencies. Introduction of a leaky prediction in the interframe loop preserves the coder resilience to cell loss. Optimum value of the leak factor is found to be in the range of 0.85-0.95. It is shown that while both Twin-H.261 and Inter-Enhance codecs generate bit rates close to the one-layer coder, their resilience to cell loss closely follows the normal two-layer coder.>

A block-based subband image coder that exploits the ability to perform decimation in the discrete cosine transform (DCT) domain to effect a pyramidal data structure is described. The proposed "DCT pyramid" has a distinct feature of improved image rendition properties without the associated blocking artifacts at low bit-rates.

The coding gain of biorthogonal wavelet transforms is investigated. The influence of the regularity and number of vanishing moments of these wavelets on the coding gain is analysed. More than 80 wavelet transforms are assessed. The results show that as long as the coding gain of a wavelet transform is reasonably high, the influence of both regularity and number of vanishing moments on the coding gain is low. On the other hand, for wavelet transforms with coding gains less than 1, the coding gain decreases when the regularity decreases. It is also shown that, in some cases, the coding gain of biorthogonal wavelet transforms is larger than the theoretical maximum for orthogonal transforms, obtained with ideal filters. In addition, unlike orthogonal transforms, whose coding gain in octave decompositions is always smaller than the one obtained with uniform band decompositions with the same number of stages and filter banks, the coding gain of biorthogonal transforms in octave band decompositions can be either larger or smaller than in the uniform case.>

Texture classification applied to the frame difference signals can be used to design adaptive algorithms for a variety of video applications. In this paper an adaptive block matching motion estimation algorithm based on the interframe texture analysis is presented. The algorithm adaptively changes the size and shape of the search window of each block depending on the values of textural features. The chosen features are extracted from the temporal difference histogram and provide information about the speed of the moving objects and the most likely direction of their motion. Applications of the proposed algorithm to the normal and hierarchical motion estimators are shown. Compared to the known techniques, the presented methods reduce the computational effort by saving unnecessary searches while offering higher performance.>

Three different versions of a two-layer video coding technique are considered. In the first method the generated bit rate of both layers are variable (VBR-VBR). The second method employs a smoothing buffer and a control mechanism to produce a constant bit rate at the base layer, but that of the second layer is still variable (CBR-VBR). The third method is similar to the second method but the base layer rate within the buffer size is made variable (semi-CBR-VBR). The performance of these methods under the coding efficiency and their interactions with ATM networks are investigated. It is shown that the second layer under VBR-VBR has the minimum network delay and loss, while at the base-layer, CBR-VBR has the best performance. Both layers under semi-CBR-VBR method have all the advantages of the other two methods without having their disadvantages. The semi-CBR-VBR codec does not need network policing and hence is an ideal candidate for ATM networks.>

Methods for the interpolation of lost cells in asynchronous-transfer-mode (ATM) networks are studied. It is shown that use of motion-compensated previous frames gives the best results. The quality of the interpolated pictures improves if the motion vectors truly represent the actual motion in the scene. This is only possible with a two-layer coding scheme, where the motion vectors can be delivered to the decoder through the base-layer guaranteed channel. In derivation of the motion vectors at the encoder, use of uncoded input picture frames outperforms the conventional method of motion extraction from the previous coded pictures, despite the lower bit rate of the latter to the former. Depending on the quality of the base layer and the scene activity, the signal-to-noise ratio (SNR) in the cell-loss-interpolated areas can be improved by up to 10 dB.>

Aliasing artifacts of subband coded pictures at low bitrates can be eliminated by adding the information from the higher bands that cancel aliasing at the lower band. This is done by DCT coding the high-frequency bands and selecting the coefficients with the highest energy. The optimum scanning directions of these coefficients for each band are given. As an application of these ideas, a method for the transmission of high-resolution still pictures at 64 kb/s based in a combination of subband analysis/synthesis and a modified H261 video codec is proposed. The progressive buildup obtained with this method is pleasant to the human observer, and is four times faster than if the DCT alone were used.>

Measurements of subjective picture impairment as a function of network loading in a simulated ATM network are reported. The simulation indicated that cells tend to be discarded in bursts, the frequency and severity of which can be related to the loading by a threshold model. The effect of the discards on broadcast-style video, coded using a single-layer H.261-type method, was found to be a function of scene content and movement at the instant of occurrence. If the visibility of cell discards is maintained at or below threshold in worst-case scenes, the study indicated that network loadings around 55% for a multiplex of 16 video sources and around 70% for a multiplex of 48 video sources are achievable.

The output from an H.261 or MPEG video coder consists of macroblocks of differing lengths. Two methods of packing the macroblocks into ATM cells are considered, depending on whether or not part of an uncompleted macroblock may be carried over to the next cell. It is shown that while due to cell loss for a single source loosely packed cells result in a smaller number of lost macroblocks, in a multiplexed network closely packed cells have a marginally better performance.>

An adaptive video codec which can maintain compatibility between the existing digital circuit switched networks and future ATM networks is described. It is based on two-layer coding principles, where in the base layer the H.261 standard video codec is employed. It is shown that a discrete cosine transform (DCT) in the second layer is unnecessary and can be replaced by a simple requantizer which finely quantizes the residue quantization distortions of the transform coefficients of the base layer. Characteristics of this quantizer in terms of nature of quantization and step sizes are studied. Efficient coding and addressing of residual distortions are described. Finally, a two-dimensional variable length code for the quantized residual distortions is given.>

A fast block-matching algorithm for motion estimation is presented. It is based on a logarithmic step where, in each search step, only four locations are tested. For a motion displacement of w pels/frame, this technique requires 5+4 log/sub 2/w computations to locate the best match. Using sequences of CIF standard pictures, the interframe motion compensated prediction error with this technique is compared to the other fast methods. The computational complexity of this algorithm is also compared against those methods.>

Two-layer conditional-replenishment coding of video signals over a variable-bit-rate (VBR) network is described. A slotted-ring network based on an Orwell protocol is assumed, where transmission of certain packets is guaranteed. The two-layer coder produces two output bit streams: the first bit stream contains all the important structural information in the image and is accommodated in the guaranteed capacity of the network, while the second adds the necessary quality finish. The performance of the coder is tested with CIF standard sequences and broadcast-quality pictures. The portion of the VBR channel allocated to the lower layer as guaranteed bandwidth is examined. Using broadcast-quality pictures, statistics were obtained on the performance of this system for different choices of bit rate in the lower layer. The effect of lost packets is shown on CIF standard picture sequences. It is shown that the coder performs well for a guaranteed channel rate as low as 10-20% of the total bit rate.>