Efficient and perceptually meaningful quality assessment is a fundamental requirement for image and video processing, compression, and streaming systems. This article shows that, in the context of Discrete Cosine Transform ( DCT)-based compressed images, Structural Similarity Index ( SSIM ) can be approximated from global Peak Signal to Noise Ratio (PSNR) or Mean Square Error ( MSE) using local statistics derived only from the reference image. While prior work assumes access to local MSE, we propose two approaches to approximate local MSE by redistributing the global MSE using variance or standard-deviation-based weighting. Experiments on the Kodak and Xiph Subset1 datasets across a range of JPEG quality levels demonstrate that both approaches provide accurate and robust SSIM approximations, substantially outperforming the global MSE baseline. The proposed framework is designed to extend naturally to video, where reference-derived statistics can be amortized across multiple encodes of the same content.

This special issue presents recent advances in multimedia systems research showcased at ACM Multimedia Systems 2024 and its co-located workshops. The selected papers span adaptive and immersive video streaming, low-latency and scalable delivery architectures, and innovations in video coding and processing. Together, they illustrate the rapid progress and broad impact of emerging techniques across the multimedia stack.

Light field displays enable immersive visual environments by delivering accurate depth and 3D representation of the content. Due to the constrained viewing angle of light field displays, the alignment between the angular coverage of the captured scene with the field of view (FoV) of the display is essential. This paper employs the Looking Glass Factory light field display—with a 53-degree FoV—to visualize content captured at the same 53-degree angular extent provided in the KULF-TT53 light field dataset. In total, 77 stimuli are used in the test, comprising original contents and impaired version of them, which are encoded with HEVC, and AV1 at 1-5 Mbps bitrate range. Two subjective quality assessment metrics of MOS and DMOS are used to represent the obtained scores from the subjects. The evaluation of the results shows that AV1-encoded stimuli showcase better quality compared with HEVC, with statistically significant quality differences, at the considered bitrates. The per-subjects results of this study are made available upon publication of this work (Q-KULF-TT53 dataset).

Abstract Quality assessment is a key element for the evaluation of hardware and software involved in image and video acquisition, processing, and visualization. In the medical field, user-based quality assessment is still considered more reliable than objective methods, which allow the implementation of automated and more efficient solutions. Regardless of increasing research on this topic in the last decade, defining quality standards for medical content remains a non-trivial task, as the focus should be on the diagnostic value assessed by expert viewers rather than the perceived quality from naïve viewers, and objective quality metrics should aim at estimating the first rather than the latter. In this paper, we present a survey of methodologies used for the objective quality assessment of medical images and videos, dividing them into visual quality-based and task-based approaches. Visual quality-based methods compute a quality index directly from visual attributes, while task-based methods, being increasingly explored, measure the impact of quality impairments on the performance of a specific task. A discussion on the limitations of state-of-the-art research on this topic is also provided, along with future challenges to be addressed.

360-degree video streaming for VR visualisation is characterised by large transmission data volume and stringent interactive latency demands; hence guaranteeing suitable transmission quality, while meeting the existing constraints, is highly challenging. This paper addresses the relevant grand challenge presented at MMSys 2024 on 360-degree video on-demand streaming, aiming at designing and implementing a 360-degree video on-demand streaming solution using the open-source evaluation platform E3PO. The proposed solution incorporates several strategies including viewport prediction, tiling, encoding tile selection for streaming and upsampling. The source code for the proposed solution will be publicly available on Github1.

The aim of this paper is to show how the structural similarity metric SSIM for image quality assessment can be seen in many cases, such as DCT-based compressed images and video, as a content-aware version of the peak signal-to-noise ratio (PSNR). In fact, under some assumptions described in the paper, the first can be derived directly from the latter based on a single content-dependent parameter, i.e. the variance of the image / video frame, not requiring joint access to original and compressed image/frames. Results show the validity of the assumptions and high accuracy for the derivation / estimation, in particular at quality levels of interest in real use cases.

Over the years, many subjective and objective quality assessment datasets have been created and made available to the research community. However, there is no standard process for documenting the various aspects of the dataset, such as details about the source sequences, number of test subjects, test methodology, encoding settings, etc. Such information is often of great importance to the users of the dataset as it can help them get a quick understanding of the motivation and scope of the dataset. Without such a template, it is left to each reader to collate the information from the relevant publication or website, which is a tedious and time-consuming process. In some cases, the absence of a template to guide the documentation process can result in an unintentional omission of some important information. This paper addresses this simple but significant gap by proposing a datasheet template for documenting various aspects of sub-jective and objective quality assessment datasets for multimedia data. The contributions presented in this work aim to simplify the documentation process for existing and new datasets and improve their reproducibility. The proposed datasheet template is available on GitHub1, along with a few sample datasheets of a few open-source audiovisual subjective and objective datasets.

In modern-era video streaming systems, videos are streamed and displayed on a wide range of devices. Such devices vary from large-screen UHD and HDTVs to medium-screen Desktop PCs and Laptops to smaller-screen devices such as mobile phones and tablets. It is well known that a video is perceived differently when displayed on different devices. The viewing experience for a particular video on smaller screen devices such as smartphones and tablets, which have high pixel density, will be different with respect to the case where the same video is played on a large screen device such as a TV or PC monitor. Being able to model such relative differences in perception effectively can help in the design of better quality metrics and in the design of more efficient and optimized encoding profiles, leading to lower storage, encoding, and transmission costs. However, to the best of our knowledge, open-source datasets providing subjective scores for the same content when viewed on multiple devices with different screen sizes do not exist, thus limiting a proper evaluation of the existing quality metrics for such multi-screen video streaming applications. This paper addresses this research gap by presenting a new, open-source dataset consisting of subjective ratings for various encoded video sequences of different resolutions and bitrates (quality) when viewed on three devices of varying screen sizes: TV, Tablet, and Mobile. Along with the subjective scores, an evaluation of some of the most famous and commonly used open-source objective quality metrics is also presented. It is observed that the performance of the metrics varies a lot across different device types, with the recently standardized ITU-T P.1204.3 Model, on average, outperforming their full-reference counterparts. The dataset consisting of the videos, along with their subjective and objective scores, is available freely on Github1.

Gaming video streaming services have grown tremendously in the past few years, with higher resolutions, higher frame rates and HDR gaming videos getting increasingly adopted among the gaming community. Since gaming content as such is different from non-gaming content, it is imperative to evaluate the performance of the existing encoders to help understand the bandwidth requirements of such services, as well as further improve the compression efficiency of such encoders. Towards this end, we present in this paper GamingHDRVideoSET, a dataset consisting of eighteen 10-bit UHD-HDR gaming videos and encoded video sequences using four different codecs, together with their objective evaluation results. Additionally, the paper discusses the codec compression efficiency of most widely used practical encoders, i.e., x264 (H.264/AVC), x265 (H.265/HEVC) and libvpx (VP9), as well the recently proposed encoder libaom (AV1), on 10-bit, UHD-HDR content gaming content. Our results show that the latest compression standard AV1 results in the best compression efficiency, followed by HEVC, H.264, and VP9.

Dynamic vision sensors (DVSs) are emerging neuromorphic visual capturing devices, with great advantages in terms of low-power consumption, wide dynamic range, and high temporal resolution in diverse applications, such as autonomous driving, robotics, tactile sensing, and drones. The capturing method results in lower data rates than conventional video. Still, such data can be further compressed. Recent research has shown great benefits of temporal data aggregation on event-based vision data utilization. According to recent results, time aggregation of DVS data not only reduces the data rate but improves classification and object detection accuracy. In this work, we propose a compression strategy, time-aggregation-based lossless video encoding for neuromorphic vision sensor data (TALVEN), which utilizes temporal data aggregation, arrangement of the data in a specific format and lossless video encoding techniques to achieve high compression ratios. The detailed experimental analysis on outdoor and indoor datasets shows that our proposed strategy achieves superior compression ratios than the best state-of-the-art strategies.

Subjective tests for the assessment of the quality of experience (QoE) are typically run with a pool of subjects providing their opinion scores using a 5-point scale. The subjects’ mean opinion score (MOS) is generally assumed as the best estimation of the average score in the target population. Indeed, for a large enough sample, we may assume that the mean of the variations across the subjects approaches zero, but this is not the case for the limited number of subjects typically considered in subjective tests. In this paper, we propose an approach based on generalized linear models (GLMs) for estimation of the population average QoE. The motivating dataset is composed of the individual scores assigned by 25 subjects to a set of gaming videos evaluated under different resolutions and compression ratios. The approach recognizes the multinomial nature of the data and allows for correlation between scores of the same subject. The resulting estimated average QoE is shown to follow more credible patterns than the MOS, particularly for higher bitrates, for which the model estimates present more coherent behavior. Similar convincing results are found on a second dataset, showing the validity of the approach.

Dynamic Vision Sensors (DVS) are emerging neuromorphic visual capturing devices, with great advantages in terms of low power consumption, wide dynamic range, and high temporal resolution in diverse applications. The capturing method results in lower data rates than conventional video. Still, such data can be further compressed. This is an emerging research area and a performance comparison of different compression strategies for these data is still missing. This paper addresses lossless compression strategies for data output by neuromorphic visual sensors. We compare the performance of a number of strategies, including the only strategy developed specifically for such data and other more generic data compression strategies, tailored here to the case of neuromorphic data. We perform the comparison in terms of compression ratio, as well as compression and decompression speed. According to the detailed experimental analysis, LZMA achieves the best compression ratio among all the considered strategies. On the other hand, Brotli achieves the best trade-off between speed (compression and decompression) and compression ratio.

Neuromorphic sensors, a.k.a. dynamic vision sensors (DVS) or silicon retinas, do not capture full images (frames) at a fixed rate, but asynchronously capture spikes indicating changes of brightness in the scene, following the principles of biological vision and perception in mammals. DVS sensing and processing produces a data representation where the scene can be represented with a very high time resolution with a limited number of bits (an inherent data compression is performed at the time of acquisition). Such representation can be used locally to derive actionable responses and selected parts can be transmitted and then processed in another network location. Due to these features, such sensors represent an excellent choice as visual sensing technology for next-generation Internet-of-Things, e.g. in surveillance, drone technology, and robotics. It is in fact becoming evident that in this framework acquiring, processing, and transmitting frame-based video is inefficient in terms of energy consumption and reaction times, in particular in some scenarios. Hence, we explore here the feasibility of advanced Machine to Machine (M2M) communications systems that directly capture, compress and transmit spike-based visual information to cloud computing services in order to produce content classification or retrieval results with extremely low power and low latency.

Existing works in the field of quality assessment focus separately on gaming and non-gaming content. Along with the traditional modeling approaches, deep learning based approaches have been used to develop quality models, due to their high prediction accuracy. In this paper, we present a deep learning based quality estimation model considering both gaming and non-gaming videos. The model is developed in three phases. First, a convolutional neural network (CNN) is trained based on an objective metric which allows the CNN to learn video artifacts such as blurriness and blockiness. Next, the model is fine-tuned based on a small image quality dataset using blockiness and blurriness ratings. Finally, a Random Forest is used to pool frame-level predictions and temporal information of videos in order to predict the overall video quality. The light-weight, low complexity nature of the model makes it suitable for real-time applications considering both gaming and non-gaming content while achieving similar performance to existing state-of-the-art model NDNetGaming. The model implementation for testing is available on GitHub1.

With the increasing demand in video traffic, delivering appropriate Quality of Experience may appear as a challenge for highly popular services such as Twitch.tv and YouTube-Gaming. This paper presents the evaluation of three widely used encoders, namely H.264, H.265 and AV1, for the case of gaming videos, assuming streaming scenario. The performance of the codecs has been assessed in terms of objective video quality metrics (PSNR, SSIM and VMAF) and subjective video quality assessment. The encoding settings were kept similar for all the three aforementioned encoders. In the considered setting, AV1 achieves a better level of video quality both in terms of objective and subjective VQA, for almost all bitrates and content considered. The improvement is remarkable in particular for the lower range of bitrates considered.

Recently, streaming of gameplay scenes has gained much attention, as evident with the rise of platforms such as Twitch.tv and Facebook Gaming. These streaming services have to deal with many challenges due to the low quality of source materials caused by client devices, network limitations such as bandwidth and packet loss, as well as low delay requirements. Spatial video artifact such as blockiness and blurriness as a result of as video compression or up-scaling algorithms can significantly impact the Quality of Experience of end-users of passive gaming video streaming applications. In this paper, we investigate solutions to enhance the video quality of compressed gaming content. Recently, several super-resolution enhancement techniques using Generative Adversarial Network (e.g., SRGAN) have been proposed, which are shown to work with high accuracy on non-gaming content. Towards this end, we improved the SRGAN by adding a modified loss function as well as changing the generator network such as layer levels and skip connections to improve the flow of information in the network, which is shown to improve the perceived quality significantly. In addition, we present a performance evaluation of improved SRGAN for the enhancement of frame quality caused by compression and rescaling artifacts for gaming content encoded in multiple resolution-bitrate pairs.

This paper presents a comparative performance evaluation of the newly proposed AV1 Image File Format (AVIF) vs. other state-of-the art image codecs, for natural, synthetic and gaming images. The codecs are compared in terms of Rate-quality curves and BD-Rate savings considering different quality metrics. AVIF results in the best overall performance considering both 4:2:0 and 4:4:4 chroma sub-sampling encoded images.

Various ocular diseases, such as cataract, diabetic retinopathy, and glaucoma have affected a large proportion of the population worldwide. In ophthalmology, fundus photography is used for the diagnosis of such retinal disorders. Nowadays, the set‐up of fundus image acquisition has changed from a fixed position to portable devices, making acquisition more vulnerable to distortions. However, a trustworthy diagnosis solely relies upon the quality of the fundus image. In recent years, fundus image quality assessment (IQA) has drawn much attention from researchers. This study presents a detailed survey of the fundus IQA research. The survey covers a comprehensive discussion on the factors affecting the fundus image quality and the real‐time distortions. The fundus IQA algorithms have been analysed on the basis of the methodologies used and divided into three classes, namely: (i) similarity‐based, (ii) segmentation‐based, and (iii) machine learning based. In addition, limitations of state of the art in this research field are also presented with the possible solutions. The objective of this study is to provide a detailed information about the fundus IQA research with its significance, present status, limitations, and future scope. To the best of the authors’ knowledge, this is the first survey paper on the fundus IQA research.

Gaming as a popular system has recently expanded the associated services, by stepping into live streaming services. Live gaming video streaming is not only limited to cloud gaming services, such as Geforce Now, but also include passive streaming, where the players' gameplay is streamed both live and ondemand over services such as Twitch.tv and YouTubeGaming. So far, in terms of gaming video quality assessment, typical video quality assessment methods have been used. However, their performance remains quite unsatisfactory. In this paper, we present a new No Reference (NR) gaming video quality metric called NR-GVQM with performance comparable to state-of-the-art Full Reference (FR) metrics. NR-GVQM is designed by training a Support Vector Regression (SVR) with the Gaussian kernel using nine frame-level indexes such as naturalness and blockiness as input features and Video Multimethod Assessment Fusion (VMAF) scores as the ground truth. Our results based on a publicly available dataset of gaming videos are shown to have a correlation score of 0.98 with VMAF and 0.89 with MOS scores. We further present two approaches to reduce computational complexity.

Event-based cameras are very promising sensors for high speed mobile robots, self-driving cars and drones. Event-based Vision Sensors output a sequence of events data rather than a video bitstream as for classical cameras. The data rate output by such sensors depends upon the type of scene and camera motion speed. Estimating the data rate associated to such sensors is needed for the appropriate design of transmission systems. This paper suggests considering information on the scene content to support such estimation. The considered content parameter shows a high correlation with the event rate, appearing as a promising parameter for event rate and hence data rate estimation.

Recent years have seen a tremendous increase in video traffic with the rise of Over The Top (OTT) services. Along with traditional Video on demand (VoD) streaming services (e.g., Netflix, YouTube), live video services (e.g., Twitch. tv, YouTubeGaming, Facebook Live) have also resulted in a tremendous share of Internet traffic. Among the live streaming services, gaming video streaming has a major share, with Twitch.tv alone currently responsible for the fourth highest peak Internet traffic in the US. As a consequence of this, and due to the fact that gaming videos are artificial and synthetic, it is worth investigating the specificity of gaming videos in relation to compression and the consequent end user QoE. In this paper, we present an objective and subjective quality comparison study for regular videos and gaming videos, with 30 video sequences (15 per type), encoded using the state of the art encoder HEVC. We discuss the similarity and dissimilarity between the two video types and also discuss how these observations can be used to improve the end user QoE.

Medical professionals are viewing an increasing number of images and videos in their clinical routine. However, various types of distortions can affect medical imaging data, and therefore impact the viewers' experienced quality and their clinical practice. Thus it is necessary to quantify this impact and understand how the viewers, i.e., medical experts, perceive the quality of (distorted) images and videos. In this paper, we present an up-to-date review of the methodologies used in the literature for the subjective quality assessment of medical images and videos and discuss their merits and drawbacks depending on the use case.

Due to recent advent of light field visualization, ac-quisition/creation, encoding, transmission, rendering and quality assessment of 3D light field content has gained momentum. In particular, large light field displays need content with large field of view, and with high spatial and angular quality. Accordingly, subjective and objective quality evaluation studies have been conducted to examine spatial, angular and spatio-angular aspects of light field visualization. Recently, the effect of various zooming levels of the displayed content, as well as regions of interest on Quality of Experience (QoE) has also been explored. However, there has been no systematic attempt to see how the features of the content itself affect the visualization quality. In this work, we attempt to examine the effects of some primitive features of the content on subjective QoE. The results are based on a subjective study conducted on a large light field display, offering virtually continuous horizontal parallax.

In this paper, we present a high-angular-resolution data-set created using a turntable arrangement. Seven distinct objects, positioned on an automated turntable, were captured from three camera positions for every half degree of rotation, generating 720 images for each camera position. For each object, the camera positions were registered to the coordinate system of the middle camera. Intrinsic parameters of the camera were also estimated. A data-set of this kind is instrumental for research in a variety of areas, such as light field visualization, manifold learning, visual quality assessment, evaluation of preferred object orientation etc. Due to the availability of three-view stereo, this data-set could be useful for studying view interpolation techniques.

This paper focuses on passive Quality of Experience (QoE) monitoring at user end devices as a necessary activity of the ISP (Internet Service Provider) for an effective quality-based service delivery. The contribution of the work is threefold. Firstly, we highlight the opportunities and challenges for the QoE monitoring of the Over-The-Top (OTT) applications while investigating the available interfaces for monitoring the deployed applications at the end-device. Secondly, we propose a multilayer passive QoE monitor for OTT applications at the user terminal with ISPs prospect. Five layers are considered: user profile, context, resource, application and network layers. Thirdly, we consider YouTube as a case study for OTT video streaming applications in our experiments for analyzing the impact of the monitoring cycle on the user end device resources, such as the battery, RAM and CPU utilization at end user device.

We present a new approach for image and video quality evaluation in telemedicine applications. Our approach relies on analyzing the quality of a pre-known reduced size logo embedded in an unused part of the medical ultrasound frame. The method is tested using two different objective metrics, namely: the Peak Signal to Noise Ratio (PSNR) and the Structural SIMilarity index metric (SSIM). We show that the presented method, not needing the original frame to predict the quality, achieves a high correlation coefficient (more than 0.9) for the different quality metrics used. We also present relationships between the quality derived via the logo and via the original frame and we assess the overhead in data transmission resulting from the compressed logo data and its protection overhead.

Gaming videos are increasingly being streamed live over the Internet, as evident by the increasing popularity of services such as Twitch.tv and YouTube-Gaming, with Twitch.tv alone consisting of approx. two million streamers and over nine million daily active users. We present here an objective evaluation of eight most popular games encoded using H.264/MPEG-AVC, H.265/MPEG-HEVC and VP9 encoders for live game video streaming applications as currently used by Twitch.tv and YouTube-Gaming. The results are reported in terms of three objective video quality metrics (PSNR, SSIM, VIFp), Bjontegaard-Delta Bitrate (BD-BR) analysis, and encoding duration. For the encoding settings and the encoders used, in terms of BD-BR analysis, H.265/MPEG-HEVC is found to provide the best compression efficiency but is 2.6 times slower than H.264/MPEG-AVC. The magnitude of bitrate savings for VP9 compared to H.264/MPEG-AVC is found to be highly dependent on the content type, with H.264/MPEG-AVC resulting in higher average bitrate savings with an encoding speed four times faster than VP9.

The smooth motion parallax of light field displays is one of the most critical enablers of the glasses-free auto-stereoscopic 3D experience. This smoothness originates from an acceptable degree of angular resolution, which is derived from the number of views visualized in the given field of view. However, if human observers view the content without any movement, the requirement for angular resolution can be lower. In this paper, we introduce the results of a subjective quality assessment of visual content displayed on a light field cinema. The angular resolution of the content was different in each test condition, and participants were located in fixed positions during the experiment.

There is an ongoing debate in the research community over the improved visual quality of UHD video in comparison to the still widely-deployed HD standard. It is the inspiration of many scientific studies, yet UHD displays and services are continuously spreading on the consumer market. This paper presents the results of a subjective paired-comparison test with both upscaled HD and UHD video sequences, investigating the primary research question of whether UHD can offer a significant improvement over HD for common viewing conditions. In our study, subjects rated their visual preference on video-only clips without lossy encoding. In addition we studied cognitive biases, by presenting a label of what users were about to see (HD or UHD) before the sequence, purposely manipulating the labels in some conditions to suggest different resolutions than actually shown. Finally, we investigated the impact of different rating scales on the precision of the results. Our studies show that HD clips appear indistinguishable from UHD clips when the rating scale chosen is not fine-grained enough. We also found that the labeling effect has a significant impact on the perceived quality, overriding the actual visual perception. Even with a precise scale, users cannot detect major improvements of UHD compared to HD, which may let us question the added value offered by UHD.

The ever-increasing network traffic and user expectations at reduced cost make the delivery of high Quality of Experience (QoE) for multimedia services more vital than ever in the eyes of Internet Service Providers (ISPs). Real-time quality monitoring, with a focus on the user, has become essential as the first step in cost-effective provisioning of high quality services. With the recent changes in the perception of user privacy, the rising level of application-layer encryption and the introduction and deployment of virtualized networks, QoE monitoring solutions need to be adapted to the fast changing Internet landscape. In this contribution, we provide an overview of state-of-the-art quality monitoring models and probing technologies, and highlight the major challenges ISPs have to face when they want to ensure high service quality for their customers.

Limited radio resources and increasing user demands make wireless resource allocation a challenging task. As a user moves, his channel quality varies significantly depending on his location. In this paper we show that wireless link quality can be predicted on a large time scale in terms of Path Loss values by using street and coverage maps. The predicted wireless link quality information can be used by the network operators for optimized resource allocation. The feasibility of the proposed approach is demonstrated by extensive simulations advocating this data driven approach for channel prediction.

This work focuses on Quality of Experience (QoE) based scheduling of multiple video flows on the downlink of LTE/LTE-A system. The LTE/LTE-A network architecture is not built for QoE-based video delivery. LTE/LTE-A enables Quality of Service (QoS) based traffic prioritization. QoS allows network operators to split traffic into flows based on attributes such as traffic type (streaming video, real-time video or VoIP) and traffic needs (packet delay, jitter or throughput). In this work, we consider scalable video layers mapped to QoE-based priority classes at the core network. At the Radio Access Network (RAN), packets of different priority classes receive different forwarding treatment (scheduling and queue management). In the recent approaches, the QoE marked packets are dropped by utilizing a packet dropping algorithm, and scheduled by employing a QoS-aware scheduler. In this work, we proposed a QoE-based priority class aware scheduling approach which, under congestion, does not require a packet dropping algorithm. Furthermore, the proposed approach not only conforms to the LTE/LTE-A hierarchical architecture but also delivers to the end users a better QoE as delivered by the state-of-the-art cross-layer approaches.

The papers in this special issue focus on cloud computing-based video processing and content sharing. With the rapid growth of IPTV and mobile video applications and driven by urgent demands from industry and users, video processing and content sharing technologies have received significant research attention in recent years. Cloud-based video processing and content sharing networks are promising technologies to orchestrate large-scale and efficient video distribution between mobile clients and multimedia cloud systems. The objective of this special issue is to identify and promote advancements in media cloud-based video processing and content sharing technologies to advance current and enable future anywhere and anytime video processing and streaming applications.

We propose an optimal radio resource allocation (RRA) scheme for scalable H.264/SVC multi-user video transmission over downlink orthogonal frequency division multiple Access (OFDMA)-based cognitive radio (CR) networks. Our framework adopts a new probabilistic approach to mitigate the total imposed interference by cognitive users on the licensed spectrum. We consider two fundamental network service objectives, i.e., the number of satisfied users and the video quality. We devise the 3-dimensional scalable quality of the H.264/SVC video transmission for an OFDMA-based CR network and develop efficient suboptimal algorithms to solve the probabilistic constrained mixed discrete-continuous non-linear programming (MDCNLP) problem. Simulation results indicate that the proposed quality-aware scheme can achieve up to 1.3 dB increase in the average peak signal-to-noise ratio (PSNR) per user over the conventional non-quality-aware RRA algorithms.

A stochastic radio resource allocation (RRA) algorithm is designed to maximize the total transmission rate of orthogonal frequency-division multiple access (OFDMA) cognitive radios (CRs) with hybrid (i.e., joint underlay and overlay) spectrum access strategy. Our novel solution incorporates the probabilities of channel availability obtained through spectrum sensing for allocating power and subcarrier in a multi-user multi-band environment. In order to protect the licensed users from harmful intervention under imperfect sensing information, stochastic transmit and interference power constraints are imposed on the CRs. The performance of the proposed RRA algorithm and advantages over the conventional hard-decision-based approaches are demonstrated via simulation.

Reducing the queue length and queuing delay in the buffer which in turn reduces congestion and latency in the network are of great importance for real-time video services. In this paper, we propose a perceptual quality-aware queue management mechanism to shorten the queue length and waiting time of video traffic in a finite buffer. Our proposed approach prioritizes video layers in a stream in accordance with their hierarchical structure and selectively drops packets from different layers under a target user perceived quality constraint. This in turn reduces network load and latency and increases the capacity to serve more concurrent streams while satisfying users' perceptual quality requirements. We develop a statistical queuing model to express the delay requirements of video layers with an equivalent cross-layer constraint We then formulate the queue management problem as a minimization of queuing delay using a selective packet drop approach under perceptual quality and queue expiry time provisioning. Numerical results show that our proposed approach outperforms the widely deployed random early detection (RED) and tail drop (ID) queue management schemes in terms of reducing queuing delay and queue length in the buffer, while increasing the fraction of perceived quality-satisfied users.

3D medical video was forecasted to be one of the groundbreaking 3D video applications. These range from tele-consultation to 3D robotics surgery. Enabling 3D video in e-health applications results in the provision of more natural viewing conditions, improved diagnosis and accurate interventions in surgical procedures. The deployment of 3D video services in healthcare is made possible to some extent by the advanced capturing devices (e.g., 3D endoscopes), recent advances in wireless communication technologies (e.g., LTE-Advanced(LTE-A)) and 3D video display technologies. Remote robotic assisted surgery and surgery training (education for surgeons) can benefit in particular from 3D video technologies due to the added dimension of depth. This paper analyzes the quality of compressed 3D surgical video. Moreover, asymmetric encoding of 3D medical video without compromising the medical quality of experience (M-QoE) is investigated in this paper. The quality of the compressed 3D medical video with the proposed method is evaluated using a comprehensive subjective quality evaluation test involving 12 medical surgeons. The results show a slightly better perception with the proposed asymmetric coding method compared to reference symmetric compression method, however the difference is statistically insignificant.

Recent advancements in 3D imaging, coding, compression, storage, transmission, and error concealment techniques enable wide usage of 3D video/image applications. Video quality assessment plays a major role in improving the perceived quality at the receiver side, since such information can be used at the transmitter or in the different network nodes for system optimization `on-the-fly'. Most of the objective quality metrics in use are Full-Reference (FR), requiring the original video sequance for comparison. In the case of quality assessment for stereoscopic video, both left and right views need to be considered. In this paper, we introduce a novel Reduced-Reference (RR) quality metric for stereoscopic video using 3D structural tensors, based on the fact that the Human Visual System (HVS) is more sensitive to the structural information present in the scene. This method incorporates a new saliency detection method by considering spatial and temporal aspects of the video sequance. The Correlation Coefficient (CC) calculated for the obtained results shows that the values of the derived metric are well correlated with the corresponding subjective test results.

The quality of experience and quality of service provided in the healthcare sector are critical in evaluating the reliable delivery of the healthcare services provided. Medical images and videos play a major role in modern e-health services and have become an integral part of medical data communication systems. The quality evaluation of medical images and videos is an essential process, and one of the ways of addressing it is via the use of quality metrics. In this paper, we evaluate the performance of seven state-of-the-art video quality metrics with respect to compressed medical ultrasound video sequences. We study the performance of each video quality metric in representing the diagnostic quality of the video, by evaluating the correlation of each metric with the subjective opinions of medical experts. The results indicate that the visual information fidelity, structural similarity index, and universal quality index metrics show good correlation with the subjective scores provided by medical experts. The tests also investigate the performance of the emerging video compression standard, high-efficiency video coding-HEVC, for medical ultrasound video compression. The results show that, using HEVC with the considered ultrasound video sequences, a diagnostically reliable compressed ultrasound video can be obtained for compression with values of the quantization parameter up to 35.

The five papers in this special section cover a wide range of topics, from mobile health systems to service-oriented decision support in medical systems and service quality evaluation.

Enabling 3D video in e-health applications results in the provision of more natural viewing conditions and improved diagnosis. Recent advances in wireless communications, network technologies and 3D video capture/display mechanisms enable the replacement of 2D video applications of current healthcare services with 3D video, to provide improved perception and diagnosis. Remote surgery and surgery training can benefit in particular from 3D video due to the added dimension of depth. This paper envisages a wireless health system which transmits 3D surgical video over next generation wireless networks. Asymmetric coding of 3D video to provide effective medical services over band limited wireless networks is investigated in this paper. The quality of the compressed 3D medical video with the proposed method is measured both objectively and subjectively for a range of compression levels.

This paper presents an analysis of medical video streaming based on the Dynamic Adaptive Streaming over HTTP (DASH) standard from the Moving Picture Experts Group (MPEG).We propose two adaptation approaches at the client and evaluate parameters of the Quality of Experience (QoE), i.e., the quality perceived by the users, by investigating the number and duration of re-buffering events, the added time in the playout, and the Peak Signal-to-Noise Ratio (PSNR) of the medical video frames. The results show that the selection of MPEG-DASH related parameters and the adaptation approach at the client result in improved quality of the medical video streaming. In the end, we discuss how the elaborated approaches and results could be utilized to facilitate and enhance telemedicine applications.

Healthcare services are increasingly using medical images and video in their applications. Quality assessment can be performed via image quality evaluation metrics. However, due to the specific nature of the associated data, quality evaluation of medical images poses several issues. In this paper, we provide an insight into medical image quality evaluation, by discussing the issues faced and the current trends in the literature. Based on the analysis of the medical and technical literature, three broad categories of quality evaluation metrics are presented. The different methodologies are compared in different environments and services and recommendations for future research are provided.

This paper proposes a novel segment-based teletraffic model for dynamic adaptive video streaming over HTTP (DASH), based on the MPEG-DASH standard. The functionality of the standard's framework is mapped on the proposed model for which the probabilities of buffer overflow, empty buffer and active buffer are adopted as relevant performance metrics. These metrics describe the MPEG-DASH streaming process from the teletraffic viewpoint and are observed for different segment sizes of the encoded video at the server and different buffer sizes at the client. The results show that small segment sizes of the encoded video increase the probabilities of empty buffer and active buffer, while decreasing the probability of buffer overflow, whereas large segment sizes are suitable for decreasing the probabilities of empty buffer and active buffer, while increasing the probability of buffer overflow. Regardless of the segment sizes, the performance metrics are further improved when the buffer size at the client is increased.

In this paper, we propose a strategy for resource allocation for different traffic classes at the Medium Access Control (MAC) layer of wireless systems based on Orthogonal Frequency Division Multiple Access (OFDMA), such as the recent Long-Term Evolution (LTE) wireless standard. In order to achieve inter-class fairness, we propose a modification of the Virtual Token Modified Largest Weighted Delay First (VT-M-LWDF) and Modified Largest Weighted Delay First (M-LWDF) rules. Through simulation, we show that the proposed scheduler introduces remarkable multi-objective improvement of the Quality of Service (QoS) performance parameters, i.e., Packet Loss Rate (PLR), average throughput, fairness index and system spectral efficiency, among different classes of traffic such as video, VoIP and best-effort.

We consider in this paper downlink scheduling at the MAC layer for delay sensitive traffic in wireless systems based on Orthogonal Frequency Division Multiple Access (OFDMA), such as the recent LTE/LTE-A wireless standard. In the literature, time diversity is exploited in scheduling by using the Head of Line (HoL) packet delay and time-averaged bit throughput. However, under diverse channel conditions and variable rate characteristics of video traffic, the average throughput along with the HoL packet delay does not provide a fair system. We advocate here that, instead of utilizing the average throughput, the utilization of time-averaged channel quality along with the HoL packet delay and Packet Loss Rate (PLR) can reduce the Quality of Service (QoS) violations of low channel quality users and thus provide a fair system.

We consider in this paper downlink scheduling at the MAC layer for delay sensitive traffic in wireless systems based on Orthogonal Frequency Division Multiple Access (OFDMA). We propose a downlink scheduling strategy for scalable video transmission to multiple users over OFDMA systems, such as the recent LTE/LTE-A wireless standard. We use a payoff metric based on video frame importance in conjunction with our proposed scheduler. The priority function on each Physical Resource Block (PRB), considering video frame importance and bit throughput, efficiently exploits multi-user diversity as compared to state-of-the-art strategies. Simulation results confirm the efficient utilization of time and frequency diversity by showing an improvement in the objective video quality of each user as compared to state-of-the-art strategies.

We consider in this paper downlink scheduling at the Medium Access Layer (MAC) layer for delay sensitive traffic in wireless systems based on Orthogonal Frequency Division Multiple Access (OFDMA). We refer in particular to next generation wireless systems such as 3GPP-LTE. We propose an Opportunistic Packet Loss Fair (OPLF) scheduling algorithm based on calculating a simple dynamic priority function which depends on the Head of Line (HOL) packet delay, the packet loss rate (PLR) and the achievable instantaneous downlink rate of each user. This algorithm overcomes the main limitations of existing algorithms, achieving better performance than state-of-the art algorithms, such as Modified Largest Weighted Delay First (M-LWDF), Proportional Fair (PF), and Packet Loss Fair (PLF), in terms of throughput, PLR and fairness among users.

The 11 papers in this special issue cover a range of topics and can be logically organized in three groups, focusing on QoE-aware media protection, QoE assessment and modelling, and multi-user-QoE management.

Scalable video coding (SVC) has aroused a wide interest in the areas of video coding and transmission technology, since it provides desirable features for heterogeneous error-prone network environments. The layered video structure allows not only adaptation to the avail able bandwidth but also a device adaptation capability via multiple decodable sub-streams. In this paper, we focus on SVC Medium-Grain Scalability (MGS) and investigate the end user's quality of experience (QoE) in error-prone transmission conditions. We describe our error detection and concealment implementation on the JSVM 9.15 reference decoder and assess its effectiveness through several quality indicators. Results show an improvement of the proposed concealment strategy when compared to the standard Frame Copy (FC) and Interpolation, in terms of Peak Signal-to-Signal Noise Ratio (PSNR) and Structural Similarity Index Metric (SSIM) quality metrics. This improvement is evident in the case of concealment on quality enhancement layers, since video fluidity is preserved assuring an acceptable QoE to the end users.

3D video adds the sensation of depth to conventional video in order to provide better natural viewing conditions for users. However, 3D video communication applications are in its early stage due to limitations of channel bandwidth and to the packet losses and errors due to congestion and bad channel conditions. In order to provide a better service to 3D video users, the compression and transmission system parameters can be changed "on the fly". The measured 3D video quality at the receiver-side can be used as feedback information to fine tune the system. This paper proposes a quality evaluation method for real-time trans mission of 3D video depth maps associated with colour plus depth based 3D video. The extracted edge information together with luminance and contrast information are employed to evaluate the quality degradation in this method. This work is motivated by the fact that edges and contours of the depth map can represent different depth levels and hence can be used in measuring the structural degradation of 3D depth maps. The performance of the method is evaluated across a range of compression levels and PLRs (Packet Loss Rates) and shows accurate results compared to its counterpart Full-Reference (FR) quality metric.

When 3D video is transmitted over lossy channels, different strategies can be adopted at the different layers of the communication protocol stack to cope with such errors, in a cross-layer operation framework. At the receiver side, effective error concealment methods are necessary to overcome the quality loss due to erroneous packets. The existing concealment techniques for conventional 2D video can be individually applied for each component/view associated with 3D video. Novel error concealment methods which utilize inherent characteristics of 3D video are also emerging. This paper proposes a colour/texture initiated error concealment method for colour plus depth based 3D video. The error concealment process starts from the colour image sequence as accurate texture information is required to render good quality 2D/3D video with the DIBR (Depth Image-Based-Rendering) method. Since the depth map represents the basic structure of the corresponding colour image, the proposed method utilizes colour image concealment to conceal the errors in the associated depth map as well. This colour/texture image based temporal error concealment method is tested over a backward compatible Scalable Video Coding (SVC) architecture and a simulated packet loss network. The objective and subjective results/illustrations show that improved 2D/3D video quality can be achieved with the proposed method.

3D video quality assessment is a key task in transmission of immersive video, both for the assessment of end-to-end system performance and for quality evaluation “on the fly”. For example, measured 3D video quality at the receiver side can be used as feedback information for controlling transmission parameters. However, measuring video quality at the receiver side using Full-Reference quality metrics (e.g. PSNR) is less practical due to the need of original image sequence at the receiver. Hence, Reduced-Reference and No-Reference quality metrics are being developed to evaluate the quality. Since edges and contours of the depth map represent different depth levels, this paper proposes a Reduced-Reference quality metric for the depth maps associated with colour plus depth 3D video using edge detection. Edge/Side information (i.e. binary edge mask) generated using sobel filtering is utilized. Results show that the proposed metric can be used in place of a Full-Reference metric with an acceptable accuracy.

M-health is an emerging area of research and one of the key challenges in future research in this area is medical video streaming over wireless channels. Contrasting requirements of almost lossless compression and low available bandwidth have to be tackled in medical quality video streaming in ultrasound and radiology applications. On one side, compression techniques need to be conservative, in order to avoid removing perceptively important information; on the other side, error resilience and correction should be provided, with the constraint of a limited bandwidth. A quality-driven, network-aware approach for joint source and channel coding based on a controller structure specifically designed for enhanced video streaming in a robotic teleultrasonography system is presented. The designed application based on robotic teleultrasonography is described and the proposed method is simulated in a wireless environment in two different scenarios; the video quality improvement achievable through the proposed scheme in such an application is remarkable, resulting in a peak signal-to-noise ratio (PSNR) improvement of more than 4 dB in both scenarios.

The design of an efficient mobile healthcare system using 3.5G and 4G wireless networks is a challenging problem especially for bandwidth demanding telemedical applications. In this paper, we focus on the concept of medical quality of service (m-QoS) applied to a typical bandwidth demanding m-health application. Based on this concept, we propose a novel multiobjective rate-control mechanism for the optimized delivery of diagnostically acceptable ultrasound video images over 3.5G wireless networks. The performance of the proposed algorithm has been evaluated via both simulations and experimental studies. The proposed optimal rate control algorithm achieved performance improvements that are compatible with the medical QoS requirements.

In this paper applications and requirements of mobile broadband multimedia healthcare services are presented, with the focus on specific research challenges. Such pervasive multimedia healthcare applications include hospital tele-consultation, health monitoring, intelligent emergency management systems, pervasive healthcare data access, and ubiquitous mobile telemedicine. After a survey of the past and undergoing research and demonstration projects in the area, recent challenges and future research directions of comprehensive wireless multimedia health monitoring are presented, including context awareness, content awareness, quality issues, security and reliability, and autonomous and adaptable operation.

We present new tight bounds for evaluating the performance of sync word-based frame synchronization algorithms in the periodically embedded case. We consider antipodal signaling with coherent detection over additive white Gaussian noise and both optimal and suboptimal search techniques. Our bounds are very close to results obtained through simulation and tend asymptotically (for increasing signal-to-noise ratios) to the exact performance.

Frame synchronization is classically treated in literature for equiprobable data symbols. In this case the optimum metric has been derived both in the case of periodically and aperiodically embedded synchronization words and the corresponding performance has been evaluated. In this paper we evaluate the performance of sequential frame synchronization for non- equiprobable data symbols, assuming no a-priori information about data distribution. Results show that the performance may sensibly differ from the case of equiprobable data symbols and strongly depends on the chosen synchronization word.

We present a framework for the analysis of frame synchronization based on synchronization words (SWs), where the detection is based on the following sequential algorithm. The received samples are observed over a window of length equal to the SW; over this window, a metric (e.g., correlation) is computed; an SW is declared if the computed metric is greater than a proper threshold, otherwise the observation window is time-shifted one sample. We assume a Gaussian channel, antipodal signaling, equally distributed data symbols, and coherent detection, where soft values are provided to the frame synchronizer. We state the problem starting from the hypothesis testing theory, deriving the optimum metric [optimum likelihood ratio test (LRT)] according to the Neyman-Pearson lemma. When the data distribution is unknown, we design a simple and effective test based on the generalized LRT (GLRT). We also analyze the performance of the commonly used correlation metric, both with "hard" and "soft" values at the synchronizer input. We show that synchronization can be greatly improved by using the LRT and GLRT metrics instead of correlation and that, among correlation-based tests, sometimes hard correlation is better than soft correlation. The obtained closed-form expressions allow the derivation of the receiver operating characteristic (ROC) curves for the LRT and GLRT synchronizers, showing a remarkable gain with respect to synchronization based on correlation metric.

Abstract — In this paper we study frame synchronization based on sync word in the general case when the frame size is unknown and possibly variable on a frame by frame basis. We assume a Gaussian channel with binary symbols and coherent decoding, where soft values are provided to the frame synchronizer. Ac-cording to the hypothesis testing theory, the optimum likelihood ratio test (LRT) is derived using the Neyman-Pearson lemma. Then, the performance of the test is analytically assessed. The obtained closed form expressions allow also the derivation of the receiver operating characteristic (ROC) curves for the LRT synchronizer, showing that a remarkable gain is obtained with respect to synchronization based on correlation metric. I.

Future wireless video transmission systems would consider OFDM (orthogonal frequency division multiplexing) as basic modulation technique due to its robustness and low complexity implementation in the presence of frequency selective channels. Recently, adaptive bit loading techniques have been applied to OFDM showing good performance gains in cable transmission systems. In this paper adaptive loading techniques are applied to an OFDM wireless system in the 5 GHz band for efficient layered multimedia traffic transmission. A low complexity scheme is introduced and compared with other solutions to realize unequal error protection both at coding and modulation level. The large impact of this technique in terms of video quality is evaluated for MPEG-4 video transmission.

Future wireless video transmission systems will consider OFDM (orthogonal frequency division multiplexing) as the basic modulation technique due to its robustness and low complexity implementation in the presence of frequency selective channels. Adaptive bit loading techniques have been applied to OFDM showing good performance gains in cable transmission systems. In this paper adaptive loading techniques are applied to a HIPERLAN2-like wireless system at 5 GHz for efficient multimedia traffic transmission. Moreover, the classical water-filling algorithm is extended to the multi-layer case. The semi-analytical results obtained show a large improvement respect to the non-adaptive case, also taking different channel state information up-date rates into account. The impact of this technique in terms of video quality is also evaluated for MPEG-4 video transmission.

Robust video transmission over wireless channels is very critical due to channel impairments and to their effect on the compressed bitstream. Error detection is a clue point as, if errors are detected, their concealment may be performed. In spite of that, data assisted error detection is often not sufficient to guarantee error detection is correctly performed in any case. A joint source-channel error detection approach is thus proposed in the paper, where standard compatibility is kept, as the source decoder needs not to be modified in order to exploit the proposed technique. This technique may be useful in any case of packet transmission where data assisted error detection is particularly difficult to be performed for some packet partition and compatibility to a standard is required. With the proposed technique, information about the source coded bitstream may be exploited in order to reduce the redundancy added and the amount of data to be discarded. In particular, the case of MPEG-4 video transmission is analyzed in the paper.

Considering the limitations due to channel impairments on the transmission of MPEG-4 video over error prone channels, a technique to apply unequal error protection (UEP) for the MPEG-4 video bitstream is proposed, where UEP is performed through rate compatible punctured convolutional codes (RCPC), with rates chosen according to the perceived importance of bits. The data partitioning tool available in the MPEG-4 standard is exploited, in order to provide a stronger protection for the most significant partitions. The main problem in the application of such a scheme is the fact that packets, like partitions, are not the same length, thus the UEP scheme should be dynamically changed for each packet and the knowledge of each partition length is required. In order to cope with this problem, proportional unequal error protection (P-UEP) is proposed: as the length of each field is not known at the receiver, a proportional scheme may be used, given the length of the packet. The possibility of considering different code rates for the protection of different frames is also proposed: I frames and P frames may be coded with different rates according to their different structure and sensitivity to channel errors.