A new reference-free, objective, video quality prediction model that takes into account video content type to predict the quality of streamed high efficiency video coding (HEVC) encoded video sequences is proposed. Research has shown that for the same encoder settings and network quality of service (NQoS), the video quality differs for different types of video content . This indicates that, in addition to encoder settings and NQoS, there may be other key parameters that impact video quality. In this work, we hypothesized that video content type is one of the key parameters that may impact the quality of streamed videos. Based on this assertion, temporal information is extracted from the motion vector (MV) information inherent in the encoded video bitstreams and spatial information is extracted from the quantisation parameter (QP) and the number of bits (Bits) of coded intra (I) and predictive (P) frames to develop a metric that quantifies the content type of different video sequences . The content type metric is subsequently used together with encoding QP setting and network packet loss rate (PLR) to develop a reference -free objective video quality prediction model for streamed HEVC encoded video sequences. This model has an accuracy of 92% when the model predicted values of sequences not used in model derivation are compared with mean opinion score (MOS) obtained through subjective method.

Recently, crowdsourcing has emerged as a cheaper and quicker alternative to traditional laboratory based Quality of Experience (QoE) evaluation for video streaming services. Crowdsourcing is a process of recruiting anonymous members of the public to solve/perform different tasks without supervision. This approach of seeking solutions from the public has been enhanced by ubiquitous internet connectivity, low cost and the ease to recruit workers without geographical restrictions. Although crowdsourcing makes it possible to save cost and reach a large group of people to perform subjective video quality testing, challenges such as validity of results and the trustworthiness of workers still remain unresolved. In this paper, we attempt to address some of these challenges by developing a screening algorithm that is able to determine the validity and trustworthiness of crowdsourcing video QoE evaluation results. This algorithm is based on evaluator's extracted data (e.g. Network IP addresses, network devices and browser information, time spent on the crowdsourcing platform, date and grading patterns). To determine the performance of this algorithm, we carried out a separate controlled laboratory based subjective video quality testing. This test enables us to determine how screened crowdsourcing results correlate with lab based results. Preliminary results show that crowdsourcing results can be improved by up to 59% when the proposed screening algorithm is applied. Moreover, the results support our assertion that crowdsourcing can provide reliable video QoE measurements if proper screening is performed on the test results.

In the past, many video quality assessment methods have been proposed to predict video quality non-intrusively from different application and/or network related parameters. One of the important parameters which have been identified to have a significant impact on video quality is the video content type. To determine video content type, researchers have used different methods such as the extraction of video motion features from the encoded bitstream, the calculation of pixel-wise differences between consecutive frames and human vision systems. In this paper, we developed a new motion amount (MA) metric for defining and determining video content type for the newly released HEVC encoding standard. This metric is based on a combination of the counting number of extracted motion vectors from an encoded bitstream, the magnitude and the complexity of video sequences. Accuracy of the metric was determined by performing a direct correlation between MA, video bitrate and Peak Signal-to-Noise Ratio (PSNR). We also performed a performance comparison with existing pixel-wise approach of defining video content type. Preliminary results show that our metric outperformed the pixel-wise approach of content type definition. Based on the new metric, we further developed an HEVC encoded video quality prediction model that takes into account the MA and the Quantization Parameter (QP). We achieved an accuracy of 96% when our model predicted PSNR values are compared with full reference PSNR measurement.