All-Intra coding is a video compression method used in recent codecs where each frame in a YUV sequence is compressed independently. This approach enables high editing precision, random frame access, and low-latency processing, making it ideal for video editing, digital cinema, surveillance, medical imaging, and live production workflows. Recent work showed that multidimensional transpose of YUV sequences prior to encoding increases homogeneity of the frames and therefore reduces the bitrate. Such a solution is sequence-dependent and therefore, it is not suitable for all YUV sequences. In this work, we propose a classification approach in which the suitability of YUV sequences for multidimensional transpose is assessed prior to committing to full encoding. Features are first extracted from the sequences at various resolutions ranging from $\mathbf{4} \boldsymbol{\times} \mathbf{4}$ to $\mathbf{6 4 \times 6 4}$ pixels. The features include the statistical entropy and measures of dispersion such as variances and interquartile ranges. The proposed multi-scale feature extraction is inspired from the recursive quadratic split of coding units in video compression. Consequently, video frames are classified individually, and the video they belong to is labeled based on a majority vote of the frame classifications. Tested using a leave-one-sequence-out strategy on video sequences with different resolutions and textures, the proposed classification solution successfully distinguishes sequences that are suitable for multidimensional transpose prior to encoding. It is also shown that the proposed multi-resolution feature extraction solution is more suitable for this classification problem than existing CNNs. Furthermore, it is shown that the computational time of the proposed solution is around third of what is required for a lightweight first pass encoding as reported in the literature.

Gait recognition is gaining popularity as it can recognize people in a non-intrusive and a non-contact manner. However, gait recognition is known for its susceptibility to clothing conditions. In this paper, we propose a solution specific to clothing conditions in the Gulf region where Abaya and Kandura are considered traditional clothing. The paper proposes a solution capable of training users based on traditional clothing and recognizing them in Western style clothing and vice-a-versa. The solution uses depth imaging, optical flow, accumulated motion and Discrete Cosine Transformation (DCT). Motion is calculated from consecutive images where the magnitudes and phases of motion vectors are accumulated into separate matrices. DCT and zonal coding is then applied to these matrices to form one concise feature vector that represents a walk. Experimental results, with 38 participants, showed that the proposed method is suitable for gait recognizing with such clothing constraints. The average classification accuracy is 88%. In comparison to an existing method, it is shown that the proposed method results in much more accurate recognition results yet at a higher computational cost.

A data embedding solution in HEVC videos is proposed by modifying the partitioning of coding units (CUs). The partitions of a CU are first represented as a sequence of binary flags. The flags pertaining to 16 × 16 sub CUs are used as a cover for data embedding, where 6 or 4 message bits are embedded per CU. The data embedding algorithm guarantees that a maximum of one partition is modified per message segment, therefore, in a given CU, either 0, 1 or 2 partitions are modified. The proposed solution is assessed in terms of message payload, number of modified partitions, loss in video quality as indicated by the PSNR results, mean objective scores and excessive bitrate. The proposed solution can embed messages with up to an average payload of 32.6 kbit/s with a corresponding average distortion of <0.5 dB. Comparisons with existing solutions reveal that the proposed solution maintains similar message payloads with less modifications of CU partitioning and at the same time resulting in less distortions for the cover video.

In this paper, we propose a glove-based Arabic sign language recognition system using a novel technique for sequential data classification. We compile a sensor-based dataset of 40 sentences using an 80-word lexicon. In the dataset, hand movements are captured using two DG5-VHand data gloves. Data labeling is performed using a camera to synchronize hand movements with their corresponding sign language words. Low-complexity preprocessing and feature extraction techniques are applied to capture and emphasize the temporal dependence of the data. Subsequently, a Modified k-Nearest Neighbor (MKNN) approach is used for classification. The proposed MKNN makes use of the context of feature vectors for the purpose of accurate classification. The proposed solution achieved a sentence recognition rate of 98.9%. The results are compared against an existing vision-based approach that uses the same set of sentences. The proposed solution is superior in terms of classification rates while eliminating restrictions of vision-based systems.

The new video coding standard, High Efficiency Video Coding (HEVC), was developed to succeed the current standard, H.264/AVC, as the state of the art in video compression. However, there is a lot of legacy content encoded with H.264/AVC. This paper proposes and evaluates several transcoding algorithms from the H.264/AVC to the HEVC format. In particular, a novel transcoding architecture, in which the first frames of the sequence are used to compute the parameters so that the transcoder can learn the mapping for that particular sequence, is proposed. Then, two types of mode mapping algorithms are proposed. In the first solution, a single H.264/AVC coding parameter is used to determine the outgoing HEVC partitions using dynamic thresholding. The second solution uses linear discriminant functions to map the incoming H.264/AVC coding parameters to the outgoing HEVC partitions. This paper contains experiments designed to study the impact of the number of frames used for training in the transcoder. Comparisons with existing transcoding solutions reveal that the proposed work results in lower rate-distortion loss at a competitive complexity performance.

The emerging video coding standard, HEVC, was developed to replace the current standard, H.264/AVC. However, in order to promote inter-operability with existing systems using the H.264/AVC, transcoding from H.264/AVC to the HEVC codec is highly needed. This paper presents a transcoding solution that uses machine learning techniques in order to map H.264/AVC macroblocks into HEVC coding units (CUs). Two alternatives to build the machine learning model are evaluated. The first uses a static training, where the model is built offline and used to transcode any video sequence. The other uses a dynamic training, with two well-defined stages: a training stage and a transcoding stage. In the training stage, full re-encoding is performed while the H.264/AVC and the HEVC information are gathered. This information is then used to build a model, which is used in the transcoding stage to classify the HEVC CU partitioning. Both solutions are tested with well-known video sequences and evaluated in terms of rate-distortion (RD) and complexity. The proposed method is on average 2.26 times faster than the trivial transcoder using fast motion estimation, while yielding a RD loss of only 3.6% in terms of bitrate.

This paper proposes an efficient MPEG-2 to High Efficiency Video Coding (HEVC) video transcoder. The objective of the transcoder is to migrate the abundant MPEG-2 video content to the emerging HEVC video coding standard. The transcoder introduces a content-based machine learning solution to predict the depth of the HEVC coding units. The proposed transcoder utilizes full re-encoding to find a mapping between the incoming MPEG-2 coding information and the outgoing HEVC depths of the coding units. Once the model is built, a switch to transcoding mode occurs. Hence, the model is content based and varies from one video sequence to another. The transcoder is compared against full re-encoding using the default HEVC fast motion estimation. Using HEVC test sequences, it is shown that a speedup factor of up to 3 is achieved, while reducing the bitrate of the incoming video by around 50%. In comparison to full re-encoding, an average of 3.9% excessive bitrate is encountered with an average PSNR drop of 0.1 dB. Since this is the first work to report on MPEG-2 to HEVC video transcoding, the reported results can be used as a benchmark for future transcoding research.

This paper proposes two data hiding approaches using compressed MPEG video. The first approach hides message bits by modulating the quantization scale of a constant bitrate video. A payload of one message bit per macroblock is achieved. A second order multivariate regression is used to find an association between macroblock-level feature variables and the values of a hidden message bit. The regression model is then used by the decoder to predict the values of the hidden message bits with very high prediction accuracy. The second approach uses the flexible macroblock ordering feature of H.264/AVC to hide message bits. Macroblocks are assigned to arbitrary slice groups according to the content of the message bits to be hidden. A maximum payload of three message bits per macroblock is achieved. The proposed solutions are analyzed in terms of message extraction accuracy, message payload, excessive bitrate and quality distortion. Comparisons with previous work reveal that the proposed solutions are superior in terms of message payload while causing less distortion and compression overhead.

In this paper, we propose a novel biometric based on utilizing microwave images constructed from measuring a voltage that is related to the properties of the reflected microwave signal. The proposed technique requires collecting scans of the users' hands using an open-ended rectangular waveguide operating at a frequency of 9.6 GHz in the near-field region. The microwave scan reflects the dielectric properties of the hand affected by the thickness, shape and structure of the hand at every point in the scan. The proposed technique is safe as it uses low power microwave signals in the range of 10-20 dBm. An in-house microwave hand-image database is collected using a microwave scanning system. The database is comprised of 370 images collected from 37 users. Radon and DCT transformations are used to extract distinguishing features from microwave images. Classification is done with polynomial classifiers. In the context of the polynomial classifier, a variety of dimensionality reduction schemes are examined. Amongst these schemes, spectral regression is found to give the best results in terms of accuracy, dimensionality of feature space and classifier stability. A user identification rate of 94.7% was achieved with a feature vector size of 36 elements only.

This letter proposes a multipass system for predicting the Structural Similarity Indices (SSIM) of compressed video. The prediction is performed in the absence of the original reference and it is carried out on macroblock basis. Distinguishing macroblock features are extracted from compressed bit streams and reconstructed videos. Optimal feature sets are then computed using stepwise regression. The multipass system uses reduced model polynomial regression for quality prediction. At intermediate passes, the predicted SSIM indices are concatenated to the original set of features allowing for higher prediction accuracy. The polynomial weights used for predicting the SSIM indices are computed in the final pass. Comparisons with existing work reveal that the proposed solution is more consistent and offers higher prediction accuracy.

This letter proposes a no-reference MacroBlock (MB) level PSNR identification of compressed MPEG video. Features are extracted on MB basis from video bitstreams and reconstructed images. The identification problem is formalized using reduced model polynomial networks. The letter proposes a two-step identification solution in which supervised spectral regression is used to reduce the dimensionality of the feature vector prior to model estimation. Two identification scenarios are presented in the experimental results, namely, video sequence dependent and video sequence independent identification. Based on the various video sequences used in the experiments, it is shown that the average mean absolute difference between the actual and the identified PSNRs is 1 dB and 1.6 dB for the sequence dependent and sequence independent identification respectively.

This paper presents an extension to video error resiliency based on distributed source coding. The paper proposes a frequency domain transcoder and a compressed domain transcoder for boosting the error resiliency of pre-encoded video using distributed source coding techniques. In both transcoders it is proposed to identify regions of interest in the pre-encoded video. In the compressed domain transcoder, the regions of interests are retained as is, whilst in the frequency domain transcoder, such regions are reproduced at a lower fidelity by requantization or ideal low pass filtering. Channel coding follows, and the parity bits are transmitted in an error resiliency supplementary bitstream. At the destination, the paper proposes a pre-decoding solution that regenerates the low fidelity ROIs and corrects the received bit stream using the aforementioned supplementary stream. The corrected bitstream is then repacked and sent to the destination decoder. In comparison with traditional FEC techniques, experimental results show robust resiliency against packet losses up to a loss ratio of 15%.

This paper proposes a two tier feature extraction approach for the recognition of video-based isolated Arabic sign language gestures. In the first tier, the prediction error of the image sequence is binarized and collapsed into two unidirectional accumulated differences images. In the second tier of feature extractions, two approaches are applied to the accumulated differences images: frequency domain transformation, and radon transformation. We apply such feature extractions on each of the accumulated differences images and then concatenate the resultant feature vectors. Alternatively, the accumulated differences images are concatenated prior to the second tier of feature extractions. The paper reports on the classification results of both solutions using Fisher's linear discriminants. Comparisons with existing work reveal that up to 39% of the misclassifications have been corrected.

This paper presents various spatio-temporal feature-extraction techniques with applications to online and offline recognitions of isolated Arabic Sign Language gestures. The temporal features of a video-based gesture are extracted through forward, backward, and bidirectional predictions. The prediction errors are thresholded and accumulated into one image that represents the motion of the sequence. The motion representation is then followed by spatial-domain feature extractions. As such, the temporal dependencies are eliminated and the whole video sequence is represented by a few coefficients. The linear separability of the extracted features is assessed, and its suitability for both parametric and nonparametric classification techniques is elaborated upon. The proposed feature-extraction scheme was complemented by simple classification techniques, namely, K nearest neighbor (KNN) and Bayesian, i.e., likelihood ratio, classifiers. Experimental results showed classification performance ranging from 97% to 100% recognition rates. To validate our proposed technique, we have conducted a series of experiments using the classical way of classifying data with temporal dependencies, namely, hidden Markov models (HMMs). Experimental results revealed that the proposed feature-extraction scheme combined with simple KNN or Bayesian classification yields comparable results to the classical HMM-based scheme. Moreover, since the proposed scheme compresses the motion information of an image sequence into a single image, it allows for using simple classification techniques where the temporal dimension is eliminated. This is actually advantageous for both computational and storage requirements of the classifier.

This paper proposes a novel multilayer video transcoding approach for multicasting pre-encoded video to heterogeneous end-systems via diverse grouping of networks. Multilayer transcoding is first addressed by means of multiquality or SNR scalability of the MPEG-2 standard. Frequency domain transcoding and drift-compensated transcoding are derived from the closed-loop and multiloop SNR scalabilities, respectively. The proposed transcoding architectures are verified in terms of eliminating picture drift whilst preserving compatibility with the MPEG-2 SNR decoder. Multilayer transcoding is then addressed by means of multiresolution or spatial scalability of the MPEG-2 standard that supports different video formats. The transcoder retains the full resolution of the incoming video stream in its enhancement layer while generating a low spatio-temporal resolution base-layer compatible with the H.263 video format. Hence providing both multilayer transcoding and video format portability. The resultant video layers are shown to be free from drift with PSNR results comparable to those of the respective scalable encoders.

In this paper, the motion parameters of the bidirectionally predicted pictures (B-pictures) of MPEG-1,2 are exploited for concealment of large portions of corrupted anchor pictures (and vice versa) that might arise due to channel errors or packet losses. To further enhance the quality of the concealed pictures, we propose two methods of constraining the motion vectors of the B-pictures that strengthen the tie between them and those of the anchor pictures in the same picture subgroup. In one method, the macroblock decisions on the last B-picture in each subgroup is constrained to be bidirectional if those of the other B-pictures are not, such that the derived motion vectors for the concealment of the anchor picture are always composed from the forward and backward motion vectors of the bidirectional motions. Second, the bidirectional motion vectors of the B-pictures in each subgroup is constrained such that the vectorial sum of their forward and backward motion vectors results in an accurate motion prediction of the anchor picture. The experimental results show that while the composed motion vectors improve the quality of concealment over the conventional methods by more than 3-4 dB, another 2 dB improvement can be achieved by constraining the generation of the bidirectional motion vectors.

The work carried out by the authors on heterogeneous video transcoding (see Shanableh, T. and Ghanbari, M., IEEE Trans. on Multimedia, vol.2, no.2, p.101-10, 2000) is extended to the DCT domain. The heterogeneous video transcoder was mainly proposed for achieving portability between different encoding formats, namely, from MPEG-1/2 into H.261/3 with lower spatio-temporal resolutions. The motion compensation (MC) loops of the decoder-encoder cascade are now simplified by eliminating the DCT/IDCT pairs whilst performing picture decimation and MC in the DCT domain (MC-DCT). Two transcoding architectures are proposed, in one, both the MC-loops are in the DCT-domain, while in the other, it is shown that by implementing the encoder's MC-loop in the pixel domain higher flexibility and better results are achieved. Methods for accelerating the MC-DCT are then divided into two broad categories of lossless and lossy methods. It is shown that by utilising shared information among successive motion compensated macroblocks, better results over those reported in the literature are achieved.

Video streaming over a hostile environment such as the Internet, where the available bandwidth varies in time, requires the source to adjust its transmission rate dynamically. Moreover, inevitable packet losses cannot be totally avoided and, hence, the video streaming schemes should be complemented with appropriate error concealment techniques. We highlight the vitality of the bi-directionally predicted pictures in the MPEG coded bitstreams to applications of video streaming and error concealment. In particular, we show how the properties and the motion parameters of such pictures can be exploited for efficient video transcoding into lower resolutions and different encoding formats for pre-encoded video streams. At the decoder side, however, we also emphasize on the important role of the bidirectional prediction on interframe error concealment.

In this paper, transcoding of pre-encoded MPEG-1,2 video into lower bit rates is realized through altering the coding algorithm into H.261/H.263 standards with lower spatial resolutions. We extract and compose a set of candidate motion vectors, from the incoming bit stream, to comply with the encoding format of the output bit stream. For the spatial resolution reduction, we generate one motion vector out of a set of input motion vectors operating on the higher spatial resolution image. We discuss the impact of motion estimation refinement on the new motion vectors and show that for all cases a simple half-pixel refinement is sufficient for near-optimum results.

A novel interframe error concealment technique is proposed for video packet losses in IP environments. The proposed technique benefits from the bidirectional motion parameters of bidirectionally predicted pictures 'B-pictures' to conceal large portions (up to 100%) of corrupted anchor pictures. The lost anchor picture is interframe concealed by backward tracking its motion to its anchor picture through the intermediate bidirectional motion of B-pictures. Furthermore, we show that by constraining the motion compensation of the B-pictures to strengthen the tie between their bidirectional motion and that of the future anchor pictures, better concealment of the latter is achieved. The paper then elaborates on the impact of the proposed constrained-encoding in terms of encoding efficiency and concealment quality.

In this work, transcoding of pre-encoded MPEG-1, 2 video into lower bit rates is realized through altering the coding algorithm into H.261/H.263 standards with lower spatio-temporal resolutions. For this heterogeneous transcoding, we extract and compose a set of candidate motion vectors, from the incoming bit stream, to comply with the encoding format of the output bit stream. For the spatial resolution reduction we generate one motion vector out of a set of input motion vectors operating on the higher spatial resolution image. Finally, for the temporal resolution reduction we compose new motion vectors from the dropped frames motion vectors. Throughout the paper, we discuss the impact of motion estimation refinement on the new motion vectors and show that for all cases a simple half-pixel refinement is sufficient for near-optimum results.