Holographic-type communication demands multi-Gbps throughput and ultra-low latency, posing significant challenges to current 5G networks. This demo work presents a programmable testbed built on the P7 network emulator, which overcomes the throughput constraints of conventional platforms. Our demonstration deploys three edge computing slices—one at a far edge and two at near edges—to support volumetric media streaming directly to VR head-mounted display. By facilitating precise adjustment of network metrics such as latency and packet loss across the P7-enabled slices, the testbed allows users to visually observe the resulting effects on holographic content in VR. Although automated measurement of Quality of Experience (QoE) parameters are not incorporated, the testbed provides a valuable experimental environment where researchers can subjectively evaluate the relationship between network conditions and perceived quality, thereby gaining insights into optimizing network performance for enhanced Quality of Service (QoS).

Demand for wireless video streaming services increases with users expecting to access high-quality video streaming experiences. Ensuring Quality of Experience (QoE) is quite challenging due to varying bandwidth and time constraints. Since most of today’s mobile devices are equipped with multiple network interfaces, one promising approach is to benefit from multipath communications. Multipathing leads to higher aggregate bandwidth and distributing video traffic over multiple network paths improves stability, seamless connectivity, and QoE. However, most of current transport protocols do not match the requirements of video streaming applications or are not designed to address relevant issues, such as networks heterogeneity, head-of-line blocking, and delay constraints. In this comprehensive survey, we first review video streaming standards and technology developments. We then discuss the benefits and challenges of multipath video transmission over wireless. We provide a holistic literature review of multipath wireless video streaming, shedding light on the different alternatives from an end-to-end layered stack perspective, reviewing key multipath wireless scheduling functions, unveiling trade-offs of each approach, and presenting a suitable taxonomy to classify the state-of-the-art. Finally, we discuss open issues and avenues for future work.

Point cloud compression has been studied in standard bodies and we are here concerned with the Moving Picture Experts Group video-based point cloud compression (V-PCC) solution. Plenoptic point clouds (PPC) is a novel volumetric data representation wherein points are associated with colors in all viewing directions to improve realism. It is sampled as a number ($N_{c}$) of attribute colors per point. We propose a new method for the efficient video-based compression of PPC that is backwards compatible with the existing single-color V-PCC decoder. V-PCC generates three image atlases which are encoded using an image/video encoder. We assume there may be a reference color which is to be encoded as the main payload. We generate$N_{c}+3$atlases and we produce$N_{c}$differential images against the reference color image. Those difference images are pixel-wise transformed using an$N_{c}$-point discrete cosine transform, generating$N_{c}$transformed atlases which are encoded, forming the secondary payload. Such secondary information is the plenoptic enhancement to the point cloud. If there is no reference attribute, we skip the differences and use the lowest frequency of the transformed atlases as the main payload. Results are presented that show an unrivaled performance of the proposed method.

Recently finalized, the MPEG Video-based Point Cloud Compression (V-PCC) standard leverages existing video codecs to compress point clouds. This approach relies on existing video coding hardware accelerators to enable its fast adoption. However, the processing steps to project 3D point clouds into 2D frames still have a considerable complexity. Thus, we propose two modifications to TMC2, V-PCC’s test model, considering the memory access pattern: one reducing unnecessary memory allocation and the other increasing data locality through a set of pre-processing steps. Our approach achieved up to 46.31% encoding self-time reduction without changing the resulting bitstream. This paper also provides an analysis of our implementation that may help future V-PCC codecs achieve real-time encoding.

In recent years image compression techniques based on deep learning have achieved great success and their performances are gradually reaching the methods crafted by experts, such as JPEG, WebP, and Better Portable Graphics (BPG). A technique that is fundamental for modern image and video codecs is intra prediction, which takes advantage of local redundancy to predict the pixels from previously encoded neighbors. In this paper, we use Convolutional Neural Networks (CNN) to develop a new intra-picture prediction mode. More specifically, we propose a multi-mode intra prediction approach that uses two CNN-based prediction modes and all intra modes previously implemented in the High Efficiency Video Coding (HEVC) standard. We also propose a bit allocation technique that increases the bitstream only if the reconstruction error is significantly reduced. Experimental results evince a significant and consistent performance increase compared to other approaches that use a similar backbone architecture, with 28% bitrate reduction compared to the baseline codec.

This work investigates a distortion scalable learned image encoder that, differently from other works, only encodes those most prominent feature maps that are found to benefit rate-distortion performance. This way, better suiting the texture complexity of content being coded and providing a competitive distortion scalable solution. Experimental results show that the proposed solution performs comparably to HEVC Intra with respect to the perceptually-inspired structural similarity (SSIM) index while being able to be optimized to suit a particular application.

Bandwidth constraints and high end-to-end delays are real challenges for achieving and sustaining high quality mobile video streaming services. Diverse multipath transmission techniques are being investigated as possible solutions, since recent developments have enabled mobile devices users to receive video data simultaneously over multiple interfaces (e.g., LTE and WiFi). While some multipath protocols have been recently standardized for this purpose (e.g., MPTCP), being network layer protocols they cannot properly handle challenging transmission scenarios subject to packet losses and congestion, such as lossy wireless channels. In this work, we adopt the MPEG Media Transport (MMT) protocol to propose an improvement for mobile multipath video streaming solutions. MMT is an application layer protocol with inherent hybrid media delivery properties. We propose a novel path-and-content-aware scheduling strategy for MMT by means of full cooperation between network metrics and video content features. Our strategy provides better models to adaptively cope with unstable communication channel conditions and to improve the final user quality of experience (QoE). For the experimental evaluation, we used NS3-DCE to simulate a realistic multipath network scenario which includes channel error models and background traffic. Results for two video sequences are presented in terms of PSNR, SSIM, goodput, delay and packet loss rates. When compared with a simple scheduling strategy for the traditional multipath MMT, our approach yields significant packet loss rate reductions (~90%) and video quality improvements of around 12 dB for PSNR and 0.15 for SSIM.

In this work we study the 4D sparsity of light fields using as main tool the 4D- Discrete Cosine Transform. We analyze the two JPEG Pleno light field datasets, namely the lenslet-based and the High-Density Camera Array (HDCA) datasets. The results suggest that the lenslets datasets exhibit a high 4D redundancy, with a larger inter-view sparsity than the intra-view one. For the HDCA datasets, there is also 4D redundancy worthy to be exploited, yet in a smaller degree. Unlike the lenslets case, the intra-view redundancy is much larger than the inter-view one. The results and conclusions of this first study for light field imaging may have a strong impact on the current and future design of efficient codecs for this type of emerging data, notably in the context of the JPEG Pleno standard.

Light fields aim to represent visual information in 3D space. They are 4D structures that contain the images of a given scene from a sampled 2D range of viewpoints. When acquired using a lenslet camera, in addition to the ordinary intra-view redundancy, these views have a great deal of inter-view redundancy. In this work we propose a light field codec that fully exploits the 4D redundancy of light fields by using a 4D transform and hexadeca-trees. It initially divides the light field into 4D blocks and computes a 4D Discrete Cosine Transform of each one. Then the transform coefficients of the 4D block are grouped using hexadeca-trees on a bitplane-by-bitplane basis, and the generated stream is encoded using an adaptive arithmetic coder. The proposed codec has been employed to encode the JPEG Pleno lenslet light fields. The rate-distortion results have been assessed using test conditions comparable to the ones presented at the ICIP 2017 Light Field Coding Grand Challenge. The proposed codec, despite being conceptually simple, achieves competitive rate-distortion performance.

Automatically detecting violence in videos is paramount for enforcing the law and providing the society with better policies for safer public places. In addition, it may be essential for protecting minors from accessing inappropriate contents on-line, and for helping parents choose suitable movie titles for their children. However, this is an open problem as the very definition of violence is subjective and may vary from one society to another. Detecting such nuances from video footages with no human supervision is very challenging. Clearly, when designing a computer-aided solution to this problem, we need to think of efficient (quickly harness large troves of data) and effective detection methods (robustly filter what needs special attention and further analysis). In this vein, we explore a content description method for violence detection founded upon temporal robust features that quickly grasp video sequences, automatically classifying violent videos. The used method also holds promise for fast and effective classification of other recognition tasks (e.g., pornography and other inappropriate material). When compared to more complex counterparts for violence detection, the method shows similar classification quality while being several times more efficient in terms of runtime and memory footprint.

We consider the joint source-channel coding problem of a 3D video transmitted over an AWGN channel. The goal is to minimize the total number of bits, which is the sum of the number of source bits and the number of forward error correction bits, under two constraints: the quality of the primary view and the quality of the secondary view must be greater than or equal to a predetermined threshold at the receiver. The quality is measured in terms of the expected PSNR of an entire decoded group of pictures. A MVC (multiview coding) encoder is used as the source encoder, and rate compatible punctured turbo codes are utilized for protection of the encoded 3D video over the noisy channel. Equal error protection and unequal error protection are compared for various 3D video sequences and noise levels.

Hierarchical transforms are widely used in image and video coding to produce multilevel decomposition of signals. After applying these transforms, same level signals are typically uncorrelated. However, there is often still significant cross level information. Traditionally, this cross-level information is exploited at the entropy coding step, but not at the transform step. The main contribution of this work is an energy compaction technique/transform that can also exploit these cross-resolution-level structural similarities. The core idea of the technique is to include in the hierarchical transform a number of adaptive basis functions derived from the lower resolution of the signal. A full image codec was developed in order to measure the performance of the new transform. Results are presented in terms of transform coding gain, energy concentration and distortion versus rate curves compared with standard JPEG, JPEG 2000 and JPEG XR.

We explore several local and global strategies for adaptive scan ordering of transform coefficients in JPEG XR/HD Photo. This codec applies a global adaptive scan-order heuristic with respect to an effective localized predictor. The global ordering heuristic, although simple, performs as well as localized techniques that are computationally significantly more complex. We conclude that effective localized prediction not only minimizes but also essentially randomizes coefficient residuals, so that a global statistic is sufficient to deliver near-optimal compression performance.

In this paper we analyze practical issues related to an adversarial inversion of biometric templates constructed without any cryptographically-secure protection features. The inversion in most cases is considered an ill-defined problem as the template captures only a small subset of the physical presence of a specific biometric trait. We apply our practical approach to an existing iris-based biometric system and demonstrate that “inverted” iris images pass matching tests even when encoded like barcodes.