The next generation Audio Video Coding Standard (AVS4) introduces enhanced intra prediction tools to further improve spatial coding efficiency. To improve directional adaptability for luma prediction and enhance cross-component correlation modeling for chroma, this paper proposes an intra prediction mode optimization method based on directional enhancement and cross-component fusion.

This paper aims to reduce the complexity of encoding screen content videos using Versatile Video Coding (VVC). To address the significant increase in encoding time caused by Screen Content Coding (SCC) tools in VVC, we propose a fast intra mode decision algorithm based on Adaptive Multi-Size Convolutional Neural Network (AMS-CNN).

The Geometric Partitioning Mode (GPM) in Versatile Video Coding (VVC) significantly enhances inter prediction efficiency by employing non-rectangular partitions. However, the Motion Vectors (MVs) for these irregular sub-partitions still follow the derivation of rectangular blocks, which often leads to weakly correlated MV candidates and motion mismatch. To address this issue, we propose a unified Geometric Centroid-based Correlation Model (GCCM) to quantify the correlation between GPM sub-partitions and their neighboring regions. By jointly considering geometric centrality and boundary connectivity, GCCM provides an effective metric to guide motion derivation for irregular sub-partitions. Based on GCCM, we redesign the Temporal MV Prediction (TMVP) derivation at the sub-partition level using the most correlated collocated position, and further perform correlation-guided Template Matching (TM) for MV refinement. Experimental results on VTM-12.0 demonstrate that the proposed method achieves 1.94% and 1.14% bit rate savings on Low-delay B and Random Access configurations, respectively.

360$^\circ$video streaming emerges as an innovative video presentation form that offers users an immersive and interactive experience, where the quality of experience (QoE) is a vital indicator to measure user viewing perception. In multicategory 360$^\circ$video streaming, existing QoE-driven approaches typically assume a fixed request distribution to enhance users' average QoE, prioritizing the optimization of edge caching and bitrate selection decisions for the video category with a larger request number. Inevitably, these unfair approaches would lead to average QoE reduction in real-world scenarios, in which the request distribution exhibits significant variations and is challenging to predict accurately. To this end, we propose a fairness-aware 360$^\circ$video streaming strategy in cloud-edge collaboration networks for improving users' average QoE. Specifically, we first formulate the joint edge caching and bitrate selection problem as a multi-agent cooperative input-driven Markov decision process to maximize users' average QoE and guarantee QoE fairness for users. Subsequently, we devise an adaptive learning-based multi-agent deep reinforcement learning (MADRL) approach, which can adaptively adjust the learning rate of each agent according to the dynamic user request distribution, thus helping agents make optimal decisions. Finally, experimental results on real-world datasets show that the proposed algorithm significantly improves users' average QoE while ensuring QoE fairness for users.

Versatile Video Coding (VVC) inherits Screen Content Coding (SCC) tools such as Intra Block Copy (IBC) and Palette mode (PLT) from High Efficiency Video Coding Screen Content Coding (HEVC-SCC), which is known as VVC-SCC. VVC-SCC can effectively improve the efficiency of screen content encoding, but it can also lead to higher encoding complexity. In order to reduce the encoding complexity of VVC-SCC, we design a Hierarchical Feature Fusion Convolutional Neural Network (HFF-CNN) for predicting the current CU best intra prediction mode. The encoder determines the current CU intra prediction mode based on the network out put best prediction mode, angle intra prediction mode indexes, and adjacent CU mode probability, skipping unnecessary rate distortion cost calculations and speeding up the encoding process. Experimental results show that the proposed model reduces the intra frame encoding time of VCC-SCC by 36.6% while increasing the average BDBR by 0.44%. Compared to state-of-the-art algorithms, it exhibits a better balance between the rate distortion performance and the encoding complexity.

The ever-increasing quality of experience (QoE) demand for video streaming has prompted the integration of video super-resolution and multi-access edge computing networks (MEC). With super-resolution, the low-resolution frames can be reconstructed into high-resolution ones by edge node and end device collaboratively, which is beneficial in improving QoE. However, the existing works focus on designing video streaming strategies in single-user scenarios, which cannot be applied to multi-user scenarios due to the resource contention among users, as well as the huge solution space of coupled bitrate selection and workload share between edge-end. To fill this gap, we propose a collaborative video streaming strategy with super-resolution in multi-user MEC networks, named Co-Video, to maximize the average QoE by making optimal bitrate selection and workload share. We first formulate the problem as an optimization problem towards maximum average QoE, where the QoE incorporates playback delay, video quality, and smoothness. Then, we transform the optimization problem into a partially observable Markov decision process (POMDP) and exploit the Co-Video strategy based on the multi-agent soft actor-critic (MASAC) algorithm. Specifically, Co-Video utilizes the branching actor network to converge to good policy stably. Finally, trace-driven simulations on real-world bandwidth traces demonstrate that Co-Video outperforms the state-of-the-art baselines.

Joint service caching and workload scheduling plays an important role in ultra-dense mobile edge computing (MEC) networks to satisfy the stringent requirements of latency-critical services by leveraging the aggregated edge resources (e.g., storage and computing resources) located near the users. However, most of the existing methods incorporating popularity-based and/or size-aware caching strategies fail to match the resource demands of user requests with the heterogeneous resources of edge nodes, leading to heavier cloud load and higher latency. It becomes even worse when user requests exhibit dynamic variations over time. To address these issues, we propose an online approach for resource-efficient joint service caching and workload scheduling in ultra-dense MEC networks, called CoShare. The core idea is to fully utilize the heterogeneous resources of the edge layer to further reduce the cloud load and thus the service latency. First, we formulate the joint service caching and workload scheduling problem as a mixed integer nonlinear programming problem with the goal of minimizing the cloud load. Then, an online algorithm is developed to transform this optimization problem into a series of per-slot sub-problems by leveraging Lyapunov optimization. Next, to solve these sub-problems, we design a cacheability-based alternating iterative algorithm utilizing Gibbs sampling, in which the cacheability indicator considers both service resource demands and service popularity. Finally, simulation results show that CoShare can effectively exploit available edge resources to achieve lower cloud loads compared to other strategies.

360° video streaming has gained increasing attraction in the current popular virtual reality, AR, and MR applications, which can provide users with an immersive experience. In tile-based 360° video streaming, edge caching and bitrate selection strategies are jointly designed to improve users’ Quality of Experience (QoE), which incorporates video quality and rebuffer. However, the existing QoE-driven approaches use a unified QoE function to guide the decisions of edge caching and bitrate selection, which neglect the impact of video quality and rebuffer on different categories of 360° videos, thus failing to provide high-average QoE for users. In this article, we propose a multiagent deep-reinforcement-learning-based joint edge caching and bitrate selection strategy for multicategory 360° video streaming to improve users’ average QoE. The key idea is to employ different edge caching and bitrate selection strategies for different video categories to enable fine-grained performance optimization. Based on multicategory 360° video streaming, we first model a joint edge caching and bitrate selection problem as a multiagent cooperative Markov decision process with the goal of maximizing users’ average QoE. Next, an Field-of-View-aware multiagent soft actor–critic (FA-MASAC) algorithm is designed to help agents collaboratively learn optimal edge caching and bitrate selection decisions in a distributed way, in which each video category is treated as an agent. Finally, experimental results on real-world data sets show that our proposed strategy can greatly benefit users’ average QoE compared to existing strategies.

360° video streaming, as an immersive and interactive form of video presentation, has consistently strived for higher video quality and lower latency. A lot of effort has been devoted to improving video quality and reducing latency by utilizing the storage or computing resources of the edge/end layer with edge caching and super-resolution (SR) techniques, respectively. However, it has been conspicuously ignored by existing work that fully leveraging the aggregated caching and computing resources of both edge and end layers can further improve video quality and reduce latency. To this end, in this paper, we propose a MADRL-based Edge-Client collaborative Caching and SR (ECCSR) strategy for high-quality low-latency 360° video streaming. First, we construct a Quality of Experience (QoE) function that involves not only video quality, temporal smoothness, and rebuffering time, but also device energy consumption. Subsequently, we formulate the problem of edge-client collaborative caching and SR as a multi-agent cooperative Markov decision process with the goal of maximizing users’ average QoE. Furthermore, to cope with the decision coupling between agents, an adaptive learning-based multi-agent double actors regularized critics (AL-MADARC) algorithm is developed to help agents make optimal collaborative caching and SR decisions. Through extensive experiments using real-world datasets, we show that ECCSR makes a great improvement in users’ average QoE compared to existing strategies.

Nowadays, not only wearable and portable devices but also aerial and ground robots can be made smaller, lighter, cheaper, and thus as large as hundreds of them may form a swarm to participate in a complicated cooperative application, such as searching, rescuing, mapping, and war-battling. Devices and robots in such a swarm have three important features, namely, large number, high mobility and short distance, hence they form a dynamic and dense wireless network. Successful swarm cooperative applications require low latency communications and real-time localization. This paper proposes to use ultra-wideband (UWB) radio technology to implement both functionalities, because UWB is very time-sensitive that an accurate distance can be calculated using the transmission and reception timestamps of data messages. A UWB swarm ranging protocol is designed to achieve simultaneously wireless data communication and swarm ranging that allows a device/robot to compute the distances to all the peer neighbors at the same time. This protocol is designed for dynamic and dense networks, meanwhile it can also be used in various wireless networks and implemented on various types of devices/robots including low-end ones. In our experiment, this protocol is implemented on Crazyflies, STM32 microcontroller powered micro drones, with onboard UWB wireless transceiver chips DW1000. Extensive real-world experiments are conducted to verify the proposed protocol on various performance aspects, with a total of 9 Crazyflie drones in a compact area. The implemented swarm ranging protocol is open-sourced athttps://github.com/SEU-NetSI/crazyflie-firmware

Nowadays, aerial and ground robots, wearable and portable devices are becoming smaller, lighter, cheaper, and thus popular. It is now possible to utilize tens and thousands of them to form a swarm to complete complicated cooperative tasks, such as searching, rescuing, mapping, and battling. A swarm usually contains a large number of robots or devices, which are in short distance to each other and may move dynamically. So this paper studies the dynamic and dense swarms. The ultra-wideband (UWB) technology is proposed to serve as the fundamental technique for both networking and localization, because UWB is so time sensitive that an accurate distance can be calculated using timestamps of the transmit and receive data packets. A UWB swarm ranging protocol is designed in this paper, with key features: simple yet efficient, adaptive and robust, scalable and supportive. This swarm ranging protocol is introduced part by part to uncover its support for each of these features. It is implemented on Crazyflie 2.1 drones, STM32 microcontrollers powered aerial robots, with onboard UWB wireless transceiver chips DW1000. Extensive real world experiments are conducted to verify the proposed protocol with a total of 9 Crazyflie drones in a compact area.

In the tile-based 360-degree video streaming, it is essential to predict future viewport and to allocate higher bitrates to tiles inside the predicted viewport to optimize the Quality of Experience (QoE) of the users. However, the majority of existing work focuses on short-term viewport prediction, which is prone to rebuffering in dynamic network conditions. On the other hand, the recently developed on-policy Deep Reinforcement Learning (DRL)-based bitrate allocation approaches suffer from poor sample efficiency. To address these issues, in this paper we present a tile-based adaptive 360-degree video streaming system, named LS360, which consists of long-term viewport prediction and adaptive bitrate allocation. First, we propose a Long Short-Term Memory (LSTM)-based viewport prediction model to make use of the heatmap feature from all users’ previous movement information and the target user’s fixation movement feature to improve prediction accuracy. Next, we employ the off-policy Soft Actor-Critic (SAC) algorithm to make optimal tile bitrate allocation decisions by taking the predicted long-term viewport, playback buffer, and bandwidth-related information into account. Experiments on real-world datasets demonstrate that LS360 outperforms state-of-the-art streaming algorithms in terms of long-term viewport prediction accuracy and QoE under different bandwidth conditions.

In mobile edge computing (MEC) systems with energy harvesting, the mobile devices are empowered with the energy that harvested from renewable energy sources. On the other hand, mobile devices can offload their computation-intensive tasks to the MEC server to further save energy and reduce the task execution latency. However, the energy harvested is unstable and the mobile devices have to make sure that the energy should not be run out. Moreover, the wireless channel condition between the mobile device and the MEC server is dynamically changing, leading to unstable communication delay. Considering the energy constraints and unstable communication delay, the benefit of computation offloading is limited. In this paper, we investigate D2D-assisted computation offloading for mobile edge computing systems with energy harvesting. In our method, the mobile device is allowed to offload its tasks to the MEC server with the help of its neighbor node. More Specifically, the neighbor node acts as a relay to help the mobile device to communicate with the MEC server. Our goal is to minimize the average task execution time by selecting an optimal execution strategy for each task, i.e., whether to execute the task locally, or offload it to the MEC server directly, or offload it to the MEC server with the help of the most suitable neighbor node, or just to drop it. We propose a low-complexity online algorithm, which stem from Lyapunov Optimization-based Dynamic Computation Offloading (LODCO) algorithm, to solve this problem. Extensive simulations verified the effectiveness of the proposed algorithm, where the average task execution time is reduced around 50% as compared to that of the original LODCO algorithm.