Current congestion control algorithms (CCAs) are inefficient in wireless networks due to the lack of distinction of congestion and wireless packet losses. In this work, we propose a simple yet effective learning-based wireless loss diagnosis (WiLD) solution for enhancing wireless congestion control. WiLD uses a neural network (NN) to accurately distinguish between wireless packet loss and congestion packet loss. To seamlessly cooperate with rule-based CCAs and make real-time decisions, we further implement WiLD in Linux kernel to avoid the frequent kernel-space communication. Specifically, we use a lightweight NN for inference and propose an integer quantization for WiLD deployment in various Linux versions. Real-world experiments and simulations demonstrate that WiLD can accurately differentiate the wireless and congestion packet loss with negligible CPU overhead (around 1% of WiLD vs. around 100% of learning-based algorithms such as Vivace and Aurora) and fast inference time (45% less compared to TensorFlow Lite). When combined with Cubic, WiLD-Cubic can achieve around 792%, 536%, 412%, 231%, 218%, 108%, 85% and 291% throughput improvement compared with BBRv2, Cubic, Westwood, Copa, Copa+, Vivace, Aurora and Indigo in the real network environment.

Existing delay-based congestion control algorithms such as Vegas and Copa can effectively achieve low latency only in scenarios where the number of flows is relatively small. In this paper, we design a novel low delay congestion control (LDCC) algorithm for the Internet to achieve minimal low latency while maintaining high throughput. The LDCC framework first adopts a neural network to predict the current queue length, leveraging all relevant information carried via TCP. Then, we design a source-side algorithm dynamically adjusting the congestion window with the predicted queue length as a congestion signal while keeping the queue length low. We further present a fluid model for the performance analysis of LDCC. Our steady-state analysis reveals that the bottleneck queue length is proportional to the square root of the number of flows$(\sqrt{N})$. In contrast, the bottleneck queue length of other delay-based congestion control algorithms such as Vegas and Copa is much large and proportional linearly to the number of flows ($N$). The advantage of LDCC becomes more significant as the number of flows increases. Both real-world experiments and simulations validate the effectiveness of LDCC. Compared with the latest delay-based congestion control algorithm Copa+, when the number of flows increases from 10 to 1000, the queue delay of LDCC is reduced by 70 % to 91 %.

This paper introduces an end-to-end video transmission system that integrates a neural video codec and a novel rate control algorithm combined with a QoE model. The proposed algorithm dynamically adjusts quantization parameters in response to real-time network conditions and optimize QoE. Experimental results demonstrate that the proposed rate control algorithm significantly improves bitrate allocation, achieving a balanced trade-off between video quality and computational latency. Compared to traditional rate control methods, the system improves QoE by 56%.

3D immersive telepresence systems have dramatically transformed the way users communicate, yet existing point cloud and mesh-based approaches require large amounts of data transmission. Although recent 3D facial semantic-driven techniques can be used to reduce the network burden, they face the critical problem of the high computational cost of facial semantic extraction. To solve the problem, we design and implement an innovative bimodal semantic-driven real-time 3D telepresence system which leverages the low-cost audio-driven semantics for facial expression extraction and head movement semantics for 3D interaction. To improve the efficiency and accuracy of semantic information processing, we propose a speech separation module and optimize the data reception strategy. To optimize the quality of video content reconstruction, we employ frame interpolation and super-resolution techniques, and further propose an online resource scheduling algorithm to balance the rendering, interpolation, and super-resolution processes with limited terminal resources. Experimental results demonstrate that our system can achieve 1K rendering resolution and ~46FPS frame rate with ultra-low network bandwidth (375kbps, approximately 0.32% compared to the point cloud-based approach) and low latency (about 78% semantic feature extraction latency compared to the 3D facial semantic-driven approach) in the campus network.

3D Gaussian Splatting (3DGS), as the cutting-edge static three-dimensional (3D) content generation technology, revolutionizes the speed and fidelity of 3D model construction and provides immense potential for various applications, including e-commerce, 3D exhibitions, and virtual tourism. However, our pioneering analysis of firsthand user experiments uncovers a critical challenge: the unique user behavior patterns of static 3D scenarios render existing immersive video streaming solutions inadequate. To be concrete, the frequent switches between active and inactive states impair viewport prediction accuracy, while the fast glance and slow view pattern provides an opportunity for further quality of experience (QoE) improvement. To tackle these problems, this paper introduces innovative designs for static 3D video streaming. Specifically, we devise a viewport prediction and error correction mechanism on the client side to restore the user viewport with a low cost. Furthermore, we design a dynamic frame rate and bitrate control algorithm to improve user QoE under various network conditions. We implement the first 3DGS-enabled immersive static 3D video streaming system based on an edge-rendered architecture, ensuring efficient rendering and encoding on the edge server while providing broad accessibility for various client-side devices through a web browser. Extensive testing under real-world network conditions and with various kinds of devices demonstrates that the proposed approach exhibits robust and rapid adaptability to fluctuating network conditions, improving user QoE by over 20%, reducing interactive latency by 89%, and minimizing the stall duration by 26% compared to existing low-latency streaming solutions.

Recently, Deep Neural Networks (DNNs) based methods for modern video delivery systems have been acquiring remarkable performance due to their lower bandwidth requirements and better video quality. Specifically, these methods divide a low-resolution video into chunks, train a specific super-resolution model for each chunk on the server, and then stream low-resolution video chunks and corresponding content-aware models to the clients. The client uses their computing power to run the inference of models to super-resolve the low-resolution video chunks. However, the heavy model can achieve better super-resolution quality due to its high overfitting ability, which substantially poses a challenge for client-side computation power, increasing storage and consuming more bandwidth resources for data transmission. On the other hand, although some existing lightweight networks can deploy well to resource-constrained devices, the quality of the video generated is significantly subpar. To reconcile this, we propose a Partial Convolution based padding-based Residual Feature Network (PCRFN). The main idea is to use three partial convolution based padding layers for residual local feature learning to simplify network architecture, which achieves a better trade-off between model performance and efficiency. Moreover, we also analyze the impact of different activation functions and attention mechanisms on our network, which is an excellent guideline for designing a lightweight network. Extensive experimental results show that the proposed PCRFN achieves a better trade-off against state-of-the-art methods in terms of inference times, visual quality, and model complexity. Besides, we also conducted experiments applying our network to the latest neural video delivery, achieving comparatively good video quality and faster inference times across different video lengths. Therefore, PCRFN is a more suitable and efficient super-resolution network for neural video delivery. The Code is available is at https://github.com/Wenbin-Tian/PCRFN.

Apart from the promising potential, federated learning (FL) faces challenges, such as high communication costs and client heterogeneity. Although numerous works have been proposed to address these issues, they lack a holistic perspective to balance all requirements. Moreover, these solutions have not fully utilized the underlying computation capability and network resources, resulting in suboptimal tradeoffs between communication efficiency and inference accuracy. To overcome these challenges, we propose FDL: a federated distillation (FD) learning framework that combines FD and FL to fully utilize computation and network resources. We theoretically prove the convergence bound of the proposed FDL framework. Furthermore, to minimize the training time while maintaining inference accuracy, we design HAD: a heterogeneity-aware FL/FD selection algorithm that determines the total communication rounds and selects the set of FL and FD nodes in each communication round. The optimality of HAD is also theoretically proved. The FDL framework and HAD algorithm together minimize the training time while satisfying the inference accuracy in a heterogeneous and dynamic environment. Extensive experiments on various learning algorithms and data sets show that the proposed FDL-HAD solution can obtain the optimal selection decision in overwhelmingly less selection time compared with the Gurobi solver and can reduce the overall training time by at least 44.8% compared with FL solutions with the same inference accuracy.

To provide high user QoE while maintaining low bandwidth waste, it is important to design adaptive bitrate and preloading algorithms for short video streaming. Current solutions either have relatively low performance, as observed in heuristic algorithms, or suffer the problem of poor generalization, as seen in deep reinforcement learning (DRL)-based algorithms. To address this issue, we propose CLAPS, a curriculum learning-based DRL model that enhances the generalization of the DRL model across a wide range of data, while ensuring high performance. CLAPS introduces a comprehensive metric that measures the curriculum difficulty by combing the performance gap between an existing heuristic algorithm and the DRL model with the prediction error of network bandwidth. Moreover, we design a training scheduler to control sampling proportion based on Markov transition probabilities to address the model forgetting problem. Extensive evaluations using real video datasets and network traces including 5G, 4G, and Wi-Fi demonstrate that CLAPS outperforms all the baseline algorithms. Specifically, CLAPS improves the overall performance by 10.04%-13.84% and the generalization by 22.52% -35.77% compared to the best-performing DRL baseline.

Mobile edge computing is a promising framework for mobile virtual reality (VR) game. Although there are several existing studies on the edge assisted mobile VR game system, they lack the consideration of provisioning services with satisfactory QoE to a large number of users. In this paper, we consider the problem of providing QoE-oriented edge assisted mobile VR game as a service to multiple users, with a comprehensive QoE concern of both visual and delay aspects. Due to the unique features of mobile VR game, the problem is formulated into a Mixed Integer Quadratically Constrained Quadratic Programming (MIQCQP) problem. We show that the problem is NP-hard with object placement decision and rendering level selection decision quadratically coupling together. To solve this problem, we propose the Alternating Directions Method of Multipliers (ADMM) algorithm which can iteratively decouple the quadratic terms and reform the problem into the efficiently solvable MIQCQP-1 (i.e., MIQCQP with one constraint) problem. Trace driven simulation shows that our algorithm fits the edge assisted mobile VR game scenario well with fast computation time (at least 4 orders of magnitude less computation time compared to Gurobi solver) and good performance (at least 18% of user visual QoE improvement compared to other mobile VR scheme).

To improve the performance of Automatic Speech Recognition (ASR), it is common to deploy an error correction module at the post-processing stage to correct recognition errors. In this paper, we propose 1) an error correction model, which takes account of both contextual information and phonetic information by dual-channel; 2) a self-supervised learning method for the model. Firstly, an error region detection model is used to detect the error regions of ASR output. Then, we perform dual-channel feature extraction for the error regions, where one channel extracts their contextual information with a pre-trained language model, while the other channel builds their phonetic information. At the training stage, we construct error patterns at the phoneme level, which simplifies the data annotation procedure, thus allowing us to leverage a large scale of unlabeled data to train our model in a self-supervised learning manner. Experimental results on different test sets demonstrate the effectiveness and robustness of our model.

Large model size and high computational complexity prevent the neural machine translation (NMT) models from being deployed to low resource devices (e.g. mobile phones). Due to the large vocabulary, a large storage memory is required for the word embedding matrix in NMT models, in the meantime, high latency is introduced when constructing the word probability distribution. Based on reusing the word embedding matrix in the softmax layer, it is possible to handle the two problems brought by large vocabulary at the same time. In this paper, we propose Partial Vector Quantization (P-VQ) for NMT models, which can both compress the word embedding matrix and accelerate word probability prediction in the softmax layer. With P-VQ, the word embedding matrix is split into two low dimensional matrices, namely the shared part and the exclusive part. We compress the shared part by vector quantization and leave the exclusive part unchanged to maintain the uniqueness of each word. For acceleration, in the softmax layer, we replace most of the multiplication operations with the efficient looking-up operations based on our compression to reduce the computational complexity. Furthermore, we adopt curriculum learning and compact the word embedding matrix gradually to improve the compression quality. Experimental results on the Chinese-to-English translation task show that our method can reduce 74.35% of parameters of the word embedding and 74.42% of the FLOPs of the softmax layer. Meanwhile, the average BLEU score on the WMT test sets only drops 0.04.

Conversion rate (CVR) prediction is becoming increasingly important in the multi-billion dollar online display advertising industry. It has two major challenges: firstly, the scarce user history data is very complicated and non-linear; secondly, the time delay between the clicks and the corresponding conversions can be very large, e.g., ranging from seconds to weeks. Existing models usually suffer from such scarce and delayed conversion behaviors. In this paper, we propose a novel deep learning framework to tackle the two challenges. Specifically, we extract the pre-trained embedding from impressions/clicks to assist in conversion models and propose an inner/self-attention mechanism to capture the fine-grained personalized product purchase interests from the sequential click data. Besides, to overcome the time-delay issue, we calibrate the delay model by learning dynamic hazard function with the abundant post-click data more in line with the real distribution. Empirical experiments with real-world user behavior data prove the effectiveness of the proposed method.

Internet-of-Things (IoT) big data streaming applications, such as video surveillance and automatic driving, tend to use mobile-edge computing (MEC) infrastructure to enhance their performance and augment their functionalities. Although extensive previous studies have worked on offloading requests to MEC servers, none of them has comprehensively and thoroughly considered the important features of IoT data streaming applications (i.e., component dependency and dynamic arrival) and the infrastructure provisioning (i.e., capacity constraint and colocation interference). In this article, we consider the offloading problem for dynamically arrived IoT data streaming requests on MEC servers in real time. We model it as a delay-sensitive multiuser multiresource online offloading problem respecting component dependency and capacity constraint. The problem is NP-hard with offloading decisions coupling together. To solve it, we decouple the problem into component placement problem and request scheduling problem and propose a two-stage DPGPD algorithm with polynomial time complexity. We show the first stage dynamic programming (DP) algorithm is the optimal solution and the second-stage greedy primal-dual (GPD) algorithm is asymptotic optimal. The simulation results show that our solution is effective yet efficient compared to benchmark solutions. (DP provides the optimal placement layout with 12× less decision time of Gurobi; and GPD provides the asymptotic optimal scheduling with 5× less average waiting time compared to least work left (LWL) in heavy workload.) We implement a dedicated prototype and exploit several representative big data streaming applications to evaluate it. Lab-scale experiment shows that our solution can provide over 3× less total completion time compared to local execution.

The live streaming is more challenging than on-demand streaming, because the low latency is also a strong requirement in addition to the trade-off between video quality and jitters in playback. To balance several inherently conflicting performance metrics and improve the overall quality of experience (QoE), many adaptation schemes have been proposed. Bitrate adaptation is one of the major solution for video streaming under time-varying network conditions, which works even better combining with some latency control methods, such as adaptive playback rate control and frame dropping. However, it still remains a challenging problem to design an algorithm to combine these adaptation schemes together. To tackle this problem, we propose a hybrid control scheme for adaptive live streaming, namely HYSA, based on heuristic playback rate control, latency-constrained bitrate control and QoE-oriented adaptive frame dropping. The proposed scheme utilizes Kaufman's Adaptive Moving Average (KAMA) to predict segment bitrates for better rate decisions. Extensive simulations demonstrate that HYSA outperforms most of the existing adaptation schemes on overall QoE.

To realize mobile virtual reality (VR) group gaming services which are currently hampered by the prohibitive bandwidth and the stringent delay requirements, we investigate the problem of provisioning such services using the emerging mobile edge cloudlet (MEC) networks with a distributed content rendering architecture. The underlying dynamic rendering-module placement problem requires to optimize the service’s operational cost and the users’ end-to-end performance, involving multiple intertwined conflicting system objectives that are discrete, nonconvex, and higher degree polynomial functions with coupled decisions and arbitrary user dynamics over time. We solve this online placement problem by leveraging model predictive control (MPC) and overcoming the aforementioned challenges over each prediction window. We explore the connection between the placement problem and the minimal$s$-$t$cut problem in graph theory and solve the former via solving a series of instances of the latter. We formally prove the performance guarantee of our approach. We also conduct extensive trace-driven evaluations and demonstrate the superior practical performance of our MPC-based approach compared to thede factopractices and the state-of-the-art alternatives.

Population activity measurement by calcium imaging can be combined with cellular resolution optogenetic activity perturbations to enable the mapping of neural connectivity in vivo. This requires accurate inference of perturbed and unperturbed neural activity from calcium imaging measurements, which are noisy and indirect, and can also be contaminated by photostimulation artifacts. We have developed a new fully Bayesian approach to jointly inferring spiking activity and neural connectivity from in vivo all-optical perturbation experiments. In contrast to standard approaches that perform spike inference and analysis in two separate maximum-likelihood phases, our joint model is able to propagate uncertainty in spike inference to the inference of connectivity and vice versa. We use the framework of variational autoencoders to model spiking activity using discrete latent variables, low-dimensional latent common input, and sparse spike-and-slab generalized linear coupling between neurons. Additionally, we model two properties of the optogenetic perturbation: off-target photostimulation and photostimulation transients. Using this model, we were able to fit models on 30 minutes of data in just 10 minutes. We performed an all-optical circuit mapping experiment in primary visual cortex of the awake mouse, and use our approach to predict neural connectivity between excitatory neurons in layer 2/3. Predicted connectivity is sparse and consistent with known correlations with stimulus tuning, spontaneous correlation and distance.

Calcium imaging permits optical measurement of neural activity. Since intracellular calcium concentration is an indirect measurement of neural activity, computational tools are necessary to infer the true underlying spiking activity from fluorescence measurements. Bayesian model inversion can be used to solve this problem, but typically requires either computationally expensive MCMC sampling, or faster but approximate maximum-a-posteriori optimization. Here, we introduce a flexible algorithmic framework for fast, efficient and accurate extraction of neural spikes from imaging data. Using the framework of variational autoencoders, we propose to amortize inference by training a deep neural network to perform model inversion efficiently. The recognition network is trained to produce samples from the posterior distribution over spike trains. Once trained, performing inference amounts to a fast single forward pass through the network, without the need for iterative optimization or sampling. We show that amortization can be applied flexibly to a wide range of nonlinear generative models and significantly improves upon the state of the art in computation time, while achieving competitive accuracy. Our framework is also able to represent posterior distributions over spike-trains. We demonstrate the generality of our method by proposing the first probabilistic approach for separating backpropagating action potentials from putative synaptic inputs in calcium imaging of dendritic spines.

The paper presents a perceptual optimized adaptive HTTP streaming scheme to improve the quality of experience (QoE). In addition to barely controlling the bandwidth and buffer size in existing works, this paper integrates the video saliency based adaptation to improve the perceptual quality for end users. Resources (e.g., buffer) are well managed using saliency cues to ensure the smooth quality under a given network bandwidth. Algorithms are implemented on top of the open-source DASH platform - dash.js to demonstrate our superiority compared with the default throughput-based adaptation and well-known BOLA method. Our work can be a potential enhancement for current DASH standard to offer the smooth and perceptual optimized adaptive streaming in mobile networks.

The error resilience and low-complexity video encoding are two major requirements of real-time visual communications on mobile devices. To address the two requirements simultaneously, this paper presents a fast mode decision algorithm for the error resilient video coding in packet loss environment. The proposed algorithm is a two-step method: early skip mode decision and early intra mode decision. Different from the existing methods for early skip mode decision, the proposed method takes the error-propagation distortion into account in estimating the coding cost. Considering the intra blocks are frequently used to terminate the error propagations, we also propose a method to fast estimate the intra block coding cost, so that the intra mode can be early determined. Overall, the proposed method can significantly reduce the encoding time while keeping the coding efficiency similar to the rate-distortion optimized mode decision method.

Routing and resource allocation are two major research directions for cloud computing, especially for improving the response time in multimedia cloud computing. In this paper, we propose network model for the transmission time between data centers and design routing algorithms for multimedia cloud computing. We further propose resource allocation with the goal to minimize the resource cost. We show some first simulation results of our proposed algorithms.

Although new TCP congestion control algorithms have been proposed for high-speed network in recent years, standard TCP is still the most widely used on the Internet. The performance of applications strongly depends on the behavior of TCP, especially the TCP congestion control algorithms. As different operating systems may implement standard TCP differently in detail, our objective is to investigate and compare the performance of standard TCP implementations on different operating systems, particularly on Windows 7 and Linux. In this paper, we introduce our experimental methodology and show some exemplary results in our experiments in terms of throughput, fairness and so forth. Surprisingly, we find that TCP implementation in Linux probes slightly less packet drop than Windows 7, however, it increases the congestion window slower than Windows 7. Moreover, intra-implementation flows or inter-implementation flows are generally fair in both Windows 7 and Linux. More differences in detail are also found in the paper.

Multimedia streaming applications are traditionally delivered over UDP. Recent measurements show that more and more multimedia streaming data are over TCP as web-based TV, P2P streaming, video sharing websites are getting increasingly popular. To improve the quality of experience (QoE) for users and to cope with variability in TCP throughput, streaming applications typically implement buffers. Yet, for improving the QoE and the streaming quality, e.g., playback continuity and timeliness, it is critical to dimension buffers and the initial buffering delay appropriately. In this paper, we first develop a model for TCP streaming systems and an analytical framework to assess the QoE. Our emphasis is on buffer occupancy, which depends on the TCP arriving rate and the playout rate (the coding rate). We observe that TCP window “bounds”, namely congestion window sizes immediately before a triple duplicate or timeout event, allow to distinguish the minimum and maximum buffer occupancy for TCP streaming systems. As confirmed by experiments, the proposed analytical framework allows to estimate the frequency of buffer overflow or underflow events if buffer sizes and the initial buffering delays are known parameters, or conversely, to dimension the buffer and delay appropriately. We further extend our model and analysis for P2P multicast streaming systems. Simulations and experiments in real networks validate our proposed analytical framework in terms of underflow/overflow probabilities and delay.

In this paper, we propose a fully distributed network model for rate control in application layer multicast based on the utility-price model to maximize the aggregate utilities, and accordingly design an original primal algorithm and a typical dual algorithm both with very small messaging overhead.

In this paper, we study the topic of distributed and optimal congestion control for scalable video streams in application-layer multicast (ALM). We propose a TCP-friendly, fully distributed synchronous algorithm based on the utility-price model which maximizes the global utilities for the streams in the application-layer multicast tree. With the help of numerical study, we show that our proposed algorithm optimizes the overall video quality for fine-grained scalable streams, while minimizing the messaging overhead in the application-layer multicast channel.

In this comments paper, we revisit the network model introduced in Cui, Y., et al (2006). We discuss the inaccuracy of the model and, to correct the network model, we propose to apply directed capacity constraints for directed flows. Based on a comparison of numerical results, we show that the corrected model leads to better accuracy than the original model.

This paper presents a media- and TCP-friendly rate-based congestion control algorithm (MTFRCC) for scalable video streaming in the Internet. The algorithm integrates two new techniques: i) a utility-based model using the rate-distortion function as the application utility measure for optimizing the overall video quality; and ii) a two-timescale approach of rate averages (long-term and short-term) to satisfy both media and TCP-friendliness. We evaluate our algorithm through simulation and compare the results against the TCP-friendly rate control (TFRC) algorithm. For assessment, we consider five criteria: TCP fairness, responsiveness, aggressiveness, overall video quality, and smoothness of the resulting bit rate. Our simulation results manifest that MTFRCC performs better than TFRC for various congestion levels, including an improvement of the overall video quality.