In Split Federated Learning (SFL), the clients collaboratively train a model with the help of a server by splitting the model into two parts. Part-1 is trained locally at each client and aggregated by the aggregator at the end of each round. Part-2 is trained at a server that sequentially processes the intermediate activations received from each client. We study the phenomenon of catastrophic forgetting (CF) in SFL in the presence of data heterogeneity. In detail, due to the nature of SFL, local updates of part-1 may drift away from global optima, while part-2 is sensitive to the processing sequence, similar to forgetting in continual learning (CL). Specifically, we observe that the trained model performs better in classes (labels) seen at the end of the sequence. We investigate this phenomenon with emphasis on key aspects of SFL, such as the processing order at the server and the cut layer. Based on our findings, we propose Hydra, a novel mitigation method inspired by multi-head neural networks and adapted for the SFL setting. Extensive numerical evaluations show that Hydra outperforms baselines and methods from the literature.

The fundamental caching problem in networks asks to find an allocation of contents to a network of caches with the aim of maximizing the cache hit rate. Despite the problem's importance to a variety of research areas - including not only content delivery, but also edge intelligence and inference - and the extensive body of work on empirical aspects of caching, very little is known about the exact boundaries of tractability for the problem beyond its general NP-hardness. We close this gap by performing a comprehensive complexity-theoretic analysis of the problem through the lens of the parameterized complexity paradigm, which is designed to provide more precise statements regarding algorithmic tractability than classical complexity. Our results include algorithmic lower and upper bounds which together establish the conditions under which the caching problem becomes tractable.

Parallel Split Learning (SL) allows resource-constrained devices that cannot participate in Federated Learning (FL) to train deep neural networks (NNs) by splitting the NN model into parts. In particular, such devices (clients) may offload the processing task of the largest model part to a computationally powerful helper, and multiple helpers may be employed and work in parallel. In hybrid federated and split learning (HFSL), on the other hand, devices can participate in the training process through any of the two protocols (SL and FL), depending on the system's characteristics. This could considerably reduce the maximum training time over all clients (makespan), especially in highly heterogeneous scenarios. In this paper, we study the joint problem of the training protocol selection, client-helper assignments, and scheduling decisions, to minimize the training makespan. We prove this problem is NP-hard and propose two solution methods: one based on the decomposition of the problem by leveraging its inherent symmetry, and a second fully scalable one. Through numerical evaluations using our testbed's measurements, we build a solution strategy comprising these methods. Moreover, this strategy finds a near-optimal solution and achieves a shorter makespan than the baseline schemes by up to 71%.

Split learning (SL) has been recently proposed as a way to enable resource-constrained devices to train multi-parameter neural networks (NNs) and participate in federated learning (FL). In a nutshell, SL splits the NN model into parts, and allows clients (devices) to offload the largest part as a processing task to a computationally powerful helper. In parallel SL, multiple helpers can process model parts of one or more clients, thus, considerably reducing the maximum training time over all clients (makespan). In this paper, we focus on orchestrating the workflow of this operation, which is critical in highly heterogeneous systems, as our experiments show. In particular, we formulate the joint problem of client-helper assignments and scheduling decisions with the goal of minimizing the training makespan, and we prove that it is NPhard. We propose a solution method based on the decomposition of the problem by leveraging its inherent symmetry, and a second one that is fully scalable. A wealth of numerical evaluations using our testbed’s measurements allow us to build a solution strategy comprising these methods. Moreover, we show that this strategy finds a near-optimal solution, and achieves a shorter makespan than the baseline scheme by up to 52.3%.

Recommendations are employed by Content Providers (CPs) of streaming services in order to boost user engagement and their revenues. Recent works suggest that nudging recommendations towards cached items can reduce operational costs in the caching networks, e.g., Content Delivery Networks (CDNs) or edge cache providers in future wireless networks. However, cache-friendly recommendations could deviate from users' tastes, and potentially affect the CP's revenues. Motivated by real-world business models, this work identifies the misalignment of the financial goals of the CP and the caching network provider, and presents a network-economic framework for recommendations. We propose a cooperation mechanism leveraging the Nash bargaining solution that allows the two entities to jointly design the recommendation policy. We consider different problem instances that vary on the extent these entities are willing to share their cost and revenue models, and propose two cooperative policies, CCR and DCR, that allow them to make decisions in a centralized or distributed way. In both cases, our solution guarantees reaching a fair and Pareto optimal allocation of the cooperation gains. Moreover, we discuss the extension of our framework towards caching decisions. A wealth of numerical experiments in realistic scenarios show the policies lead to significant gains for both entities.

In the context of on-demand video streaming services, both the caching and the recommendation decisions have an impact on the user satisfaction, and thus, financial implications for the Content Provider (CP). The idea of co-designing these decisions has been recently proposed in the literature as a way to minimize delivery costs and traffic at the backbone Internet. However, related work does not take into account that every content exists in multiple versions/streaming qualities, or at best treats each version as a separate content, when it comes to caching. In this paper, we explore how transcoding a content at the edge could avoid placing multiple related versions of this content in the same cache, thus better utilizing capacity (leading to an increase of the CP's profit). To this end, we formulate the problem of jointly deciding on caching, recommendations, and user-transcoder assignments with the goal of increasing the profit (revenue minus the incurred costs). We propose an iterative algorithm that is based on a decomposition of the formulated problem into two subproblems. We show that both subproblems, although NP-hard, are equivalent to problems in the literature for which algorithms with approximation guarantees exist. Our numerical evaluations in realistic scenarios show that the proposed policy leads to important financial gains of up to 29% when compared to the scenario where edge transcoding is not exploited.

Streaming platforms, like Netflix and YouTube, strive to offer high streaming quality (SQ), in terms of bitrate, delays, etc., to their users. Meanwhile, a significant share of content consumption of these platforms is heavily influenced by recommendations. In this setting, the user’s overall experience is a product of both the user’s interest in a recommended content,i.e., the recommendation quality (RQ), and the SQ of this content. However, network decisions (like caching) that affect the SQ are usually made without considering the recommender’s actions. Likewise, recommendations are chosen independently of the potential delivery quality. In this paper, we define a metric of streaming experience (MoSE) that captures the fundamental tradeoff between the SQ and RQ. We aim to jointly optimize caching and recommendations in a generic network of caches, with the objective of maximizing this metric. This is in line with the recent trend for content providers to simultaneously act as Content Delivery Network owners, implying that the same entity may handle both caching and recommendation decisions. We formulate this joint optimization problem and prove that it can be approximated up to a constant factor. To the best of our knowledge, this is the first polynomial algorithm to achieve a constant approximation ratio for the joint problem. Moreover, our numerical experiments show important performance gains of our algorithm over baseline schemes and existing algorithms in the literature.

Recommender systems have been established as a key component of video streaming services, shaping up to 80% of content requests. Hence, recommendations are employed by the Content Providers (CPs) of these services to increase the viewing time and their revenues. Furthermore, it has been recently suggested that recommendations could be a means to reduce the operational costs of the Content Delivery Networks (CDNs) when they are related to already cached items, i.e., when they are cache-friendly. Clearly, these conflicting objectives, i.e., increasing revenue for the CP and reducing costs for the CDN, can create tensions between the two entities, and hence, prevent the full utilization of recommendations. In this work, we propose a model for capturing these tradeoffs, and an economic mechanism, based on the Nash bargaining solution, for reconciling the potentially conflicting objectives of the CP and the CDN. Our scheme enables the CP and CDN to jointly design the recommendations in a way that balances the revenue gains and cost savings, ensuring a fair and Pareto optimal split of the accrued benefits for both entities. Our numerical experiments in realistic scenarios show that the proposed scheme leads to important financial gains of up to 30%.

On streaming platforms such as Youtube and Netflix, recommendations influence a large share of content consumption. In this context, use rexperience depends on both the quality of the recommendations (QoR) and the quality of service (QoS) of the delivered content. However, network decisions (such as caching) affecting QoS are usually made without explicit knowledge of the recommender's actions. Similarly, recommendation decisions are made without considering the potential delivery quality of the recommended content. In this paper, we propose to jointly optimize caching and recommendations in a generic network of caches, towards maximizing the quality of experience (QoE). This coincides with the recent trend for large content providers to also act as Content Delivery Network (CDN) owners. We formulate this joint optimization problem and prove that it can be approximated up to a constant. To the best of our knowledge, this is the first polynomial algorithm to achieve a constant approximation ratio for the joint problem. Our numerical experiments show important performance gains of the proposed algorithm over baseline schemes and existing algorithms.