LLM serving providers typically offer a suite of structurally similar models, known as model families, such as the open-source Llama2 series featuring 7B, 13B, and 70B models. While numerous optimizations for LLM serving have been proposed, the potential for leveraging synergies between models within the same family has not been thoroughly explored. This paper introduces MFS, an innovative multi-tiered LLM model family serving system to exploit the structural similarities and parameter redundancies across different scales of models within a family. By utilizing a novel fine-tuning technique called Knowledge Precipitation, MFS restructures the largest model in a family to encapsulate smaller models within its architecture, enabling a unified multi-tiered serving pipeline. Based on the multi-tiered model, MFS realizes a highly parallelized tiered-level batching approach, significantly enhancing system efficiency. It also enables the sharing of intermediate features and KV-cache between models and facilitates multi-level sampling techniques during the inference phase. Experimental results demonstrate that MFS achieves substantial improvements over existing methods, including a 56.1% reduction in end-to-end token generation latency and a 47.8% decrease in GPU memory footprint without compromising the quality of generated content.

Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) cooperating with Priority Flow Control (PFC) has been widely deployed in production datacenters to enable low latency, lossless transmission. At the same time, modern datacenters typically offer parallel transmission paths between any pair of end-hosts, underscoring the importance of load balancing. However, the well-studied load balancing mechanisms designed for lossy datacenter networks (DCNs) are ill-suited for such lossless environments. Through extensive experiments, we are among the first to comprehensively inspect the interactions between PFC and load balancing, and uncover that existing fine-grained rerouting schemes can be counterproductive to spread the congested flows among more paths, further aggravating PFC’s head-of-line (HoL) blocking. Motivated by this, we present FLB, a Fine-grained Load Balancing scheme for lossless DCNs. At its core, FLB employs threshold-free rerouting to effectively balance traffic load and improve link utilization during normal conditions and leverages timely congested flow isolation to eliminate HoL blocking on non-congested flows when congestion occurs. To handle complex multi-bottleneck scenarios, we further introduce FLB*, which incorporates an enhanced congestion-point-aware isolation mechanism using Congestion Point Identifiers (CPI) to eliminate HoL blocking among different congested flows.We have fully implemented a FLB prototype, and our evaluation results show that FLB reduces PFC PAUSE rate by up to 96% and avoids HoL blocking, translating to up to 45% improvement in goodput over CONGA+DCQCN and 40%, 36%, 29% and 18% reduction in average flow completion time (FCT) over LetFlow+Swift, MP-RDMA, Proteus+DCQCN and LetFlow+PCN, respectively.

Personalized federated learning (PFL) has recently gained significant attention for its capability to address the poor convergence performance on highly heterogeneous data and the lack of personalized solutions of traditional federated learning (FL). Existing mainstream approaches either perform personalized aggregation based on a specific model architecture to leverage global knowledge or achieve personalization by exploiting client similarities. However, the former overlooks the discrepancies in client data distributions by indiscriminately aggregating all clients, while the latter lacks fine-grained collaboration of classifiers relevant to local tasks. In view of this challenge, we propose a Personalized Federated learning method for Enhancing Collaboration among Similar Classifiers (PFedCS), which aims at improving the client’s accuracy on local tasks. Concretely, it is achieved by leveraging awareness of the client classifier similarities to address the above problems. By iteratively measuring the distance of the classifier parameters between clients and clustering with each client as a cluster center, the central server adaptively identifies the collaborating clients with similar data distributions. In addition, a distance-constrained aggregation method is designed to generate customized collaborative classifiers to guide local training. As a result, extensive experimental evaluations conducted on three datasets demonstrate that our method achieves state-of-the-art performance.

Efficient Input/Output (I/O) data path between NICs and CPUs/DRAMs is critical for supporting datacenter applications with high-performance network transmission, especially as link speed scales to 100Gbps and beyond. Traditional I/O acceleration strategies, such as Data Direct I/O (DDIO) and Remote Direct Memory Access (RDMA), perform suboptimally due to the inefficient utilization of the Last-Level Cache (LLC). This paper presents CEIO, a novel cache-efficient network I/O architecture that employs proactive rate control and elastic buffering to achieve zero LLC misses in the I/O data path while ensuring the effectiveness of DDIO and RDMA under various network conditions. We have implemented CEIO on commodity SmartNICs and incorporated it into widely-used DPDK and RDMA libraries. Experiments with well-optimized RPC framework and distributed file system under realistic workloads demonstrate that CEIO achieves up to 2.9× higher throughput and 1.9× lower P99.9 latency over prior work.

Due to the high operational complexity and limited deployment scale of lossless RDMA networks, the community has been exploring efficient RDMA communication over lossy fabrics. State-of-the-art (SOTA) lossy RDMA solutions implement a simplified selective repeat mechanism in RDMA NICs (RNICs) to enhance loss recovery efficiency. However, these solutions still face performance challenges, such as unavoidable ECMP hash collisions and excessive retransmission timeouts (RTOs). In this paper, we revisit RDMA reliability with the goals of being independent of PFC, compatible with packet-level load balancing, free from RTO, and friendly to hardware offloading. To this end, we propose DCP, a transport architecture that co-designs both the switch and RNICs, fully meeting the design goals. At its core, DCP-Switch introduces a simple yet effective lossless control plane, which is leveraged by DCP-RNIC to enhance reliability support for high-speed lossy fabrics, primarily including header-only-based retransmission and bitmap-free packet tracking. We prototype DCP-Switch using P4 switch and DCP-RNIC using FPGA. Extensive experiments demonstrate that DCP achieves 1.6× and 2.1× performance improvements, compared to SOTA lossless and lossy RDMA solutions, respectively.

The right to be forgotten mandates that machine learning models enable the erasure of a data owner's data and information from a trained model. Removing data from the dataset alone is inadequate, as machine learning models can memorize information from the training data, increasing the potential privacy risk to users. To address this, multiple machine unlearning techniques have been developed and deployed. Among them, approximate unlearning is a popular solution, but recent studies report that its unlearning effectiveness is not fully guaranteed. Another approach, exact unlearning, tackles this issue by discarding the data and retraining the model from scratch, but at the cost of considerable computational and memory resources. However, not all devices have the capability to perform such retraining. In numerous machine learning applications, such as edge devices, Internet-of-Things (IoT), mobile devices, and satellites, resources are constrained, posing challenges for deploying existing exact unlearning methods. In this study, we propose a Constraint-aware Adaptive Exact Unlearning System at the network Edge (CAUSE), an approach to enabling exact unlearning on resource-constrained devices. Aiming to minimize the retrain overhead by storing sub-models on the resource-constrained device, CAUSE inno-vatively applies a Fibonacci-based replacement strategy and updates the number of shards adaptively in the user-based data partition process. To further improve the effectiveness of memory usage, CAUSE leverages the advantage of model pruning to save memory via compression with minimal accuracy sacrifice. The experimental results demonstrate that CAUSE significantly outperforms other representative systems in realizing exact unlearning on the resource-constrained device by 9.23%-80.86%, 66.21%-83.46%, and 5.26%-194.13% in terms of unlearning speed, energy consumption, and accuracy.

In recent years, the proliferation of Connected Autonomous Vehicles (CAV) has revolutionized the transportation industry. However, these vehicles often face limitations in terms of local computing resources, leading to the need for offloading interactive-intensive application tasks to servers for processing. Traditional paradigm has its limitations in meeting the demands of massive task processing. The combination of Web3.0 and edge computing offers users high-reliable, low-latency, and highly flexible services. Nevertheless, the new paradigm also presents its own challenges such as ensuring privacy data protection, and reducing the time and energy costs associated with task offloading. To tackle these challenges, an edge task offloading framework based on consortium blockchain for CAVs has been developed. Within this framework, a consortium blockchain-based interaction-intensive task offloading method, called CBIToMe, has been designed. CBIToMe specifically addresses the multi-stage nature of interactive-intensive CAV tasks and aims to minimize task completion time and offloading costs, particularly when the waiting time for interaction is uncertain. Additionally, CBIToMe effectively utilizes consortium blockchain technology to safeguard the CAV privacy data. Results from experiments conducted in various scenarios demonstrate that CBIToMe outperforms three representative methods, showcasing its superior performance.

Summary With the rapid development of the Internet of things (IoT) and mobile communication technology, the amount of data related to industrial Internet of things (IIoT) applications has shown a trend of explosive growth, and hence edge‐cloud collaborative environment becomes one of the most popular paradigms to place the IIoT applications data. However, edge servers are often heterogeneous and capacity limited while having lower access delay, so there is a contradiction between capacity and latency while using edge storage. Additionally, when IIoT applications deployed crossing edge regions, the impact of data replication and data privacy should not be ignored. These factors often pose challenges to proposing an effective data placement strategy to take full advantage of edge storage. To address these challenges, an effective data placement strategy for IIoT applications is designed in this article. We first analyze the data access time and data placement cost in an edge‐cloud collaborative environment, with the consideration of data replication and data privacy. Then, we design a data placement strategy based on ‐constraint and Lagrangian relaxation, to reduce the data access time and meanwhile limit the data placement cost to an ideal level. As a result, our proposed data placement strategy can effectively reduce data access time and control data placement costs. Simulation and comparative analysis results have demonstrated the validity of our proposed strategy.

Abstract In the past few years, latency‐sensitive task computing over the industrial internet of things (IIoT) has played a key role in an increasing number of intelligent applications, such as intelligent self‐driving vehicles and unmanned aircraft systems. The edge computing paradigm provides a basic functional infrastructure for across‐domain business task computing on distributed edge servers. With this observation, a trade‐off between the mobile devices and the fixed edge servers is needed to run moving task computing in a low‐latency way. Given this challenge, an intelligent server selection method, named SeeMe, is proposed in this paper. Technically speaking, this method aims at minimizing the communication capacity and the transferring capacity in a multiobjective optimization way to find a low‐latency edge server. The experiments and comparison analysis verify the availability of our method.

Abstract The advent of federated learning (FL) has presented a viable solution for distributed training in edge environment, while simultaneously ensuring the preservation of privacy. In real‐world scenarios, edge devices may be subject to label noise caused by environmental differences, automated weakly supervised annotation, malicious tampering, or even human error. However, the potential of the noisy samples have not been fully leveraged by prior studies on FL aimed at addressing label noise. Rather, they have primarily focused on conventional filtering or correction techniques to alleviate the impact of noisy labels. To tackle this challenge, a method, named DETECTION, is proposed in this article. It aims at effectively detecting noisy clients and mitigating the adverse impact of label noise while preserving data privacy. Specially, a confidence scoring mechanism based on local intrinsic dimensionality (LID) is investigated for distinguishing noisy clients from clean clients. Then, a loss function based on prototype contrastive learning is designed to optimize the local model. To address the varying levels of noise across clients, a LID weighted aggregation strategy (LA) is introduced. Experimental results on three datasets demonstrate the effectiveness of DETECTION in addressing the issue of label noise in FL while maintaining data privacy.

Edge computing (EC) enables low-latency services by pushing computing resources to the network edge. Due to the geographic distribution and limited capacities of edge servers, EC systems face the challenge of edge distributed denial-of-service (DDoS) attacks. Existing systems designed to fight cloud DDoS attacks cannot mitigate edge DDoS attacks effectively due to new attack characteristics. In addition, those systems are typically activated upon detected attacks, which is not always realistic in EC systems. DDoS mitigation needs to be cohesively integrated with workload migration at the edge to ensure timely responses to edge DDoS attacks. In this paper, we present EdgeShield, a novel DDoS mitigation system that leverages edge servers’ computing resources collectively to defend against edge DDoS attacks without the need for attack detection. Aiming to maximize system throughput over time without causing significant service delays, EdgeShield monitors service delays and migrates workloads across an EC system with adaptive mitigation strategies. The experimental results show that EdgeShield significantly outperforms state-of-the-art solutions in both system throughput and service delays.

Abstract Watching the live video on a bus/train/tram has become an important pattern for people to enjoy their travel time. Due to the complex environment changes and obstacles caused by the rapid movement, the network state of the user device is often unstable. A large number of interruptions and resolution reductions seriously affect the user's watching experience. Through direct transmission between devices, Device‐to‐Device (D2D) can effectively improve video quality. Accurate decision for helper device selection is the key to optimize video streaming services based on D2D technology. However, the existing methods through D2D rarely consider the states and the intention of the helper devices when selecting cooperative devices, such as the watching status, the state of charge, the state of cache, and the connection quality. This is highly probable to reduce the efficiency of D2D transmission and the cooperation enthusiasm of helper devices. In addition, the existing methods seldom consider the multiple network interfaces in the device. In view of these challenges, we propose a dynamic smart decision method for device selection in D2D‐assisted multi‐path video transmission network (named DS‐DAMP). Specifically, this method takes the current states and the resource constraints into consideration to select the appropriate helper devices for video cooperative transmission, so as to improve the cooperation satisfaction of helper devices and optimize the video quality of the requesting device. Besides, we make full use of multiple network interfaces to realize multi‐path parallel transmission between devices. A large number of experiments have proved the good performance of our method in terms of video quality and network utility.

Mobile edge computing (MEC) is emerging as a novel computing paradigm that pushes network resources (such as computation and storage resources) away from the centralized data center to distributed edge servers. By hiring various resources of nearby edge servers, the MEC provides high-bandwidth and low-latency network services for mobile users. As numerous mobile users may compete for limited edge servers’ resources, to improve the resource utilization of the MEC system, it is very critical to investigate an effective user allocation policy. Previous studies mainly focus on investigating offline user allocation policies. However, mobile users may arrive online, and the MEC should be able to allocate these users online too. In a real-world MEC environment, online allocation decisions should not be made entirely in the dark. The historical user requests which may contain powerful hints about future user requests, can be adopted to assist in making allocation decisions. In this paper, we take the historical data into account and study the history-assisted online user allocation strategy. Specifically, we formulate the user allocation problem with a comprehensive model and show its hardness. Then, we present an online algorithm named HOUA to allocate mobile users according to both the online arrived user requests and the historical user requests. The competitive ratio of HOUA is proved. To further verify the effectiveness of HOUA, we conduct experiments on a widely-used real-world dataset. We show that HOUA can allocate more mobile users and achieve high resource rental revenue compared with the other approaches.

Abstract Generally, a location‐based service (LBS) often contains the location attribute, content attribute, time‐stamp, and range. From the perspective of privacy protection, the location attribute and the content attribute are the key attributes and need to be protected. However, existing privacy protection methods focus excessively on the location attribute and ignore the content attribute contained in the LBS, which discloses the user's private information. In view of this challenge, a content‐aware privacy protection method, called the CPP method that considers the content attribute is proposed. Specifically, the CPP method is based on using k‐anonymity to generate dummy content attributes to protect the private content. As is shown in an experiment constructed on real‐world data, the CPP method can indeed improve the effect of privacy protection.

With the emergence of various convolutional neural network (CNN)-based applications and the rapid growth of CNN model scale, the resource-constricted end devices can hardly deploy CNN-based applications. Current work optimizes the CNN model on edge servers and deploys the optimized model on devices in an edge computing environment. However, most of them only optimize the resource consumption within or across models solely, whereas neglecting the other side. In this article, we propose a novel CNN-based resource optimization approach (CroApp) that not only optimizes the resource consumption within the CNN model but also pays attention to resource optimization across the applications. Specifically, we adopt model compression as the “inner-model” optimization method, as well as computation sharing as the “intermodel” optimization method. First, during “inner-model” optimization, the CroApp prunes unnecessary parameters within the model on edge servers to reduce the scale of the model. Then, during “intermodel” optimization, the CroApp trains a set of shareable models based on the pruned model and sends these shareable models to end devices. Finally, the CroApp adaptively adjusts the shared models to reduce resource consumption. The experimental results show that the CroApp outperforms the state-of-the-art approaches in terms of resource reduction, scalability, and application performance.

Edge computing provides task offloading services to extend the computational capacity of mobile users and reduce task latency. The deadline-awareness offloading algorithm plays a key role in guaranteeing the quality of service (QoS) requirement. Prior studies mainly focus on tasks with strict deadlines. However, some tasks may not always have to be finished before hard deadlines, e.g., multimedia tasks. Tasks with soft deadlines can miss their primary deadlines, but not by too much. This has not been properly considered by existing offloading approaches. In this paper, we propose CONFECT to offload tasks with mixed deadlines. We formulate the problem and prove its hardness. Then, we propose two online algorithms with proven competitive ratios to solve the problem collectively, including an algorithm that assigns tasks to edge servers and an algorithm that adjusts the task execution order on each server. Extensive experiments show that CONFECT outperforms five baseline algorithms.