Serverless computing offers a compelling cloud model for online inference services. However, existing serverless platforms lack efficient support for GPUs, hindering their ability to deliver high-performance inference. In this article, we present Torpor , a serverless platform for GPU-efficient, low-latency inference. To enable efficient sharing of a node’s GPUs among numerous inference functions, Torpor maintains models in main memory and dynamically swaps them onto GPUs upon request arrivals (i.e., late binding with model swapping). Torpor uses various techniques, including asynchronous API redirection, GPU runtime sharing, pipelined model execution, and efficient GPU memory management, to minimize latency overhead caused by model swapping. Additionally, we design an interference-aware request scheduling algorithm that utilizes high-speed GPU interconnects to meet latency service-level objectives (SLOs) for individual inference functions. We have implemented Torpor and evaluated its performance in a production environment. Utilizing late binding and model swapping, Torpor can concurrently serve hundreds of inference functions on a worker node with 4 GPUs, while achieving latency performance comparable to native execution, where each model is cached exclusively on a GPU. Pilot deployment in a leading commercial serverless cloud shows that Torpor reduces the GPU provisioning cost by 70% and 65% for users and the platform, respectively.

Machine learning (ML), driven by prominent paradigms such as centralized and federated learning, has made significant progress in various critical applications. However, its remarkable success has been accompanied by various attacks. Recently, the model hijacking attack has shown that ML models can be hijacked to execute tasks different from their original tasks, which increases both accountability and parasitic computational risks. Nevertheless, thus far, this attack has only focused on centralized learning. In this work, we broaden the scope of this attack to the federated learning domain, where multiple clients collaboratively train a global model without sharing their data. Specifically, we present the first-of-its-kind hijacking attack against the global model in federated learning, namely HijackFL. The adversary aims to force the global model to perform a different task (called hijacking task) from its original task without the server or benign client noticing. To accomplish this, unlike existing methods that use data poisoning to modify the target model’s parameters, HijackFL searches for pixel-level perturbations based on their local model (without modifications) to align hijacking samples with the original ones in the feature space. When performing the hijacking task, the adversary applies these perturbations to the hijacking samples, compelling the global model to identify them as original ones and predict them accordingly. Extensive experiments demonstrate HijackFL significantly outperforms baselines, e.g., 92.75% vs. 10%. We further investigate the factors that affect its performance and discuss possible defenses to mitigate its impact.

Serverless applications are typically composed of function workflows in which multiple short-lived functions are triggered to exchange data in response to events or state changes. Current serverless platforms coordinate and trigger functions by following high-level invocation dependencies but are oblivious to the underlying data exchanges between functions. This design is neither efficient nor easy to use in orchestrating complex workflows – developers often have to manage complex function interactions by themselves, with customized implementation and unsatisfactory performance. Therefore, we argue that function orchestration should follow a data-centric approach. In our design, the platform provides a data bucket abstraction to hold the intermediate data generated by functions. Developers can use a rich set of data trigger primitives to control when and how the output of each function should be passed to the next functions in a workflow. By making data consumption explicit and allowing it to trigger functions and drive the workflow, complex function interactions can be easily and efficiently supported. We presentPheromone– a scalable, low-latency serverless platform following this data-centric design. Compared to well-established commercial and open-source platforms,Pheromonecuts the latencies of function interactions and data exchanges by orders of magnitude, scales to large workflows, and enables easy implementation of complex applications.

Gradient compression is a promising approach to alleviating the communication bottleneck in data parallel deep neural network (DNN) training by significantly reducing the data volume of gradients for synchronization. While gradient compression is being actively adopted by the industry (e.g., Facebook and AWS), our study reveals that there are two critical but often overlooked challenges: 1) inefficient coordination between compression and communication during gradient synchronization incurs substantial overheads, and 2) developing, optimizing, and integrating gradient compression algorithms into DNN systems imposes heavy burdens on DNN practitioners, and ad-hoc compression implementations often yield surprisingly poor system performance. In this paper, we propose a compression-aware gradient synchronization architecture,CaSync, which relies on flexible composition of basic computing and communication primitives. It is general and compatible with any gradient compression algorithms and gradient synchronization strategies and enables high-performance computation-communication pipelining. We further introduce a gradient compression toolkit,CompLL, to enable efficient development and automated integration of on-GPU compression algorithms into DNN systems with little programming burden. Lastly, we build a compression-aware DNN training frameworkHiPresswithCaSyncandCompLL.HiPressis open-sourced and runs on mainstream DNN systems such as MXNet, TensorFlow, and PyTorch. Evaluation via a 16-node cluster with 128 NVIDIA V100 GPUs and a 100 Gbps network shows thatHiPressimproves the training speed over current compression-enabled systems (e.g., BytePS-onebit, Ring-DGC and PyTorch-PowerSGD) by 9.8%-69.5% across six popular DNN models.

Federated learning (FL) is increasingly deployed among multiple clients to train a shared model over decentralized data. To address privacy concerns, FL systems need to safeguard the clients' data from disclosure during training and control data leakage through trained models when exposed to untrusted domains. Distributed differential privacy (DP) offers an appealing solution in this regard as it achieves a balanced tradeoff between privacy and utility without a trusted server. However, existing distributed DP mechanisms are impractical in the presence of client dropout, resulting in poor privacy guarantees or degraded training accuracy. In addition, these mechanisms suffer from severe efficiency issues.

Relaxing strong consistency plays a vital role in achieving scalability and availability for geo-replicated web applications. However, making relaxation correct in modern implementations, typically written in dynamic languages and utilizing high-level object-oriented database abstractions, remains a challenge, despite the existence of numerous proposed analysis tools.

Many organizations own data but have limited machine learning expertise (data owners). On the other hand, organizations that have expertise need data from diverse sources to train truly generalizable models (model owners). With the advancement of machine learning (ML) and its growing awareness, the data owners would like to pool their data and collaborate with model owners, such that both entities can benefit from the obtained models. In such a collaboration, the data owners want to protect the privacy of its training data, while the model owners desire the confidentiality of the model and the training method that may contain intellectual properties. Existing private ML solutions, such as federated learning and split learning, cannot simultaneously meet the privacy requirements of both data and model owners.

The increased use of deep neural networks has stimulated the growing demand for cloud-based model serving platforms. Serverless computing offers a simplified solution: users deploy models as serverless functions and let the platform handle provisioning and scaling. However, serverless functions have constrained resources in CPU and memory, making them inefficient or infeasible to serve large neural networks-which have become increasingly popular. In this paper, we present Gillis, a serverless-based model serving system that automatically partitions a large model across multiple serverless functions for faster inference and reduced memory footprint per function. Gillis employs two novel model partitioning algorithms that respectively achieve latency-optimal serving and cost-optimal serving with SLO compliance. We have implemented Gillis on three serverless platforms-AWS Lambda, Google Cloud Functions, and KNIX-with MXNet as the serving backend. Experimental evaluations against popular models show that Gillis supports serving very large neural networks, reduces the inference latency substantially, and meets various SLOs with a low serving cost.

Gradient compression is a promising approach to alleviating the communication bottleneck in data parallel deep neural network (DNN) training by significantly reducing the data volume of gradients for synchronization. While gradient compression is being actively adopted by the industry (e.g., Facebook and AWS), our study reveals that there are two critical but often overlooked challenges: 1) inefficient coordination between compression and communication during gradient synchronization incurs substantial overheads, and 2) developing, optimizing, and integrating gradient compression algorithms into DNN systems imposes heavy burdens on DNN practitioners, and ad-hoc compression implementations often yield surprisingly poor system performance.

Communication networks need to be both adaptive and scalable. The last few years have seen an explosive growth of software-defined networking (SDN) and network function virtualization (NFV) to address this need. Both technologies help enable networking software to be decoupled from the hardware so that software functionality is no longer constrained by the underlying hardware and can evolve independently. Both SDN and NFV aim to advance a software-based approach to networking, where networking functionality is implemented in software modules and executed on a suitable cloud computing platform. Achieving this goal requires the virtualization paradigm used in these services that play an important role in the transition to software-based networks. Consequently, the corresponding computing platforms accompanying the virtualization technologies need to provide the required agility, robustness, and scalability for the services executed. Serverless computing has recently emerged as a new paradigm in virtualization and has already significantly changed the economics of offloading computations to the cloud. It is considered as a low-latency, resource-efficient, and rapidly deployable alternative to traditional virtualization approaches, such as those based on virtual machines and containers. Serverless computing provides scalability and cost reduction, without requiring any additional configuration overhead on the part of the developer. In this paper, we explore and survey how serverless computing technology can help building adaptive and scalable networks and show the potential pitfalls of doing so.

A distributed join is a fundamental operation for processing massive datasets in parallel. Unfortunately, computing an equi-join over such datasets is very resource-intensive, even when done in parallel. Given this cost, the equi-join operator becomes a natural candidate for optimization using approximation techniques, which allow users to trade accuracy for latency. Finding the right approximation technique for joins, however, is a challenging task. Sampling, in particular, cannot be directly used in joins; naïvely performing a join over a sample of the dataset will not preserve statistical properties of the query result.

IoT-enabled devices continue to generate a massive amount of data. Transforming this continuously arriving raw data into timely insights is critical for many modern online services. For such settings, the traditional form of data analytics over the entire dataset would be prohibitively limiting and expensive for supporting real-time stream analytics. In this work, we make a case for approximate computing for data analytics in IoT settings. Approximate computing aims for efficient execution of workflows where an approximate output is sufficient instead of the exact output. The idea behind approximate computing is to compute over a representative sample instead of the entire input dataset. Thus, approximate computing- based on the chosen sample size - can make a systematic tradeoff between the output accuracy and computation efficiency. This motivated the design of APPROXIOT- a data analytics system for approximate computing in IoT. To realize this idea, we designed an online hierarchical stratified reservoir sampling algorithm that uses edge computing resources to produce approximate output with rigorous error bounds. To showcase the effectiveness of our algorithm, we implemented APPROXIOT based on Apache Kafka and evaluated its effectiveness using a set of microbenchmarks and real-world case studies. Our results show that APPROXIOT achieves a speedup 1:3×-9:9× with varying sampling fraction of 80% to 10% compared to simple random sampling.

A common practice to increase the reliability of a cloud application is to deploy redundant instances. Unfortunately such redundancy efforts can be undermined if the application's instances share common dependencies. This paper presents ReCloud, a novel system that can efficiently find a reliable deployment plan for cloud applications. ReCloud considers and avoids common dependencies shared across application instances that may lead to correlated failures, and works with applications that even have complex internal structures. ReCloud utilizes various pieces of available dependency information (e.g., hardware, software and/or network dependencies) about the cloud infrastructure to quantitatively assess the reliability of the application's deployment plan with rigorous error bounds. This assessment further enables ReCloud to find a deployment plan that balances between reliability and other criteria such as application performance and resource utilization. We implemented a fully functional system. The experimental results show that, even in a large cloud environment with more than 27K hosts, ReCloud needs only 30 seconds to find a deployment plan that is one order of magnitude more reliable than the common practice.

Approximate computing aims for efficient execution of workflows where an approximate output is sufficient instead of the exact output. The idea behind approximate computing is to compute over a representative sample instead of the entire input dataset. Thus, approximate computing --- based on the chosen sample size --- can make a systematic trade-off between the output accuracy and computation efficiency.

Major cloud computing operators provide powerful monitoring tools to understand the current (and prior) state of the distributed systems deployed in their infrastructure. While such tools provide a detailed monitoring mechanism at scale, they also pose a significant challenge for the application developers/operators to transform the huge space of monitored metrics into useful insights. These insights are essential to build effective management tools for improving the efficiency, resiliency, and dependability of distributed systems.

There is a growing body of research on mechanisms for preserving online user privacy while still allowing aggregate queries over private user data. A common approach is to store user data at users' devices, and to query the data in such a way that a differentially private noisy result is produced without exposing individual user data to any system component. A particular challenge is to design a system that scales well while limiting how much the malicious users can distort the result. This paper presents SplitX, a high-performance analytics system for making differentially private queries over distributed user data. SplitX is typically two to three orders of magnitude more efficient in bandwidth, and from three to five orders of magnitude more efficient in computation than previous comparable systems, while operating under a similar trust model. SplitX accomplishes this performance by replacing public-key operations with exclusive-or operations. This paper presents the design of SplitX, analyzes its security and performance, and describes its implementation and deployment across 416 users.

Today, websites commonly use third party web analytics services t obtain aggregate information about users that visit their sites. This information includes demographics and visits to other sites as well as user behavior within their own sites. Unfortunately, to obtain this aggregate information, web analytics services track individual user browsing behavior across the web. This violation of user privacy has been strongly criticized, resulting in tools that block such tracking as well as anti-tracking legislation and standards such as Do-Not-Track. These efforts, while improving user privacy, degrade the quality of web analytics. This paper presents the first design of a system that provides web analytics without tracking. The system gives users differential privacy guarantees, can provide better quality analytics than current services, requires no new organizational players, and is practical to deploy. This paper describes and analyzes the design, gives performance benchmarks, and presents our implementation and deployment across several hundred users.

BGP Route Reflectors (RR), which are commonly used to help scale Internal BGP (iBGP), can produce oscillations, forwarding loops, and path inefficiencies. ISPs avoid these pitfalls through careful topology design, RR placement, and link-metric assignment. This paper presents Address-Based Route Reflection (ABRR): the first iBGP solution that completely solves all oscillation and looping problems, has no path inefficiencies, and puts no constraints on RR placement. ABRR does this by emulating the semantics of full-mesh iBGP, and thereby adopting the correctness and path efficiency properties of full-mesh iBGP. Both traditional Topology-Based Route Reflection (TBRR) and ABRR take a divide-and-conquer approach. While TBRR scales by making each RR responsible for all prefixes from some fraction of routers, ABRR scales by making each RR responsible for some fraction of prefixes from all routers. We have implemented a fully functional ABRR prototype. Using BGP data from a Tier-1 ISP, our analytical and implementation results show that ABRR's scaling and convergence properties compare positively with traditional TBRR.

IP Routers use sophisticated forwarding table (FIB) lookup algorithms that minimize lookup time, storage, and update time. This paper presents SMALTA, a practical, near-optimal FIB aggregation scheme that shrinks forwarding table size without modifying routing semantics or the external behavior of routers, and without requiring changes to FIB lookup algorithms and associated hardware and software. On typical IP routers using the FIB lookup algorithm Tree Bitmap, SMALTA shrinks FIB storage by at least 50%, representing roughly four years of routing table growth at current rates. SMALTA also reduces average lookup time by 25% for a uniform traffic matrix. Besides the benefits this brings to future routers, SMALTA provides a critical easy-to-deploy one-time benefit to the installed base should IPv4 address depletion result in increased routing table growth rate. The effective cost of this improvement is a sub-second delay in inserting updates into the FIB once every few hours. We describe SMALTA, prove its correctness, measure its performance using data from a Tier-1 provider as well as Route-Views. We also describe an implementation in Quagga that demonstrates its ease of implementation.

Online social networking systems are rapidly becoming popular on the Internet for users to share, organize and locate interesting content. However, these systems have increasingly been employed as ideal platforms to spread spam and irrelevant content, abusing the valuable human attention and service resource. We propose a social reputation model to guide users to browse the desirable content. First, we compute the statistical correlation between different users to distinguish various user interests, then, since a user's friends are usually trustworthy and share the similar interest, we further exploit the inherent friend relationships to perform the reliable social enhancements of vote history extension and efficient reputation estimation. In addition to providing a strong incentive for user cooperation, our model can handle the practical problems of inactive users, unpopular content and Sybil attacks effectively and efficiently. Our evaluation on a large-scale realistic network validates our analysis, and shows that our social reputation model can help users find the desirable content in various scenarios with a precision of around 94%.

Online social networking systems are rapidly becoming popular for users to share, organize and locate interesting content. However, these systems have increasingly been employed as platforms to spread spam and irrelevant content, abusing valuable human attention and service resource. In this paper, we propose a social reputation model to guide users to browse desirable content. First, we compute the statistical correlation between different users to distinguish various user interests; then, since a user's friends are usually trustworthy and share similar interest, we further exploit the inherent friend relationships to perform reliable social enhancements of vote history extension and efficient reputation estimation. Our social reputation model provides strong incentives for user cooperation, and moreover, our model can handle practical problems of inactive users, unpopular content and Sybil attacks effectively and efficiently. Our evaluation on a large-scale network validates our analysis, and shows that our social reputation model can help users find the desirable content in various scenarios with a precision of 94%.

Tagging systems are known to be particularly vulnerable to tag spam. Due to the self-organization and self-maintenance nature of Peer-to-Peer (P2P) overlay networks, users in the P2P tagging systems are more vulnerable to tag spam than the centralized ones. This paper proposes SpamResist, a novel social reliability-based mechanism. For each tag search, SpamResist client groups the search respondents into two categories, namely unfamiliar peers and interacted peers according to the fact whether the client has interacted with such respondents. For the two different categories of peers, the client computes their reliability degrees, and then utilizes these reliability degrees as weights to rank search results. To obtain higher quality search results, we propose a socially-enhanced mechanism, considering social friends can share their previous experience and help improve both the performance and convergence of SpamResist. Finally, the experimental results illustrate that SpamResist can effectively defend against tag spam and work better than the existing search models in P2P tagging systems.

Deceptive behaviors of peers in peer-to-peer (P2P) content sharing systems have become a serious problem due to the features of P2P overlay networks such as anonymity, self-organization, etc. This paper presents Sorcery, a novel active challenge-response mechanism based on the notion that one side of interaction with dominant information can detect whether the other side is telling a lie. To make each client obtain the dominant information, our approach introduces social network to the P2P content sharing system; thus, the client can establish friend-relationships with peers who are either acquaintances in reality or those reliable online friends. Using the confidential voting histories of friends as own dominant information, the client can challenge the content providers with the overlapping votes of both his friends and the content provider, thus detecting whether the content provider is a deceiver. Moreover, Sorcery provides the punishment mechanism which can reduce the impact brought by deceptive behaviors, and our work also discusses some key practical issues. The experimental results illustrate that Sorcery can effectively address the problem of deceptive behaviors, and work better than the existing reputation models.

The growing popularity of web applications in the last few years has led users to give the management of their data to online application providers, which will endanger the security and privacy of the users. In this paper, we present WebIBC, which integrates public key cryptography into web applications without any browser plugins. The public key of WebIBC is provided by identity based cryptography, eliminating the need of public key and certificate online retrieval; the private key is supplied by the fragment identifier of the URL inspired by BeamAuth. The implementation and performance evaluation demonstrate that WebIBC is secure and efficient both in theory and practice.

Poisoning attacks in the Peer-to-Peer (P2P) content sharing service have become a serious security problem on the global Internet due to the features of P2P systems such as self-organization, self-maintenance, etc. In this paper, we propose a novel poisoning-resistant security framework based on the notion that the content providers would be the only trusted sources to verify the integrity of the requested content. To provide the mechanisms of availability and scalability, a content provider publishes the information of his shared contents to a group of content maintainers self-organized in a security overlay, so that a content requestor can verify the integrity of the requested content from the associated content maintainers. Two defense functions are first carried out - filtering out malicious activities and selecting the authentic content version. Then, the content requestor can perform the content integrity verification while downloading and take prompt protection actions to handle content poisoning attacks. To further enhance the system performance, we devise a scalable probabilistic verification scheme. The evaluation results illustrate that our framework can effectively and efficiently defend against content poisoning in various scenarios.

Current peer-to-peer architectures generally can be grouped into three categories: centralized architectures that utilize central directory servers to process queries, decentralized structured architectures that accurately build an underlying topology to support distributed hash table efficiently, and decentralized unstructured architectures that impose no structure on the topology and typically propagate queries to neighbors for searching. Aiming at integrating the flexibility of unstructured architectures with the regularity of structured architectures, we propose a hybrid overlay structure based on virtual node. Especially, the hybrid architecture utilizes virtual nodes to build a distributed ring with random links. We can use the distributed ring to perform short jumps, and apply random links to long jumps. With our hybrid design, keyword searching, even multi-keyword searching, can be performed efficiently; both popular and rare keywords can be quickly located. Furthermore, our architecture is robust to the change of system scale, and it can work well with low maintenance cost in the dynamic environment.