LLM agents increasingly rely on tool calling, and the Model Context Protocol (MCP) standardizes it between agents and tool providers, reducing integration cost and driving rapid growth in tool scale. Yet a standardized interface does not make tool access work at production scale: legacy services are not MCP-callable, fast protocol evolution creates compatibility cost, large tool sets exhaust the context window, and stateful sessions complicate load balancing. We solve these with a shared control point, a centralized MCP Gateway System that makes MCP operational at cloud scale. The gateway breaks the direct-connect data plane and consolidates legacy API integration, protocol bridging, access control, and session-aware routing, while scaling out elastically at low per-call overhead. It scales agent tool access to thousands of cloud operations.

While essential for stateful modern workloads like Large Language Model agents and IoT services, long-lived connections impede cloud infrastructure agility by complicating maintenance and load balancing. Existing connection migration solutions either lack support for industrial-grade encrypted traffic or fail to prevent packet loss during handover in active production environments. To address this gap, we propose Zephyr, a zero-loss and transparent TLS connection migration framework for cross-node migration between servers with different addresses. Zephyr ensures transport-layer consistency by orchestrating an eBPF-based packet buffering mechanism to safely intercept in-flight data. At the application layer, rather than deeply modifying standard TLS libraries, Zephyr creatively reuses the native session resumption mechanism via a “fake client” strategy to reconstruct complex cryptographic states without client involvement. Implemented in widely-used industrial stacks (Nginx and OpenSSL), Zephyr achieves connection migration with approximately 4.1 ms downtime and strict zero packet loss. This approach enables seamless infrastructure optimization without disrupting cloud services.

Current NFVs assign packets to CPU cores at flow granularity, where each flow is pinned to a single CPU. This approach is efficient under most scenarios but has exposed limitations when handling elephant flows. These “heavy hitters” overwhelm single cores, creating bottlenecks that affect overall throughput and degrade service quality. As networks scale to higher-speed links and core-rich CPUs, these imbalances become more severe. In this paper, we propose ParaFlowO, an architecture that Parallelizes processing elephant Flows across multiple CPU cores while preserving in-Order delivery. ParaFlowO breaks elephant flows into flowlets and dynamically rotates them across multiple cores. It integrates a lightweight reordering mechanism to preserve packet order and controls parallelism to mitigate contention on shared state. Preliminary evaluations show that ParaFlowO offers a practical solution to mixed-grained parallelism in stateful middleboxes.

The transition to Data Processing Unit (DPU)-centric architectures has become the de-facto standard in modern cloud networks, enabling infrastructure offload and improved host resource utilization. However, the fixed hardware limits of DPUs increasingly fail to keep pace with the rapid growth of host compute density and network-intensive workloads. As a result, when DPU resources are saturated, host compute capacity often remains underutilized due to insufficient network provisioning.

Checkpointing is becoming a hotspot of interest in both academia and industry as the primary fault-tolerance method for large model training. However, existing checkpoint designs are tightly coupled with the training process, leading to interruptions that reduce overall training efficiency. To reduce the impact of checkpoints on training, this paper presents FlowCheck, a novel checkpointing system that decouples checkpoint operations from the training process, enabling checkpoint saving without blocking the training. Specifically, FlowCheck updates the checkpoints by extracting complete gradient information from the network traffic of normal training. FlowCheck deploys a traffic-mirroring network to support this design. To utilize mirrored traffic for checkpointing operations, two key challenges need to be addressed. First, we need to achieve precise identification and extraction of gradient packets from training traffic. Second, the transmission on the mirror link is unreliable due to its inability to trigger retransmission upon packet loss. Through two key designs: (1) packet-counting-based traffic identification, and (2) packet redundancy recovery mechanism, FlowCheck implements an efficient checkpointing system using the existing training network and solves the above two challenges. Experiments and estimations verify that FlowCheck achieves checkpoint operations with zero impact on training, and demonstrate that FlowCheck achieves over 98% effective training time under practical fault conditions.

Single Root I/O Virtualization (SR-IOV) technology has advanced in recent years and can simultaneously satisfy the network requirements of high data plane performance, high deployment density, and fast startup for applications in traditional containers. However, it falls short with secure containers, which have become the mainstream choice in multi-tenant clouds. SR-IOV requires secure containers to use passthrough I/O for higher data plane performance, which hinders the container startup performance and prevents its usage in time-sensitive tasks like serverless computing. In this paper, we advocate that the startup performance of SR-IOV enabled secure containers can be further boosted, making SR-IOV suitable for building a Container Network Interface (CNI) for secure containers. We first dissect the end-to-end concurrent startup process and identify three key bottlenecks that lead to the slow startup, including Virtual Function I/O device set management, Direct Memory Access memory mapping, and Virtual Function (VF) driver initialization. We then propose a CNI named FastIOV that addresses these bottlenecks through lock decomposition, unnecessary mapping skipping, decoupled zeroing, and asynchronous VF driver initialization. Our evaluation shows that FastIOV reduces the overhead of enabling SR-IOV for secure containers by 96.1%, achieving 65.7% and 75.4% reductions in the average and 99th percentile end-to-end startup time.

Layer-7 load balancers (L7 LBs) improve service performance, availability, and scalability in public clouds. They rely on I/O event notification mechanisms such as epoll to dispatch connections from the kernel to userspace workers. However, early epoll versions suffered from the thundering herd problem. Epoll exclusive (available since Linux 4.5) mitigates this but introduces LIFO wakeups, causing connection concentration on a few workers. Reuseport (Linux 3.9) hashes connections across workers but suffers from hash collisions and lacks awareness of worker load. Since each worker serves multi-tenant traffic, inter-worker load balancing is critical to avoid worker overload and preserve tenant performance isolation.

Cloud providers use SmartNIC-accelerated virtual switches (vSwitches) to offer rich network functions (NFs) for tenant VMs. Constrained by limited SmartNIC resources, it is a challenge to provide sufficient network performance for high-demand VMs. Meanwhile, we observed a significant number of idle vSwitches in the data center, which led us to consider leveraging them to build a remote resource pool for high-demand virtual NICs (vNICs). In this work, we propose Nezha, a distributed vSwitch load sharing system. Nezha reuses the existing idle SmartNICs to handle the excess load from the local SmartNIC without adding new devices. Nezha offloads stateless rule/flow tables to the remote, while keeping states locally. This eliminates the need for state synchronization, facilitating load sharing and failover. The deployment cost of Nezha is only a small fraction of that required to deploy new devices. Data collected from production show that our CPS capability bottleneck has shifted from the vSwitch to the VM kernel stack, with #concurrent flows and #vNICs increased by up to 50.4x and 40x, respectively.

Alibaba Cloud's centralized gateways relied heavily on high-capacity switching ASICs, but the abrupt halt of Tofino chip evolution in Jan 2023 forced us to seek alternatives that can meet the requirements of performance, supply-chain security, code reuse, and resource efficiency. After evaluating multiple options, we developed Albatross, our 3rd gen cloud gateway based on FPGA and x86 CPUs. Albatross delivers FPGA-based packet-level load balancing to the host CPUs to prevent CPU core overload, manages large reorder buffers under high-latency jitters (100μs) during complex cloud service processing, and resolves head-of-line (HOL) blocking from packet losses or software exceptions in CPUs. To avoid being overloaded by heavy hitters due to anomalies or attacks, it also implements a two-stage rate limiter for millions of tenants with only 2MB of FPGA memory. To maximize resource utilization, Albatross uses containerization to host multiple gateway instances and designs a BGP proxy to lessen the BGP peering overhead on uplink switches caused by high-density container deployments. After hundreds of man-months of development, a single Albatross node can process 80~120Mpps of cloud network traffic with an average latency of 20μs, reducing gateway and sandbox infra costs by 50%.

This paper presents ZooRoute, a tenant-transparent, fast failure recovery service that requires no modifications to physical devices. ZooRoute leverages the overlay layer and enables traffic flows to bypass failures by altering source ports (srcPorts) in packet headers during encapsulation. To enable deployment in large-scale cloud networks, ZooRoute proposes: 1) On-demand probing to efficiently monitor a vast number of hosts while minimizing telemetry costs. 2) Table compression to record the states of numerous paths with limited on-chip resources. 3) A device-sensing mechanism to prevent unnecessary reconnections in stateful forwarding. Deployed in Alibaba Cloud for 18 months, ZooRoute has significantly improved network reliability, reducing cumulative outage time by 92.71%.

Load balancers (LBs) are crucial in cloud environments, ensuring workload scalability. They route packets destined for a service (identified by a virtual IP address, or VIP) to a group of servers designated to deliver that service, each with its direct IP address (DIP). Consequently, LBs significantly impact the performance of cloud services and the experience of tenants. Many academic studies focus on specific issues such as designing new load balancing algorithms and developing hardware load balancing devices to enhance the LB's performance, reliability, and scalability. However, we believe this approach is not ideal for cloud data centers for the following reasons: (i) the increasing demands of users and the variety of cloud service types turn the LB into a bottleneck; and (ii) continually adding machines or upgrading hardware devices can incur substantial costs. In this paper, we propose the Next Generation Load Balancer (NGLB), designed to bypass the TCP connection datapath from the LB, thereby eliminating latency overheads and scalability bottlenecks of traditional cloud LBs. The LB only participates in the TCP connection establishment phase. The three key features of our design are: (i) the introduction of anactive address learningmodel to redirect traffic and bypass the LB, (ii) amulti-tenant isolationmechanism for deployment within multi-tenant Virtual Private Cloud networks, and (iii) a distributed flow control method, known ashierarchical connection cleaner, designed to ensure the availability of backend resources. The evaluation results demonstrate that NGLB reduces latency by 16% and increases nearly 3× throughput. With the same LB resources, NGLB improves 10× rate of new connection establishment. More importantly, five years of operational experience has proven NGLB's stability for high-bandwidth services.

Programmable switches have been recently proposed as dataplane solutions for public clouds. However, the conflict of limited on-chip memory and massive forwarding rules in cloud networks hinders the large-scale deployments. We argue that building compact forwarding tables for programmable switches is a viable option to this problem. In this paper, as a first step, we explore the feasibility of compact data structures for VPC routing tables (VRTs) and propose QuarkTable as a solution. The idea of QuarkTable builds upon the existence of redundancy in the prefixes of VRTs, supported by extensive analysis of real-world VRTs collected from six geographically distributed regions of Alibaba Cloud. By cutting the VRT into two partitions and encoding the upper part of the longer prefixes, QuarkTable effectively shortens the length of each entry and reduces the overall memory consumption. We demonstrate the effectiveness of QuarkTable by showing its proximity to the entropy bound of real-world VRTs. Experiments on six VRTs from Alibaba Cloud show practical memory savings up to 33.5% and 30.2% for SRAM and TCAM, respectively.

The virtual switch (vSwitch) serves as a fundamental element in cloud network, critical for high-performance and strongly isolated inter-VM forwarding in local and external networks. Similar to other multicore systems, a vSwitch with multiple cores also faces the issue of core load imbalance. As a major cloud provider, we pinpoint four cases of core load imbalance within the vSwitch in our cloud, stemming from unequal traffic distribution across virtual queues and RSS buckets, as well as from traffic patterns like heavy hitters and micro-bursts. To tackle the different load imbalance cases, we present vSwitchLB, a vSwitch load balance framework. Specifically, we introduce a load imbalance detection module, accompanied by dedicated techniques designed to address each specific type of imbalance. Our preliminary evaluation shows that vSwitchLB can accurately classify different load imbalances encountered in the vSwitch on our cloud and then prevent any single core of vSwitch from being flooded and overwhelmed.

In this paper, we use empirical measurements to show that container network startup is a key factor that contributes to the slow startup of secure containers in multi-tenant clouds, especially in the scenario of serverless computing, where the issue is pronounced by high-volume concurrent container invocations. We conduct extensive and detailed analysis on existing Container Network Interface (CNI) plugins and show that even the fastest one doubles the startup time from the no-network scenario. We show that the major cause of the blowup in total startup time is that enabling networking significantly increases the contention among different startup stages, particularly for global Linux kernel locks, including the Routing Table NetLink (RTNL) mutex lock and various spin locks. We reveal that contending for these locks hinders startup performance in three ways, including directly increasing stage time, causing poor pipeline overlap and wasting CPU resources. To mitigate such kernel lock contention, we propose a multi-stage concurrency control mechanism based on Bayesian optimization to limit the concurrency of each contended stage. Our results show that this lightweight mechanism can effectively reduce the end-to-end container startup time by 18.8% with negligible extra overhead.

In recent years, service mesh frameworks have gained significant popularity in building microservice-based applications. A key component of these frameworks is a proxy in each K8s pod, named sidecar, which handles inter-pod traffic. Our empirical measurement reveals that such per-pod sidecars cause numerous problems, including intrusion into the user pod, excessive resource occupation, significant overhead in managing many sidecars, and performance degradation caused by passing traffic through the sidecar.

Service time is defined as the time taken for a courier to deliver a package to the doorstep of a customer after leaving his vehicle and includes the time taken to return back to his vehicle. Service time is a particularly important parameter in logistics job planning as together with travel time, they determine the number of jobs that can be planned for a day. Traditionally, logistics and transportation companies rely on planners to give a manual estimate of service time for different jobs. However, this process is time consuming and inaccurate. As such, a data-driven, automated Service Time Prediction (STP) method is proposed in this study to provide faster and more accurate service time predictions based on historical data. It does so by first determining historical service time from GPS data by detecting when a service is being conducted. Then, given the historical service times, a KNN regression model is used to provide predictions of service time for new jobs. This study was conducted using two sets of data from 2019 provided to us by two separate logistics and transportation companies based in Singapore. The type of goods delivered are FMCG products.

Identifying traffic accident clusters is vital in helping road users and policymakers make better decisions in managing accident risks. Traffic accidents contain both spatial and temporal dimensions and their interaction should be analyzed to have a better understanding of the nature of the clusters. Similar studies conducted in this area rely on manually sorting data into time buckets before conducting spatial analysis on each of the buckets. While this better than a purely spatial or temporal analysis, the temporal clusters defined by the researcher may not be statistically significant or reveal meaningful space-time interactions. In this paper, we describe the use of Kulldorff's space-time scan statistics to identify traffic accident spatiotemporal clusters. The method identifies clusters by using a scanning cylinder that is varying in size to search for accident cases which are close together in both space and time. The null hypothesis is that the cases are assumed to have constant risk over space and time and follow the Poisson distribution. The Poisson generalized likelihood ratio was determined for each cylinder as a measure of the evidence that it is a hotspot. The clusters were then statistically evaluated using Monte Carlo hypothesis testing. This study was conducted on the 2016 United Kingdom traffic accident dataset and the results show that this method is able to pin point the exact location, size and period of statistically significant clusters.

With wide applications of various types of sensors and web-based software, massive location and time related logistics data is available. The massive spatial and temporal data significantly enhances visibility of modem logistics activities, however, integrating continuous time information to explore spatio-temporal pattern is an outstanding challenge. In this research, we develop a data mining method using image-based method to convert massive spatio-temporal information into heatmap based image sequence where spatial data can be embedded into image frames while image sequence represent time domain. By developing architecture of deep learning neural network with three-dimensional (3D) kernels for different convolution layers, 3D convolutional computation can be applied to images with certain temporal depth to extract dynamic spatio-temporal patterns. The method was validated with real taxi data of New York City. Experimental results demonstrated that dynamic logistics patterns could be identified with historical spatio-temporal information. Comparison of clusters generated by different learning methods showed that the proposed method could produce more accurate spatial-temporal pattern.

With recent development of telemetry technology, various types of sensors have been used in current logistics and transportation industry to automate coordination among logistics companies, deliverers and customers. Global Positioning Sensor (GPS) has been widely used to track delivers' location. However, real-time and on-site environmental information plays a key role in making optimal decisions for vehicle routing and transport navigation. In this study, we propose and develop a data mining method which generates optimal routes based on global spatio-temporal pattern knowledge and local scene information extracted from large-scale and realtime images captured by dash cameras. Scene recognition is used to generate location-based scene information. A temporally weighted route mining model establishing transportation time distribution patterns can be used to produce optimal routes. Experimental results demonstrated that image data from location of different types of road segments could be converted to geospatial information used for spatio-temporal pattern generation.

In recent years, supply chain risk management has captivated both academicians and business practitioners interest, due to increasing catastrophic events and supply chain disruptions. However, the risk management process is highly complex because of the stochastic and dynamic nature and ever growing complexity of supply chains. As the ultimate goal of most enterprises is generating and increasing revenues on the long run, it is valuable to know the effect of specific supply chain risk positions and risk management practices on company sales. In this paper, secondary data on supply chain risk is analyzed and the key risk management strategies responsible for increased company sales are revealed. The novelty of this paper lies in developing a quantifying data mining approach to provide a comprehensive understanding of supply chain risk management (SCRM) and pinpoint focus areas for revenue seeking enterprises. The results prove and showcase that our methodology is capable of providing actionable insights, which were previously unknown or unaddressed.

Realistic transportation time estimation for urban logistics is challenging due to large amount of historical spatial connections and high variability of transportation time caused by inconsistent traffic situations varying in space and time. In this paper, we propose a probability based method using temporal distribution patterns to estimate logistical transportation time among locations in an urban road network. The method explores historical logistics data including location and time data to construct temporally weighted transportation time patterns in spatial domain. It enables a point-based distributed temporal pattern to be extended to probabilistic area-based spatio-temporal pattern. The experimental results demonstrated that the estimated transportation time fell within vicinity of historical temporal records. The method can be used to generate a map of spatial distribution of transportation time which may provide support in decision making process for urban logistics planning and management.

Big Data revolution has transformed business models of many organizations to include the usage of big data analytics. Big Data are believed to be the key basis of competition and growth in today's world whereby huge amounts of data are created daily. One of the main challenges of Big Data is not mainly about the storage of the data but how to blend the different varieties or sources of data together and turn them into values. As the nature of supply chain is complex and dynamic, data are stored in various forms or managed independently. The data have their own naming convention as the data from the different nodes in the supply chain seldom communicate with each other. Some of the challenges of data blending are the lack of unique identifiers to merge the data together and the lack of training data or domain knowledge to understand the criteria to blend the data. In this paper, an automatic filtering and sorting similarity metric, Term Frequency-Inverse Document Frequency (TF-IDF) Ratcliff/Obershelp is proposed. The method is able to handle the issue of same entity with different naming conventions and allow word filtering. The experiment results show that the proposed TF-IDF Ratcliff/Obershelp is able to improve the performance of the data blending.

Optimal logistics hub deployment is a strategic challenge in logistics planning and management. Selecting a proper location for the logistics hub could be significantly impacted by long-term geospatial characteristics of logistics operations including spatial distribution of target customers, convenience of traffic access and operational cost. This paper describes a method using clustering of weighted spatial patterns to find optimal locations for logistics hubs deployment. The underlying concept of this method is that an optimal location of the hub could be determined by logistics operation patterns mined from logistical spatial and temporal data. A logistics spatial pattern can be produced by spatial association rules mining and clustering. The spatial patterns weighted by characteristics of logistics operations are then be clustered to generate the final hub location. In this study, the method is validated with a real data sets of pick-up and delivery business. The experimental results demonstrated that the method was able to generate an optimal location for logistics hub deployment with reduced travel distance to frequent customers' locations.

Automatic identification system (AIS) has been widely equipped on vessels for maritime communication, positioning and traffic monitoring. The comprehensive data obtained by AIS provides spatio-temporal traces depicting the vessels' trajectories and can be used as a coherent source of information for vessels' behavior and the overall maritime traffic analysis, in supporting of the better traffic planning and service optimization. However, it is challenging to process and analysis such a large amount of AIS data that is associated with a great variety of vessels. In this paper, we propose an unsupervised data mining method using density-based strategy to analyze vessels' trajectories and extract the traffic patterns from historical AIS data. It starts with stops and moves identification from vessels' trajectories, followed by the extraction of stationary areas of interest from the stops and the detection of the main traffic routes from the moves using density-based clustering method, which takes both the speed and direction into consideration. Experiments on the real AIS data demonstrate the effectiveness of this work.

In this paper, the radio parameters are designed for OFDM-based millimeter-wave (mmWave) communication systems. Firstly, design principles are presented to meet 5G requirements. Secondly, the radio parameters including TTI length, CP length and sub-carrier spacing for mmWave bands are discussed. In particular the impact of phase noise on subcarrier spacing is analyzed in detail. It shows that the phase noise dominates the sub-carrier spacing design. Based on the measured phase noise model, minimum sub-carrier spacing of 250 kHz and 500 kHz is inferred for typical mmWave bands (30 GHz and 70 GHz). At last a numerology with 600 kHz sub-carrier spacing and 512 OFDM symbols per subframe (1 millisecond) is proposed as a unified one for all mmWave bands, which can satisfy the design principles of mmWave systems.

In order to develop an effective and user-friendly control method for surgical robotic system, we propose a new framework of cognitive engine to supervise and regulate the surgical processes. The framework aims to make the surgical processes understandable by both human operators and robots. A prototype cognitive engine was implemented using ontology and SPARQL query language on JAVA and tested in ex-vivo phantom experiments with a robotic RF needle insertion system. The prototype cognitive engine has successfully guided the robot in execution of surgical procedures.

Route mining for busy waterways is a challenging task. Complicated shipping routes may be generated due to vessels of different types congesting in a narrow water way, frequently changing navigational direction and weaving through multiple crossing traffic. The traditional way using visual bearing and ship-stationed techniques may mitigate hazards of ship collision but lack macroscopic information for safe and efficient shipping navigation. In this paper, we proposed a spatio-temporal mining method to explore vessels' shipping patterns in Singapore Strait. The frequent shipping routes can be automatically extracted using a local polynomial regression based algorithm. Time series clustering across spatial areas is used to associate spatial pattern with temporal pattern. The aim of this study is to provide support for decision-making process in optimal shipping route planning and maritime traffic management. Mapping the pattern information to a virtual geographical information platform enables users to intuitively acquire the knowledge of vessels' shipping patterns.

In this paper, we propose a joint channel estimation and beamforming (BF) scheme for the wide band millimeter wave (mmWave) cellular system. Specifically, low complexity compressive sensing (CS) based estimation algorithm is used to estimate the sparse mmWave channels. Based on the estimated channel, low complexity BF scheme is proposed to adapt to the channel for data communications. The algorithm is designed by considering the practical hardware and channel constraints. Furthermore, the complexity is very low without matrix inverse and singular value decomposition (SVD) compared with traditional algorithms, which is well suitable to practical realization. Finally, we show by simulations that the performance of proposed scheme is close to the optimal scheme with extremely less overhead.