Existing data-plane verifiers face severe performance challenges in verifying hyper-scale underlay data center networks (DCNs) – centralized verifiers often fail to fully exploit the parallelism offered by the modern multi-core CPUs, while distributed verifiers suffer from high overhead due to task distribution and inter-node communication. To overcome these limitations, this paper introduces HeTu, a high-performance centralized parallel data-plane verifier specifically for verifying hyper-scale underlay DCNs. HeTu achieves ultra-fast verification through three key designs: (1) a fully parallel verification framework with small graph construction overhead, (2) a new binary decision diagram management strategy that enables full parallelism by using separated storage and selectively indexing and caching network-level predicates to reduce redundant operations, and (3) an optimized forwarding graph model that aggregates parallel tasks to eliminate redundant computation. Extensive evaluations on synthetic FatTree and large-scale production datasets show that HeTu outperforms state-of-the-art algorithms in runtime by 100× to 6000×, demonstrating its superior scalability and efficiency in data-plane verification of hyper-scale DCNs.

Network configuration verifiers can proactively reason about a network's correctness to prevent network outages. However, even recent efforts have proposed algorithms to "scale up" the verification to several thousand switches, these algorithms still cannot be used for networks with more than 10K switches or 1000M routes, which is common for large service providers. In this paper, instead of further scaling up the verification limited to a single server, we study how to "scale out" the verification using the resources of multiple servers. To achieve this, we propose S2, a distributed verifier for network configurations. S2 partitions the network model and distributes the verification tasks, i.e., control plane simulation and data plane verification, to run on multiple servers in parallel. Additionally, S2 uses prefix sharding during control plane simulation to further reduce the memory footprint on each server. We implement a prototype of S2 based on Batfish, the state-of-the-art network verifier. Based on real datacenter topologies of a large service provider and synthetic FatTree topologies, we show that S2 can verify networks with 10K routers and 1000M routes within 2 hours.

At internet scale companies like ByteDance, data is generated and consumed at enormously high speed by many different applications. Achieving low latency on such big data jobs is an important problem. However, the naive approach of aggregating all the data required by a job to a single location is not always feasible in a geo-distributed environment. Similarly, existing approaches in geo-distributed job scheduling often try to minimize WAN usage, which may come at the cost of latency. Another crucial element to ensure low latency is resource load balancing among DCs, which enables flexibility in job scheduling and avoids resource bottlenecks. Therefore, to minimize latency, optimizing job completion time (JCT) while maintaining resource utilization balance is important. To this end, we propose ResLake , a global scheduling platform for data-intensive workloads. ResLake aims to reduce JCT of geo-distributed applications while balancing the compute (CPU/Memory) and storage (Disk) usages across DCs and efficiently using WAN interconnections. We have deployed ResLake in ByteDance's production for over 1.5 years. ResLake has scheduled billions of jobs since its deployment. We find that ResLake improves JCT of jobs by at least 20%, and can improve resource utilization balance across DCs by up to 53%.

Business processes are designed to achieve business goals under procedural rules by orchestrating tasks, information and documents. Managing data inconsistency in business processes is a challenging task. If not managed properly, business will face negative financial consequences. From literatures, BPMN modelling approaches deal with inconsistency problem by patterns; data provenance approaches analyze data generated in business process and investigate the reachability between data points. Although substantial works have been done, the data inconsistency problem has not been properly resolved. In particular, it is still lacking of modelling language and resolution for inconsistency caused by multiple starting points of business processes and dynamics of business processes execution. This paper provides data consistency solution in two aspects: a business process modelling in enriched business workflow notation with data states and temporal properties, and a workflow query algorithm to discovery data inconsistency issue.

The precondition for stepping-stone analysis is to record network events through network monitors. Little work has been done on how to place monitors. In this paper, we propose the technique for the optimal placement of passive monitors in a network where there are constraints on the number of available monitors for deployment. The placement problem is defined in terms of information theory metrics. For a given number of monitors and network topology, average entropy and "worst-case" entropy that describe the remaining uncertainty in the origin of an attack when monitors work perfectly are considered as the optimal object. A brief proof that the worst-case deployment problem is NP-complete is presented. Greedy algorithms based on graph centrality heuristics for finding high quality deployments are introduced to solve this problem. An automatic monitor placement tool, which implements our approach, is developed and we use real network topology in the experiments to evaluate our results.