GPU clusters serve as a cornerstone of high-performance computing and support a wide range of batch jobs with complex resource demands. However, the diverse requirements of batch jobs and resource heterogeneity present significant challenges to efficient scheduling. Existing approaches either rely on static rules or overlook the interdependencies among virtual machines introduced by resource heterogeneity, making it difficult to address the diverse resource demands and dynamic load balancing. In this paper, we propose a novel scheduling model based on Graph Neural Networks (GNNs) and Double Deep Q-Networks (DDQNs), termed GNN-DDQN, for batch job load balancing and scheduling of multi-dimensional resources (e.g. GPU, CPU and memory) in heterogeneous clusters. We propose a system model that integrates batch job resource requests, multi-dimensional resource configurations, and a multi-objective optimization framework. The scheduling problem is formulated as a Markov Decision Process based on this model. A GNN is employed to effectively capture the interdependencies among virtual machines, while a DDQN optimizes scheduling decisions using a dynamic target network update mechanism. Extensive experiments are conducted using two real-world Alibaba cluster traces. Results demonstrate the effectiveness and generalization capabilities of the proposed scheduling model. Compared to baseline methods, the results confirm its superiority in load balancing, job latency, fairness, and time efficiency.

Per-flow size measurement is key to many streaming applications and management systems, particularly in high-speed networks. Performing such measurement on the data plane of a network device at the line rate requires on-chip memory and computing resources that are shared by other key network functions. It leads to the need for very compact and fast data structures, called sketches, which trade off space for accuracy. Such a need also arises in other application context for extremely large data sets. The goal of sketch design is two-fold: to measure flow size as accurately as possible and to do so as efficiently as possible (for low overhead and thus high processing throughput). The existing sketches can be broadly categorized to multi-update sketches and single update sketches. The former are more accurate but carry larger overhead. The latter incur small overhead but their accuracy is poor. This paper proposes a Single update Sketch with a Variable counter Structure (SSVS), a new sketch design which is several times faster than the existing multi-update sketches with comparable accuracy, and is several times more accurate than the existing single update sketches with comparable overhead. The new sketch design embodies several technical contributions that integrate the enabling properties from both multi-update sketches and single update sketches in a novel structure that effectively controls the measurement error with minimum processing overhead.

With the rapid development of the Internet of Things (IoT), massive high-speed data streams are produced every moment, making accurate size estimation a challenging task. Many sketches have been proposed to summarize real-time high-speed data streams and provide per-flow size estimations. However, sketches have to share the memory units to fit in limited on-chip space, inevitably introducing noises to all flows and resulting in over-estimation problems. Prior work adopts an average denoising strategy to remove the same noise from raw sketch estimations. However, they overlook that the noise distribution is highly skewed, leading to inaccurate results for most flows. This paper proposes an efficient Adaptive Noise Correction (ANC) framework, which analyzes the noise of each flow on a case-by-case basis and provides accurate size estimations. The key of our design is to build an ML model to predict a weight coefficient that indicates the noises in raw estimations, which is conducted for each flow by analyzing the neighbor flows whose memory units overlap with the given flow. Then we introduce a novel Probabilistic Cold Filter to block the tiny flows and assist in noise correction. Experimental results based on real Internet traces show that our framework can effectively remove the noises for different sketches, showing better estimation accuracy than the state-of-the-art.

The traditional Gaussian mixture background model failed to build a reasonable background in complex scenarios, so this paper proposes an improved self-adaptive Gaussian mixture background modeling which integrates the difference method and adaptive threshold segmentation to improve the traditional one. In the proposed model, the difference method is applied to achieve the segmentation of changing area and background area, and different weight update policies are used for different areas. Background area updates background model with a fixed update rate. The changing region is divided into moving target area and background show area with the fusion of adaptive threshold; the background show area has a large update rate, allowing the previously obscured parts to recover rapidly; the moving target area won’t build new Gaussian components for the Gaussian mixture model. Experiments show that the video sequence algorithm with uncertainties of building background model has good adaptability. It helps to improve computing speed to a large extent and it can also respond to the change of the actual scene quickly.

Software-defined datacenters combine centralized resource management, software-defined networking, and virtualized infrastructure to meet diverse requirements of cloud computing. To fully realizing their capability in traffic engineering and flow-based bandwidth management, it is critical for the switches to measure network traffic for both individual flows between virtual machines and aggregate flows between clusters of physical or virtual machines. This paper proposes a novel hierarchical traffic measurement scheme for software-defined datacenter networks. It measures both aggregate flows and individual flows that are organized in a hierarchy with an arbitrary number of levels. The measurement is performed based on a new concept of hierarchical virtual counter arrays, which record each packet only once by updating a single counter, yet the sizes of all flows that the packet belongs to will be properly updated. We demonstrate that the new measurement scheme not only supports hierarchical traffic measurement with accuracy, but does so with memory efficiency, using a fewer number of counters than the number of flows.

The Internet has become a fundamental platform for virtually all social, economical and security activities in modern societies. Monitoring widespread events on the Internet has many important applications in social trend studies and distributed intrusion/attack detection. This paper studies the problem of distributed joint detection of widespread events observed by the collaborating network devices (called watchers). In order to work with a large number of watchers, the recent work requires a central coordinator to help detect the common events. The central coordinator however has the problems of single- point of failure, fairness and trust issue in coordinator placement, and communication bottleneck at the coordinator. This paper proposes a fully- distributed solution for joint detection of common events based on a peer-to-peer model without using a central coordinator. Our design adopts an iterative set-join process that follows the structure of a hypercube, which reduces the communication complexity from O(n m) to O(m log n), where m is the size of the largest event set at any device and n is the number of collaborating devices.

Future RFID technologies will go far beyond today's widely used passive tags. Battery-powered active tags are likely to gain more popularity due to their long operational ranges and richer on-tag resources. With integrated sensors, these tags can provide not only static identification numbers but also dynamic, real-time information such as sensor readings. This paper studies a general problem of how to design efficient polling protocols to collect such real-time information from a subset M of tags in a large RFID system. We show that the standard, straightforward polling design is not energy-efficient because each tag has to continuously monitor the wireless channel and receive O(|M|) tag IDs, which is energy-consuming. Existing work is able to cut the amount of data each tag has to receive by half through a coding design. In this paper, we propose a tag-ordering polling protocol (TOP) that can reduce per-tag energy consumption by more than an order of magnitude. We also reveal an energy-time tradeoff in the protocol design: per-tag energy consumption can be reduced to O(1) at the expense of longer execution time of the protocol. We then apply partitioned Bloom filters to enhance the performance of TOP, such that it can achieve much better energy efficiency without degradation in protocol execution time. Finally, we show how to configure the new protocols for time-constrained energy minimization.

Collecting information from mobile peer-to-peer (P2P) networks has important civilian and military applications. One problem is to determine the cardinality, i.e., the number of nodes, in a large mobile system. In a stationary wireless network, it can be trivially solved through a flooding-based query. However, the problem becomes much more challenging for mobile P2P networks whose topologies are constantly changing. In this paper, we present two novel statistical methods, called the circled random walk and the tokened random walk, to address this interesting problem. The circled random walk is simpler to implement and works well in networks of high mobility, whereas the tokened random walk works well with high or low mobility. These methods provide cardinality estimation by involving only a small subset of the nodes. They make tradeoff between overhead and estimation accuracy. The estimation error can be made arbitrarily small at the expense of larger overhead.

RFID tags have many important applications in automated warehouse management. One example is to monitor a set of tags and detect whether some tags are missing - the objects to which the missing tags are attached are likely to be missing, too, due to theft or administrative error. Prior research on this problem has primarily focused on efficient protocols that reduce the execution time in order to avoid disruption of normal inventory operations. This paper makes several new advances. First, we observe that the existing protocol is far from being optimal in terms of execution time. We are able to cut the execution time to a fraction of what is currently needed. Second, we study the missing-tag detection problem from a new energy perspective, which is very important when battery-powered active tags are used. The new insight provides flexibility for the practitioners to meet their energy and time requirements.

Future RFID technologies will go far beyond today's widely-used passive tags. Battery-powered active tags are likely to gain more popularity due to their long operational ranges and richer on-tag resources. With integrated sensors, these tags can provide not only static identification numbers but also dynamic, real-time information such as sensor readings. This paper studies a general problem of how to design efficient polling protocols to collect such real-time information from a subset M of tags in a large RFID system. We show that the standard, straightforward polling design is not energy-efficient because each tag has to continuously monitor the wireless channel and receive O(|M|) tag IDs, which is energy-consuming. Existing work is able to cut the amount of data each tag has to receive by half through a coding design. In this paper, we propose a tag-ordering polling protocol (TOP) that can reduce per-tag energy consumption by more than an order of magnitude. We also reveal an energy-time tradeoff in the protocol design: per-tag energy consumption can be reduced to O(1) at the expense of longer execution time of the protocol. Finally, we apply partitioned Bloom filters to enhance the performance of TOP, such that it can achieve much better energy efficiency without degradation in protocol execution time.

Collecting information from mobile wireless networks has important civilian and military applications. One problem is to determine the number of nodes in a large wireless system. In a stationary wireless network, it can be trivially solved through a flooding-based query. However, the problem becomes much more challenging for mobile ad-hoc networks whose topologies are constantly changing. In this paper, we present two novel statistical methods, called the circled random walk and the tokened random walk, to address this interesting problem. These methods provide an estimation by involving only a small subset of the nodes. They make tradeoff between the overhead and the estimation accuracy. The estimation error can be made arbitrarily small at the expense of larger overhead.

End-to-end fairness in a sensor network ensures that data from each sensor has an equal (or weighted) chance to reach the sink. It eliminates the problem that data flows from sensors at some locations (close to the sink) obtain most network bandwidth, while flows from sensors at other locations (far away from the sink) are starved. Existing fairness solutions assume single-path routing, which reduces achievable throughput of the network. In this paper, we propose a multipath fairness solution (MFS) that achieves end-to-end fairness among competing flows through fully distributed operations. MFS is easy to implement, which is advantageous in a resource-scarce sensor network. More importantly, it achieves much higher network throughput and better fairness among the flows.

rdquoApplication-level multicast is a promising alternative to IP multicast due to its independence from the IP routing infrastructure and its flexibility in constructing the delivery trees. The existing overlay multicast systems either support a single data source or have high maintenance overhead when multiple sources are allowed. They are inefficient for applications that require any-source multicast with varied host capacities and dynamic membership. This paper proposes ACOM, an any-source capacity-constrained overlay multicast system, consisting of three distributed multicast algorithms on top of a non-DHT overlay network with simple structures (random overlay with a non-DHT ring) that are easy to manage as nodes join and depart. The nodes have different capacities, and they can support different numbers of direct children during a multicast session. No explicit multicast trees are maintained on top of the overlay. The distributed execution of the algorithms naturally defines an implicit, roughly balanced, capacity-constrained multicast tree for each source node. We prove that the system can deliver a multicast message from any source to all nodes in expected O(logcn) hops, which is asymptotically optimal, where c is the average node capacity and n is the number of members in a multicast group.