Performance-optimized datacenter networks aim to handle more efficiently the growing East-West intra-cluster traffic of BigData applications. The demanding latency constraints and traffic patterns of these applications expose the inherent bottlenecks of the often oversubscribed datacenter network topologies, favoring in stead the full-bisectional bandwidth fat-trees. And yet their topological benefits may remain unrealized in practical deployments, if such fabrics use single path or flow-level (ECMP hashing) multipath routing. Here we model in detail on Layer 2 the routing performance of modern fat-tree networks using stochastic permutations of bursty traffic. We first analytically simplify and then validate by accurate simulation models that the throughputs for ‘static’ d-mod-k and for ECMP-like multipath routing are 63% and 47%, respectively. We also find that ECMP routing results in a wide spread of link loads under random permutation traffic, which manifests as a 3x throughput reduction for 30% of the flows. Furthermore, ECMP can lead to collisions of mouse and elephant flows, often increasing the flow completion time (FCT) of delay-sensitive flows by a factor of 10. In contrast, packet-based multipath outperforms all the others in this study.

This paper presents two complementary techniques to manage the power consumption of large-scale systems with a packet-switched interconnection network. First, we propose Thrifty Interconnection Network (TIN), where the network links are activated and de-activated dynamically with little or no overhead by using inherent system events to timely trigger link activation or de-activation. Second, we propose Network Power Shifting (NPS) that dynamically shifts the power budget between the compute nodes and their corresponding network components. TIN activates and trains the links in the interconnection network, just-in-time before the network communication is about to happen, and thriftily puts them into a low-power mode when communication is finished, hence reducing unnecessary network power consumption. Furthermore, the compute nodes can absorb the extra power budget shifted from its attached network components and increase their processor frequency for higher performance with NPS. Our simulation results on a set of real-world workload traces show that TIN can achieve on average 60% network power reduction, with the support of only one low-power mode. When NPS is enabled, the two together can achieve 12% application performance improvement and 13% overall system energy reduction. Further performance improvement is possible if the compute nodes can speed up more and fully utilize the extra power budget reinvested from the thrifty network with more aggressive cooling support.

We propose Thrifty Interconnection Network (TIN), where the network links are activated and de-activated dynamically to save power with little or no overhead by using inherent system events to overlap the link activation or de-activation time. Our simulation results on a set of real world HPC workload traces show on average 35% network power reduction.

The efficiency of computer network has been regarded as a bottleneck in parallel computing paradigm. It is important to have efficient methodology to obtain network performance measures, especially for a large scale system, i.e. exa-scale system. Communication performance is often investigated by the static complexity analysis based on a given network topology or a detailed network simulation, which is often time consuming. To provide a dynamic and scalable communication performance measure, we first propose an aggregate multi-stage queueing network model to capture the application's communication load and derive the closed-form system performance, i.e. throughput and delay. Trace simulation results obtained from a sophisticated simulator, Venus, show that the proposed model is accurate, yet simple. Secondly, we develop a link bandwidth optimization framework, which optimally allocates/distributes link bandwidth across the network to maximize the system communication throughput. Specifically, we apply the derived optimal bandwidth allocation on dimensioning link bandwidth of an exploratory direct network and slimming fat-tree network. Our results show that the proposed methodology is cost-effective in providing system performance and design explorations for the existing and the next-generation network system.

This paper considers a general parallel buffered packet switch (PBPS) architecture which is based on multiple packet switches operating independently and in parallel. A load-balancing mechanism is used at each input to distribute the traffic to the parallel switches. The buffer structure of each of the parallel packet switches is based on either a dedicated, a shared, or a buffered-crosspoint output-queued architecture. As in such PBPS multipath switches, packets may get out of order when they travel independently in parallel through these switches, a resequencing mechanism is necessary at the output side. This paper addresses the issue of evaluating the minimum resequence-queue size required for a deadlock-free lossless operation. An analytical method is presented for the exact evaluation of the worst-case resequencing delay and the worst-case resequence-queue size. The results obtained reveal their relation, and demonstrate the impact of the various system parameters on resequencing

A single-stage nonblocking N*N packet switch with both output and input queuing is considered. The limited queuing at the output ports resolves output port contention partially. Overflow at the output queues is prevented by a backpressure mechanism and additional queuing at the input ports. The impact of the backpressure effect on the switch performance for arbitrary output buffer sizes and for N to infinity is studied. Two different switch models are considered: an asynchronous model with Poisson arrivals and a synchronous model with Bernoulli arrivals. The investigation is based on the average delay and the maximum throughput of the switch. Closed-form expressions for these performance measures are derived for operation with fixed size packets. The results demonstrate that a modest amount of output queuing, in conjunction with appropriate switch speedup, provides significant delay and throughput improvements over pure input queuing. The maximum throughput is the same for the synchronous and the asynchronous switch model, although the delay is different. >

A survey of high-performance switch fabric architectures which incorporate fast packet switching as their underlying switching technique to handle various traffic types is presented. A descriptive overview of the major activities in this rapidly evolving field of telecommunications is given. The switch fabrics are classified into the following categories: banyan and buffered banyan-based fabrics, sort-banyan-based fabrics fabrics with disjoint-path topology and output queuing, crossbar-based fabrics, time division fabrics with common packet memory, and fabrics with shared medium.>

An architecture is described for a high-performance switching fabric that can accommodate circuit-switched and packet-switched traffic in a unified manner. The switch fabric is self-routing and uses fixed-length minipackets within the switching fabric for all types of connections. Its architecture provides full input/output connectivity paths with FIFO (first-in-first-out) queueing at each output. The connection paths are non-overlapping, so there is no internal blocking. Because of output queueing, there is also practically no output port blocking. The uniformity in architecture allows the construction of any size fabric from a single basic module which could be realized on a single chip. Larger-size configurations can be realized either as single-stage configuration.>