This paper introduces the newly developed Infini-Band (IB) Management Queue capability, used by the Host Channel Adapter (HCA) to manage network task data flow dependancies, and progress the communications associated with such flows. These tasks include sends, receives, and the newly supported wait task, and are scheduled by the HCA based on a data dependency description provided by the user. This functionality is supported by the ConnectX-2 HCA, and provides the means for delegating collective communication management and progress to the HCA, also known as collective communication offload. This provides a means for overlapping collective communications managed by the HCA and computation on the Central Processing Unit (CPU), thus making it possible to reduce the impact of system noise on parallel applications using collective operations. This paper further describes how this new capability can be used to implement scalable Message Passing Interface (MPI) collective operations, describing the high level details of how this new capability is used to implement the MPI_Barrier collective operation, focusing on the latency sensitive performance aspects of this new capability. This paper concludes with small scale benchmark experiments comparing implementations of the barrier collective operation, using the new network offload capabilities, with established point-to-point based implementations of these same algorithms, which manage the data flow using the central processing unit. These early results demonstrate the promise this new capability provides to improve the scalability of high-performance applications using collective communications. The latency of the HCA based implementation of the barrier is similar to that of the best performing point-to-point based implementation managed by the central processing unit, starting to outperform these as the number of processes involved in the collective operation increases.

Redundant traffic dispersal exploits the topological redundancy of networks and improves load balancing by replicating each message or partitioning it into several "data" packets and generating several "redundant" ones; all are then sent over different paths to the destination. The redundancy overcomes the "weakest link" problem, but increases the load. This paper introduces "prioritized dispersal", whereby "redundant" packets receive lower priority than the "data" ones. Moreover, the use of non-FCFS queuing policies for the redundant packets leads to the timely arrival of at least a fraction of them even under heavy load. Queuing-theoretic analysis shows the new schemes to substantially outperform non-prioritized ones in terms of both the blocking probability and that of delay exceeding a specified limit. One possible use of prioritized dispersal, which is discussed in this paper, is to improve the quality of service for best-effort traffic in ATM networks with multiple paths between nodes. Another is in conjunction with ad hoc path trunking. Additional likely uses include parallel access to mirrored data sites and reliable multicast.

The most prominent single-hop interconnection (SHI) topology is the single broadcast channel, or bus, which is used in local area networks (LANs) such as Ethernet. The uniform-traffic capacity of switchless, non-bus-oriented, fiber-optic SHIs among N stations, each equipped with a small number of transmitters and receivers, can be as high as /spl Theta/(log/sub 2/N) concurrent transmissions on a single wavelength with round-robin scheduling in a time-slotted system. However, their capacity with the slotted ALOHA access scheme does not increase with N. (The capacity of bus-oriented interconnections, in contrast, varies across time-slotted access schemes by, at most, a factor of e.) This paper quantifies the contribution of several factors to capacity. Merely avoiding destructive interference with ongoing receptions contributes, at most, a factor of e over slotted ALOHA, the same as in bus-oriented interconnections. For an interconnection among two-transmitter, single-receiver stations, whose capacity is log/sub 2/N with global scheduling and 2/e with slotted ALOHA, also avoiding transmissions to blocked receivers increases capacity to, at most, log/sub 2/log/sub 2/N. These results suggest that the added complexity of non-bus-oriented SHIs may be warranted only if they are operated in ways that permit the selection of "good" combinations of (source, destination) pairs for concurrent transmission, and further research should focus on those.