With increasing sizes of DNN (Deep Neural Network) models making them exceed the memory of a single device (GPU), model parallelism-based training has become paramount, splitting a model across multiple devices. Unfortunately, today’s model parallelism approaches often result in memory-unbalanced allocations of the model across the multiple devices, with some devices’ memory heavily utilized while others remain underused. This imbalance limits deployments from reaching high batch sizes, triggers Out of Memory (OOM) errors earlier, and underutilizes resources. We present Wainscot, a model parallelism solution that produces memory-balanced placements of a DNN model across multiple devices, without noticeable increases in step time. We explore and empirically compare different granularities of rebalancing: operators, operator groups, and subgraphs. Experiments with diverse DNNs across a wide range of batch sizes demonstrate that compared to state-of-the-art model parallelism systems, Wainscot reduces maximal peak memory (across devices) by 47.94%, with a modest increase of 1.62% in step time.

Molecular communication has recently gained a lot of interest due to its potential to enable micro-implants to communicate by releasing molecules into the bloodstream. In this paper, we aim to explore the molecular communication channel through theoretical and empirical modeling in order to achieve a better understanding of its characteristics, which tend to be more complex in practice than traditional wireless and wired channels. Our study reveals two key new characteristics that have been overlooked by past work. Specifically, the molecular communication channel exhibits non-causal inter-symbol-interference and a long delay spread, that extends beyond the channel coherence time, which limit decoding performance. To address this, we design, μ-Link a molecular communication protocol and decoder that accounts for these new insights. We build a testbed to experimentally validate our findings and show that μ-Link can improve the achievable data rates with significantly lower bit error rates.

In a coherent time-varying MIMO multiple access channel (MAC), the ergodic sum-capacity can be achieved by suitable power control, which adapts the transmit covariance matrices based on the available channel state information (CSI). We consider a MIMO-MAC model where each transmitter is only aware of its own fading realization. For this distributed MAC, we propose decentralized power control schemes. First, some key structural properties on the optimal power control laws are derived, followed by an upper bound on the sum-capacity. Numerical results show that the proposed power control schemes closely characterize the sum-capacity in several regimes of interest.