Permutation invariant neural networks are a promising tool for predictive modeling of set data. We show, however, that existing architectures struggle to perform well when they are deep. In this work, we mathematically and empirically analyze normalization layers and residual connections in the context of deep permutation invariant neural networks. We develop set norm, a normalization tailored for sets, and introduce the “clean path principle” for equivariant residual connections alongside a novel benefit of such connections, the reduction of information loss. Based on our analysis, we propose Deep Sets++ and Set Transformer++, deep models that reach comparable or better performance than their original counterparts on a diverse suite of tasks. We additionally introduce Flow-RBC, a new single-cell dataset and real-world application of permutation invariant prediction. We open-source our data and code here: https://github.com/rajesh-lab/deep_permutation_invariant.

Human red blood cells (RBCs) lose ∼30% of their volume and ∼20% of their hemoglobin (Hb) content during their ∼100-day lifespan in the bloodstream. These observations are well-documented, but the mechanisms for these volume and hemoglobin loss events are not clear. RBCs shed hemoglobin-containing vesicles during their life in the circulation, and this process is thought to dominate the changes in the RBC physical characteristics occurring during maturation. We combine theory with single-cell measurements to investigate the impact of vesiculation on the reduction in volume, Hb mass, and membrane. We show that vesicle shedding alone is sufficient to explain membrane losses but not volume or Hb losses. We use dry mass measurements of human RBCs to validate the models and to propose that additional unknown mechanisms control volume and Hb reduction and are responsible for ∼90% of the observed reduction. RBC population characteristics are used in the clinic to monitor and diagnose a wide range of conditions including malnutrition, inflammation, and cancer. Quantitative characterization of cellular maturation processes may help in the early detection of clinical conditions where maturation patterns are altered.

Blood is a dense suspension of soft non-Brownian cells of unique importance. Physiological blood flow involves complex interactions of blood cells with each other and with the environment due to the combined effects of varying cell concentration, cell morphology, cell rheology, and confinement. We analyze these interactions using computational morphological image analysis and machine learning algorithms to quantify the non-equilibrium fluctuations of cellular velocities in a minimal, quasi-two-dimensional microfluidic setting that enables high-resolution spatio-temporal measurements of blood cell flow. In particular, we measure the effective hydrodynamic diffusivity of blood cells and analyze its relationship to macroscopic properties such as bulk flow velocity and density. We also use the effective suspension temperature to distinguish the flow of normal red blood cells and pathological sickled red blood cells and suggest that this temperature may help to characterize the propensity for stasis in Virchow's Triad of blood clotting and thrombosis.