Statistical heterogeneity in federated learning poses two major challenges: slow global training due to conflicting gradient signals, and the need of personalization for local distributions. In this work, we tackle both challenges by leveraging recent advances in \emph{linear mode connectivity} --- identifying a linearly connected low-loss region in the parameter space of neural networks, which we call solution simplex. We propose federated learning over connected modes (\textsc{Floco}), where clients are assigned local subregions in this simplex based on their gradient signals, and together learn the shared global solution simplex. This allows personalization of the client models to fit their local distributions within the degrees of freedom in the solution simplex and homogenizes the update signals for the global simplex training. Our experiments show that \textsc{Floco} accelerates the global training process, and significantly improves the local accuracy with minimal computational overhead in cross-silo federated learning settings.

Selecting the right resources for big data analytics jobs is hard because of the wide variety of configuration options like machine type and cluster size. As poor choices can have a significant impact on resource efficiency, cost, and energy usage, automated approaches are gaining popularity. Most existing methods rely on profiling recurring workloads to find near-optimal solutions over time. Due to the cold-start problem, this often leads to lengthy and costly profiling phases. However, big data analytics jobs across users can share many common properties: they often operate on similar infrastructure, using similar algorithms implemented in similar frameworks. The potential in sharing aggregated profiling runs to collaboratively address the cold start problem is largely unexplored. We present Karasu, an approach to more efficient resource configuration profiling that promotes data sharing among users working with similar infrastructures, frameworks, algorithms, or datasets. Karasu trains lightweight performance models using aggregated runtime information of collaborators and combines them into an ensemble method to exploit inherent knowledge of the configuration search space. Moreover, Karasu allows the optimization of multiple objectives simultaneously. Our evaluation is based on performance data from diverse workload executions in a public cloud environment. We show that Karasu is able to significantly boost existing methods in terms of performance, search time, and cost, even when few comparable profiling runs are available that share only partial common characteristics with the target job.

Abstract The growing electricity demand of IT infrastructure has raised significant concerns about its carbon footprint. To mitigate the associated emissions of computing systems, current efforts therefore increasingly focus on aligning the power usage of software with the availability of clean energy. To operate, such carbon‐aware applications require visibility and control over relevant metrics and configurations of the energy system. However, research and development of novel energy system abstraction layers and interfaces remain difficult due to the scarcity of available testing environments: Real testbeds are expensive to build and maintain, while existing simulation testbeds are unable to interact with real computing systems. To provide a widely applicable approach for developing and testing carbon‐aware software, we propose a method for integrating real applications into a simulated energy system through software‐in‐the‐loop simulation. The integration offers an API for accessing the energy system, while continuously modeling the computing system's power demand within the simulation. Our system allows for the integration of physical as well as virtual compute nodes, and can help accelerate research on carbon‐aware computing systems in the future.

Despite constant improvements in efficiency, today's data centers and networks consume enormous amounts of energy and this demand is expected to rise even further. An important research question is whether and how fog computing can curb this trend. As real-life deployments of fog infrastructure are still rare, a significant part of research relies on simulations. However, existing power models usually only target particular components such as compute nodes or battery-constrained edge devices.Combining analytical and discrete-event modeling, we develop a holistic but granular energy consumption model that can determine the power usage of compute nodes as well as network traffic and applications over time. Simulations can incorporate thousands of devices that execute complex application graphs on a distributed, heterogeneous, and resource-constrained infrastructure. We evaluated our publicly available prototype LEAF within a smart city traffic scenario, demonstrating that it enables research on energy-conserving fog computing architectures and can be used to assess dynamic task placement strategies and other energy-saving mechanisms.

Depending on energy sources and demand, the carbon intensity of the public power grid fluctuates over time. Exploiting this variability is an important factor in reducing the emissions caused by data centers. However, regional differences in the availability of low-carbon energy sources make it hard to provide general best practices for when to consume electricity. Moreover, existing research in this domain focuses mostly on carbon-aware workload migration across geo-distributed data centers, or addresses demand response purely from the perspective of power grid stability and costs.

Random Forests (RFs) are a powerful machine learning technique used for various applications including classification, regression, clustering, and manifold learning. The interpretation of a given Random Forest usually relies on statistical values, such as the distribution of path length, leaf impurity, leaf size, etc. All those measures focus on specific aspects and are incapable to provide a holistic understanding of the RF. In this paper, we propose a two-dimensional, easy-to-grasp visualization technique that follows a botanical approach and illustrates several key parameters necessary to understand why a given RF performs in a certain way. The method allows customized mappings of RF characteristics to visual properties and provides the possibility to interactively analyze the forest structure. This allows to determine trees that perform extraordinarily well or bad, to analyze the reasons for their performance, and thus to gain insights into how to change parameter setting to increase performance or efficiency.