Federated learning (FL) allows model training from local data collected by edge/mobile devices while preserving data privacy, which has wide applicability to image and vision applications. A challenge is that client devices in FL usually have much more limited computation and communication resources compared to servers in a data center. To overcome this challenge, we propose PruneFL -a novel FL approach with adaptive and distributed parameter pruning, which adapts the model size during FL to reduce both communication and computation overhead and minimize the overall training time, while maintaining a similar accuracy as the original model. PruneFL includes initial pruning at a selected client and further pruning as part of the FL process. The model size is adapted during this process, which includes maximizing the approximate empirical risk reduction divided by the time of one FL round. Our experiments with various datasets on edge devices (e.g., Raspberry Pi) show that: 1) we significantly reduce the training time compared to conventional FL and various other pruning-based methods and 2) the pruned model with automatically determined size converges to an accuracy that is very similar to the original model, and it is also a lottery ticket of the original model.

Supervised machine learning (ML) algorithms are aimed at maximizing classification performance under available energy and storage constraints. They try to map the training data to the corresponding labels while ensuring generalizability to unseen data. However, they do not integrate meaning-based relationships among labels in the decision process. On the other hand, natural language processing (NLP) algorithms emphasize the importance of semantic information. In this article, we synthesize the complementary advantages of supervised ML and NLP algorithms into one method that we refer to as SECRET (Semantically Enhanced Classification of REal-world Tasks). SECRET performs classifications by fusing the semantic information of the labels with the available data: it combines the feature space of the supervised algorithms with the semantic space of the NLP algorithms and predicts labels based on this joint space. Experimental results indicate that, compared to traditional supervised learning, SECRET achieves up to 14.0 percent accuracy and 13.1 percent F1 score improvements. Moreover, compared to ensemble methods, SECRET achieves up to 12.7 percent accuracy and 13.3 percent F1 score improvements. This points to a new research direction for supervised classification based on incorporation of semantic information.

Many image and vision applications require a large amount of data for model training. Collecting all such data at a central location can be challenging due to data privacy and communication bandwidth restrictions. Federated learning is an effective way of training a machine learning model in a distributed manner from local data collected by client devices, which does not require exchanging the raw data among clients. A challenge is that among the large variety of data collected at each client, it is likely that only a subset is relevant for a learning task while the rest of data has a negative impact on model training. Therefore, before starting the learning process, it is important to select the subset of data that is relevant to the given federated learning task. In this paper, we propose a method for distributedly selecting relevant data, where we use a benchmark model trained on a small benchmark dataset that is task-specific, to evaluate the relevance of individual data samples at each client and select the data with sufficiently high relevance. Then, each client only uses the selected subset of its data in the federated learning process. The effectiveness of our proposed approach is evaluated on multiple real-world image datasets in a simulated system with a large number of clients, showing up to 25% improvement in model accuracy compared to training with all data.

We address energy-efficient placement of data and analytics components of composite analytics services on a wireless network to minimize execution-time energy consumption (computation and communication) subject to compute, storage and network resource constraints. We introduce an expressive analytics service hypergraph model for representing k-ary composability relationships (k ≥ 2) between various analytics and data components and leverage binary quadratic programming (BQP) to minimize the total energy consumption of a given placement of the analytics hypergraph nodes on the network subject to resource availability constraints. Then, after defining a potential energy functional Φ(·) to model the affinities of analytics components and network resources using analogs of attractive and repulsive forces in physics, we propose a decentralized Metropolis Monte Carlo (MMC) sampling method which seeks to minimize Φ by moving analytics and data on the network. Although Φ is non-convex, using a potential game formulation, we identify conditions under which the algorithm provably converges to a local minimum energy equilibrium placement configuration. Trace-based simulations of the placement of a deep-neural-network analytics service on a realistic wireless network show that for smaller problem instances our MMC algorithm yields placements with total energy within a small factor of BQP and more balanced workload distributions; for larger problems, it yields low-energy configurations while the BQP approach fails.

Large-scale distributed computing systems often contain thousands of distributed nodes (machines). Monitoring the conditions of these nodes is important for system management purposes, which, however, can be extremely resource demanding as this requires collecting local measurements of each individual node and constantly sending those measurements to a central controller. Meanwhile, it is often useful to forecast the future system conditions for various purposes such as resource planning/allocation and anomaly detection, but it is usually too resource-consuming to have one forecasting model running for each node, which may also neglect correlations in observed metrics across different nodes. In this paper, we propose a mechanism for collecting and forecasting the resource utilization of machines in a distributed computing system in a scalable manner. We present an algorithm that allows each local node to decide when to transmit its most recent measurement to the central node, so that the transmission frequency is kept below a given constraint value. Based on the measurements received from local nodes, the central node summarizes the received data into a small number of clusters. Since the cluster partitioning can change over time, we also present a method to capture the evolution of clusters and their centroids. As an effective way to reduce the amount of computation, time-series forecasting models are trained on the time-varying centroids of each cluster, to forecast the future resource utilizations of a group of local nodes. The effectiveness of our proposed approach is confirmed by extensive experiments using multiple real-world datasets.

In this paper, we present a novel incremental and decremental learning method for the least-squares support vector machine (LS-SVM). The goal is to adapt a pre-trained model to changes in the training dataset, without retraining the model on all the data, where the changes can include addition and deletion of data samples. We propose a provably exact method where the updated model is exactly the same as a model trained from scratch using the entire (updated) training dataset. Our proposed method only requires access to the updated data samples, the previous model parameters, and a unique, fixed-size matrix that quantifies the effect of the previous training dataset. Our approach can significantly reduce the storage requirement of model updating, preserve the privacy of unchanged training samples without loss of model accuracy, and enhance the computational efficiency. Experiments on real-world image dataset validate the effectiveness of our proposed method.

Acoustic signals contain rich information of the environment. They can be used for detecting anomalous events such as in automated machine monitoring. In this demonstration, we present our acoustic anomaly detection system that captures acoustic signals and classifies them using machine learning techniques. Our system includes a server for sound management and model training, a mobile client for sound capturing and real-time classification, and a workbench that acts as a user interface. We will show the full operational pipeline of our system in this demonstration.

Machine learning is a promising technology for many modern applications. To train an effective machine learning model, a large amount of data is required. However, data may be created in different organizations and sharing data across organizational boundaries is difficult due to privacy concerns and communication bandwidth limitations. Data summarization is a technique for reducing the amount of data that needs to be shared, while preserving characteristics in the data that are useful for training machine learning models. In this paper, we present an overview of data summarization techniques, which can be useful for machine learning across organizational boundaries. We also discuss some possible applications related to these data summarization techniques and challenges for future research.

Sensor networks are commonly adopted to collect a variety of measurements in indoor and outdoor settings. However, collecting such measurements from every node in the network, although providing high accuracy and resolution of the phenomena of interest, may easily cause sensors’ battery depletion. In this article, we show that measurement correlation can be successfully exploited to reduce the amount of data collected in the network without significantly sacrificing the monitoring accuracy. In particular, we propose an online adaptive measurement technique with which a subset of nodes are dynamically chosen as monitors while the measurements of the remaining nodes are estimated using the computed correlations. We propose an estimation framework based on jointly Gaussian distributed random variables, and we formulate an optimization problem to select the monitors under a total cost constraint. We show that the problem is NP-Hard and propose three efficient heuristics. We also develop statistical approaches that automatically switch between learning and estimation phases to take into account the variability occurring in real networks. Simulations carried out on real-world traces show that our approach outperforms previous solutions based on compressed sensing, and it can be successfully applied to the real application of solar irradiance prediction of photovoltaics systems.

Deep neural networks (DNNs) are emerging as important drivers for GPU (Graphical Processing Unit) usage. Routinely, now, cloud offerings include GPU-capable VMs, and GPUs are used for training and testing DNNs. A popular way to run inference (or testing) tasks with DNNs is to use middleware called a serving system. Tensorflow-Serving (TF-Serving) is an example of a DNN serving system. In this paper, we consider the problem of carefully scheduling multiple concurrent DNNs in a serving system on a single GPU to achieve fairness or service differentiation objectives, a capability crucial to cloud-based TF-Serving offerings. In scheduling DNNs, we face two challenges: how to schedule, and switch between, different DNN jobs at low overhead; and, how to account for their usage. Our system, Olympian, extends TF-Serving to enable fair sharing of a GPU across multiple concurrent large DNNs at low overhead, a capability TF-Serving by itself is not able to achieve. Specifically, Olympian can run concurrent instances of several large DNN models such as Inception, ResNet, GoogLeNet, AlexNet and VGG, provide each with an equal share of the GPU, while interleaving them at timescales of 1-2 ms, and incurring an overhead of less than 2%. It achieves this by leveraging the predictability of GPU computations to profile GPU resource usage models offline, then using these to achieve low overhead switching between DNNs.

We present a method to locate faults in service-based software systems hosted on mobile ad hoc networks (MANETs). In such systems, computations are structured as interdependent services distributed across the network, collaborating to satisfy client requests. Faults, which may occur at either or both the service and network layers, propagate by cascading through some subset of the services, from their root causes back to the clients that initiate requests. Fault localization in this environment is especially challenging because the systems are typically subject to a wider variety and higher incidence of faults than those deployed in fixed networks, the resources available to collect and store analysis data are severely limited, and many of the sources of faults are by their nature transient. Our method makes use of service-dependence and fault data that are harvested in the network through decentralized, run-time observations of service interactions and fault symptoms. We have designed timingand Bayesian-based reasoning techniques to analyze the data in the context of a specific fault propagation model. The analysis provides a ranked list of candidate fault locations. Through extensive simulations, we evaluate the performance of our method in terms of its accuracy in correctly ranking root causes under a wide range of operational conditions.

In this paper, we report on a mobile-health intervention study for COPD patients. Mobile devices are used to continuously record relevant patient symptoms, such as lung function, cough intensity, sputum color, vital-signs and activity. Furthermore, behavioral support is provided by a bi-directional communication channel and virtual assistant communication. The goal is to train a predictive algorithm to prevent exacerbations and to provide personalized virtual coaching to improve medication adherence and activity level.

Mobile edge clouds (MECs) are small cloud-like infrastructures deployed in close proximity to users, allowing users to have seamless and low-latency access to cloud services. When users move across different locations, their service applications often need to be migrated to follow the user so that the benefit of MEC is maintained. In this paper, we propose a layered framework for migrating running applications that are encapsulated either in virtual machines (VMs) or containers. We evaluate the migration performance of various real applications under the proposed framework.

We present a framework and a set of algorithms for determining faults in networks when large scale outages occur. The design principles of our algorithm, netCSI, are motivated by the fact that failures are geographically clustered in such cases. We address the challenge of determining faults with incomplete symptom information due to a limited number of reporting nodes. netCSI consists of two parts: a hypotheses generation algorithm, and a ranking algorithm. When constructing the hypothesis list of potential causes, we make novel use of positive and negative symptoms to improve the precision of the results. In addition, we propose pruning and thresholding along with a dynamic threshold value selector, to reduce the complexity of our algorithm. The ranking algorithm is based on conditional failure probability models that account for the geographic correlation of the network objects in clustered failures. We evaluate the performance of netCSI for networks with both random and realistic topologies. We compare the performance of netCSI with an existing fault diagnosis algorithm, MAX-COVERAGE, and demonstrate an average gain of 128 percent in accuracy for realistic topologies.

Network monitoring is an essential component of network operation and, as the network size increases, it usually generates a significant overhead in large scale networks such as sensor and data center networks. In this paper, we show that measurement correlation often exhibited in real networks can be successfully exploited to reduce the network monitoring overhead. In particular, we propose an online adaptive measurement technique with which a subset of nodes are dynamically chosen as monitors while the measurements of the remaining nodes are estimated using the computed correlations. We propose an estimation framework based on jointly Gaussian distributed random variables, and formulate an optimization problem to select the monitors which minimize the estimation error under a total cost constraint. We show that the problem is NP-Hard and propose three efficient heuristics. In order to apply our framework to real-world networks, in which measurement distribution and correlation may significantly change over time, we also develop a learning based approach that automatically switches between learning and estimation phases using a change detection algorithm. Simulations carried out on two real traces from sensor networks and data centers show that our algorithms outperforms previous solutions based on compressed sensing and it is able to reduce the monitoring overhead by 50% while incurring a low estimation error. The results further demonstrate that applying the change detection algorithm reduces the estimation error up to two orders of magnitude.

The dependencies among the components of service-oriented software applications hosted in a mobile ad hoc network (MANET) are difficult to determine due to the inherent loose coupling of the services and the transient communication topologies of the network. Yet understanding these dependencies is critical to making good management decisions, since dependence data underlie important analyses such as fault localization and impact analysis. Current methods for discovering dependencies, developed primarily for fixed networks, assume that dependencies change only slowly and require relatively long monitoring periods as well as substantial memory and communication resources, all of which are impractical in the MANET environment. We describe a new dynamic dependence discovery method designed specifically for this environment, yielding dynamic snapshots of dependence relationships discovered through observations of service interactions. We evaluate the performance of our method in terms of the accuracy of the discovered dependencies, and draw insights on the selection of critical parameters under various operational conditions. Although operated under more stringent conditions, our method is shown to provide results comparable to or better than existing methods.

We propose a greedy algorithm, Cluster-MAX-COVERAGE (CMC), to efficiently diagnose large-scale clustered failures. We primarily address the challenge of determining faults with incomplete symptoms. CMC makes novel use of both positive and negative symptoms to output a hypothesis list with a low number of false negatives and false positives quickly. CMC requires reports from about half as many nodes as other existing algorithms to determine failures with 100 percent accuracy. Moreover, CMC accomplishes this gain significantly faster (sometimes by two orders of magnitude) than an algorithm that matches its accuracy. When there are fewer positive and negative symptoms at a reporting node, CMC performs much better than existing algorithms. We also propose an adaptive algorithm called Adaptive-MAX-COVERAGE (AMC) that performs efficiently during both independent and clustered failures. During a series of failures that include both independent and clustered, AMC results in a reduced number of false negatives and false positives.

In this paper, we propose an algorithm to efficiently diagnose large-scale clustered failures. The algorithm, Cluster-MAX-COVERAGE (CMC), is based on greedy approach. We address the challenge of determining faults with incomplete symptoms. CMC makes novel use of both positive and negative symptoms to output a hypothesis list with a low number of false negatives and false positives quickly. CMC requires reports from about half as many nodes as other existing algorithms to determine failures with 100% accuracy. Moreover, CMC accomplishes this gain significantly faster (sometimes by two orders of magnitude) than an algorithm that matches its accuracy. Furthermore, we propose an adaptive algorithm called Adaptive-MAX-COVERAGE (AMC) that performs efficiently during both kinds of failures, i.e., independent and clustered. During a series of failues that include both independent and clustered, AMC results in a reduced number of false negatives and false positives.

Fault localization in general refers to a technique for identifying the likely root causes of failures observed in systems formed from components. Fault localization in systems deployed on mobile ad hoc networks (MANETs) is a particularly challenging task because those systems are subject to a wider variety and higher incidence of faults than those deployed in fixed networks, the resources available to track fault symptoms are severely limited, and many of the sources of faults in MANETs are by their nature transient. We present a method for localizing the faults occurring in service-based systems hosted on MANETs. The method is based on the use of dependence data that are discovered dynamically through decentralized observations of service interactions. We employ both Bayesian and timing-based reasoning techniques to analyze the data in the context of a specific fault propagation model, deriving a ranked list of candidate fault locations. We present the results of an extensive set of experiments exploring a wide range of operational conditions to evaluate the accuracy of our method.

In this paper we present a framework and a set of algorithms for determining faults in networks when large scale outages occur. The design principles of our algorithm, netCSI, are motivated by the fact that failures are geographically clustered in such cases. We address the challenge of determining faults with incomplete symptom information due to a limited number of reporting nodes in the network. netCSI consists of two parts: hypotheses generation algorithm, and ranking algorithm. When constructing the hypotheses list of potential causes, we make novel use of the positive and negative symptoms to improve the precision of the results. The ranking algorithm is based on conditional failure probability models that account for the geographic correlation of the network objects in clustered failures. We evaluate the performance of netCSI for networks with both random and realistic topologies. We compare the performance of netCSI with an existing fault diagnosis algorithm, MAX-COVERAGE, and achieve an average gain of 128\% in accuracy for realistic topologies.

The vitality and utility of a network are affected significantly by the network management system (NMS) that is used to administer and monitor the network. However, models that can characterize the quality of a NMS are generally missing in the literature. In this paper, we introduce the concept of quality of monitoring (QoM), provide a mathematical formulation based on stochastic processes that can be used to model a network monitoring system and define QoM metrics based on this formulation. A formal analysis of the proposed framework along various metrics is also provided, along with a case study of its application to network monitoring in a mobile ad hoc network.

Received power measurements at spatially distributed monitors can be usefully exploited to deduce various characteristics of active wireless transmitters. In this paper, we study the problem of “blind” estimation of a wireless node's transmit power utilizing solely received power measurements at spatially distributed monitors, without any prior knowledge about the transmitter's location or any statistical characterization of its transmit power. We first consider a deterministic setup and utilize a geometrical approach to obtain fundamental limitations on estimating the transmit power and location of an unknown wireless node. We show that a regular placement of monitors, though appealing, does not provide sufficient measurement diversity to yield a unique estimate. We then extend the setup to consider wireless fading and present a theoretical analysis of maximum likelihood (ML) estimate, which is analytically shown to be asymptotically optimal. Finally, we provide numerical results comparing the performance of the estimator through simulations and on a dataset of field measurements.

We study blind estimation of transmission power of a node based on received power measurements obtained under wireless fading. Specifically, the setup consists of a set of monitors that measure the signal power received from the transmitter, and the goal is to utilize these measurements to estimate the transmission power in the absence of any prior knowledge of the transmitter's location or any statistical distribution of its power. Towards this end, we exploit spatial diversity in received-power measurements and cooperation among the multiple monitoring nodes; based on theoretical analysis we obtain the Maximum Likelihood (ML) estimate, derive fundamental geometrical insights and show that this estimate is asymptotically optimal. Finally, we provide numerical results comparing the performance of the estimators through simulations and on a data-set of field measurements.

Discovering the topology of a network and detecting link state changes (e.g.: link failures) is an essential element for various network management and monitoring tasks. In this paper, we investigate scalable mechanisms to monitor the topology and link states of networks based on information available in network nodes' routing tables. We first present an algorithm that infers the network topology based on the full or partial information about network distances between nodes, based on which we obtain a scalable network topology discovery solution via a novel use of random walk in graphs. We then present scalable algorithms to detect the state changes of remote links by monitoring the routing tables of a small fraction of the routers, where the routers to be monitored are selected by a greedy approach to an NP-complete Tree Cover problem. We show the efficacy and scalability of our topology monitoring algorithms through experimental evaluation performed both on synthetic topologies and on a large topology data-set from a real enterprise network.

Transmit-power estimation is an important part in power-aware designs of mobile ad-hoc networks (MANETs). In this paper, we consider the cooperation among multiple monitor-nodes to estimate the transmit power of other nodes. Utilizing a geometric approach, we characterize the theoretical performance of such cooperative monitoring schemes and propose transmit-power estimation techniques with different number of cooperating nodes. We introduce the novel concept of confidence region that provides a fundamental confidence level for the accuracy of the power estimation and enables the development of techniques for allocating network monitors. Finally, we present a simple, distributed cooperative estimation scheme for a large-scale wireless network and give illustrative simulation results to quantify its performance.

To increase the utilization of the available frequency channel space in 802.11-based wireless mesh networks, recent work has explored solutions based on multi-radio stations. This paper reports on our design and experimental study of a distributed, self-stabilizing mechanism that assigns channels to multi-radio nodes in wireless mesh networks. We take a modular approach by decoupling the channel selection decision from the data forwarding mechanism, which makes our solution readily applicable to real-world operation when used with emerging multi-radio routing solutions. We demonstrate the efficacy of our protocol on a real-world, 14-node testbed comprised of nodes, each equipped with an 802.11a card and an 802.11g card. We show via extensive measurements on our testbed that our channel assignment algorithm improves the network capacity by 50% in comparison to a homogeneous channel assignment and by 20% in comparison to a random assignment.

Emerging large scale distributed networking systems, such as P2P file sharing systems, sensor networks, and ad hoc wireless networks, require replication of content, functionality, or configuration to enact or optimize communication tasks. The placement of these replicated resources can significantly impact performance. We present a novel self-stabilizing, fully distributed, asynchronous, scalable protocol that can be used to place replicated resources such that each node is "close" to some copy of any object. We describe our protocol in the context of a graph with colored nodes, where a node's color indicates the replica/task that it is assigned. Our combination of theoretical results and simulation prove stabilization of the protocol, and evaluate its performance in the context of convergence time, message transmissions, and color distance. Our results show that the protocol generates colorings that are close to the optimal under a set of metrics, making such a protocol ideal for emerging networking systems.

Quality of service for high-bandwidth or delay-sensitive applications in the Internet, such as streaming media and online games, can be significantly improved by replicating server content. We present a decentralized algorithm that allocates server resources to replicated servers in large-scale client-server networks to reduce network distance between each client and the nearby replicated server hosting the resources of interest to that client. Preliminary simulation results show that our algorithm converges quickly to an allocation that reduces the expected client-server distance by almost half compared to the distance when the assignment of replicated servers is done at random.

In this paper, we present a novel distributed algorithm that dynamically selects game servers for a group of game clients participating in large scale interactive online games. The goal of server selection is to minimize server resource usage while satisfying the real-time delay constraint. We develop a synchronization delay model for interactive games and formulate the server selection problem, and prove that the considered problem is NP-hard. The proposed algorithm, called zoom-in-zoom-out, is adaptive to session dynamics (e.g. clients join and leave) and lets the clients select appropriate servers in a distributed manner such that the number of servers used by the game session is minimized. Using simulation, we present the performance of the proposed algorithm and show that it is simple yet effective in achieving its design goal. In particular, we show that the performance of our algorithm is comparable to that of a greedy selection algorithm, which requires global information and excessive computation.

Motivated by the need to enable easier data sharing and curb rising storage management costs, storage systems are becoming increasingly consolidated and thereby shared by a large number of users and applications. In such environments, service differentiation becomes increasingly important. Since caching is a fundamental and pervasive technique employed to improve the performance of storage systems, providing differentiated services from a storage cache is a crucial component of the entire end-to-end QoS solution. In this paper we discuss a QoS architecture for a shared storage proxy cache which can provide long-term hit rate assurances to competing classes. The proposed architecture consists of three components: (a) per-class feedback controllers that track the performance of each class, (b) a fairness controller that allocates excess resources fairly in the case when all goals are met, and (c) a contention resolver that decides cache allocation in the case when at least one class does not meet its target hit rate. We compare the performance of various feedback per-class controllers, and provide guidelines for designing QoS mechanisms for such a dynamic environment.

Emerging large scale distributed networking systems, such as P2P file sharing systems, sensor networks, and ad hoc wireless networks, require replication of content, functionality, or configuration to enact or optimize communication tasks. The placement of these replicated resources can significantly impact performance. We present a novel self-stabilizing, fully distributed, asynchronous, scalable protocol that can be used to place replicated resources such that each node is "close" to some copy of any object. We describe our protocol in the context of a graph with colored nodes, where a node's color indicates the replica/task that it is assigned. Our combination of theoretical results and simulation prove stabilization of the protocol, and evaluate its performance in the context of convergence time, message transmissions, and color distance. Our results show that the protocol generates colorings that are close to the optimal under a set of metrics, making such a protocol ideal for emerging networking systems.