In this paper, we propose and analyze SQuARM-SGD, a communication-efficient algorithm for decentralized training of large-scale machine learning models over a network. In SQuARM-SGD, each node performs a fixed number of local SGD steps using Nesterov’s momentum and then sends sparsified and quantized updates to its neighbors regulated by a locally computable triggering criterion. We provide convergence guarantees of our algorithm for general (non-convex) and convex smooth objectives, which, to the best of our knowledge, is the first theoretical analysis for compressed decentralized SGD with momentum updates. We show that the convergence rate of SQuARM-SGD matches that of vanilla SGD. We empirically show that including momentum updates in SQuARM-SGD can lead to better test performance than the current state-of-the-art which does not consider momentum updates.

We develop a Distributed Event-Triggered Stochastic GRAdient Descent (DETSGRAD) algorithm for solving non-convex optimization problems typically encountered in distributed deep learning. We propose a novel communication triggering mechanism that would allow the networked agents to update their model parameters aperiodically and provide sufficient conditions on the algorithm step-sizes that guarantee the asymptotic mean-square convergence. The algorithm is applied to a distributed supervised-learning problem, in which a set of networked agents collaboratively train their individual neural networks to perform image classification, while aperiodically sharing the model parameters with their one-hop neighbors. Results indicate that all agents report similar performance that is also comparable to the performance of a centrally trained neural network, while the event-triggered communication provides significant reduction in inter-agent communication. Results also show that the proposed algorithm allows the individual agents to classify the images even though the training data corresponding to all the classes are not locally available to each agent.

This paper presents a model-free solution to distributed transmit beamforming using mobile agents. Each agent is equipped with an antenna and the agents represent the individual elements in an antenna array. The agents are tasked to coordinate their relative location, phase offsets, and amplitude to construct a desired beam-pattern. As a prospective solution, we propose a model-free optimization algorithm based on real-time feedback that does not require a model that maps the control parameters (relative location, phase offsets, and amplitude) to a radiation pattern. We evaluate the performance of proposed approach for different motion constraints. Numerical results are presented to validate the theory.

This paper is concerned with the problem of bearings based multi-agent circumnavigation of a maneuvering target. Agents are assumed to have access to their own individual bearing measurements as well as the ones from their immediate neighbors. The aim is to devise a distributed algorithm to estimate the position of the maneuvering target and drive the agents to circumnavigate the target while forming a regular polygon and keeping a desired distance from the target. Analytic results show that the algorithm attains exponential convergence of the agents to a neighborhood of the desired polygon, where the neighborhood size depends on the target speed. Simulation results illustrate the efficacy of the approach.

Motivated by decentralized approaches to machine learning, we propose a collaborative Bayesian learning algorithm taking the form of decentralized Langevin dynamics in a non-convex setting. Our analysis show that the initial KL-divergence between the Markov Chain and the target posterior distribution is exponentially decreasing while the error contributions to the overall KL-divergence from the additive noise is decreasing in polynomial time. We further show that the polynomial-term experiences speed-up with number of agents and provide sufficient conditions on the time-varying step-sizes to guarantee convergence to the desired distribution. The performance of the proposed algorithm is evaluated on a wide variety of machine learning tasks. The empirical results show that the performance of individual agents with locally available data is on par with the centralized setting with considerable improvement in the convergence rate.

This paper presents a novel distributed algorithm for tracking a highly maneuvering target using bearing or direction of arrival measurements. The proposed approach is built on the dynamic average-consensus algorithm, which allows a networked group of agents (nodes) to reach consensus on the global average of a set of local time-varying signals in a distributed fashion. Since the average-consensus error corresponding to the presented dynamic average-consensus algorithm converges to zero in finite time, the proposed distributed algorithm guarantees that the tracking error converges to zero in finite time. Numerical simulations are provided to illustrate the effectiveness of the proposed algorithm.

In wireless sensor networks, estimating a global parameter from locally obtained measurements via local interactions is known as the distributed parameter estimation problem. Solving these problems often require the deployment of distributed optimization algorithms that rely on a constant exchange of information among the sensor nodes. This makes such distributed algorithms vulnerable to attackers or malicious nodes that want to gain access to private information regarding the network. Based on the sliding mode control scheme, here we present a novel approach to infer sensitive information (e.g., gradient or private parameters of the local objective function) regarding a node of interest by intercepting the communication between the nodes. The effectiveness of the proposed approach is illustrated in a representative example of distributed event localization using an acoustic sensor network.

This paper presents the formulation and analysis of a novel distributed maximum likelihood algorithm that utilizes a first-order optimization scheme. The proposed approach utilizes a static average consensus algorithm to reach agreement on the initial condition to the iterative optimization scheme and a dynamic average consensus algorithm to reach agreement on the gradient direction. The current distributed algorithm is guaranteed to exponentially recover the performance of the centralized algorithm. Though the current formulation focuses on maximum likelihood algorithm built on first-order methods, it can be easily extended to higher order methods. Numerical simulations validate the theoretical contributions of the paper.

This paper develops an algorithm that exploits picture-oriented social networks to localize urban events. We choose picture-oriented networks because taking a picture requires physical proximity, thereby revealing the location of the photographed event. Furthermore, most modern cell phones are equipped with GPS, making picture location, and time metadata commonly available. We consider Instagram as the social network of choice and limit ourselves to urban events (noting that the majority of the world population lives in cities). The paper introduces a new adaptive localization algorithm that does not require the user to specify manually tunable parameters. We evaluate the performance of our algorithm for various real-world datasets, comparing it against a few baseline methods. The results show that our method achieves the best recall, the fewest false positives, and the lowest average error in localizing urban events.

Physical events in the real world are known to trigger reactions and then discussions in online social media. Mining these reactions through online social sensors offers a fast and low cost way to understand what is happening in the physical world. In some cases, however, further study of the affected population's emotional state can improve this understanding. In our study we analyzed how car commuters react on Twitter while stuck in heavy traffic. We discovered that the online social footprint does not necessarily follow a strict linear correlation with the volume of a traffic jam. Through our analysis we offer a potential explanation: people's mood could be an additional factor, apart from traffic severity itself, that leads in fluctuations of the observed reaction in social media. This finding can be important for social sensing applications where external factors, like sentiment, also contribute on how humans react.

Recent sensor network literature investigated the use of social networks as sensor networks, and formulated a physical event localization problem from social network data. This paper improves on the above results by formulating a joint localization problem of events and sources, leveraging the fact that sources on social networks often have a location affinity: They tend to comment more on events in their locations of interest. While social networks, such as Twitter, do not offer source location information for the majority of sources, we show that our algorithms for jointly inferring source and event location significantly improve localization quality by mutually enhancing location estimation of both events and sources. We evaluate the performance of our algorithm both in simulation and using Twitter data about current events. The results show that joint inference of source and event location allows us to localize many more of the events identified in real-world datasets.

Uncovering subgraphs with an abnormal distribution of attributes reveals much insight into network behaviors. For example in social or communication networks, diseases or intrusions usually do not propagate uniformly, which makes it critical to find anomalous regions with high concentrations of a specific disease or intrusion. In this paper, we introduce a probabilistic model to identify anomalous subgraphs containing a significantly different percentage of a certain vertex attribute, such as a specific disease or an intrusion, compared to the rest of the graph. Our framework, gAnomaly, models generative processes of vertex attributes and divides the graph into regions that are governed by background and anomaly processes. Two types of regularizers are employed to smoothen the regions and to facilitate vertex assignment. We utilize deterministic annealing EM to learn the model parameters, which is less initialization-dependent and better at avoiding local optima. In order to find fine-grained anomalies, an iterative procedure is further proposed. Experiments show gAnomaly outperforms a state-of-the-art algorithm at uncovering anomalous subgraphs in attributed graphs.

This paper presents the formulation of distributed Kalman-Bucy filter algorithm for a network of autonomous sensors, which is modeled as a connected undirected graph. Development of the distributed Kalman-Bucy filter is formulated as two average consensus problems in terms of weighted inverse of measurement noise covariance matrices and weighted measurements. The proposed algorithm utilizes the static average consensus protocol to solve the first consensus problem and the proportional-integral based dynamic average consensus protocol to solve the latter. The distributed Kalman-Bucy filter algorithm is optimal in the sense that the performance of the proposed algorithm asymptotically approaches that of a centralized filter. Numerical simulations are presented to demonstrate the performance of the proposed scheme.

Distributed sensor networks are quickly gaining recognition as viable embedded computing platforms. Current techniques for programming sensor networks are cumbersome, inflexible, and low-level. We introduce EnviroTrack, an object-based distributed middleware system that raises the level of programming abstraction by providing a convenient and powerful interface to the application developer geared towards tracking the physical environment. EnviroTrack is novel in its seamless integration of objects that live in physical time and space into the computational environment of the application. Performance results demonstrate the ability of the middleware to track realistic targets.