We introduce EVENFLOW, a novel toolkit of synchronization protocols for data-parallel training of neural nets using the Parameter Server (PS) paradigm. EVENFLOW achieves both timely and accurate global model updates in streaming settings. Instead of leaving stragglers out of the global model to avoid delays (asynchronous protocol) or using laggy synchronizations of all learners (synchronous protocol), EVENFLOW establishes data-driven mechanisms that allow the PS paradigm to decide when a synchronization is necessary, i.e., the global model may have changed beyond an allowed tolerance value. EVENFLOW models this problem as a distributed, thresholded function monitoring task and decomposes it to local filters monitored independently by each learner. When a learner finds its local filter violated, only then a synchronization is triggered. Our experiments show that EVENFLOW combines the virtues of both the vanilla (synchronous, asynchronous) protocols. EVENFLOW offers the rapid training times of asynchronous, with mostly equal or even improved accuracy compared to synchronous.

In this work we introduce a novel, reversible data summarization technique, namely the Reverse Random Hyperplane Projection (RRHP) scheme. RRHP is particularly useful in Wireless Sensor Network (WSN) settings because it enables individual sensors to compress their local data streams before transmitting them across the WSN. In that, RRHP saves communication and, thus, the residual energy of battery-powered sensors. Then, when the compressed sensor data streams reach a base station, the reversibility property of RRHP can be used to regain approximations of the original sensor streams to perform all kinds of data mining tasks. We provide formal theoretic guarantees on how RRHP directly trades the amount of compression for the approximation of original sensor streams’ desired properties. We experimentally prove that RRHP is useful for performing various kinds of data mining tasks, over sensor data streams, by dramatically reducing the amount of communicated data, simultaneously achieving high accuracy.