We design a decentralized stochastic optimization algorithm for machine-to-machine (M2M) networks, in which machines in the networks can attend the optimization process in a randomly-unexpected manner. This property of random attendance aims at capturing unexpected events that prevent the machines to successfully communicate with each other. The property, beside simulating the unforeseen nature of the networks, additionally helps reduce the communication cost of machines while still maintaining a reasonable optimization outcome. Our contribution gives practical insights for deploying optimization algorithms over machines in complex environments, including the Internet of Things.

This paper considers the scenario that multiple data owners wish to apply a machine learning method over the combined dataset of all owners to obtain the best possible learning output but do not want to share the local datasets owing to privacy concerns. We design systems for the scenario that the stochastic gradient descent (SGD) algorithm is used as the machine learning method, because SGD (or its variants) is at the heart of recent deep learning techniques over neural networks. Our systems differ from the existing systems in the following features: 1) any activation function can be used, meaning that no privacy-preserving-friendly approximation is required; 2) gradients computed by SGD are not shared but the weight parameters are shared instead; and 3) robustness against colluding parties even in the extreme case that only one honest party exists. One of our systems requires a shared symmetric key among the data owners (trainers) to ensure the secrecy of the weight parameters against a central server. We prove that our systems, while privacy preserving, achieve the same learning accuracy as SGD and, hence, retain the merit of deep learning with respect to accuracy. Finally, we conduct several experiments using benchmark datasets and show that our systems outperform the previous system in terms of learning accuracies.