Computational methods for identifying hidden structures in high-order data are critical for exploratory data analysis tasks. This work proposes a joint dimensionality reduction and co-clustering algorithm for tensors. A compressed representation of a tensor is obtained via a Tucker-like decomposition model, whose factor matrices capture the tensor co-clustering structure. Factor matrices correspond to the cluster centroids of the tensor fibers per mode, whose entries interact nonlinearly to build the tensor approximation. The algorithm, developed based on the alternating-direction method of multipliers, has computational complexity similar to that of a single Tucker decomposition.

Clustering is a basic task in a variety of machine learning applications. Partitioning a set of input vectors into compact, well separated subsets can be severely affected by the presence of model incompatible inputs called outliers. The present paper develops robust clustering algorithms for jointly partitioning the data and identifying the outliers. The novel approach relies on translating scarcity of outliers to sparsity in a judiciously defined domain, to robustify three widely used clustering schemes: hard K-means, fuzzy K-means, and probabilistic clustering. Cluster centers and assignments are iteratively updated in closed form. The developed outlier aware algorithms are guaranteed to converge, while their computational complexity is of the same order as their outlier-agnostic counterparts. Preliminary simulations validate the analytical claims.

This paper deals with clustering of spatially distributed data using wireless sensor networks. A distributed low-complexity clustering algorithm is developed that requires one-hop communications among neighboring nodes only, without local data exchanges. The algorithm alternates iterations over the variables of a consensus-based version of the global clustering problem. Using stability theory for time-varying and time-invariant systems, the distributed clustering algorithm is shown to be bounded-input bounded-output stable with an output arbitrarily close to a fixed point of the algorithm. For distributed hard K-means clustering, convergence to a local minimum of the centralized problem is guaranteed. Numerical examples confirm the merits of the algorithm and its stability analysis.

This paper develops algorithms to train linear support vector machines (SVMs) when training data are distributed across different nodes and their communication to a centralized node is prohibited due to, for example, communication overhead or privacy reasons. To accomplish this goal, the centralized linear SVM problem is cast as the solution of coupled decentralized convex optimization subproblems with consensus constraints on the parameters defining the classifier. Using the method of multipliers, distributed training algorithms are derived that do not exchange elements from the training set among nodes. The communications overhead of the novel approach is fixed and fully determined by the topology of the network instead of being determined by the size of the training sets as it is the case for existing incremental approaches. An online algorithm where data arrive sequentially to the nodes is also developed. Simulated tests illustrate the performance of the algorithms.

The present paper develops a decentralized expectation-maximization (EM) algorithm to estimate the parameters of a mixture density model for use in distributed learning tasks performed with data collected at spatially deployed wireless sensors. The E-step in the novel iterative scheme relies on local information available to individual sensors, while during the M-step sensors exchange information only with their one- hop neighbors to reach consensus and eventually percolate the global information needed to estimate the wanted parameters across the wireless sensor network (WSN). Analysis and simulations demonstrate that the resultant consensus-based distributed EM (CB-DEM) algorithm matches well the resource- limited characteristics of WSNs and compares favorably with existing alternatives because it has wider applicability and remains resilient to inter-sensor communication noise.