Stein Variational Gradient Descent (SVGD) is a widely used sampling algorithm that has been successfully applied in several areas of Machine Learning. SVGD operates by iteratively moving a set of $n$ interacting particles (which represent the samples) to approximate the target distribution. Despite recent studies on the complexity of SVGD and its variants, their long-time asymptotic behavior (i.e., after numerous iterations $k$) is still not understood in the finite number of particles regime. We study the long-time asymptotic behavior of a noisy variant of SVGD. First, we establish that the limit set of noisy SVGD for large $k$ is well-defined. We then characterize this limit set, showing that it approaches the target distribution as $n$ increases. In particular, noisy SVGD avoids the variance collapse observed for SVGD. Our approach involves demonstrating that the trajectories of noisy SVGD closely resemble those described by a McKean-Vlasov process.

Actor-critic methods integrating target networks have exhibited a stupendous empirical success in deep reinforcement learning. However, a theoretical understanding of the use of target networks in actor-critic methods is largely missing in the literature. In this paper, we reduce this gap between theory and practice by proposing the first theoretical analysis of an online target-based actor-critic algorithm with linear function approximation in the discounted reward setting. Our algorithm uses three different timescales: one for the actor and two for the critic. Instead of using the standard single timescale temporal difference (TD) learning algorithm as a critic, we use a two timescales target-based version of TD learning closely inspired from practical actor-critic algorithms implementing target networks. First, we establish asymptotic convergence results for both the critic and the actor under Markovian sampling. Then, we provide a finite-time analysis showing the impact of incorporating a target network into actor-critic methods.

Although Adam is a very popular algorithm for optimizing the weights of neural networks, it has been recently shown that it can diverge even in simple convex optimization examples. Several variants of Adam have been proposed to circumvent this convergence issue. In this work, we study the Adam algorithm for smooth nonconvex optimization under a boundedness assumption on the adaptive learning rate. The bound on the adaptive step size depends on the Lipschitz constant of the gradient of the objective function and provides safe theoretical adaptive step sizes. Under this boundedness assumption, we show a novel first order convergence rate result in both deterministic and stochastic contexts. Furthermore, we establish convergence rates of the function value sequence using the Kurdyka-Lojasiewicz property.

The Douglas Rachford algorithm is an algorithm that converges to a minimizer of a sum of two convex functions. The algorithm consists in fixed point iterations involving computations of the proximity operators of the two functions separately. The paper investigates a stochastic version of the algorithm where both functions are random and the step size is constant. We establish that the iterates of the algorithm stay close to the set of solution with high probability when the step size is small enough. Application to structured regularization is considered.

Optimization of mobile networks is a crucial task for operators aiming to offer a high quality of service. In this paper, we focus on the handover (HO) problem. An HO is the procedure through which a user changes its serving cell, ensuring service continuity for mobile users. The HO procedure is typically governed by several variables: the received signals powers and the HO parameters (offsets and hysteresis) corresponding to the different cells. HO parameters allow to modify the cell borders in order to balance loads, and prevent radio link failures and HO ping-pongs. There are two approaches for defining HO parameters: one is to use per-cell parameters and the other is to use per-cell-pair parameters. Per-cell-pair parameters provide more degrees of freedom in HO parameter optimization as we benefit from independent parameters for each neighbour. However, we show that it can lead to instabilities in the form of continuous chained HO, i.e., several HOs that follow one shortly after another, among three or more cells. Then, we show that using the same HO parameters toward all neighbouring cells, i.e., per-cell parameters, is risk free of such problems. We provide numerical results outlining the tendencies of this risk.

An important problem of today's mobile network operators is to bring down the capital expenditures and operational expenditures. One strategy is to automate the parameter tuning on the small cells through the so-called self-organizing network (SON) functionalities, such as cell range expansion, mobility robustness optimization, or enhanced Inter-Cell Interference Coordination. Having several of these functionalities in the network will surely create conflicts, as, for example, they may try to change the same parameter in the opposite directions. This raises that the need for an SON COordinator (SONCO) meant to arbitrate the parameter change requests of the SON functions, ensuring some degree of fairness. It is difficult to anticipate the impact of accepting several simultaneous requests. In this paper, we provide a SONCO design based on reinforcement learning (RL) as it allows us to learn from previous experiences and improve our future decisions. Typically, RL algorithms are complex. To reduce this complexity, we employ two flavors of function approximation and provide a study-case. Results show that the proposed SONCO design is capable of biasing this fairness among the SON functions by means of weights attributed to the SON functions. Also, we evaluate the tracking capability of the algorithms.

In trying to meet the demands of traffic hungry users, mobile network operators are faced with increased CAPital EXpenditures (CAPEX) and OPerational EXpenditures (OPEX). The Self Organizing Network (SON) functions have been introduced by 3GPP as a means to cut down these costs. There are mainly 3 categories of such functions: self-configuration, self-optimization and self-healing. In this paper we focus on the second which represents the SON functions performing a runtime optimization of the network. We center our attention on LTE heterogeneous networks. Having several SON functions in a network may lead to conflicts and potentially to bad network Key Performance Indicators (KPIs). Thus a troubleshooting mechanism has to be envisaged. Such a mechanism typically contains 3 steps: fault detection, cause diagnosis and solution deployment. In this paper we tackle the first two and we study the feasibility of using the Naive Bayes Classifier (NBC) in order to build a framework for SON Conflict Diagnosis (SONCD). We provide numeric results proving the feasibility of the framework.

The continuously increasing traffic demand faces us with increased CAPital Expenditures (CAPEX) and Operational Expenditure (OPEX). Self Organizing Network (SON) functions aim to lower these costs by automating the network tuning. A SON instance is a realization of a SON function which can tune one or a set of cells. Having several uncoordinated SON functions in the network creates a risk for conflicts and instability. This raises the need for a SON Coordinator (SONCO) meant to deal with these issues. In this work we consider that on each cell we have one SON instance of each SON function. We present the design of a SONCO which arbitrates conflicts based on weights attributed to the SON functions. The design makes use of Reinforcement Learning (RL) with function approximation. We provide a low complexity approximation of the action-value function based on a number of parameters that scales linearly with the number of cells. We present a study case with the Mobility Load Balancing (tuning the Cell Individual Offset (CIO)) and Mobility Robustness Optimization (tuning the CIO and the handover hysteresis) functions, where the SONCO deals with the conflicts on the CIOs. Numerical results prove that we can orchestrate the SON functions through SONCO configurations that reflect different operator policies.

Consider a wireless network formed by fixed or mobile nodes. Each node seeks to estimate its own position based on noisy measurements of the relative distance with other nodes. In a centralized batch mode, positions can be retrieved (up to a rigid transformation) by applying Principal Component Analysis (PCA) on a so-called similarity matrix built from the relative distances. In this paper, we propose a distributed on-line algorithm allowing each node to estimate its own position based limited exchange of information in the network. Our framework encompasses the case of sporadic measurements and random link failures. We prove the consistency of our algorithm in the case of fixed sensors. Our numerical results also demonstrate the attractive performance of the algorithm for tracking the positions of mobile sensors. Simulations are conducted on a wireless sensor network testbed.

With the emergence of Self-Organizing Network (SON) functions network operators are faced with a practical problem: coordination of SON instances. The SON functions are usually designed in a standalone manner, i.e. they do not take into account the possibility that other instances of the same or different SON functions may be running in the network. This creates the risk of conflicts and network instability. Therefore a SON COordinator (SONCO) is needed. In this paper we design an operator centric SONCO that sees the SON instances as black-boxes, i.e. it does not know the algorithm inside the SON functions. Our aim is to improve the network stability (i.e. number of parameter changes) for SON instances of the same SON function. We employ Reinforcement Learning (RL) in order to profit from the information on the past SONCO decisions. We simplify the expression of the action-value function and we use state aggregation to further reduce the required state space, making it scale linearly with the number of coordinated cells. We provide a study case with the Mobility Load Balancing (MLB) function independently instantiated on every cell. The results show that the proposed SONCO improves the network stability.

In Long Term Evolution(LTE) networks one of the main focuses is on automating the network optimization. This is done through so called Self Organizing Network (SON) functions like Mobility Load Balancing (MLB), Mobility Robustness Optimization(MRO) and others. A SON instance is a realization of a SON function that governs (optimizes) one or a cluster of eNBs. The SON functions are built in a standalone manner, i.e. without considering the existence of other SON instances. So they do not necessarily operate in a coordinated fashion, especially in a network where different SON instances may come from different vendors. Thus we face a risk of generating conflicts and instabilities in the network and so this raises the need for a SON COordinator (SONCO) . The SONCO, built from an operator point of view, sees the SON instances as black boxes and has a very limited amount of information on them. In these conditions the SONCO has to solve conflicts and improve the network stability. In this paper we propose a Reinforcement Learning (RL) based solution for coordinating SON instances that run independently on neighboring eNBs and we provide results for a case study with MLB instances. We analyze the scalability of our solution and we provide numerical results showing how improvements in network stability can be obtained.

This paper is devoted to investigate binary classification in a distributed and on-line setting. In the Big Data era, datasets can be so large that it may be impossible to process them using a single processor. The framework considered accounts for situations where both the training and test phases have to be performed by taking advantage of a network architecture by the means of local computations and exchange of limited information between neighbor nodes. An online learning gossip algorithm (OLGA) is introduced, together with a variant which implements a node selection procedure. Beyond a discussion of the practical advantages of the algorithm we promote, the paper proposes an asymptotic analysis of the accuracy of the rules it produces, together with preliminary experimental results.

In this paper, a distributed stochastic approximation algorithm is studied. Applications of such algorithms include decentralized estimation, optimization, control or computing. The algorithm consists in two steps: a local step, where each node in a network updates a local estimate using a stochastic approximation algorithm with decreasing step size, and a gossip step, where a node computes a local weighted average between its estimates and those of its neighbors. Convergence of the estimates toward a consensus is established under weak assumptions. The approach relies on two main ingredients: the existence of a Lyapunov function for the mean field in the agreement subspace, and a contraction property of the random matrices of weights in the subspace orthogonal to the agreement subspace. A second-order analysis of the algorithm is also performed under the form of a central limit Theorem. The Polyak-averaged version of the algorithm is also considered.

We consider a flotilla composed of N autonomous mobile agents. Each agent ignores its absolute position but collects noisy measurements of the relative position of other agents. It has a convex utility function which depends on its control variables and on some global variables, typically, the average motion of all agents during a unit of time. The utility function of an agent is unknown by other agents. The aim is to determine the control variables which minimize the sum of all utility functions. The minimization is subject to constraints: speed limitations, collision avoidance and maintenance of the flock. We provide sufficient convex and separable conditions which guarantee that constraints are satisfied. These conditions allow to search for a minimizer of the aggregate utility function in a distributed fashion. The proposed algorithm simultaneously involves a gradient descent locally performed by the agents and a random gossip step which allows to find an agreement on the global motion strategy. In the case when there is a large number of transmissions during each unit of time, the algorithm is proved to converge almost surely to the desired minimizer, under mild conditions on the gossip protocol. When transmissions become less frequent, the minimization problem is no longer likely to be solved exactly. However, we propose an adaptive algorithm capable of tracking the minimizer. Its behavior is characterized through detailed simulations.

In this work, a static relaying protocol, called Decode or Quantize and Forward (DoQF), is introduced for half duplex single-relay networks, and its performance is studied in the context of communications over slow fading wireless channels. The proposed protocol is inspired by the so-called Compress-and-Forward (CF) but only needs statistical Channel State Information at the Transmitter (CSIT). First, we analyze the behavior of the outage probability Poof the proposed protocol as the SNR p tends to infinity. In this case, we prove that ρ2Poconverges to a constant ξ. We refer to this constant as the outage probability gain and we derive its closed-form expression for a general class of wireless channels that includes Rayleigh and Rice. We furthermore prove that the DoQF protocol has the best achievable outage gain in the wide class of half-duplex static relaying protocols and we minimize ξ w.r.t the power allocation to the source and the relay and the durations of the slots. Next, we focus on Rayleigh channels to derive the Diversity-Multiplexing Tradeoff (DMT) of the DoQF. Our results show that the DoQF achieves the 2 by 1 MISO DMT upper-bound for multiplexing gains r <; 0.25.

The paper addresses the convergence of a decentralized Robbins-Monro algorithm for networks of agents. This algorithm combines local stochastic approximation steps for finding the root of an objective function, and a gossip step for consensus seeking between agents. We provide verifiable sufficient conditions on the stochastic approximation procedure and on the network so that the decentralized Robbins-Monro algorithm converges to a consensus. We also prove that the limit points of the algorithm correspond to the roots of the objective function. We apply our results to Maximum Likelihood estimation in sensor networks.

This paper introduces a unified framework for the detection of a single source with a sensor array in the context where the noise variance and the channel between the source and the sensors are unknown at the receiver. The Generalized Maximum Likelihood Test is studied and yields the analysis of the ratio between the maximum eigenvalue of the sampled covariance matrix and its normalized trace. Using recent results from random matrix theory, a practical way to evaluate the threshold and thep-value of the test is provided in the asymptotic regime where the numberKof sensors and the numberNof observations per sensor are large but have the same order of magnitude. The theoretical performance of the test is then analyzed in terms of Receiver Operating Characteristic (ROC) curve. It is, in particular, proved that both Type I and Type II error probabilities converge to zero exponentially as the dimensions increase at the same rate, and closed-form expressions are provided for the error exponents. These theoretical results rely on a precise description of the large deviations of the largest eigenvalue of spiked random matrix models, and establish that the presented test asymptotically outperforms the popular test based on the condition number of the sampled covariance matrix.

This paper investigates the effect of quantization on the performance of the Neyman-Pearson test. It is assumed that a sensing unit observes samples of a correlated stationary ergodic multivariate process. Each sample is passed through anN-point quantizer and transmitted to a decision device which performs a binary hypothesis test. For any false alarm level, it is shown that the miss probability of the Neyman-Pearson test converges to zero exponentially as the number of samples tends to infinity, assuming that the observed process satisfies certain mixing conditions. The main contribution of this paper is to provide a compact closed-form expression of the error exponent in the high-rate regime, i.e., when the numberNof quantization levels tends to infinity, generalizing previous results of Gupta and Hero to the case of nonindependent observations. Ifdrepresents the dimension of one sample, it is proved that the error exponent converges at rateN2/dto the one obtained in the absence of quantization. As an application, relevant high-rate quantization strategies which lead to a large error exponent are determined. Numerical results indicate that the proposed quantization rule can yield better performance than existing ones in terms of detection error.

In this paper, we propose a resource allocation algorithm for the downlink of sectorized two-dimensional (2-D) OFDMA cellular networks assuming statistical Channel State Information (CSI) and fractional frequency reuse. The proposed algorithm can be implemented in a distributed fashion without the need to any central controlling units. Its performance is analyzed assuming fast fading Rayleigh channels and Gaussian distributed multicell interference. We show that the transmit power of this simple algorithm tends, as the number of users grows to infinity, to the same limit as the minimal power required to satisfy all users' rate requirements i.e., the proposed resource allocation algorithm is asymptotically optimal. As a byproduct of this asymptotic analysis, we characterize a relevant value of the reuse factor that only depends on an average state of the network.

This paper investigates the decentralized detection of spatially correlated processes using the Neyman-Pearson test. We consider a network formed by a large number of sensors, each of them observing a random data vector. Sensors' observations are non-independent, but form a stationary process verifying mixing conditions. Each vector-valued observation is quantized before being transmitted to a fusion center which makes the final decision. For any false alarm level, it is shown that the miss probability of the Neyman-Pearson test converges to zero exponentially as the number of sensors tends to infinity. A compact closed-form expression of the error exponent is provided in the high-rate regime i.e., when fine quantization is applied. As an application, our results allow to determine relevant quantization strategies which lead to large error exponents.

We address the optimization issue of the outage probability in Rician block fading channels with N transmit antennas and 1 receive antenna. It is already known that the eigenvectors of the covariance matrix of the transmit signal that minimizes the outage probability are the normalized mean vector of the Rician channel and N-1 orthonormal vectors belonging to the space orthogonal to the range of the mean vector. Our contribution is to optimize the eigenvalues. At high SNR, we show that the part of the power allocated to the orthogonal space of the range of the mean vector should be uniformly distributed amongst its N-1 eigenvectors. In addition, thanks to an original approximation of the outage probability, we characterize the part of power associated with the mean vector. We numerically observe improvement achieved by the proposed allocation scheme compared to uniform power allocation.

In the context of wireless relay networks operating on slow fading channels, the outage probability optimization is of central importance. It is often hard to give a closed form expression of the outage probability Pofor all possible values of the Signal to Noise Ratio (SNR). On the other hand, it is possible to analyze the behaviour of Poin the asymptotic regime where the SNR rho converges to infinity. In this regime, rhoN+1Pousually converges to a constant xi where N is the number of relays. This paper presents a general method for deriving and minimizing xi with respect to the power distribution between the source and the relays, and with respect to the durations of the slots specified by the relaying protocol. Convexity of xi with respect to the design parameters is shown. The method applies to a general class of radio channels that includes the Rayleigh and the Rice channels as particular cases. Decode-and-forward as well as amplify-and-forward protocols are considered in the half duplex mode. While the proposed approach is designed for the high SNR regime, simulations show that outage probability is reduced in a similar proportion at moderate SNR.

In this paper, we address the joint data-aided estimation of the frequency offsets and channel coefficients in the uplink MIMO-OFDMA systems. A novel low complexity ML-based joint estimator is proposed which is applicable for any subcarrier assignment scheme (SAS). It is shown that the proposed estimator is asymptotically efficient, i.e., the variance of the estimates achieves the performance bounds for large number of subcarriers. Moreover, this estimator is generalized in such a way that the resulting estimators achieve the performance bounds for reasonable training sizes. Simulation results sustain all our claims.

In this paper, we investigate the data-aided estimation of frequency offsets and channel coefficients in the uplink transmission of MIMO-OFDMA systems. A compact and informative expression of the deterministic Cramer-Rao bound (CRB) is derived for large training sequence sizes. It is shown that the asymptotic performance bounds do not depend on the number of users. Next, we bring to the fore the training strategies which minimize the asymptotic performance bounds and which are therefore likely to lead to accurate estimates of the parameters. In particular, for a given user, it is shown that accurate frequency offset estimates are likely to be obtained by introducing relevant correlation between training sequences sent at different antennas. On the other hand, accurate channel estimation is achieved when training sequences sent at different antennas are uncorrelated

In this paper, a blind estimator of the technical parameters of continuous phase modulated (CPM) signals is proposed. It consists in estimating jointly the modulation index, the symbol period and the frequency offset. It is based on the following observations. First, the inverse of the index is the smallest positive real number a CPM signal should be raised to in order to generate a deterministic harmonic signal; second, the frequencies of the harmonic signal are simply related to the symbol period and the carrier frequency. The practical implementation of this joint estimator is described and the asymptotic behavior of the estimation error is studied. If N is the number of signaling intervals, the estimate of the modulation index is shown to converge to a non-Gaussian distribution at rate 1/N, while the estimate of the frequency offset and the estimate of the symbol period converge at rate 1/N/sup 3/2/. We also investigate the case where the modulation index and the symbol period are available at the receiver side. An estimator of the frequency offset is proposed by adapting the above joint estimator to the latter case. The asymptotic behavior of this estimator is studied and compared with the case where all of the parameters are unknown, so as to evaluate the possible degradation of the performance due to the ignorance of certain technical parameters. Simulations results sustain our theoretical claims.

In this paper, a new nondata aided estimator of the technical parameters of continuous phase modulated (CPM) signals is proposed. It consists in estimating jointly the modulation index, the symbol period and the frequency offset. It is based on the following observations. Firstly, the inverse of the index is the smallest positive real number a CPM signal should be raised to in order to generate a deterministic harmonic signal; secondly, the frequencies of the harmonic signal are simply related to the symbol period and the carrier frequency. The asymptotic behavior of the estimation error is studied. If N is the number of signaling intervals, the estimate of the modulation index is shown to converge to a nonGaussian distribution at rate 1/N, while the estimate of the frequency offset and the estimate of the symbol period converge at rate 1/N/sup 3/2/. Simulations results sustain our theoretical claims.

In this paper, a new non data aided estimator of the modulation index of a CPM signal is proposed. It is based on the observation that the inverse of the index is the smallest positive real number to which the CPM signal should be expanded in order to generate a sinusoid of frequency 1 over 2Ts. The asymptotic behavior of the estimate is studied. If N is the sample size, the estimate is shown to converge to a non Gaussian distribution at rate 1 over N. Simulations results sustain our theoretical claims.