The generative adversarial network (GAN) aims to approximate an unknown distribution via a parameterized neural network (NN). While GANs have been widely applied in reinforcement and semi-supervised learning as well as computer vision tasks, selecting their parameters often needs an exhaustive search and only a few selection methods can be proved to be theoretically optimal. One of the most promising GAN variants is the Wasserstein GAN (WGAN). Prior work on optimal parameters for WGAN is limited to the linear-quadratic-Gaussian (LQG) setting, where the NN is linear and the data is Gaussian. In this paper, we focus on the characterization of optimal WGAN parameters beyond the LQG setting. We derive closed-form optimal parameters for one-dimensional WGANs with non-linear sigmoid and ReLU activation functions. Extensions to high-dimensional WGANs are also discussed. Empirical studies show that our closed-form WGAN parameters have good convergence behavior with data under both Gaussian and Laplace distributions.

This paper approaches the unsupervised learning problem by gradient descent in the space of probability density functions. A main result shows that along the gradient flow induced by a distribution-dependent ordinary differential equation (ODE), the unknown data distribution emerges as the long-time limit. That is, one can uncover the data distribution by simulating the distribution-dependent ODE. Intriguingly, the simulation of the ODE is shown equivalent to the training of generative adversarial networks (GANs). This equivalence provides a new "cooperative" view of GANs and, more importantly, sheds new light on the divergence of GANs. In particular, it reveals that the GAN algorithm implicitly minimizes the mean squared error (MSE) between two sets of samples, and this MSE fitting alone can cause GANs to diverge. To construct a solution to the distribution-dependent ODE, we first show that the associated nonlinear Fokker-Planck equation has a unique weak solution, by the Crandall-Liggett theorem for differential equations in Banach spaces. Based on this solution to the Fokker-Planck equation, we construct a unique solution to the ODE, using Trevisan's superposition principle. The convergence of the induced gradient flow to the data distribution is obtained by analyzing the Fokker-Planck equation.

The Byzantine distributed quickest change detection (BDQCD) is studied, where a fusion center monitors the occurrence of an abrupt event through a bunch of distributed sensors that may be compromised. We first consider the binary hypothesis case where there is only one post-change hypothesis and prove a novel converse to the first-order asymptotic detection delay in the large mean time to a false alarm regime. This converse is tight in that it coincides with the currently best achievability shown by Fellouris et al.; hence, the optimal asymptotic performance of binary BDQCD is characterized. An important implication of this result is that, even with compromised sensors, a 1-bit link between each sensor and the fusion center suffices to achieve asymptotic optimality. To accommodate multiple post-change hypotheses, we then formulate the multi-hypothesis BDQCD problem and again investigate the optimal first-order performance under different bandwidth constraints. A converse is first obtained by extending our converse from binary to multi-hypothesis BDQCD. Two families of stopping rules, namely the simultaneous d-th alarm and the multi-shot d-th alarm, are then proposed. Under sufficient link bandwidth, the simultaneous d-th alarm, with d being set to the number of honest sensors, can achieve the asymptotic performance that coincides with the derived converse bound; hence, the asymptotically optimal performance of multi-hypothesis BDQCD is again characterized. Moreover, although being shown to be asymptotically optimal only for some special cases, the multi-shot d-th alarm is much more bandwidth-efficient and energy-efficient than the simultaneous d-th alarm. Built upon the above success in characterizing the asymptotic optimality of the BDQCD, a corresponding leader-follower Stackelberg game is formulated and its solution is found.

Sequential change point detection with multiple decentralized sensors is studied. Each sensor makes a local decision based on its own observations and reports it through a bandlimited link to a fusion center, which then decides whether the change has occurred. Since sensors in many applications such as cyber-physical systems are prone to a number of attacks such as Byzantine attacks, combating such a security breach becomes one of the most crucial issues. Previous works on sequential change detection under Byzantine attacks only focus on binary-hypothesis case, which significantly limits the applicability. In this paper, we consider the extension to the multi-hypothesis setting. We show that naively extending the existing method from the binary case to the multi-hypothesis one can result in a catastrophic event preventing the fusion center from making a conclusive decision. Thus we propose the other two new methods by allowing each sensor to cast multiple local alarms, and both can avoid this catastrophic event and improve the asymptotic detection delay. In analyzing detection delays of our multi-hypothesis schemes, we also show that for each hypothesis, asymptotically, it suffices to focus on the competing hypothesis that is closest in Kullback-Leibler distance. Through large sensor analysis, we also show that as the number of honest sensors grows, one of the proposed scheme, called the simultaneous rule, approaches the optimal performance within a factor of 2.

The Byzantine distributed sequential change detection (BDSCD) problem is studied, where a fusion center monitors an abrupt event occurring at an unknown time through a bunch of distributed sensors. It is assume that a part of the sensors are compromised and each sensor, honest or compromised, communicates with the fusion center via a noiseless link. A new converse for this problem is presented whose first-order asymptotic delay subject to a certain false alarm rate coincides with the currently best known result achieved by the consensus rule proposed by Fellouris et al. This result characterizes the first-order asymptotic performance of BDSCD and shows that 1-bit links suffice to achieve the asymptotic optimality. The proof of the converse involves constructing an attack strategy, called the reverse attack, introducing a genie that gives the fusion center the identities of a subset of honest sensors and observations at each sensor used for generating its local report, and transforming the problem into an equivalent non-Byzantine sequential change detection but with reduced number of honest sensors.