Determinantal point processes (DPPs) are random configurations of points with tunable negative dependence. Because sampling is tractable, DPPs are natural candidates for subsampling tasks, such as minibatch selection or coreset construction. A \emph{coreset} is a subset of a (large) training set, such that minimizing an empirical loss averaged over the coreset is a controlled replacement for the intractable minimization of the original empirical loss. Typically, the control takes the form of a guarantee that the average loss over the coreset approximates the total loss uniformly across the parameter space. Recent work has provided significant empirical support in favor of using DPPs to build randomized coresets, coupled with interesting theoretical results that are suggestive but leave some key questions unanswered. In particular, the central question of whether the cardinality of a DPP-based coreset is fundamentally smaller than one based on independent sampling remained open. In this paper, we answer this question in the affirmative, demonstrating that \emph{DPPs can provably outperform independently drawn coresets}. In this vein, we contribute a conceptual understanding of coreset loss as a \emph{linear statistic} of the (random) coreset. We leverage this structural observation to connect the coresets problem to a more general problem of concentration phenomena for linear statistics of DPPs, wherein we obtain \emph{effective concentration inequalities that extend well-beyond the state-of-the-art}, encompassing general non-projection, even non-symmetric kernels. The latter have been recently shown to be of interest in machine learning beyond coresets, but come with a limited theoretical toolbox, to the extension of which our result contributes. Finally, we are also able to address the coresets problem for vector-valued objective functions, a novelty in the coresets literature.

We investigate the procedure of semi-parametric maximum likelihood estimation under constraints on summary statistics. Such a procedure results in a discrete probability distribution supported on the data points that maximizes the likelihood among all distributions supported on the data points satisfying the specified constraints (called estimating equations). The resultant distribution is an approximation of the underlying population distribution. The study of such empirical likelihood estimation originates from the seminal work of Owen (1998 and 2001). We investigate this procedure in the setting of misspecified (or biased) constraints, i.e., when the null hypothesis is not true. We establish that the behavior of the optimal weight distribution under such misspecification differ markedly from their properties under the null, i.e., when the estimating equations are correctly specified (or unbiased). This is manifested by certain “singularities” in the optimal distribution, that are not observed under the null. Furthermore, we establish an anomalous behavior of the log-likelihood based Wilks’ statistic, which, unlike under the null, does not exhibit a chi-squared limit. In the Bayesian setting, we establish the posterior consistency of procedures based on these ideas, where instead of a parametric likelihood, an empirical likelihood is used to define the posterior distribution. In particular, we show that this posterior, as a random probability measure, rapidly converges, with explicit convergence guarantees, to the delta measure at the true parameter value. We also illustrate implications of our results in diverse settings such as degeneracies in exponential random graph models (ERGM) for random networks (Chatterjee and Diaconis, 2013, and Mukherjee, 2020), empirical procedures where the constraints are themselves estimated from data (Hjort, 2009), and to approximate Bayesian computation based procedures (Chaudhuri et al., 2020). A novel feature of our work is to connect the likelihood maximization problem to critical points of random polynomials. This yields the mass of the singular weight in the optimal weight distribution as the leading term in a canonical expansion of a critical point of a random polynomial. Our work unveils the possibility that similar random polynomial based techniques could be effective in analyzing a wide class of problems in related areas.

We propose a novel stochastic network model, called Fractal Gaussian Network (FGN), that embodies well-defined and analytically tractable fractal structures. Such fractal structures have been empirically observed in diverse applications. FGNs interpolate continuously between the popularpurely randomgeometric graphs (a.k.a. the Poisson Boolean network), and random graphs with increasingly fractal behavior. In fact, they form a parametric family ofsparserandom geometric graphs that are parametrized by a fractality parameter$\nu $which governs the strength of the fractal structure. FGNs are driven by the latent spatial geometry of Gaussian Multiplicative Chaos (GMC), a canonical model of fractality in its own right. We asymptotically characterize the expected number of edges, triangles, cliques and hub-and-spoke motifs in FGNs, unveiling a distinct pattern in their scaling with the size parameter of the network. We then examine the natural question of detecting the presence of fractality and the problem of parameter estimation based on observed network data, in addition to fundamental properties of the FGN as a random graph model. We also explore fractality in community structures by unveiling a natural stochastic block model in the setting of FGNs. Finally, we substantiate our results with phenomenological analysis of the FGN in the context of available scientific literature for fractality in networks, including applications to real-world massive network data.

Disordered complex networks are of fundamental interest in statistical physics, and they have attracted recent interest as stochastic models for information transmission over wireless networks. While mathematically tractable, a network based on the regulation Poisson point process model offers challenges vis-a-vis network efficiency. Strongly correlated alternatives, such as networks based on random matrix spectra (the Ginibre network), on the other hand offer formidable challenges in terms of tractability and robustness issues. In this work, we demonstrate that network models based on random perturbations of Euclidean latticesinterpolatebetween Poisson and rigidly structured networks, and allow us to achieve thebest of both worlds: significantly improve upon the Poisson model in terms of network efficacy measured by theSignal to Interference plus Noise Ratio(abbrv. SINR) and the related concept ofcoverage probabilities, at the same time retaining a considerable measure of mathematical and computational simplicity and robustness to erasure and noise. We investigate the optimal choice of the base lattice in this model, connecting it to the celebrated problem optimality of Euclidean lattices with respect to the Epstein Zeta function, which is in turn related to notions of lattice energy. This leads us to the choice of the triangular lattice in 2D and face centered cubic lattice in 3D, whose Gaussian perturbations we consider. We provide theoretical analysis and empirical investigations to demonstrate that the coverage probability decreases with increasing strength of perturbation, eventually converging to that of the Poisson network. In the regime of low disorder, our studies suggest an approximate statistical behaviour of the coverage function near a base station as a log-normal distribution with parameters depending on the Epstein Zeta function of the lattice, and related approximate dependencies for a power-law constant that governs the network coverage probability at large thresholds. In 2D, we determine the disorder strength at which the perturbed triangular lattice (abbrv. PTL) and the Ginibre networks are theclosestmeasured by comparing their network topologies via a comparison of theirPersistence Diagramsin the total variation as well as the symmetrized nearest neighbour distances. We demonstrate that, at this very same disorder, the PTL and the Ginibre networks exhibit very similar coverage probability distributions, with the PTL performing at least as well as the Ginibre. Thus, the PTL network at this disorder strength can be taken to be an effective substitute for the Ginibre network model, while at the same time offering the advantages of greater tractability both from theoretical and empirical perspectives.

In 1-bit compressive sensing, each measurement is quantized to a single bit, namely the sign of a linear function of an unknown vector, and the goal is to accurately recover the vector. While it is most popular to assume a standard Gaussian sensing matrix for 1-bit compressive sensing, using structured sensing matrices such as partial Gaussian circulant matrices is of significant practical importance due to their faster matrix operations. In this paper, we provide recovery guarantees for a correlation-based optimization algorithm for robust 1-bit compressive sensing with partial Gaussian circulant matrices (with random column sign flips) under a generative prior, where the signal to estimate is assumed to belong to the range of a Lipschitz continuous generative model with bounded inputs. Under suitable assumptions, we match guarantees that were previously only known to hold for i.i.d. Gaussian matrices that require significantly more computation.

Stochastic gradient descent (SGD) is a cornerstone of machine learning. When the number $N$ of data items is large, SGD relies on constructing an unbiased estimator of the gradient of the empirical risk using a small subset of the original dataset, called a minibatch. Default minibatch construction involves uniformly sampling a subset of the desired size, but alternatives have been explored for variance reduction. In particular, experimental evidence suggests drawing minibatches from determinantal point processes (DPPs), tractable distributions over minibatches that favour diversity among selected items. However, like in recent work on DPPs for coresets, providing a systematic and principled understanding of how and why DPPs help has been difficult. In this work, we contribute an orthogonal polynomial-based determinantal point process paradigm for performing minibatch sampling in SGD. Our approach leverages the specific data distribution at hand, which endows it with greater sensitivity and power over existing data-agnostic methods. We substantiate our method via a detailed theoretical analysis of its convergence properties, interweaving between the discrete data set and the underlying continuous domain. In particular, we show how specific DPPs and a string of controlled approximations can lead to gradient estimators with a variance that decays faster with the batchsize than under uniform sampling. Coupled with existing finite-time guarantees for SGD on convex objectives, this entails that, for a large enough batchsize and a fixed budget of item-level gradients to evaluate, DPP minibatches lead to a smaller bound on the mean square approximation error than uniform minibatches. Moreover, our estimators are amenable to a recent algorithm that directly samples linear statistics of DPPs (i.e., the gradient estimator) without sampling the underlying DPP (i.e., the minibatch), thereby reducing computational overhead. We provide detailed synthetic as well as real data experiments to substantiate our theoretical claims.

Compressive phase retrieval is a popular variant of the standard compressive sensing problem in which the measurements only contain magnitude information. In this paper, motivated by recent advances in deep generative models, we provide recovery guarantees with near-optimal sample complexity for phase retrieval with generative priors. We first show that when using i.i.d. Gaussian measurements and an $L$-Lipschitz continuous generative model with bounded $k$-dimensional inputs, roughly $O(k \log L)$ samples suffice to guarantee that any signal minimizing an amplitude-based empirical loss function is close to the true signal. Attaining this sample complexity with a practical algorithm remains a difficult challenge, and finding a good initialization for gradient-based methods has been observed to pose a major bottleneck. To partially address this, we further show that roughly $O(k \log L)$ samples ensure sufficient closeness between the underlying signal and any {\em globally optimal} solution to an optimization problem designed for spectral initialization (though finding such a solution may still be challenging). We also adapt this result to sparse phase retrieval, and show that $O(s \log n)$ samples are sufficient for a similar guarantee when the underlying signal is $s$-sparse and $n$-dimensional, matching an information-theoretic lower bound. While these guarantees do not directly correspond to a practical algorithm, we propose a practical spectral initialization method motivated by our findings, and experimentally observe performance gains over various existing spectral initialization methods for sparse phase retrieval.

We propose a novel stochastic network model, called Fractal Gaussian Network (FGN), that embodies well-defined and analytically tractable fractal structures. Such fractal structures have been empirically observed in diverse applications. FGNs interpolate continuously between the popular purely random geometric graphs (a.k.a. the Poisson Boolean network), and random graphs with increasingly fractal behavior. In fact, they form a parametric family of sparse random geometric graphs that are parametrised by a fractality parameter $\nu$ which governs the strength of the fractal structure. FGNs are driven by the latent spatial geometry of Gaussian Multiplicative Chaos (GMC), a canonical model of fractality in its own right. We explore the natural question of detecting the presence of fractality and the problem of parameter estimation based on observed network data. Finally, we explore fractality in community structures by unveiling a natural stochastic block model in the setting of FGNs.