Quantization is a widely used compression technique for reducing the memory and computation costs of large pre-trained models. A key challenge in per-channel post-training quantization (PTQ) is selecting appropriate scaling factors to replace weight values with values from a scaled integer grid. Existing methods typically fix the scale at the outset via heuristic tuning or grid search. We proposeBeacon, a simple and effective algorithm that eliminates the need for such manual tuning. Beacon performs per-channel PTQ directly using an unscaled grid and automatically determines the optimal scaling factors by exploiting the geometry of quantizing a channel. It does not rely on back-propagation or large calibration sets. Despite its simplicity and tuning-free nature, Beacon achieves competitive performance compared to state-of-the-art methods, making it a practical solution for efficient model deployment.

We propose a new computationally efficient method for quantizing the weights of pre- trained neural networks that is general enough to handle both multi-layer perceptrons and convolutional neural networks. Our method deterministically quantizes layers in an iterative fashion with no complicated re-training required. Specifically, we quantize each neuron, or hidden unit, using a greedy path-following algorithm. This simple algorithm is equivalent to running a dynamical system, which we prove is stable for quantizing a single-layer neural network (or, alternatively, for quantizing the first layer of a multi-layer network) when the training data are Gaussian. We show that under these assumptions, the quantization error decays with the width of the layer, i.e., its level of over-parametrization. We provide numerical experiments, on multi-layer networks, to illustrate the performance of our methods on MNIST and CIFAR10 data, as well as for quantizing the VGG16 network using ImageNet data.

Binary, or one-bit, representations of data arise naturally in many applications, and are appealing in both hardware implementations and algorithm design. In this work, we study the problem of data classification from binary data obtained from the sign pattern of low-dimensional projections and propose a framework with low computation and resource costs. We illustrate the utility of the proposed approach through stylized and realistic numerical experiments, and provide a theoretical analysis for a simple case. We hope that our framework and analysis will serve as a foundation for studying similar types of approaches.

Compressed sensing is now established as an effective method for dimension reduction when the underlying signals are sparse or compressible with respect to some suitable basis or frame. One important, yet under-addressed problem regarding the compressive acquisition of analog signals is how to perform quantization. This is directly related to the important issues of how “compressed” compressed sensing is (in terms of the total number of bits one ends up using after acquiring the signal) and ultimately whether compressed sensing can be used to obtain compressed representations of suitable signals. In this paper, we propose a concrete and practicable method for performing “analog-to-information conversion”. Following a compressive signal acquisition stage, the proposed method consists of a quantization stage, based on ΣΔ (sigma-delta) quantization, and a subsequent encoding (compression) stage that fits within the framework of compressed sensing seamlessly. We prove that, using this method, we can convert analog compressive samples to compressed digital bitstreams and decode using tractable algorithms based on convex optimization. We prove that the proposed analog-to-information converter (AIC) provides a nearly optimal encoding of sparse and compressible signals. Finally, we present numerical experiments illustrating the effectiveness of the proposed AIC.

Consider the recovery of an unknown signal x from quantized linear measurements. In the one-bit compressive sensing setting, one typically assumes that x is sparse, and that the measurements are of the form sign((ai, x)) ϵ {±1}. Since such measurements give no information on the norm of x, recovery methods typically assume that ∥x∥2= 1. We show that if one allows more generally for quantized affine measurements of the form sign((ai, x) + bi), and if the vectors ai are random, an appropriate choice of the affine shifts bi allows norm recovery to be easily incorporated into existing methods for one-bit compressive sensing. In addition, we show that for arbitrary fixed x in the annulus r ∥×∥2R, one may estimate the norm ∥×∥2up to additive error δ from ≳ R4r-2δ-2such binary measurements through a single evaluation of the inverse Gaussian error function. Finally, all of our recovery guarantees can be made universal over sparse vectors in the sense that with high probability, one set of measurements and thresholds can successfully estimate all sparse vectors x in a Euclidean ball of known radius.

In this note we study the under-addressed quantization stage implicit in any compressed sensing signal acquisition paradigm. We also study the problem of compressing the bitstream resulting from the quantization. We propose using Sigma-Delta (ΣΔ) quantization followed by a compression stage comprised of a discrete Johnson-Lindenstrauss embedding, and a subsequent reconstruction scheme based on convex optimization. We show that this encoding/decoding method yields near-optimal rate-distortion guarantees for sparse and compressible signals and is robust to noise. Our results hold for sub-Gaussian (including Gaussian and Bernoulli) random compressed sensing measurements, and they hold for high bit-depth quantizers as well as for coarse quantizers including 1-bit quantization.

We study the problem of recovering sparse vectors given possibly erroneous support estimates. First, we provide necessary and sufficient conditions for weighted ℓ1minimization to successfully recovery all sparse signals whose support estimate is sufficiently accurate. We relate these conditions to the analogous ones for ℓ1minimization, showing that they are equivalent when the support estimate is 50% accurate but that the weighted ℓ1conditions are easier to satisfy when the support is more than 50% accurate. Second, to quantify this improvement, we provide bounds on the number of Gaussian measurements that ensure, with high probability, that weighted ℓ1minimization succeeds. The resulting number of measurements can be significantly less than what is needed to ensure recovery via ℓ1minimization. Finally, we illustrate our results via numerical experiments.

We study Sigma-Delta ($\Sigma\Delta$) quantization of oversampled bandlimited functions. We prove that digitally integrating blocks of bits and then down-sampling, a process known as decimation, can efficiently encode the associated $\Sigma\Delta$ bit-stream. It allows a large reduction in the bit-rate while still permitting good approximation of the underlying bandlimited function via an appropriate reconstruction kernel. Specifically, in the case of stable $r$th order $\Sigma\Delta$ schemes we show that the reconstruction error decays exponentially in the bit-rate. For example, this result applies to the 1-bit, greedy, first-order $\Sigma\Delta$ scheme.

We study recovery conditions of weightedl1minimization for signal reconstruction from compressed sensing measurements when partial support information is available. We show that if at least 50% of the (partial) support information is accurate, then weightedl1minimization is stable and robust under weaker sufficient conditions than the analogous conditions for standardl1minimization. Moreover, weightedl1minimization provides better upper bounds on the reconstruction error in terms of the measurement noise and the compressibility of the signal to be recovered. We illustrate our results with extensive numerical experiments on synthetic data and real audio and video signals.

In this note, we show that by quantizing theN-dimensional frame coefficients of signals in Rdusingrth-order Sigma-Delta quantization schemes, it is possible to achieve root-exponential accuracy in the oversampling rate λ: =N/d. In particular, we construct a family of finite frames tailored specifically for coarse Sigma-Delta quantization that admit themselves as both canonical duals and Sobolev duals. Our construction allows for error guarantees that behave ase-c√{λ}, where under a mild restriction on the oversampling rate, the constants are absolute. Moreover, we show that harmonic frames can be used to achieve the same guarantees, but with the constants now depending ond.

In this paper, we propose an algorithm to estimate the true values of saturated pixels in color images. Pixel saturation occurs when at least one color channel is clipped at some value below the full dynamic range of the scene, resulting in a loss in image fidelity. The proposed algorithm is based on the assumptions that images are nearly sparse in an appropriate transform domain, and that saturated pixels can be inferred from the structure of non-saturated neighboring pixels. Consequently, we use a hierarchical windowing algorithm which selects image regions containing relatively few saturated pixels for processing. Starting with small sized regions, and progressively increasing the size, we solve a sparsity promoting constrained ℓ1minimization problem for each selected region to recover the saturated pixels. Moreover, we provide simulation results to show the effectiveness of our algorithm.

Sparco is a framework for testing and benchmarking algorithms for sparse reconstruction. It includes a large collection of sparse reconstruction problems drawn from the imaging, compressed sensing, and geophysics literature. Sparco is also a framework for implementing new test problems and can be used as a tool for reproducible research. Sparco is implemented entirely in Matlab, and is released as open-source software under the GNU Public License.

We present theoretical results pertaining to the ability of lscrpminimization to recover sparse and compressible signals from incomplete and noisy measurements. In particular, we extend the results of Candes, Romberg and Tao (2005) to the ppminimization with certain values of p1minimization does. This is especially true when the restricted isometry constants are relatively large.