Deep learning has achieved remarkable success across a wide range of applications, such as language modeling, computer vision, recommendation systems, and robotics. However, the growing size of models and their increasing computational demands pose significant challenges, particularly for resource-constrained devices. A promising approach to address these challenges is extreme quantization, exemplified by binary and ternary neural networks. These techniques significantly reduce model size by quantizing weights and activations to 1 bit or 1.58 bits, while simplifying computation, making them well-suited for efficient deployment in resource-limited environments. This paper presents a comprehensive review of extreme quantization techniques, organized into three key areas: (1) a comparative analysis of quantizing only the weights (e.g., binary weight networks, ternary weight networks) versus quantizing both weights and activations (e.g., binary neural networks, ternary neural networks), along with a discussion of the progress and trade-offs of their approaches; (2) an examination of how extreme quantization, initially applied to convolutional neural networks, has been extended to Transformer architectures; and (3) an overview of compute-in-memory architectures optimized for binarization and ternarization, including designs based on advanced bit-cell technologies.

An m-extracting procedure produces unbiased random bits from a loaded dice with m faces, and its output rate, the average number of output per input is bounded by the Shannon entropy of the source. This information-theoretic upper bound can be achieved only asymptotically as the input size increases, by certain extracting procedures that we call asymptotically optimal. Although a computationally efficient asymptotically optimal 2-extracting procedure has been known for a while, its counterparts for m-ary input, m > 2, was found only recently, and they are still relatively complicated to describe. A binarization takes inputs from an m-faced dice and produce bit sequences to be fed into a binary extracting procedure to obtain random bits. Thus, binary extracting procedures give rise to an m-extracting procedure via a binarization. A binarization is to be called complete, if it preserves the asymptotic optimality, and such a procedure has been proposed by Zhou and Bruck. We show that a complete binarization naturally arises from a binary tree with m leaves. Therefore, there exist complete binarizations in abundance and Zhou-Bruck scheme is an instance of them. We now have a relatively simple way to obtain an asymptotically optimal and computationally efficient m-extracting procedure, from a binary one, because these binarizations are both conceptually and computationally simple. The well-known leaf entropy theorem and a closely related structure lemma play important roles in the arguments.

Extracting procedures produce unbiased random bits from biased coin flips. Binarizations take inputs from an m-faced dice and produce bit sequences to be fed into a (binary) extracting procedure to obtain random bits, and this can be done in an entropy-preserving manner, without loss of information. Such a procedure has been proposed by Zhou and Bruck [1]. We discuss a family of such entropy-preserving processes that we call complete binarizations.

Peres's algorithm produces unbiased random bits from biased coin tosses, recursively, using the famous von Neumann's method as its base. The algorithm is simple and elegant, but, at first glance, appears to work almost like magic and its generalization is elusive. We generalize the method to generate unbiased random bits from loaded dice, i.e., many-valued Bernoulli source. The generalization is asymptotically optimal in its output rate as is the original Peres's algorithm. Three-valued case is discussed in detail, and then other many-faced cases are considered.

In this paper, we characterize functions that simulate independent unbiased coin flips from independent coin flips of unknown bias. We call such functions randomizing. Our characterization of randomizing functions enables us to identify the functions that generate the largest average number of fair coin flips from a fixed number of biased coin flips. We show that these optimal functions are efficiently computable. Then we generalize the characterization, and we present a method to simulate an arbitrary rational probability distribution optimally (in terms of the average number of output digits) and efficiently (in terms of computational complexity) from outputs of many-faced dice of unknown distribution. We also study randomizing functions on exhaustive prefix-free sets.

This paper addresses the problem of constructing the scale-space aspect graph of a solid of revolution whose surface is the zero set of a polynomial volumetric density undergoing a Gaussian diffusion process. Equations for the associated visual event surfaces are derived, and polynomial curve tracing techniques are used to delineate these surfaces. An implementation and examples are presented, and limitations as well as extensions of the proposed approach are discussed.