Large language models (LLMs) show excellent performance but are compute- and memory-intensive. Quantization can reduce memory and accelerate inference. However, existing methods cannot maintain accuracy and hardware efficiency at the same time. We propose SmoothQuant, a training-free, accuracy-preserving, and general-purpose post-training quantization (PTQ) solution to enable 8-bit weight, 8-bit activation (W8A8) quantization for LLMs. Based on the fact that weights are easy to quantize while activations are not, SmoothQuant smooths the activation outliers by offline migrating the quantization difficulty from activations to weights with a mathematically equivalent transformation. SmoothQuant enables an INT8 quantization of both weights and activations for all the matrix multiplications in LLMs, including OPT, BLOOM, GLM, MT-NLG, and LLaMA family. We demonstrate up to 1.56$\times$ speedup and 2$\times$ memory reduction for LLMs with negligible loss in accuracy. SmoothQuant enables serving 530B LLM within a single node. Our work offers a turn-key solution that reduces hardware costs and democratizes LLMs.

This paper presents a fast color-to-gray conversion algorithm which robustly reproduces the visual appearance of a color image in grayscale. The conversion preserves feature discriminability and reasonable color ordering, while respecting the original lightness of colors, by simple optimization of a nonlinear global mapping. Experimental results show that our method produces convincing results for a variety of color images. We further extend the method to temporally coherent color-to-gray video conversion.

Let F ∪ {U} be a collection of convex sets in Rd such that F covers U. We show that if the elements of F and U have comparable size, in the sense that each contains a ball of radius r and is contained in a ball of radius R for some fixed r and R, then for any ε > 0 there exists Hε ⊂ F, whose size |Hε| is polynomial in 1/ε and independent of |F|, that covers U except for a volume of at most ε. The size of the smallest such subset depends on the geometry of the elements of F; specifically, we prove that it is O(1/ε) when F consists of axis-parallel unit squares in the plane and Õ(ε1--d/2) when F consists of unit balls in Rd (here, Õ(n) means O(n log n) for some constant), and that these bounds are, in the worst-case, tight up to the logarithmic factors.

Computing shadow boundaries is a difficult problem in the case of non-pointlight sources. A point is in the umbra if it does not see any part of anylight source; it is in full light if it sees entirely all the light sources;otherwise, it is in the penumbra. While the common boundary of the penumbraand the full light is well understood, less is known about the boundary of theumbra. In this paper we prove various bounds on the complexity of the umbra andthe penumbra cast by a segment or polygonal light source on a plane in the presence ofpolygon or polytope obstacles. In particular, we show that a single segment light source may cast on a plane, in thepresence of two triangles, four connected components of umbra and that two fatconvex obstacles of total complexity n can engender Ω(n) connectedcomponents of umbra. In a scene consisting of a segment light source and kdisjoint polytopes of total complexity n, we prove an Ω(nk2+k4)lower bound on the maximum number of connected components of the umbra and a O(nk3) upper bound on its complexity. We also prove that, in the presence of kdisjoint polytopes of total complexity n, some of which being light sources,the umbra cast on a plane may have Ω(n2k3 +nk5) connected components and has complexity O(n3k3).These are the first bounds on the size of the umbra in terms of both k and n. These results prove that the umbra, which is bounded by arcs of conics,is intrinsically much more intricate than the full light/penumbra boundary whichis bounded by linesegments and whose worst-case complexity is in Ω(nα(k) +km +k2) and O(nα(k) + kmα(k) +k2), where m is the complexity of the polygonallight source.