What computational structure are we building into large language models when we train them on next-token prediction? Here, we present evidence that this structure is given by the meta-dynamics of belief updating over hidden states of the data- generating process. Leveraging the theory of optimal prediction, we anticipate and then find that belief states are linearly represented in the residual stream of transformers, even in cases where the predicted belief state geometry has highly nontrivial fractal structure. We investigate cases where the belief state geometry is represented in the final residual stream or distributed across the residual streams of multiple layers, providing a framework to explain these observations. Furthermore we demonstrate that the inferred belief states contain information about the entire future, beyond the local next-token prediction that the transformers are explicitly trained on. Our work provides a general framework connecting the structure of training data to the geometric structure of activations inside transformers.

A challenge in image processing is quantifying the perceptual quality of distorted images. Solutions to this problem allow lossy compression algorithms to be more easily and accurately evaluated. Motivated by failings of mean-squared error (MSE/PSNR), Wang, Bovik, and others proposed a perceptual image measure called mean structural similarity (MSSIM), which decomposes the distortion of image patches into three components: a difference in mean luminance, a difference in luminance variance, and a difference in structure. We present a new measure, mean discrete structural similarity (MDSSIM), that replaces the structural comparison of MSSIM with the Hamming distance between suitably discretized original and distorted image patches. To assess its performance, we apply this new image measure to a standard human psychophysics dataset, the LIVE Image Quality Assessment Database (Release 2). The high correlation of MDSSIM with human scores suggests, consistent with experiment and well-known results about lossy compression, that the human visual system may be fundamentally concerned with discrete structure in natural images.