A full understanding of the behavior of a large language model (LLM) requires our grasp of its input token space. If this space differs from our assumptions, our comprehension of and conclusions about the LLM will likely be flawed. We elucidate the structure of the token embeddings both empirically and theoretically. We present a novel statistical test assuming that the neighborhood around each token has a relatively flat and smooth structure as the null hypothesis. Failing to reject the null is uninformative, but rejecting it at a specific token $\psi$ implies an irregularity in the token subspace in a $\psi$-neighborhood, $B(\psi)$. The structure assumed in the null is a generalization of a manifold with boundary called a \emph{smooth fiber bundle} (which can be split into two spatial regimes -- small and large radius), so we denote our new hypothesis test as the ``fiber bundle hypothesis.'' By running our test over several open-source LLMs, each with unique token embeddings, we find that the null is frequently rejected, and so the evidence suggests that the token subspace is not a fiber bundle and hence also not a manifold. As a consequence of our findings, when an LLM is presented with two semantically equivalent prompts, if one prompt contains a token implicated by our test, the response to that prompt will likely exhibit less stability than the other.

Topological Data Analysis (TDA) is a topic which has recently seen many applications. The goal of this special session is to highlight the bridge between signal processing, machine learning and techniques in topological data analysis. In this way, we hope to encourage more engineers to start exploring TDA and its applications. This paper briefly introduces the standard techniques used in this area, delineates the common theme connecting the works presented in this session, and concludes with a brief summary of each of the papers presented.

This letter presents a two-stage topological algorithm for recovering an estimate of a quasi-periodic function from a set of noisy measurements. The first stage of the algorithm is a topological phase estimator, which detects the quasi-periodic structure of the function without placing additional restrictions on the function. By respecting this phase estimate, the algorithm avoids creating distortion even when it uses a large number of samples for the estimate of the function.