In the graph clustering problem with a planted solution, the input is a graph on \(n\) vertices partitioned into \(k\) clusters, and the task is to infer the clusters from graph structure. A standard assumption is that clusters induce well-connected subgraphs (i.e. \(\Omega(1)\)-expanders), and connections between clusters are sparse (i.e. clusters form \(\epsilon\)-sparse cuts). Such a graph defines the clustering uniquely up to \(\approx \epsilon\) misclassification rate, and efficient algorithms for achieving this rate are known. While this vanilla version of graph clustering is extremely well studied, in the practice of graph analysis vertices of the graph are typically equipped with labels, or features, that provide additional information on cluster ids of the vertices. For example, each vertex could be equipped with a cluster label that is corrupted independently with probability \(\delta\). Using either of the two sources of information separately leads to misclassification rate \(\min \{\epsilon, \delta\}\), but can one combine the two to achieve misclassification rate \(\approx \epsilon\delta\)?

Given a vertex in a \((k,\varphi,\epsilon)\)-clusterable graph, i.e. a graph whose vertex set can be partitioned into a disjoint union of \(\varphi\)-expanders of size \(\approx n/k\) with outer conductance bounded by \(\epsilon\), can one quickly tell which cluster it belongs to? This is a classical question going back to the expansion testing problem of Goldreich and Ron’11 (the case of \(k=2\)) that has received a lot of attention in the literature. For \(k=2\) a sample of \(\approx n^{1/2+O(\epsilon/\varphi^{2})}\) logarithmic length walks from a given vertex approximately determines its cluster membership by the birthday paradox: two vertices whose random walk samples are ’close’ are likely in the same cluster, and otherwise in different clusters.

Large language models (LLMs) have achieved impressive success, but their high memory requirements present challenges for long-context token generation. In this paper we study the streaming complexity of attention approximation, a key computational primitive underlying token generation. Our main contribution is BalanceKV, a streaming algorithm for $\epsilon$-approximating attention computations based on geometric process for selecting a balanced collection of Key and Value tokens as per Banaszczyk's vector balancing theory. We complement our algorithm with space lower bounds for streaming attention computation. Besides strong theoretical guarantees, BalanceKV exhibits empirically validated performance improvements over existing methods, both for attention approximation and end-to-end performance on various long context benchmarks.