ABSTRACT Localization and place recognition are important tasks in many fields, including autonomous driving, robotics, and AR/VR applications. Local and global feature‐based solutions typically rely on exact nearest neighbour search methods, such as KD‐tree, to retrieve candidate places or frames and estimate the precise sensor position using point correspondences. However, in large‐scale applications, maintaining real‐time online processing without loss of performance can be challenging. We propose that by using an approximate nearest neighbour search method instead of exact methods, runtime can be significantly reduced without sacrificing accuracy. To demonstrate this, we developed a localization pipeline based on a keypoint voting mechanism, employing the hierarchical navigable small world (HNSW) structure as the nearest neighbour search method. Graph‐based structures like HNSW are widely used in other domains, such as recommender systems and large language models. We argue that for the use case of matching local feature descriptors, the slightly lower accuracy in terms of exact neighbours does not lead to a significant increase in localization error. We evaluated our pipeline on widely known datasets and performed parameter tuning of HNSW specifically for this use case.

This paper investigates the structure of linear operators introduced in Hernandez et al. [2023] that decode specific relational facts in transformer language models. We extend their single-relation findings to a collection of relations and systematically chart their organization. We show that such collections of relation decoders can be highly compressed by simple order-3 tensor networks without significant loss in decoding accuracy. To explain this surprising redundancy, we develop a cross-evaluation protocol, in which we apply each linear decoder operator to the subjects of every other relation. Our results reveal that these linear maps do not encode distinct relations, but extract recurring, coarse-grained semantic properties (e.g., country of capital city and country of food are both in the country-of-X property). This property-centric structure clarifies both the operators' compressibility and highlights why they generalize only to new relations that are semantically close. Our findings thus interpret linear relational decoding in transformer language models as primarily property-based, rather than relation-specific.

We explore element-wise convex combinations of two permutation-aligned neural network parameter vectors ΘA and ΘB of size d. We conduct extensive experiments by examining various distributions of such model combinations parametrized by elements of the hypercube [0,1]d and its vicinity. Our findings reveal that broad regions of the hypercube form surfaces of low loss values, indicating that the notion of linear mode connectivity extends to a more general phenomenon which we call mode combinability. We also make several novel observations regarding linear mode connectivity and model re-basin. We demonstrate a transitivity property: two models re-based to a common third model are also linear mode connected, and a robustness property: even with significant perturbations of the neuron matchings the resulting combinations continue to form a working model. Moreover, we analyze the functional and weight similarity of model combinations and show that such combinations are non-vacuous in the sense that there are significant functional differences between the resulting models.

Abstract. We consider the minimum number of lines [Formula: see text] and [Formula: see text] needed to intersect or pierce, respectively, all the cells of the [Formula: see text] chessboard. Determining these values can also be interpreted as a strengthening of the classical plank problem for integer points. Using the symmetric plank theorem of K. Ball, we prove that [Formula: see text] for each [Formula: see text]. Studying the piercing problem, we show that [Formula: see text] for [Formula: see text], where the upper bound is conjectured to be sharp. The lower bound is proven by using the linear programming method, whose limitations are also demonstrated.

We employ a toolset --- dubbed Dr. Frankenstein --- to analyse the similarity of representations in deep neural networks. With this toolset we aim to match the activations on given layers of two trained neural networks by joining them with a stitching layer. We demonstrate that the inner representations emerging in deep convolutional neural networks with the same architecture but different initialisations can be matched with a surprisingly high degree of accuracy even with a single, affine stitching layer. We choose the stitching layer from several possible classes of linear transformations and investigate their performance and properties. The task of matching representations is closely related to notions of similarity. Using this toolset we also provide a novel viewpoint on the current line of research regarding similarity indices of neural network representations: the perspective of the performance on a task.