Hyperdimensional Computing (HDC) has shown promise in time series classification by enhancing MiniROCKET, forming HDC-MiniROCKET. However, the impact of choosing a specific HDC implementation, referred to as a Vector Symbolic Architecture (VSA), within this framework is so far unexplored. This paper systematically evaluates different VSAs within HDC-MiniROCKET, analyzing their impact on classification performance, hyperparameter sensitivity, and computational efficiency. Our findings reveal that certain VSAs require a significantly broader range of hyperparameter values to achieve optimal accuracy despite similar properties. We demonstrate this effect on a synthetic dataset and validate it across a subset of the real-world benchmark UCR, showing that VSA performance varies depending on the dataset characteristics. Additionally, we investigate the trade-off between computational complexity and classification accuracy. We find that computationally efficient VSAs not only reduce processing time but also achieve comparable or superior accuracy to more complex alternatives. These insights help to select VSAs in HDC-based time series classification.

Abstract Time series classification poses significant challenges due to the inherent temporal order of the data points and the existence of sequential dependencies between them. The ROCKET family, featuring methods like MiniROCKET, MultiROCKET, and HYDRA, is currently a leading approach in this domain, leveraging convolution kernels to aggregate temporal features into encodings for linear classifiers. However, these models encode temporal features over short temporal windows and then aggregate them as an unordered set of encodings over the longer temporal window of the entire data sequence. This prevents these models from capturing any longer sequence structure. To address this design drawback, we propose integrating hyperdimensional computing into ROCKET methods to explicitly incorporate temporal order of the short-term features within the entire time series. This approach enhances the discriminative power of encodings generated by MiniROCKET, MultiROCKET, and HYDRA where longer-term structure exists in the data, leading to increased classification performance with minimal computational overhead. More specifically, we introduce a method to represent time series as high-dimensional vectors through multiplicative binding of ROCKET encodings with encodings representing temporal order, applying this approach across various ROCKET methods. Additionally, we explore different high-dimensional vector representations of temporal order, yielding diverse similarity kernels that enhance classification accuracy. Through experiments on synthetic datasets, we highlight the limitations of ROCKET methods in handling temporal dependencies and show how the methods based on hyperdimensional computing overcome these limitations. Furthermore, our extensive experimental evaluation with real-world datasets included in the recent UCR archive, validates the advantages of our approach, consistently achieving classification improvements across all ROCKET methods that integrate hyperdimensional computing. Notably, our best model achieves a relative error rate reduction of over 50% compared to the best ROCKET model on several UCR datasets.

The vector superposition operation plays a central role in Hyperdimensional Computing (HDC), enabling compositionality of hypervectors without expanding the dimensionality, unlike concatenation. However, a problem arises when the quantity of superimposed vectors surpasses a certain threshold, which is determined by the hypervector’s information capacity relative to its dimensionality. Beyond this point, cross-talk noise incrementally obscures the distinctiveness of individual hypervectors and information is lost. To solve this challenge, we introduce a novel method for weighting individual hypervectors within the superposition, ensuring that only those hypervectors crucial for a given task are prioritized. The weights are learned end-to-end using the backpropagation algorithm in a neural network. Our method is characterized by two key features: (1) The resultant weighting model is exceptionally compact, as the number of trainable weights is equal to the total number of hypervectors in the superposition; (2) The model offers enhanced explainability due to the compositional nature of its encoding. These features collectively contribute to the efficiency and effectiveness of our proposed classification approach using hyperdimensional computing. We illustrate our approach through the multi-channel time series classification task. In this framework, each channel is encoded as a hypervector-descriptor, and those are subsequently composed into a single hypervector via superposition. This superimposed vector forms the basis for training the classification model based on the neural network. Applying our approach of weighted superposition on this task improved the classification performance compared to standard superposition or concatenation of feature vectors, especially for larger numbers of channels.

Semantic segmentation is an essential step for many vision applications in order to understand a scene and the objects within. Recent progress in hyperspectral imaging technology enables the application in driving scenarios and the hope is that the devices perceptive abilities provide an advantage over RGB-cameras. Even though some datasets exist, there is no standard benchmark available to systematically measure progress on this task and evaluate the benefit of hyperspectral data. In this paper, we work towards closing this gap by providing the HyperSpectral Semantic Segmentation benchmark (HS3-Bench). It combines annotated hyperspectral images from three driving scenario datasets and provides standardized metrics, implementations, and evaluation protocols. We use the benchmark to derive two strong baseline models that surpass the previous state-of-the-art performances with and without pre-training on the individual datasets. Further, our results indicate that the existing learning-based methods benefit more from leveraging additional RGB training data than from leveraging the additional hyperspectral channels. This poses important questions for future research on hyperspectral imaging for semantic segmentation in driving scenarios. Code to run the benchmark and the strong baseline approaches are available under https://github.com/nickstheisen/hyperseg.

Classification of time series data is an important task for many application domains. One of the best existing methods for this task, in terms of accuracy and computation time, is MiniROCKET. In this work, we extend this approach to provide better global temporal encodings using hyperdimensional computing (HDC) mechanisms. HDC (also known as Vector Symbolic Architectures, VSA) is a general method to explicitly represent and process information in high-dimensional vectors. It has previously been used successfully in combination with deep neural networks and other signal processing algorithms. We argue that the internal high-dimensional representation of MiniROCKET is well suited to be complemented by the algebra of HDC. This leads to a more general formulation, HDC-MiniROCKET, where the original algorithm is only a special case. We will discuss and demonstrate that HDC-MiniROCKET can systematically overcome catastrophic failures of MiniROCKET on simple synthetic datasets. These results are confirmed by experiments on the 128 datasets from the UCR time series classification benchmark. The extension with HDC can achieve considerably better results on datasets with high temporal dependence at about the same computational effort for inference.

Image and video descriptors are an omnipresent tool in computer vision and its application fields like mobile robotics. Many hand-crafted and in particular learned image descriptors are numerical vectors with a potentially (very) large number of dimensions. Practical considerations like memory consumption or time for comparisons call for the creation of compact representations. In this paper, we use hyperdimensional computing (HDC) as an approach to systematically combine information from a set of vectors in a single vector of the same dimensionality. HDC is a known technique to perform symbolic processing with distributed representations in numerical vectors with thousands of dimensions. We present a HDC implementation that is suitable for processing the output of existing and future (deep learning based) image descriptors. We discuss how this can be used as a framework to process descriptors together with additional knowledge by simple and fast vector operations. A concrete outcome is a novel HDC-based approach to aggregate a set of local image descriptors together with their image positions in a single holistic descriptor. The comparison to available holistic descriptors and aggregation methods on a series of standard mobile robotics place recognition experiments shows a 20% improvement in average performance and > 2× better worst-case performance compared to runner-up.

Visual place recognition is the task of recognizing same places of query images in a set of database images, despite potential condition changes due to time of day, weather or seasons. It is important for loop closure detection in SLAM and candidate selection for global localization. Many approaches in the literature perform computationally inefficient full image comparisons between queries and all database images. There is still a lack of suited methods for efficient place recognition that allow a fast, sparse comparison of only the most promising image pairs without any loss in performance. While this is partially given by ANN-based methods, they trade speed for precision and additional memory consumption, and many cannot find arbitrary numbers of matching database images in case of loops in the database. In this paper, we propose a novel fast sequence-based method for efficient place recognition that can be applied online. It uses relocalization to recover from sequence losses, and exploits usually available but often unused intra-database similarities for a potential detection of all matching database images for each query in case of loops or stops in the database. We performed extensive experimental evaluations over five datasets and 21 sequence combinations, and show that our method outperforms two state-of-the-art approaches and even full image comparisons in many cases, while providing a good tradeoff between performance and percentage of evaluated image pairs. Source code for Matlab will be provided with publication of this paper.

Visual place recognition is the task of finding matchings of images that show the same place in the world. Combinations of appearance changes (e.g. changing illumination or weather) and geometric changes (e.g. viewpoint changes or occlusions) challenge existing approaches. Learning-based local image feature pipelines are a promising approach to this type of problem. We present a novel attentive feature pooling method that can be used to train a CNN to jointly detect and describe local image features. It can be trained on small or moderately sized datasets with weak supervision in a classification training setup (e.g. we use a set of 24k images of publicly available web-camera images in our experiments). We propose to use a joint loss function that combines the cross-entropy loss for the classification task with a mean squared error in order to increase the repeatability of feature detections. We show how the approach can be integrated in a place recognition pipeline and run experiments on several standard place recognition datasets. Despite the small training dataset, we demonstrate a 15% improvement in the average performance compared to the best of a number of compared state-of-the-art approaches, and, probably more importantly, a 3x improvement in the worst-case performance. Open source code is available.

In this paper, we present a novel approach for driving style classification based on time series data. Instead of automatically learning the embedding vector for temporal representation of the input data with Recurrent Neural Networks, we propose a combination of Hyperdimensional Computing (HDC) for data representation in high-dimensional vectors and much simpler feed-forward neural networks. This approach provides three key advantages: first, instead of having a “black box” of Recurrent Neural Networks learning the temporal representation of the data, our approach allows to encode this temporal structure in high-dimensional vectors in a human-comprehensible way using the algebraic operations of HDC while only relying on feed-forward neural networks for the classification task. Second, we show that this combination is able to achieve at least similar and even slightly superior classification accuracy compared to state-of-the-art Long Short-Term Memory (LSTM)-based networks while significantly reducing training time and the necessary amount of data for successful learning. Third, our HDC-based data representation as well as the feed-forward neural network, allow implementation in the substrate of Spiking Neural Networks (SNNs). SNNs show promise to be orders of magnitude more energy-efficient than their rate-based counterparts while maintaining comparable prediction accuracy when being deployed on dedicated neuromorphic computing hardware, which could be an energy-efficient addition in future intelligent vehicles with tight restrictions regarding on-board computing and energy resources. We present a thorough analysis of our approach on a publicly available data set including a comparison with state-of-the-art reference models.

Visual place recognition in changing environments is the problem of finding matchings between two sets of observations, a query set and a reference set, despite severe appearance changes. Recently, image comparison using CNNbased descriptors showed very promising results. However, the experiments in the literature typically assume a single distinctive condition within each set (e.g., reference images are captured at daytime and the query sequence is at night). In this paper, we will demonstrate that as soon as the conditions change within one set (e.g., reference is daytime and now the query is a traversal daytime-dusk-night-dawn), different places under the same condition can suddenly look more similar than same places under different conditions. As a consequence, state-of-the-art approaches like CNN-based descriptors fail. This paper discusses this practically very important problem of in-sequence condition changes and defines a hierarchy of problem setups from (1) no in-sequence changes, (2) discrete in-sequence changes, to (3) continuous in-sequence changes. We will experimentally evaluate the effect of in-sequence condition changes on two state-of-the-art CNN-descriptors and investigate unsupervised methods to improve their performance. This includes an evaluation of the importance of statistical normalization (standardization) of descriptors, which is often omitted in existing approaches but can considerably improve results for problems up to discrete in-sequence changes. To address the practical most relevant setup of continuous changes, we investigate the application of unsupervised learning methods using two PCA-based approaches from the literature and propose a novel clustering-based extension of the statistical normalization. We experimentally demonstrate that these approaches can significantly improve place recognition performance in case of continuous in-sequence condition changes. Matlab implementations of the presented approaches are available online: www.tu-chemnitz.de/etit/proaut/cont_changing_envs.

Circular Convolutional Neural Networks (CCNN) are an easy to use alternative to CNNs for input data with wrap-around structure like 360° images and multi-layer laserscans. Although circular convolutions have been used in neural networks before, a detailed description and analysis is still missing. This paper closes this gap by defining circular convolutional and circular transposed convolutional layers as the replacement of their linear counterparts, and by identifying pros and cons of applying CCNNs. We experimentally evaluate their properties using a circular MNIST classification and a Velodyne laserscanner segmentation dataset. For the latter, we replace the convolutional layers in two state-of-the-art networks with the proposed circular convolutional layers. Compared to the standard CNNs, the resulting CCNNs show improved recognition rates in image border areas. This is essential to prevent blind spots in the environmental perception. Further, we present and evaluate how weight transfer can be used to obtain a CCNN from an available, readily trained CNN. Compared to alternative approaches (e.g. input padding), our experiments show benefits of CCNNs and transferred CCNNs regarding simplicity of usage (once the layer implementations are available), performance and runtime for training and inference. Implementations for Keras with Tensorflow are provided online2.

Camera-based navigation within a given three-dimensional map enables heterogeneous robotic systems to share maps and use more abstract environment models like floor plans. This paper builds upon our previous work, and addresses the problem of how to combine 3D distance information from the map and the current visual image from the robot's camera in order to navigate within this map. The underlying assumption is that features which cause depth changes are also likely to create visual gradients. Based on this assumption, the similarity of visual image and depth images that are synthesized from the 3D map can be used to evaluate pose hypothesis. This paper integrates this idea into a Monte Carlo localization approach and additionally presents its application to path following. The presented approach is evaluated on a synthetic datasets that provides perfect knowledge of the ground truth, as well as two real-world datasets acquired by a heterogeneous robotic team: a proof-of-concept dataset in a scattered indoor environment, and a challenging corridor dataset.

A major insight from our previous work on extensive comparison of super pixel segmentation algorithms is the existence of several trade-offs for such algorithms. The most intuitive is the trade-off between segmentation quality and runtime. However, there exist many more between these two and a multitude of other performance measures. In this work, we present two new super pixel segmentation algorithms, based on existing algorithms, that provide better balanced trade-offs. Better balanced means, that we increase one performance measure by a large amount at the cost of slightly decreasing another. The proposed new algorithms are expected to be more appropriate for many real time computer vision tasks. The first proposed algorithm, Preemptive SLIC, is a faster version of SLIC, running at frame-rate (30 Hz for image size 481x321) on a standard desktop CPU. The speed-up comes at the cost of slightly worse segmentation quality. The second proposed algorithm is Compact Watershed. It is based on Seeded Watershed segmentation, but creates uniformly shaped super pixels similar to SLIC in about 10 ms per image. We extensively evaluate the influence of the proposed algorithmic changes on the trade-offs between various performance measures.

There exist almost as many superpixel segmentation algorithms as applications they can be used for. So far, the choice of the right superpixel algorithm for the task at hand is based on their ability to resemble human-made ground truth segmentations (besides runtime and availability). We investigate the equally important question of how stable the segmentations are under image changes as they appear in video data. Further we propose a new quality measure that evaluates how well the segmentation algorithms cover relevant image boundaries. Instead of relying on human-made annotations, that may be biased by semantic knowledge, we present a completely data-driven measure that inherently emphasizes the importance of image boundaries. Our evaluation is based on two recently published datasets coming with ground truth optical flow fields. We discuss how these ground optical truth fields can be used to evaluate segmentation algorithms and compare several existing superpixel algorithms.

Recently a SLAM algorithm based on biological principles (RatSLAM) has been proposed. It was proven to perform well in large and demanding scenarios. In this paper we establish a comparison of the principles underlying this algorithm with standard probabilistic SLAM approaches and identify the key difference to be an additive update step. Using this insight, we derive the novel, non-Bayesian Causal Update filter that is suitable for application in appearance-based SLAM. We successfully apply this new filter to two demanding vision-only urban SLAM problems of 5 and 66 km length. We show that it can functionally replace the core of RatSLAM, gaining a massive speed-up.

We present a fast line segment tracker which does not require any knowledge about the motion of the camera nor the structure of the observed scene. It runs on 320 times 240 pixel images at 30 Hz. We adapted the RAPiD tracker with a new way of handling multiple line hypotheses to deal with the simple model of a single line segment. We discuss the difficulty of using a chi2-test as merging criterion and also present a new approach to overcome it. Furthermore, instead of making assumptions about the camera motion, a constant velocity motion model to predict the line segment position in the following frame is used. We explain how to deal with the instability of the endpoint extraction in this motion model to avoid unintentional motion along the line. Finally, we present results on real world indoor and urban outdoor image sequences.