Efficient and accurate object pose estimation is an essential component for modern vision systems in many applications such as Augmented Reality, autonomous driving, and robotics. While research in model-based 6D object pose estimation has delivered promising results, model-free methods are hindered by the high computational load in rendering and inferring consistent poses of arbitrary objects in a live RGB-D video stream. To address this issue, we present 6DOPE-GS, a novel method for online 6D object pose estimation \& tracking with a single RGB-D camera by effectively leveraging advances in Gaussian Splatting. Thanks to the fast differentiable rendering capabilities of Gaussian Splatting, 6DOPE-GS can simultaneously optimize for 6D object poses and 3D object reconstruction. To achieve the necessary efficiency and accuracy for live tracking, our method uses incremental 2D Gaussian Splatting with an intelligent dynamic keyframe selection procedure to achieve high spatial object coverage and prevent erroneous pose updates. We also propose an opacity statistic-based pruning mechanism for adaptive Gaussian density control, to ensure training stability and efficiency. We evaluate our method on the HO3D and YCBInEOAT datasets and show that 6DOPE-GS matches the performance of state-of-the-art baselines for model-free simultaneous 6D pose tracking and reconstruction while providing a 5$\times$ speedup. We also demonstrate the method's suitability for live, dynamic object tracking and reconstruction in a real-world setting.

Visual localization is the task of accurately estimating the camera’s position in a known environment. State-of-the-art methods use the 3D structure of a scene for precise visual localization. However, 3D scene reconstruction is resource-intensive in terms of hardware requirements and computation time, making it infeasible to run on low-cost embedded hardware. Unsupervised spatial representation learning with Slow Feature Analysis (SFA) enables computationally inexpensive localization and mapping. This paper analyzes SFA-based and the well-known structure-based localization, i.e., Active Search, in two distinct settings: short-term temporal and extreme spatial generalization. We present the experimental results from an outdoor environment and compare both methods w.r.t localization accuracy and computation time. Results show that the SFA-based approach is 886x faster in mapping time and 34x faster in localization than Active Search while achieving comparable localization accuracy in our test scenario.

In outdoor scenarios changing conditions (e.g., seasonal, weather and lighting effects) have a substantial impact on the appearance of a scene, which often prevents successful visual localization. The application of an unsupervised Slow Feature Analysis (SFA) on the images captured by an autonomous robot enables self-localization from a single image. However, changes occurring during the training phase or over a more extended period can affect the learned representations. To address the problem, we propose to join long-term recordings from an outdoor environment based on their position correspondences. The established hierarchical model trained on raw images performs well, but as an extension, we extract Fourier components of the views and use that for learning of spatial representations, which reduces the computation time and makes it adequate to run on an ARM embedded system. We present the experimental results from a simulated environment and real-world outdoor recordings collected over a full year, which has effects like different day time, weather, seasons and dynamic objects. Results show an increasing invariance w.r.t. changing conditions over time, thus an outdoor robot can improve its localization performance during operation.

A model of hierarchical Slow Feature Analysis (SFA) enables a mobile robot to learn a spatial representation of its environment directly from images captured during a random walk. After the unsupervised learning phase a subset of the resulting representations are orientation invariant and code for the position of the robot. Hence, they change monotonically over space even though the variation of the sensory signals received from the environment might change drastically e.g. during rotation on the spot. Furthermore, the property of spatial smoothness allows us to infer a navigation direction by taking the difference between the measurement at the current location and a measurement at a target location. In our work we investigated the use of slow feature representations, learned for a specific environment, for the purpose of navigation. We present a straightforward method for navigation using gradient descent on the difference between two points specified in slow feature space. Due to its slowness objective, the resulting slow feature representations implicitly encode information about static obstacles, allowing a mobile robot to efficiently circumnavigate them by simply following the steepest gradient in slow feature space.

In this paper we propose an approach for unsupervised segmentation of continuous object manipulation sequences into semantically differing subsequences. The proposed method estimates segment borders based on an integrated consideration of three modalities (tactile feedback, hand posture, audio) yielding robust and accurate results in a single pass. To this end, a Bayesian approach originally applied by Fearnhead to segment one-dimensional time series data -- is extended to allow an integrated segmentation of multi-modal sequences. We propose a joint product model which combines modality-specific likelihoods to model segments. Weight parameters control the influence of each modality within the joint model. We discuss the relevance of all modalities based on an evaluation of the temporal and structural correctness of segmentation results obtained from various weight combinations.

Primates are very good at recognizing objects independent of viewing angle or retinal position, and they outperform existing computer vision systems by far. But invariant object recognition is only one prerequisite for successful interaction with the environment. An animal also needs to assess an object's position and relative rotational angle. We propose here a model that is able to extract object identity, position, and rotation angles. We demonstrate the model behavior on complex three-dimensional objects under translation and rotation in depth on a homogeneous background. A similar model has previously been shown to extract hippocampal spatial codes from quasi-natural videos. The framework for mathematical analysis of this earlier application carries over to the scenario of invariant object recognition. Thus, the simulation results can be explained analytically even for the complex high-dimensional data we employed.

We present a model for the self-organized formation of place cells, head-direction cells, and spatial-view cells in the hippocampal formation based on unsupervised learning on quasi-natural visual stimuli. The model comprises a hierarchy of Slow Feature Analysis (SFA) nodes, which were recently shown to reproduce many properties of complex cells in the early visual system []. The system extracts a distributed grid-like representation of position and orientation, which is transcoded into a localized place-field, head-direction, or view representation, by sparse coding. The type of cells that develops depends solely on the relevant input statistics, i.e., the movement pattern of the simulated animal. The numerical simulations are complemented by a mathematical analysis that allows us to accurately predict the output of the top SFA layer.