Partial differential equations (PDEs) find applications across various scientific and engineering fields.There is a growing trend for integrating physics-aware machine learning models to solve PDEs.Among them, the Finite Volume Neural Network (FINN) has proven to be efficient in uncovering latent structures in data.This study explores the capabilities of FINN in the investigation of shallow-water equations, which simulate wave dynamics in coastal regions.Specifically, we investigate the efficacy of FINN in reconstructing underwater topography.We find that FINN excels at inferring topography solely from wave dynamics, stressing the importance of application-specific inductive bias in neural network architectures.

Neuromorphic Computing (NC) and Spiking Neural Networks (SNNs) in particular are often viewed as the next generation of Neural Networks (NNs). NC is a novel bio-inspired paradigm for energy efficient neural computation, often relying on SNNs in which neurons communicate via spikes in a sparse, event-based manner. This communication via spikes can be exploited by neuromorphic hardware implementations very effectively and results in a drastic reductions of power consumption and latency in contrast to regular GPU-based NNs. In recent years, neuromorphic hardware has become more accessible, and the support of learning frameworks has improved. However, available hardware is partially still experimental, and it is not transparent what these solutions are effectively capable of, how they integrate into real-world robotics applications, and how they realistically benefit energy efficiency and latency. In this work, we provide the robotics research community with an overview of what is possible with SNNs on neuromorphic hardware focusing on real-time processing. We introduce a benchmark of three popular neuromorphic hardware devices for the task of event-based object detection. Moreover, we show that an SNN on a neuromorphic hardware is able to run in a challenging table tennis robot setup in real-time.

The locally competitive algorithm (LCA) can solve sparse coding problems across a wide range of use cases. Recently, convolution-based LCA approaches have been shown to be highly effective for enhancing robustness for image recognition tasks in vision pipelines. To additionally maximize representational sparsity, LCA with hard-thresholding can be applied. While this combination often yields very good solutions satisfying an ℓ 0 sparsity criterion, it comes with significant drawbacks for practical application: (i) LCA is very inefficient, typically requiring hundreds of optimization cycles for convergence; (ii) the use of a hard-thresholding results in a non-convex loss function, which might lead to suboptimal minima. To address these issues, we propose the Locally Competitive Algorithm with State Warm-up via Predictive Priming (WARP-LCA), which leverages a predictor network to provide a suitable initial guess of the LCA state based on the current input. Our approach significantly improves both convergence speed and the quality of solutions, while maintaining and even enhancing the overall strengths of LCA. We demonstrate that WARP-LCA converges faster by orders of magnitude and reaches better minima compared to conventional LCA. Moreover, the learned representations are more sparse and exhibit superior properties in terms of reconstruction and denoising quality as well as robustness when applied in deep recognition pipelines. Furthermore, we apply WARP-LCA to image denoising tasks, showcasing its robustness and practical effectiveness. Our findings confirm that the naive use of LCA with hard-thresholding results in suboptimal minima, whereas initializing LCA with a predictive guess results in much better outcomes. • WARP-LCA accelerates convergence and achieves superior sparsity compared to LCA. • Achieves higher PSNR and SSIM with fewer iterations than traditional LCA. • Improves denoising in classification pipelines under varying noise levels. • Enables generalizable and efficient sparse coding with predictive initialization.

The resonate-and-fire (RF) neuron, introduced over two decades ago, is a simple, efficient, yet biologically plausible spiking neuron model, which can extract frequency patterns within the time domain due to its resonating membrane dynamics. However, previous RF formulations suffer from intrinsic shortcomings that limit effective learning and prevent exploiting the principled advantage of RF neurons. Here, we introduce the balanced RF (BRF) neuron, which alleviates some of the intrinsic limitations of vanilla RF neurons and demonstrates its effectiveness within recurrent spiking neural networks (RSNNs) on various sequence learning tasks. We show that networks of BRF neurons achieve overall higher task performance, produce only a fraction of the spikes, and require significantly fewer parameters as compared to modern RSNNs. Moreover, BRF-RSNN consistently provide much faster and more stable training convergence, even when bridging many hundreds of time steps during backpropagation through time (BPTT). These results underscore that our BRF-RSNN is a strong candidate for future large-scale RSNN architectures, further lines of research in SNN methodology, and more efficient hardware implementations.

We introduce a compositional physics-aware FInite volume Neural Network (FINN) for learning spatiotemporal advection-diffusion processes. FINN implements a new way of combining the learning abilities of artificial neural networks with physical and structural knowledge from numerical simulation by modeling the constituents of partial differential equations (PDEs) in a compositional manner. Results on both one- and two-dimensional PDEs (Burgers’, diffusion-sorption, diffusion-reaction, Allen{–}Cahn) demonstrate FINN’s superior modeling accuracy and excellent out-of-distribution generalization ability beyond initial and boundary conditions. With only one tenth of the number of parameters on average, FINN outperforms pure machine learning and other state-of-the-art physics-aware models in all cases{—}often even by multiple orders of magnitude. Moreover, FINN outperforms a calibrated physical model when approximating sparse real-world data in a diffusion-sorption scenario, confirming its generalization abilities and showing explanatory potential by revealing the unknown retardation factor of the observed process.

The ability to flexibly bind features into coherent wholes from different perspectives is a hallmark of cognition and intelligence. Importantly, the binding problem is not only relevant for vision but also for general intelligence, sensorimotor integration, event processing, and language. Various artificial neural network models have tackled this problem with dynamic neural fields and related approaches. Here we focus on a generative encoder-decoder architecture that adapts its perspective and binds features by means of retrospective inference. We first train a model to learn sufficiently accurate generative models of dynamic biological motion or other harmonic motion patterns, such as a pendulum. We then scramble the input to a certain extent, possibly vary the perspective onto it, and propagate the prediction error back onto a binding matrix, that is, hidden neural states that determine feature binding. Moreover, we propagate the error further back onto perspective taking neurons, which rotate and translate the input features onto a known frame of reference. Evaluations show that the resulting gradient-based inference process solves the perspective taking and binding problem for known biological motion patterns, essentially yielding a Gestalt perception mechanism. In addition, redundant feature properties and population encodings are shown to be highly useful. While we evaluate the algorithm on biological motion patterns, the principled approach should be applicable to binding and Gestalt perception problems in other domains.

In the paper, we show how scalable, low-cost trunk-like robotic arms can be constructed using only basic 3D-printing equipment and simple electronics. The design is based on uniform, stackable joint modules with three degrees of freedom each. Moreover, we present an approach for controlling these robots with recurrent spiking neural networks. At first, a spiking forward model learns motor-pose correlations from movement observations. After training, intentions can be projected back through unrolled spike trains of the forward model essentially routing the intention-driven motor gradients towards the respective joints, which unfolds goal-direction navigation. We demonstrate that spiking neural networks can thus effectively control trunk-like robotic arms with up to 75 articulated degrees of freedom with near millimeter accuracy.

Fast and robust person detection is one of the most important tasks for robotic applications involving human interaction. Particularly in mobile robotics this task is still challenging. Though there are already reliable and real-time capable approaches, they are usually computationally expensive. They either require GPUs or multiple CPU cores in order to work properly. Furthermore, some of the approaches are designed for special environments and sensor types, which reduces general applicability. In this work, we present a robust, generic and lightweight solution for real-time 3D person detection. Since our approach requires only a single CPU thread, it can be run as a background process and is suitable for smaller robotic systems. We demonstrate applicability to indoor and outdoor scenarios using different 3D sensor types separately. Moreover, we are able to show that the proposed method outperforms other state-of-the-art approaches, including a DCNN.

This paper investigates Recurrent Neural Networks (RNNs), particularly Dynamic Cortex Memories (DCMs), an extension of Long Short Term Memories (LSTMs) for classification of 14 different ground types based on vibration data. Also a simple regularization technique called Sequence Boundary Dropout (SBD) is introduced, which effectively enlarges the training set and improves generalization. The neural networks perform in the time domain without any explicit feature computation, while previous state-of-the-art methods extract features mainly in the frequency domain. The presented approach does not require a time window, is causal, and works just-in-time, such that a classification can be done online at each new time step. Furthermore, we show that the neural networks outperform previous methods significantly in terms of classification accuracy. Finally, we demonstrate that the networks retrained with Random Activation Preservation (RAP) can classify very early - within a fraction of a second - but robustly at the same time in a continuous recognition scenario with varying classes.

The recently introduced Dynamic Cortex Memory (DCM) is an extension of the Long Short Term Memory (LSTM) providing a systematic inter-gate connection infrastructure. In this paper the behavior of DCM networks is studied in more detail and their potential in the field of gradient-based sequence learning is investigated. Hereby, DCM networks are analyzed regarding particular key features of neural signal processing systems, namely, their robustness to noise and their ability of time warping. Throughout all experiments we show that DCMs converge faster and yield better results than LSTMs. Hereby, DCM networks require overall less weights than pure LSTM networks to achieve the same or even better results. Besides, a promising neurally implemented just-in-time online signal filter approach is presented, which is latency-free and still provides an accurate filtering performance much better than conventional low-pass filters. We also show that the neural networks can do explicit time warping even better than the Dynamic Time Warping (DTW) algorithm, which is a specialized method developed for this task.

In this paper, we present a new variant of the well-known Random Sample Consensus (RANSAC) algorithm for robust estimation of model parameters. The idea of our method is based on a kind of volatile memory which is similar to the pheromone evaporation in the ant colony optimization algorithm. Therefore, we call our improved RANSAC like algorithm ANTSAC. We describe our new approach and the influence of its relevant parameters to the achieved performance in detail. ANTSAC is computationally efficient and convincingly easy to implement. It turns out that ANTSAC significantly outperforms RANSAC regarding the number of inliers after a given number of iterations. Further, we show that the advantage of ANTSAC increases with the complexity of the problem, i.e., with the number of model parameters, as well as with the relative number of outliers. ANTSAC is entirely generic, such that no further domain knowledge is required, as it is for many other RANSAC extensions. Nevertheless, we show that it is competitive to state-of-the-art methods even in domain specific scenarios.

In this paper we propose a novel approach for online mode detection, where the task is to classify ink traces into several categories. In contrast to previous approaches working on global features, we introduce a system completely relying on local features. For classification, standard recurrent neural networks (RNNs) and the recently introduced long short-term memory (LSTM) networks are used. Experiments are performed on the publicly available IAMonDo-database which serves as a benchmark data set for several researches. In the experiments we investigate several RNN structures and classification sub-tasks of different complexities. The final recognition rate on the complete test set is 98.47% in average, which is significantly higher than the 97% achieved with an MCS in previous work. Further interesting results on different subsets are also reported in this paper.