Predicting agents impacted by legal policies, physical limitations, and operational preferences is inherently difficult. In recent years, neuro-symbolic methods have emerged, integrating machine learning and symbolic reasoning models into end-to-end learnable systems. Hereby, a promising avenue for expressing high-level constraints over multi-modal input data in robotics has opened up. This work introduces an approach for Bayesian estimation of agents expected to comply with a human-interpretable neuro-symbolic model we call its Constitution. Hence, we present the Constitutional Filter (CoFi), leading to improved tracking of agents by leveraging expert knowledge, incorporating deep learning architectures, and accounting for environmental uncertainties. CoFi extends the general, recursive Bayesian estimation setting, ensuring compatibility with a vast landscape of established techniques such as Particle Filters. To underpin the advantages of CoFi, we evaluate its performance on real-world marine traffic data. Beyond improved performance, we show how CoFi can learn to trust and adapt to the level of compliance of an agent, recovering baseline performance even if the assumed Constitution clashes with reality.

In Automated Driving (AD), traffic scenarios where two agents must resolve an ordering without knowing each other's intention are critical for expanding the operational design domain of automated vehicles to urban environments. These scenarios require negotiation to determine who passes first through an interaction zone. We present a novel agreement measure and negotiation approach to resolve these ordering problems across a wide range of common scenarios. Our method emphasizes detecting and deciding when to switch between potential negotiation outcomes. Our approach extends existing behavior planners to cope with bimodal cooperative interactions, where two potentially desirable outcomes need to be considered. We evaluate our approach by providing both an illustrative scenario and extensive statistical experiments across various geometries, including oncoming narrow passages, crossing and merging scenarios. The results demonstrate that our system considerably improves the behavior in cooperative ordering scenarios compared to the baseline. Furthermore, it is also robust in the sense that it effectively handles dynamic situations where the other agent's intentions changes during the negotiation process.

Advanced Air Mobility (AAM) is a growing field that demands accurate and trustworthy models of legal concepts and restrictions for navigating Unmanned Aircraft Systems (UAS). In addition, any implementation of AAM needs to face the challenges posed by inherently dynamic and uncertain human-inhabited spaces robustly. Nevertheless, the employment of UAS beyond visual line of sight (BVLOS) is an endearing task that promises to significantly enhance today’s logistics and emergency response capabilities. Hence, we propose Probabilistic Mission Design (ProMis), a novel neuro-symbolic approach to navigating UAS within legal frameworks. ProMis is an interpretable and adaptable system architecture that links uncertain geospatial data and noisy perception with declarative, Hybrid Probabilistic Logic Programs (HPLP) to reason over the agent’s state space and its legality. To inform planning with legal restrictions and uncertainty in mind, ProMis yields Probabilistic Mission Landscapes (PML). These scalar fields quantify the belief that the HPLP is satisfied across the agent’s state space. Extending prior work on ProMis’ reasoning capabilities and computational characteristics, we show its integration with potent machine learning models such as Large Language Models (LLM) and Transformer-based vision models. Hence, our experiments underpin the application of ProMis with multi-modal input data and how our method applies to many AAM scenarios.

Kenneweg S, Deigmöller J, Cimiano P, Eggert J. Benchmarking the Ability of Large Language Models to Reason About Event Sets. In: Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management. SCITEPRESS - Science and Technology Publications; 2024: 74-82.

In this paper, we propose and analyze a method for trajectory prediction in longitudinal car-following scenarios. Hereby, the prediction is realized by a longitudinal car-following model (Intelligent Driver Model, IDM) with online estimated parameters. Previous work has shown that IDM online parameter adaptation is possible but difficult and slow, while providing only small improvement of prediction quality over e.g. constant velocity or constant acceleration baseline models.In our approach (Online IDM, OIDM), we use the difference between a parameter-specific trajectory and the real past trajectory as objective function of the optimization. Instead of optimizing the model parameters “directly”, we gain them based on a weighted sum of a set of prototype parameters, optimizing these weights.To show the benefits of the method, we compare the properties of our approach against state-of-the-art prediction methods for longitudinal driving, such as Constant Velocity (CV), Constant Acceleration (CA) and particle filter approaches on an open freeway driving dataset. The evaluation shows significant improvements in several aspects: (I) The prediction accuracy is significantly increased, (II) the obtained parameters exhibit a fast convergence and increased temporal stability and (III) the computational effort is reduced so that an online parameter adaptation becomes feasible.

In this paper, we propose and analyze a Generalized Intelligent Driver Model (GIDM) as an extension of the Intelligent Driver Model (IDM) for its applicability to model merging scenarios. For this purpose, we extend the original longitudinal car-following IDM with several terms: (1) for anticipatory acceleration capabilities, we include the most nearby backward agent, (2) we consider cars that will merge from other paths by continuous virtual projection onto the ego vehicle path, and (3) we introduce a longitudinal shift that increases the anticipatory capabilities of the model. We analyze the model in extensive simulations of merge cases measuring safety, comfort and utility. Compared to a baseline IDM, the result is a systematic improvement of the GIDM in all considered measures.

Many everyday traffic situations require cooperation among traffic participants to establish the order in which they pass a shared part of the road. Behavior planners which do not take this cooperative aspect into account properly struggle to find efficient solutions if the situation is nontrivial. Improper modelling may lead to overly aggressive or conservative behavior. In this paper, we propose an extension to state-of-the-art systems that enables behavior planners to efficiently cope with narrow passage scenarios even without car-to-car communication. The extended system is based on an asymmetry measure which takes the shared traffic space and the cooperation partners into account. This measure is then used to continuously predict which potential outcome is more likely to occur, to infer the assumed strategy of the cooperation partner, and to match the own strategy accordingly. Experiments show that the proposed system significantly reduces the cumulative passing time of the shared traffic space as compared to baseline systems. The resulting solutions are robust against variations in the behavior of both cooperation partners, and explicitly account for oblivious traffic participants which behave uncooperatively.

In this paper, we introduce Memory Nets, a knowledge representation targeted at Autonomous Intelligent Agents (IAs) operating in real world. The main focus is on a knowledge base (KB) that on the one hand is able to leverage the large body of openly available semantic information, and on the other hand allows to incrementally accumulate additional knowledge from situated interaction. Such a KB can only rely on operable semantics fully contained in the knowledge base itself, avoiding any type of hidden semantics in the KB attributes, such as human-interpretable identifier. In addition, it has to provide means for tightly coupling the internal representation to real-world events. We propose a KB structure and inference processes based on a knowledge graph that has a small number of link types with operational semantics only, and where the main information lies in the complex patterns and connectivity structures that can be build incrementally using these links. We describe the basic domain independent features of Memory Nets and the relation to measurements and actuator capabilities as available by autonomous entities, with the target of providing a KB framework for researching how to create IAs that continuously expand their knowledge about the world.

We consider the problem of correct motion planning for T-intersection merge-ins of arbitrary geometry and vehicle density. A merge-in support system has to estimate the chances that a gap between two consecutive vehicles can be taken successfully. In contrast to previous models based on heuristic gap size rules, we present an approach which optimizes the integral risk of the situation using parametrized velocity ramps. It accounts for the risks from curves and all involved vehicles (front and rear on all paths) with a so-called survival analysis. For comparison, we also introduce a specially designed extension of the Intelligent Driver Model (IDM) for entering intersections. We show in a quantitative statistical evaluation that the survival method provides advantages in terms of lower absolute risk (i.e., no crash happens) and better risk-utility tradeoff (i.e., making better use of appearing gaps). Furthermore, our approach generalizes to more complex situations with additional risk sources.

The Foresighted Driver Model (FDM) is a microscopic driver model which is based on the idea that a driver balances risk with utility. This paper deals with the modeling of advanced driving maneuvers for the FDM with a special focus on lateral positioning scenarios, such as lane changes in highway traffic. When driving at high speeds, tactical preparation for a safe lane change is of high importance. In this context, the paper presents maneuvers that allow for lane changes to be planned well in advance and carefully made without the restraint of requiring immediate action. Furthermore, the paper presents a continuous lateral control which allows driving on arbitrary paths other than the centerline, depending on the current traffic situation. Since more complex lateral maneuvers require more detailed considerations of the environment, an approach is presented to model the lane and the environmental influences. This paves the way for a modeling of variables such as lane markings, roadblocks, hard shoulders and more. Simulations illustrate how the introduced maneuvers allow successful preparation for upcoming lane changes and how traffic obstructions can be bypassed without performing a lane change but by using the continuous lateral control.

Situation recognition is a prerequisite for many advanced driver assistance systems as well as for partially and fully automated vehicles. Current situation recognition approaches focus mainly on estimating maneuvers of single scene entities. However, assessing multiple, possibly interacting, traffic participants simultaneously is crucial in complex traffic scenes and has hardly been investigated. Considering the variability and combinatorics of such scenarios, having specialized situation recognition systems covering each case directly is unrealistic. In this paper, we present a flexible framework for assessing generic traffic scenes with multiple interacting traffic participants. It is able to construct a fully interaction-respecting probabilistic situation assessment, while relying on reusable state-of-the-art single-entity-based maneuver predictions. The benefits and applicability are presented on a real-world data set. The evaluation indicates that the approach is not only able to reconstruct underlying interdependent probability distributions; it outperforms specially designed models, due to the reduced complexities of the single-entity-based recognition models.

This paper addresses the problem of future behavior evaluation and planning for ADAS in general traffic situations. Complex traffic situations require the estimation of future behavior alternatives in terms of predictive risks. Based on the predicted future dynamics of traffic scene entities, we present an approach where a continuous, probabilistic model for future risks is used to build so-called predictive risk maps. These maps indicate how risky a certain ego-car trajectory will be at different predicted times so that they can be used to directly plan the best possible future behavior. Since this optimization problem is highly non-convex we combine the risk maps with sampling-based planning algorithms of the RRT*-type to obtain future trajectories which minimize risk and maximize utility. We apply our approach to multiple risk types and various different scenarios, including inner city and highway situations.

The Intelligent Driver Model (IDM) is a microscopic, time continuous car following model for the simulation of freeway and urban traffic. Its popularity is grounded in its simplicity and its capacity to describe both single vehicle velocity profiles as well as collective traffic behavior. Nevertheless, it lacks a series of properties that would be desirable for more realistic agent models. In this paper, as an alternative and improvement to the IDM, we propose the Foresighted Driver Model (FDM), which assumes that a driver acts in a way that balances predictive risk (e.g. due to possible collisions along its route) with utility (e.g. the time required to travel, smoothness of ride, etc.). Based on a risk concept developed for full behavior planning, we introduce driver model equations from the assumption that a driver will mainly try to avoid risk maxima in time and space. We show how such a model can be used to simulate driving behavior similar to full behavior planning models and which generalizes and reaches beyond the IDM modeling scenarios.

Block-Matching stereo is commonly used in applications with low computing resources in order to get some rough depth estimates. However, research on this simple stereo estimation technique has been very scarce since the advent of energy-based methods which promise a higher quality and a larger potential for further improvement. In the domain of intelligent vehicles, especially semi-global-matching (SGM) is widely spread due to its good performance and simple implementation. Unfortunately, the big downside of SGM is its large memory footprint because it is working on the full disparity space image. In contrast to this, local block-matching stereo is much more lean. In this paper, we will introduce a novel multi-block-matching scheme which tremendously improves the result of standard block-matching stereo while preserving the low memory-footprint and the low computational complexity. We tested our new multi-block-matching scheme on the KITTI stereo benchmark as well as on the new Middlebury stereo benchmark. For the KITTI benchmark we achieve results that even surpass the results of the best SGM implementations. For the new Middlebury benchmark we get results that are only slightly worse than state-of-the-art SGM implementations.

Due to the large number and the high variability of possible traffic situations, intersections are among the most accident-prone spots in inner-city traffic. To reliably assist the driving tasks elaborated risk assessment systems are needed. Current approaches are mainly based on the prediction of possible future trajectories of the involved traffic participants. However, considering the variability and combinatorics of intersection-related traffic situations, this becomes unfeasible for limited computational resources. Here, we present a general framework for an efficient situation hypotheses selection system. The selection process is based on reasoning about whether a particular situation results in a threat for the ego vehicle's behavior. Our approach combines the results of a probabilistic situation recognition and a fast risk assessment using state-of-the-art regression methods. We show that the proposed system is able to effectively reduce the number of unnecessarily considered situation hypotheses on average by over 80%.

Motion is a crucial source of information for a variety of tasks in social interactions. The process of how humans recognize complex articulated movements such as gestures or face expressions remains largely unclear. There is an ongoing discussion if and how explicit low-level motion information, such as optical flow, is involved in the recognition process. Motivated by this discussion, we introduce a computational model that classifies the spatial configuration of gradient and optical flow patterns. The patterns are learned with an unsupervised learning algorithm based on translation-invariant nonnegative sparse coding called VNMF that extracts prototypical optical flow patterns shaped, for example, as moving heads or limb parts. A key element of the proposed system is a lateral inhibition term that suppresses activations of competing patterns in the learning process, leading to a low number of dominant and topological sparse activations. We analyze the classification performance of the gradient and optical flow patterns on three real-world human action recognition and one face expression recognition data set. The results indicate that the recognition of human actions can be achieved by gradient patterns alone, but adding optical flow patterns increases the classification performance. The combined patterns outperform other biological-inspired models and are competitive with current computer vision approaches.

In this paper, we present a new scheme for block-matching stereo. The main intention is to relax the inherent assumption that within a block the disparities are constant, because this assumption is often violated. Instead of using the matching cost of one disparity within a matching block, the best matching for several disparities are first selected for each pixel and then these best matches are combined to the final block-matching value. Results on the KITTI benchmark show that this scheme increases the performance of block-matching stereo especially for large matching windows, however, there is also a significant increase for smaller block sizes. Furthermore, we show that a straightforward combination with the appearance-aligned block-matching stereo leads to results that surpass the performance of both single techniques.

Intersections are the most accident-prone spots in the road network. In order to assist the driver in complex urban intersection situations, an ADAS will be required not only to recognize current but also to anticipate future maneuvers of the involved road users. Current approaches for intention estimation focus mainly on discerning only two intentions based on a vehicle's behavior. We argue that for distinguishing between more than two intentions not just a vehicle's kinematic behavior but also its driving situation needs to be taken into account. In our system we estimate four different intentions by modeling and recognizing driving situations in a Bayesian Network and using the behavior as additional evidence. For the behavior based estimation we present a newly engineered feature, the Anticipated Velocity at Stop line, that turned out to be a very strong indicator for the intention. Our system is evaluated on a real-world data set comprising approaches to seven different intersections on which we can show that our approach is able to estimate a driver's intention with a high accuracy.

Visual motion is a rich source of information that is directly coupled to the underlying shape of a moving object. One way to describe motion is to use optical flow fields. Due to the aperture problem, dense optical flow estimation is an ill-constraint problem, while sparse optical flow estimation looses the shape information of moving objects. Current estimation algorithms based on regularization or segmentation fail at surface deformations or when the relevant motion is less dominant then its sourrounding movements. Both is e.g. true for face movements, where small movement patterns, so called action units, need to be preserved for further image analysis. We present a novel approach to capture the characteristics of local motion patterns that is based on the brightness constancy equation of optical flow estimation in combination with feature extraction using translation invariant non-negative sparse coding. Our approach simultaneously learns basic motion patterns and estimates the flow field without requiring pretrained motion patterns from ground truth optical flow data. We show on a face expression dataset how this method can preserve weak movements even in the presence of large head movements.

Accurate stereoscopic depth estimation, in particular of the road surface area, is one of several key technologies to improve Advanced Driver Assistance Systems (ADAS). One major problem is that the quality of the stereoscopic depth measurements of the road is often poor - which is mainly attributed to a lack of texture on the road surface. Especially for patch-matching stereo algorithms, the estimated depths look irregular and bumpy. In this paper, we show that the violation of the fronto-parallel assumption is the major reason for a bad depth estimation and not a low-contrast texture on the road surface. Since patch-matching or block-matching stereo inherently assumes a constant disparity within one patch, this is violated if the cameras are oriented almost parallel to the ground, which is typically the case in ADAS, and which leads to a strong distortion of the appearance between the two cameras. In order to tackle this problem, we propose a compensation of this distortion by applying a linear warp on one of the stereo images according to the expected disparity for the planar ground. This recovers the fronto-parallel assumption and results in a very good depth estimations of road surfaces. Our experiments on the KITTI stereo benchmark demonstrate the quantitative competitiveness of the approach, while retaining the speed and simplicity of block-matching stereo approaches. Furthermore, our experiments show that the approach is very robust, achieving results for the road surface that are significantly better than standard patch-matching stereo processing without warping for a wide range of warp parameter settings.

Intersections are among the most complex traffic situations that motorists encounter, which is reflected by the fact that in Europe more than 40 percent of accidents resulting in injury occur at intersections. In order to support the driver in crossing an intersection an advanced driver assistance system is required to predict the behavior of other drivers, like acceleration and braking maneuvers, as accurately as possible. Such a prediction is a challenging task when considering the complexity and variability of situations encountered at urban intersections. We propose to tackle this problem using a two-staged approach. In the first stage the situation is decomposed into small, more manageable sets of related road users to prevent a combinatorial explosion of possibilities. For each set the road user's driving situation is estimated. In the second stage the velocity profiles of all road users are predicted, taking advantage of the previously estimated driving situation by employing prediction models that are specific to the situation type. The proposed method is evaluated on a simulated intersection situation where the two-staged approach clearly outperforms prediction methods that work without assessing driving situations first. We also show qualitative results on real-world data that confirm the benefits of our approach.

In this paper, we present a generic way to use a hierarchical representation of prediction models for adaptive tracking purposes. Each node of the hierarchy consists of an interacting multiple models (IMM) particle filter that combines local predictions with top-down predictions arriving from nodes situated higher up in the hierarchy. Such a hierarchical prediction structure provides mechanisms to automatically control the influences between the nodes of the hierarchy. We demonstrate the gain of a hierarchical 2D-3D tracking system by first using it to track 3D elliptically rotating object in an artificial scene, where in approaching and departing phases the target is inherently hard to track in pure 2D space due to large accelerations. To the contrary, the proposed hierarchical 2D- 3D tracking system successfully tracks the target, because it benefits from the ability of dynamically adapting its prediction models. In order to test the robustness of this framework and its feasibility for real-world applications, we then show in a traffic scene that we can successfully track a motorcylist from a driving car by means of this hierarchical tracking framework.

The autonomous learning of concept hierarchies is still a matter of research. Here we present a learning schema for Bayesian networks which results in a nested structure of sub- and superclass relationships. It is based on so-called parent divorcing but exploits the similarity of all nodes involved as expressed by their connectivity pattern. If the procedure is applied to simple object-property pairings a nested taxonomic hierarchy emerges. We further show how the learning procedure can be aligned with basic results from developmental psychology. For this we made a set of simulations which clearly indicate that a fixed developmental order of sensory maturation is crucial for the emerging conceptual system. The learning procedure itself is biologically plausible since it works incrementally, makes use of only local information and leads to a reduced computational effort by building a more efficient representation.

In this paper we present an approach for contour based object representation. To this end we use a curvature signal gained by a level-set segmentation method. The advantage of that curvature signal is that it generates no computational overhead as it is a byproduct of standard level-set segmentation methods. Different methods for the description of the segmented objects, so called object descriptors are presented. The object descriptors are all invariant against translation, rotation and scale of the object. Furthermore we show a sparse and memory efficient representation of the descriptors for a series of objects. Finally an approach for classification of unknown objects based on ldquomemorizedrdquo objects is proposed.

The pixel movements in an image sequence grabbed by a camera that is mounted on a mobile platform comprise the superposition of several motion components. These motion components are caused by the egomotion of the camera and by the different movements of the objects seen by the camera. Utilizing sensory information from a calibrated stereo rig and egomotion measurements of the mobile platform we develop a probabilistic framework that estimates optical flow relative to the visual flow induced by the egomotion. Despite rapid egomotion changes and a large range of pixel movements the proposed Dynamic Bayesian Network allows to infer the optical flow induced by moving objects. This is used to segregate moving individuals from static background while the stereo rig is moving. We present optical flow and figure-background segmentation results by applying this general framework to image sequences captured by the humanoid robot ASIMO while he is walking and observing moving people. 1

The basic idea of Lucas and Kanade is to constrain the local motion measurement by assuming a constant velocity within a spatial neighborhood. We reformulate this spatial constraint in a probabilistic way assuming Gaussian distributed uncertainty in spatial identification of velocity measurements and extend this idea to scale and time dimensions. Thus, we are able to combine uncertain velocity measurements observed at different image scales and positions over time. We arrive at a new recurrent optical flow filter formulated in a Dynamic Bayesian Network applying suitable factorisation assumptions and approximate inference techniques. The introduction of spatial uncertainty allows for a dynamic and spatially adaptive tuning of the constraining neighborhood. Here, we realize this tuning dependenton the local Structure Tensor of the intensity patterns of the image sequence. We demonstrate that a probabilistic combination of spatiotemporal integration and modulation of a purely local integration area improves the Lucas and Kanade estimation.

For the interaction of a mobile robot with a dynamic environment, the estimation of object motion is desired while the robot is walking and/or turning its head. In this paper, we describe a system which manages this task by combining depth from a stereo camera and computation of the camera movement from robot kinematics in order to stabilize the camera images. Moving objects are detected by applying optical flow to the stabilized images followed by a filtering method, which incorporates both prior knowledge about the accuracy of the measurement and the uncertainties of the measurement process itself. The efficiency of this system is demonstrated in a dynamic real-world scenario with a walking humanoid robot.

We develop an optical flow estimation framework that focuses on motion estimation over time formulated in a dynamic Bayesian network. It realizes a spatiotemporal integration of motion information using a dynamic and robust prior that incorporates spatial and temporal coherence constraints on the flow field. The main contribution is the embedding of these particular assumptions on optical flow evolution into the Bayesian propagation approach that leads to a computationally feasible two-filter inference method and is applicable for on and offline parameter optimization. We analyse the possibility to optimize imposed Student's t-distributed model uncertainties, which are the camera noise and the transition noise. Experiments with synthetic sequences illustrate how the probabilistic framework improves the optical flow estimation because it allows for noisy data, motion ambiguities and motion discontinuities.

In this paper we present an approach for probabilistic motion pattern segmentation. We combine level-set methods for image segmentation with motion estimations based on probability distribution functions (pdf's) calculated at each image position. To this end, we extend a region based level-set framework to exploit the motion pdf's. We then compare segmentation results of the pdf-based with those of optical-flow-based motion segmentation approaches. We found that the straightforward way of characterizing the segmented region by spatially averaging the motion measurement pdf's does not yield satisfactory results. However, describing the spatial characteristics of the motion pdf's with nonparametric density estimates enables to solve complex motion segmentation problems. In particular for situations with demanding motion patterns like partly overlapping objects and transparent motion, we show that the probabilistic approach yields better results. This confirms the idea that for motion processing it is beneficial to consistently retain the uncertainty and ambiguity of the measurement process right up to the final integration stage, instead of directly processing optical flow vectors.

In the field of intelligent systems, research and design approaches vary from predefined architectures to self-organizing systems. Regardless of the architectural approach, such systems may grow in size and complexity to levels where the capacities of people are strongly challenged. Such systems are commonly researched, designed and developed following several methods and with the help of a variety of software tools. In this paper we want to describe our research and development environment. It is composed of a set of tools that support our research and enable us to develop large scale intelligent systems used in our robots and in our test platforms. The main parts of our research and development environment are: the component models BBCM (brain bytes component model) and BBDM (brain bytes data model), the middleware RTBOS (real-time brain operating system), the monitoring system CMBOS (control-monitor brain operating system) and the design environment DTBOS (design tool for brain operating system). We will compare our research and development environment with others available on the market or still in research phase and we will describe some of our experiments

Velocity distributions are an enhanced representation of image velocity containing more velocity information than velocity vectors. In particular, non-Gaussian velocity distributions allow for the representation of ambiguous motion information caused by the aperture problem or multiple motions at motion boundaries. To resolve motion ambiguities, discrete non-Gaussian velocity distributions are suggested, which are integrated over space, time, and scales using a joint Bayesian prediction and refinement approach. This leads to a hierarchical velocity-distribution representation from which robust velocity estimates for both slow and high speeds as well as statistical confidence measures rating the velocity estimates can be computed.

This paper proposes a biologically inspired and technically implemented sound localization system to robustly estimate the position of a sound source in the frontal azimuthal half-plane. For localization, binaural cues are extracted using cochleagrams generated by a cochlear model that serve as input to the system. The basic idea of the model is to separately measure interaural time differences and interaural level differences for a number of frequencies and process these measurements as a whole. This leads to two-dimensional frequency versus time-delay representations of binaural cues, so-called activity maps. A probabilistic evaluation is presented to estimate the position of a sound source over time based on these activity maps. Learned reference maps for different azimuthal positions are integrated into the computation to gain time-dependent discrete conditional probabilities. At every timestep these probabilities are combined over frequencies and binaural cues to estimate the sound source position. In addition, they are propagated over time to improve position estimation. This leads to a system that is able to localize audible signals, for example human speech signals, even in reverberating environments.

Non-negative matrix factorization (NMF) is a very efficient parameter-free method for decomposing multivariate data into strictly positive activations and basis vectors. However, the method is not suited for overcomplete representations, where usually sparse coding paradigms apply. We show how to merge the concepts of non-negative factorization with sparsity conditions. The result is a multiplicative algorithm that is comparable in efficiency to standard NMF, but that can be used to gain sensible solutions in the overcomplete cases. This is of interest e.g. for the case of learning and modeling of arrays of receptive fields arranged in a visual processing map, where an overcomplete representation is unavoidable.

Non-negative matrix factorization (NMF) is a method for the decomposition of multivariate data into strictly positive activations and basis vectors. Here, instead of using unstructured data vectors, we assume that something is known in advance about the type of transformations that either the input data or the basis vectors may undergo. This would be the case e.g. if we assume input vectors that are translationally shifted versions of each other, but it applies to any other transformations as well. The key idea is that we factorize the data into activations and basis vectors modulo the transformations. We show that this can be done by extending NMF in a natural way. The gained factorization thus provides a transformation-invariant and compact encoding that is optimal for the given transformation constraints.

In our previous work, integral equation formulations for Mesoscopical, mathematical descriptions of dynamics of popula(cid:173) tions of spiking neurons are getting increasingly important for the understanding of large-scale processes in the brain using simula(cid:173) tions. population dynamics have been derived for a special type of spik(cid:173) ing neurons. For Integrate- and- Fire type neurons, these formula(cid:173) tions were only approximately correct. Here, we derive a math(cid:173) ematically compact, exact population dynamics formulation for Integrate- and- Fire type neurons. It can be shown quantitatively in simulations that the numerical correspondence with microscop(cid:173) ically modeled neuronal populations is excellent. 1 Introduction and motivation The goal of the population dynamics approach is to model the time course of the col(cid:173) lective activity of entire populations of functionally and dynamically similar neurons in a compact way, using a higher descriptionallevel than that of single neurons and spikes. The usual observable at the level of neuronal populations is the population(cid:173) averaged instantaneous firing rate A(t), with A(t)6.t being the number of neurons in the population that release a spike in an interval [t, t+6.t). Population dynamics are formulated in such a way, that they match quantitatively the time course of a given A(t), either gained experimentally or by microscopical, detailed simulation. At least three main reasons can be formulated which underline the importance of the population dynamics approach for computational neuroscience. First, it enables the simulation of extensive networks involving a massive number of neurons and connections, which is typically the case when dealing with biologically realistic functional models that go beyond the single neuron level. Second, it increases the analytical understanding of large-scale neuronal dynamics, opening the way towards better control and predictive capabilities when dealing with large networks. Third, it enables a systematic embedding of the numerous neuronal models operating at different descriptional scales into a generalized theoretic framework, explaining the relationships, dependencies and derivations of the respective models. Early efforts on population dynamics approaches date back as early as 1972, to the work of Wilson and Cowan [8] and Knight [4], which laid the basis for all current population-averaged graded-response models (see e.g. [6] for modeling work using these models). More recently, population-based approaches for spiking neurons were developed, mainly by Gerstner [3, 2] and Knight [5]. In our own previous work [1], we have developed a theoretical framework which enables to systematize and sim(cid:173) ulate a wide range of models for population-based dynamics. It was shown that the equations of the framework produce results that agree quantitatively well with detailed simulations using spiking neurons, so that they can be used for realistic simulations involving networks with large numbers of spiking neurons. Neverthe(cid:173) less, for neuronal populations composed of Integrate-and-Fire (I&F) neurons, this framework was only correct in an approximation. In this paper, we derive the exact population dynamics formulation for I&F neurons. This is achieved by reducing the I&F population dynamics to a point process and by taking advantage of the particular properties of I&F neurons. 2 Background: Integrate-and-Fire dynamics 2.1 Differential form We start with the standard Integrate- and- Fire (I&F) model in form of the well(cid:173) known differential equation [7] (1) which describes the dynamics of the membrane potential Vi of a neuron i that is modeled as a single compartment with RC circuit characteristics. The membrane relaxation time is in this case T = RC with R being the membrane resistance and C the membrane capacitance. The resting potential v R est is the stationary potential that is approached in the no-input case. The input arriving from other neurons is described in form of a current ji. In addition to eq. (1), which describes the integrate part of the I&F model, the neuronal dynamics are completed by a nonlinear step. Every time the membrane potential Vi reaches a fixed threshold () from below, Vi is lowered by a fixed amount Ll > 0, and from the new value of the membrane potential integration according to eq. (1) starts again. if Vi(t) = () (from below) . (2) At the same time, it is said that the release of a spike occurred (i.e., the neuron fired), and the time ti = t of this singular event is stored. Here ti indicates the time of the most recent spike. Storing all the last firing times, we gain the sequence of spikes {t{} (spike ordering index j, neuronal index i). 2.2

Models that describe qualitatively and quantitatively the activity of entire groups of spiking neurons are becoming increasingly important for biologically realistic large-scale network simulations. At the systems and areas modeling level, it is necessary to switch the basic descriptional level from single spiking neurons to neuronal assemblies. In this article, we present and review work that allows a macroscopic description of the assembly activity. We show that such macroscopic models can be used to reproduce in a quantitatively exact manner the joint activity of groups of spike-response or integrate-and-fire neurons. We also show that integral as well as differential equation models of neuronal assemblies can be understood within a single framework, which allows a comparison with the commonly used assembly-averaged graded-response type of models. The presented framework thus enables the large-scale neural network modeler to implement networks using computational units beyond the single spiking neuron without losing much biological accuracy. This article explains the theoretical background as well as the capabilities and the implementation details of the assembly approach.