Non-stationary environments subject to concept drift require the design of adaptive models that can continuously learn and update.Two primary research communities have emerged to address this challenge: Continual Learning (CL) and Streaming Machine Learning (SML).CL manages virtual drifts by learning new concepts without forgetting past knowledge, while SML focuses on real drifts, rapidly adapting to evolving data distributions.However, a unified approach is needed to balance adaptation and knowledge retention.Streaming Continual Learning (SCL) bridges the gap between CL and SML, ensuring models retain useful past information while efficiently adapting to new data.We explore key challenges in SCL, including handling temporal dependencies in data streams and adapting latent representations for personalization and knowledge editing.Additionally, we identify promising SCL benchmarks which can foster and promote a unified research effort between CL and SML. 35

We address the challenge of reward-incremental learning (RIL) within the context of continual reinforcement learning.RIL presents a novel continual learning (CL) scenario where the same data samples (observations for RL) are mapped to different classes (Q-values) at different times.This is in contrast to class-incremental CL where new sample classes may be added, but without the contradictions inherent in RIL.To tackle this issue, we propose the use of an innovative replay-based approach called adiabatic replay (AR) which is inherently suited for RL since it removes the need for large replay buffers.Based on a simple benchmark scenario for continual RL, we empirically demonstrate that RIL scenarios can be handled by our approach, in contrast to conventional DQN methods. 87

This study uncovers surprisingly effective synergies between the field of continual learning (CL) and machine unlearning (MUL).We extend the common class-incremental setting from CL to incorporate suppression requests in what we term class-incremental unlearning (CIUL).We present a light-weight approach to CIUL using replay/rehearsal-based CL approaches together with a selective replay strategy termed "Replay-To-Suppress" (RTS), where we actually make use of the catastrophic forgetting effect to achieve unlearning.In particular, we adapt a CL strategy termed adiabatic replay (AR) to achieve suppression at near-constant time complexity.We demonstrate excellent overall performance for all CL strategies extended by RTS on MNIST, F-MNIST and a latent encoded version of the challenging CIFAR and SVHN benchmarks.

Many applications of deep learning are set in an environment with perpetual change or at least with an ever-growing amount of data.In practice, deep neural network (DNNs) and large language models (LLMs) are continually trained and evaluated.They need to incorporate new data or new annotations, where one typical issue is the extensive availability of unannotated or low-quality data, coupled with a bottleneck concerning annotations and/or curated samples.In such setups, the scaling behavior of continual learning (CL) algorithms w.r.t.training time becomes critical, which is in contrast to the standard CL setting operating on small databases like MNIST, CIFAR or ImageNet.Annotations or curated samples become available progressively, e.g., because they are created by humans, or due to an ongoing exploration of the environment, and need to be progressively incorporated into models.This article explores how advancement in continual learning can improve the scalability and performance of DNNs and LLMs in such setups.One interesting aspect is to leverage dedicated (small-scale) CL techniques to achieve advantageous trade-offs between computational cost and accuracy, or how such CL methods can maintain advantageous scaling behavior w.r.t.continuous re-training on all data.99

To avoid catastrophic forgetting, many replay-based approaches to continual learning (CL) require, for each learning phase with new data, the replay of samples representing all of the previously learned knowledge. Since this knowledge grows over time, such approaches invest linearly growing computational resources just for re-learning what is already known. In this proof-of-concept study, we propose a generative replay-based CL strategy that we term adiabatic replay (AR), which achieves CL in constant time and memory complexity by making use of the (very common) situation where each new learning phase is adiabatic, i.e., represents only a small addition to existing knowledge. The employed Gaussian Mixture Models (GMMs) are capable of selective updating only those parts of their internal representation affected by the new task. The information that would otherwise be overwritten by such updates is protected by selective replay of samples that are similar to newly arriving ones. Thus, the amount of to-be-replayed samples depends not at all on accumulated, but only on added knowledge, which is small by construction. Based on the challenging CIFAR and SVHN datasets in combination with pre-trained feature extractors, we confirm AR’s superior scaling behavior while showing better accuracy than common baselines in the field.

This article proposes a flexible control strategy to improve the reliability of data-driven model-predictive control (MPC). Prediction models are usually trained on logged data, which may be incomplete, leading to model failure within the optimization algorithm of the MPC. To counter this, we propose a Gaussian Mixture Model (GMM) for detecting outliers during live operation. These outliers are found in the observed system states as well as the calculated control signals regarding the logged training data of the ai-model. If an outlier is detected, we assume that the data-driven prediction model will be inaccurate, and an appropriate fallback strategy is used instead, which includes but is not limited to an alternative controller or a complete system-shutdown. In addition, outliers are stored for subsequent improvement of the prediction model. This outlier-protected control (OPC) architecture is evaluated in terms of control quality using the integrated squared error (ISE) on three nonlinear dynamical systems. The outcomes indicate a significant enhancement in control quality in comparison to an unprotected but otherwise equivalent architecture. These findings suggest that a very simple and efficient outlier detection can greatly benefit data-driven MPC in general whenever feasible fallback strategies exist.

We introduce a novel technique to address continual reinforcement learning (CRL), i.e., reinforcement learning (RL) in non-stationary environments. This requires agents to rapidly update their policies to new statistics while avoiding the catastrophic forgetting (CF) of previous policies. In RL, CF is commonly circumvented by experience replay (ER) from a large buffer. As we show, this leads to slow updating of policies, since new statistics must be sufficiently represented in the buffer. In addition, non-stationarities can introduce contradictions that an agent needs to adapt to. Our approach, unlike traditional methods such as deep Q-networks (DQNs) with ER, enables fast reaction times under minimal memory requirements, making it suitable for real-world applications. It generalizes adiabatic replay (AR), a recently introduced generative replay (GR) method for continual learning (CL). For evaluation, we introduce two robotic simulation-based CRL benchmarks which are partitioned into tasks by environment shifts, showcasing our approach's ability to retain previously acquired policies while being able to learn novel skills.

This study is in the context of continual learning (CL) with DNNs.It compares several types of generators when performing replay, i.e., the generation of previously seen samples, to avoid catastrophic forgetting.Principal generators are generative adversarial networks (GANs) and variational autoencoders (VAEs).We evaluate these generators in various flavors (conditional, Wasserstein etc.) w.r.t.CL performance on a variety of CL tasks generated from the MNIST benchmark.Concerning generators, we find that VAEs are generally more compatible with CL than GANs.More generally, we find that replay-based CL faces counterintuitive issues for seemingly simple problems: first, that performance degrades more strongly as less information is added, and, furthermore, that performance degrades even when only known information is added.

Continual Learning (CL, sometimes also termed incremental learning) is a flavor of machine learning where the usual assumption of stationary data distribution is relaxed or omitted.When naively applying, e.g., DNNs in CL problems, changes in the data distribution can cause the so-called catastrophic forgetting (CF) effect: an abrupt loss of previous knowledge.Although many significant contributions to enabling CL have been made in recent years, most works address supervised (classification) problems.This article reviews literature that study CL in other settings, such as learning with reduced supervision, fully unsupervised learning, and reinforcement learning.Besides proposing a simple schema for classifying CL approaches w.r.t.their level of autonomy and supervision, we discuss the specific challenges associated with each setting and the potential contributions to the field of CL in general. Framework and Goals of Continual LearningContinual learning (CL) is a machine learning sub-field that studies learning under time-varying data distributions.This relaxes one of the fundamental assumptions of statistical learning theory ([1]), which states that the data follows 67

We present an empirical study on the use of continual learning (CL) methods in a reinforcement learning (RL) scenario, which, to the best of our knowledge, has not been described before. CL is a very active recent research topic concerned with machine learning under non-stationary data distributions. Although this naturally applies to RL, the use of dedicated CL methods is still uncommon. This may be due to the fact that CL methods often assume a decomposition of CL problems into disjoint sub-tasks of stationary distribution, that the onset of these sub-tasks is known, and that sub-tasks are non-contradictory. In this study, we perform an empirical comparison of selected CL methods in a RL problem where a physically simulated robot must follow a racetrack by vision. In order to make CL methods applicable, we restrict the RL setting and introduce non-conflicting subtasks of known onset, which are however not disjoint and whose distribution, from the learner's point of view, is still non-stationary. Our results show that dedicated CL methods can significantly improve learning when compared to the baseline technique of “experience replay”.

We present the Deep Convolutional Gaussian Mixture Model (DCGMM), a new probabilistic approach for image modeling capable of density estimation, sampling and tractable inference. DCGMM instances exhibit a CNN-like layered structure, in which the principal building blocks are convolutional Gaussian Mixture (cGMM) layers. A key innovation w.r.t. related models like sum-product networks (SPNs) and probabilistic circuits (PCs) is that each cGMM layer optimizes an independent loss function and therefore has an independent probabilistic interpretation. This modular approach permits intervening transformation layers to harness the full spectrum of (potentially non-invertible) mappings available to CNNs, e.g., max-pooling or half-convolutions. DCGMM sampling and inference are realized by a deep chain of hierarchical priors, where a sample generated by a given cGMM layer defines the parameters of sampling in the next-lower cGMM layer. For sampling through non-invertible transformation layers, we introduce a new gradient-based sharpening technique that exploits redundancy (overlap) in, e.g., half-convolutions. DCGMMs can be trained end-to-end by SGD from random initial conditions, much like CNNs. We show that DCGMMs compare favorably to several recent PC and SPN models in terms of inference, classification and sampling, the latter particularly for challenging datasets such as SVHN. We provide a public TF2 implementation.

Gaussian Mixture Models (GMMs) are a standard tool in data analysis. However, they face problems when applied to high-dimensional data (e.g., images) due to the size of the required full covariance matrices (CMs), whereas the use of diagonal or spherical CMs often imposes restrictions that are too severe. The Mixture of Factor analyzers (MFA) model is an important extension of GMMs, which allows to smoothly interpolate between diagonal and full CMs based on the number of factor loadings l. MFA has successfully been applied for modeling high-dimensional image data [1]. This article contributes both a theoretical analysis as well as a new method for efficient high-dimensional MFA training by stochastic gradient descent, starting from random centroid initializations. This greatly simplifies the training and initialization process, and avoids problems of batch-type algorithms such Expectation-Maximization (EM) when training with huge amounts of data. In addition, by exploiting the properties of the matrix determinant lemma, we prove that MFA training and inference/sampling can be performed based on precision matrices, which does not require matrix inversions after training is completed. At training time, the methods requires the inversion of l × l matrices only. Besides the theoretical analysis and proofs, we apply MFA to typical image datasets such as SVHN and MNIST, and demonstrate the ability to perform sample generation and outlier detection.

Continual learning (CL) is a major challenge of machine learning (ML) and describes the ability to learn several tasks sequentially without catastrophic forgetting (CF). Recent works indicate that CL is a complex topic, even more so when real-world scenarios with multiple constraints are involved. Several solution classes have been proposed [1], of which socalled replay-based approaches seem very promising due to their simplicity and robustness. Such approaches store a subset of past samples in a dedicated memory for later processing: while this does not solve all problems, good results have been obtained. In this article, we empirically investigate replay-based approaches of continual learning and assess their potential for applications. Selected recent approaches as well as own proposals are compared on a common set of benchmarks, with a particular focus on assessing the performance of different sample selection strategies. We find that the impact of sample selection increases when a smaller number of samples is stored. Nevertheless, performance varies strongly between different replay approaches. Surprisingly, we find that the most naive rehearsal-based approaches that we propose here can outperform recent state-of-the-art methods.

In this conceptual work, we present Deep Convolutional Gaussian Mixture Models (DCGMMs): a new formulation of deep hierarchical Gaussian Mixture Models (GMMs) that is particularly suitable for describing and generating images. Vanilla (i.e., flat) GMMs require a very large number of components to describe images well, leading to long training times and memory issues. DCGMMs avoid this by a stacked architecture of multiple GMM layers, linked by convolution and pooling operations. This allows to exploit the compositionality of images in a similar way as deep CNNs do. DCGMMs can be trained end-to-end by Stochastic Gradient Descent. This sets them apart from vanilla GMMs which are trained by Expectation-Maximization, requiring a prior k-means initialization which is infeasible in a layered structure. For generating sharp images with DCGMMs, we introduce a new gradient-based technique for sampling through non-invertible operations like convolution and pooling. Based on the MNIST and FashionMNIST datasets, we validate the DCGMMs model by demonstrating its superiority over flat GMMs for clustering, sampling and outlier detection.

We present Gaussian Mixture Replay (GMR), a rehearsal-based approach for continual learning (CL) based on Gaussian Mixture Models (GMM). CL approaches are intended to tackle the problem of catastrophic forgetting (CF), which occurs for Deep Neural Networks (DNNs) when sequentially training them on successive sub-tasks. GMR mitigates CF by generating samples from previous tasks and merging them with current training data. GMMs serve several purposes here: sample generation, density estimation (e.g., for detecting outliers or recognizing task boundaries) and providing a high-level feature representation for classification. GMR has several conceptual advantages over existing replay-based CL approaches. First of all, GMR achieves sample generation, classification and density estimation in a single network structure with strongly reduced memory requirements. Secondly, it can be trained at constant time complexity w.r.t. the number of sub-tasks, making it particularly suitable for life-long learning. Furthermore, GMR minimizes a differentiable loss function and seems to avoid mode collapse. In addition, task boundaries can be detected by applying GMM density estimation. Lastly, GMR does not require access to sub-tasks lying in the future for hyper-parameter tuning, allowing CL under real-world constraints. We evaluate GMR on multiple image datasets, which are divided into class-disjoint sub-tasks.

Safe optimization aims to find the optimum of an unknown function by experimentally sampling its values at selective points, without any experiment leading to a violation of prescribed safety constraints. Bayesian optimization approaches are often used here because of the need for confidence prediction. This procedure is computationally expensive: depending on the discretization resolution, even three dimensions are problematic for the average desktop computer and thus exclude safe optimization in applications with several tuning parameters. For handling high-dimensional Bayesian optimization, iterative decomposition into sub-problems is a common strategy. However, the usual definition of sub-problem focuses either on exploration (random sub-problem) or exploitation (sub-problem at the previous maximum). We present a novel approach which iteratively defines at least two sub-problems, with each sub-problem targeting either exploration or exploitation. In addition, sub-problems can be defined in terms of ellipsoid-shaped sub-spaces, whose shapes are adapted through the Gaussian process regression. Our empirical evaluation with up to 40-dimensional test functions suggests that the ellipsoidal sub-space is well suited for exploration far from local optima, while a linear sub-space has its advantages in local exploitation. Their combination outperformed all evaluated safe optimization methods and was applied to a complex power grid system.

Abstract We present an approach for efficiently training Gaussian Mixture Model (GMM) by Stochastic Gradient Descent (SGD) with non-stationary, high-dimensional streaming data. Our training scheme does not require data-driven parameter initialization (e.g., k-means) and can thus be trained based on a random initial state. Furthermore, the approach allows mini-batch sizes as low as 1, which are typical for streaming-data settings. Major problems in such settings are undesirable local optima during early training phases and numerical instabilities due to high data dimensionalities. We introduce an adaptive annealing procedure to address the first problem, whereas numerical instabilities are eliminated by an exponential-free approximation to the standard GMM log-likelihood. Experiments on a variety of visual and non-visual benchmarks show that our SGD approach can be trained completely without, for instance, k-means based centroid initialization. It also compares favorably to an online variant of Expectation-Maximization (EM)—stochastic EM (sEM), which it outperforms by a large margin for very high-dimensional data.

Optimizing an unknown objective function under uncertainty requires a balance between exploration (learn more about the objective) and exploitation (find the global optimum). Safe optimization aims to guarantee that safety requirements of the next observation to be performed are fulfilled before performing it. Common approaches are based on Gaussian process regression, also known as Kriging, as a surrogate model iteratively estimating the safety and selecting next observations by Bayesian optimization methods. The hyper-parameter setup usually requires a lot of domain-specific knowledge (if no data is available) or prior data to optimize the hyper-parameters. But it is precisely the lack of these two factors that is the main reason when safe optimization becomes interesting: If the system is unknown and random experiments to generate data are not allowed due to restrictions. We present a novel method for safe Bayesian optimization with self-adapting hyper-parameters, which requires only one safe initial observation and easily selectable initial hyper-parameters. By safely self-adapting the parameters, it is possible to find the global optimum with a reliability regarding safety requirements. Thus, the method can be used even with limited domain-specific expertise and covers a wide range of applications with a minimum of customization.

We present a processing pipeline for flow-based traffic classification using a machine learning component leveraging Deep Neural Networks (DNNs). The system is trained to predict likely characteristics of real-world traffic flows from a campus network ahead of time, e.g., a flow's throughput or duration. Training and evaluation of DNN models are continuously performed on a flow data stream collected from a university data center. Instead of the common binary classification into “mice” and “elephant” (throughput) or “short-term” and “long-term” (duration) flows, predicted flow characteristics are quantized into three classes. Various communication contexts (subset of network traffic, e.g., only TCP) and flow feature groups (subset of flow features, e.g., only a flow's 5-tuple), which are supported through an enrichment strategy, are considered and investigated. An in-depth description of the data acquisition process, including preprocessing steps and anonymization used to protect sensitive information, is given. Additionally, we employ an accelerated variant of t-distributed Stochastic Neighbor Embedding (t-SNE) to visualize network traffic data. This enables the understanding of traffic characteristics and relations between communication flows at a glance. Furthermore, possible use-cases and a high-level architecture for flow-based routing scenarios utilizing the developed pipeline are proposed.

We present a processing pipeline for flow-based throughput classification based on a machine learning component using deep neural networks (DNNs) that is trained to predict the likely bit rate of a real-world network traffic flow ahead of time. The DNN is trained and evaluated on a flow data stream as well as on a reference dataset collected from a university data center. Predicted bit rates are quantized into three classes instead of the common binary classification into “mice” and “elephant” flows. An in-depth description of the data acquisition process, including preprocessing steps and anonymization used to protect sensitive information, is given. We employ t-SNE (a state-of-the-art data visualization algorithm) to visualize network traffic data, thus enabling us to analyze and understand the characteristics of network traffic data and relations between communication flows at a glance. Additionally, an architecture for flow-based routing utilizing the developed pipeline is proposed as a possible use-case.

We present a new RGB-D database for multi-pose object recognition tasks. With the help of a multi-axis rotation framework, we are capable of capturing depth and color data of arbitrary small objects from virtually any viewpoint. In addition, recording is performed in a nearly lossless fashion, avoiding typical bleeding artifacts present in related reference data bases. This contribution presents the main advantages of our setup and contrasts it against other reference data bases. Furthermore, it outlines possible use cases and application scenarios of our data set and is complemented by experiments with standard machine learning techniques used in, e.g., object recognition tasks within the robotics domain. The experiments demonstrate the validity of our data base as they corroborate that viewpoint variance is indeed an important factor to take into account for object detection, which, from our perspective, is sometimes not considered at the required level. Detection accuracy is high if samples can be trained on data taking into account as many viewpoints as possible.

We present a publicly available benchmark database for the problem of hand posture recognition from noisy depth data and fused RGB-D data obtained from low-cost time-of-flight (ToF) sensors. The database is the most extensive database of this kind containing over a million data samples (point clouds) recorded from 35 different individuals for ten different static hand postures. This captures a great amount of variance, due to person-related factors, but also scaling, translation and rotation are explicitly represented. Benchmark results achieved with a standard classification algorithm are computed by cross-validation both over samples and persons, the latter implying training on all persons but one and testing on the remaining one. An important result using this database is that cross-validation performance over samples (which is the standard procedure in machine learning) is systematically higher than cross-validation performance over persons, which is to our mind the true application-relevant measure of generalization performance.

We present a novel hierarchical approach to multi-class classification which is generic in that it can be applied to different classification models (e.g., support vector machines, perceptrons), and makes no explicit assumptions about the probabilistic structure of the problem as it is usually done in multi-class classification. By adding a cascade of additional classifiers, each of which receives the previous classifier's output in addition to regular input data, the approach harnesses unused information that manifests itself in the form of, e.g., correlations between predicted classes. Using multilayer perceptrons as a classification model, we demonstrate the validity of this approach by testing it on a complex ten-class 3D gesture recognition task.

PROPRE is a generic and cortically inspired framework that provides online input/output relationship learning. The input data flow is projected on a self-organizing map that provides an internal representation of the current stimulus. From this representation, the system predicts the value of the output target. A predictability measure, based on the monitoring of the prediction quality, modulates the projection learning so that to favor learning of representations that are helpful to predict the output. In this article, we study PROPRE when the input/output relationship is only defined in a small subspace of the input space, that we define as a partially learnable environment. This problem, which is not typical of the machine learning field, is however crucial for the robotic developmental field. Indeed, robots face high dimensional sensory-motor environments where large areas of these sensory-motor spaces are not learnable since a motor action cannot have a consequence on every perception each time. We show that the use of the predictability measure in PROPRE leads to an autonomous gathering of local representations where the input data are related to the output value, thus providing good classification performance as the system will learn the input/output function only where it is defined.

We present a novel approach for improved hand-gesture recognition by a single time-of-flight(ToF) sensor in an automotive environment. As the sensor's lateral resolution is comparatively low, we employ a learning approach comprising multiple processing steps, including PCA-based cropping, the computation of robust point cloud descriptors and training of a Multilayer perceptron (MLP) on a large database of samples. A sophisticated temporal fusion technique boosts the overall robustness of recognition by taking into account data coming from previous classification steps. Overall results are very satisfactory when evaluated on a large benchmark set of ten different hand poses, especially when it comes to generalization on previously unknown persons.

We present a learning approach that allows to detect correspondences between visual and LIDAR measurements. In contrast to approaches that rely on calibration, we propose a learning approach that will create an implicit calibration model from training data. Our model can provide three functions: first of all, it can convert a measurement in one sensor into the coordinate system of the other, or into a distribution of probable measurements in case the transformation is not unique. Secondly, using a correspondence observation that we define, the model is able to decide if two visual/LIDAR measurements are likely to come from the same object. This is of profound importance for applications such as object detection or tracking where contributions from several sensors need to be combined. We demonstrate the feasibility of our approach by training and evaluating our system on tracklets in the KITTI database as well as on a small set of real-world scenes containing pedestrians, in which our method finds correspondences between the results of real visual and LIDAR-based detection algorithms.

In this article, we study a three-layer neural hierarchy composed of bi-directionally connected recurrent layers which is trained to perform a synthetic object recognition task. The main feature of this network is its ability to represent, transmit and fuse probabilistic information, and thus to take near-optimal decisions when inputs are contradictory, noisy or missing. This is achieved by a neural space-latency code which is a natural consequence of the simple recurrent dynamics in each layer. Furthermore, the network possesses a feedback mechanism that is compatible with the space-latency code by making use of the attractor properties of neural layers. We show that this feedback mechanism can resolve/correct ambiguities at lower levels. As the fusion of feedback information in each layer is achieved in a probabilistically coherent fashion, feedback only has an effect if low-level inputs are ambiguous.

PROPRE is a generic and modular unsupervised neural learning paradigm that extracts meaningful concepts of multimodal data flows based on predictability across modalities. It consists on the combination of three modules. First, a topological projection of each data flow on a self-organizing map. Second, a decentralized prediction of each projection activity from each others map activities. Third, a predictability measure that compares predicted and real activities. This measure is used to modulate the projection learning so that to favor the mapping of predictable stimuli across modalities. In this article, we use Kohonen map for the projection module, linear regression for the prediction one and we propose multiple generic predictability measures. We illustrate the properties and performances of PROPRE paradigm on a challenging supervised classification task of visual pedestrian data. The modulation of the projection learning by the predictability measure improves significantly classification performances of the system independently of the measure used. Moreover, PROPRE provides a combination of interesting functional properties, such as a dynamical adaptation to input statistic variations, that is rarely available in other machine learning algorithms.

We present an approach for making use of scene or situation context in object detection, aiming for state-of-the-art performance while dramatically reducing computational cost. While existing approaches are inspired by Bayes' rule, training context-independent detectors and combining them with context priors in hindsight, we propose to integrate these context priors into detector design itself, through algorithmic choices and/or pre-selection of training examples. Although such restricted detectors will, as a consequence, be valid only in regions compatible with context priors, the corresponding simplification of the object-vs-background decision problem will lead to reduced computation time and/or increased detection performance. We verify this experimentally by analyzing vehicle detection performance in a realistically simulated inner-city environment where context priors are defined by a road surface mask obtained from the simulation tool. Comparing a restricted detector, based on horizontal edges detection refined by neural network confirmation, to a generic HOG+SVM-based approach which takes into account the road context prior, we show that the restricted detector shows superior vehicle detection performance at a vastly reduced computational cost. We show qualitative results that permit the conclusion that the restricted detector will perform well on real-world scenes if appropriate road context priors are available.

Reliable road detection is a key issue for modern Intelligent Vehicles, since it can help to identify the driv-able area as well as boosting other perception functions like object detection. However, real environments present several challenges like illumination changes and varying weather conditions. We propose a multi-modal road detection and segmentation method based on monocular images and HD multi-layer LIDAR data (3D point cloud). This algorithm consists of three stages: extraction of ground points from multilayer LIDAR, transformation of color camera information to an illumination-invariant representation, and lastly the segmentation of the road area. For the first module, the core function is to extract the ground points from LIDAR data. To this end a road boundary detection is performed based on histogram analysis, then a plane estimation using RANSAC, and a ground point extraction according to the point-to-plane distance. In the second module, an image representation of illumination-invariant features is computed simultaneously. Ground points are projected to image plane and then used to compute a road probability map using a Gaussian model. The combination of these modalities improves the robustness of the whole system and reduces the overall computational time, since the first two modules can be run in parallel. Quantitative experiments carried on the public KITTI dataset enhanced by road annotations confirmed the effectiveness of the proposed method.

We describe a simple way to reduce the amount of required training data in context-based models of realtime object detection. We demonstrate the feasibility of our approach in a very challenging vehicle detection scenario comprising multiple weather, environment and light conditions such as rain, snow and darkness (night). The investigation is based on a real-time detection system effectively composed of two trainable components: an exhaustive multiscale object detector (”signal-driven detection”), as well as a module for generating object-specific visual attention (”context models”) controlling the signal-driven detection process. Both parts of the system require a significant amount of ground-truth data which need to be generated by human annotation in a time-consuming and costly process. Assuming sufficient training examples for signal-based detection, we demonstrate that a co-training step can eliminate the need for separate ground-truth data to train context models. This is achieved by directly training context models with the results of signal-driven detection. We show that this process is feasible for different qualities of signal-driven detection, and maintains the performance gains from context models. As it is by now widely accepted that signal-driven object detection can be significantly improved by context models, our method allows to train strongly improved detection systems without additional labor, and above all, cost.

We present a flexible and scalable architecture that can learn to predict the future behavior of a vehicle in inner-city traffic. While behavior prediction studies have mainly been focusing on lane change events on highways, we apply our approach to a simple inner-city scenario: approaching a traffic light. Our system employs dynamic information about the current ego-vehicle state as well as static information about the scene, in this case position and state of nearby traffic lights.

BACKGROUND: Exonic splicing enhancers (ESEs) activate nearby splice sites and promote the inclusion (vs. exclusion) of exons in which they reside, while being a binding site for SR proteins. To study the impact of ESEs on alternative splicing it would be useful to have a possibility to detect them in exons. Identifying SR protein-binding sites in human DNA sequences by machine learning techniques is a formidable task, since the exon sequences are also constrained by their functional role in coding for proteins. RESULTS: The choice of training examples needed for machine learning approaches is difficult since there are only few exact locations of human ESEs described in the literature which could be considered as positive examples. Additionally, it is unclear which sequences are suitable as negative examples. Therefore, we developed a motif-oriented data-extraction method that extracts exon sequences around experimentally or theoretically determined ESE patterns. Positive examples are restricted by heuristics based on known properties of ESEs, e.g. location in the vicinity of a splice site, whereas negative examples are taken in the same way from the middle of long exons. We show that a suitably chosen SVM using optimized sequence kernels (e.g., combined oligo kernel) can extract meaningful properties from these training examples. Once the classifier is trained, every potential ESE sequence can be passed to the SVM for verification. Using SVMs with the combined oligo kernel yields a high accuracy of about 90 percent and well interpretable parameters. CONCLUSION: The motif-oriented data-extraction method seems to produce consistent training and test data leading to good classification rates and thus allows verification of potential ESE motifs. The best results were obtained using an SVM with the combined oligo kernel, while oligo kernels with oligomers of a certain length could be used to extract relevant features.