Quantum computing leverages the unique properties of qubits and quantum parallelism to solve problems intractable for classical systems, offering unparalleled computational potential. However, optimization of quantum circuits remains critical, especially for noisy intermediate-scale quantum (NISQ) devices with limited qubits and high error rates. Genetic algorithms (GAs) provide a promising approach for efficient quantum circuit synthesis by automating optimization tasks. This work examines the impact of various mutation strategies within a GA framework for quantum circuit synthesis. By analyzing how different mutations transform circuits, it identifies strategies that enhance efficiency and performance. Experiments utilized a fitness function emphasizing fidelity, while accounting for circuit depth and T-operations, to optimize circuits with four to six qubits. Our analysis revealed that, while the "swap, addition" strategy achieved the highest fidelity scores, it consistently increased circuit depth. In contrast, combining "swap, addition, delete" strategies offers a more balanced approach, delivering near-optimal results while also having the potential of reducing circuit depth.

We apply a hybrid evolutionary algorithm to minimize the depth of circuits in quantum computing. More specifically, we evaluate two different variants of the algorithm. In the first approach, we combine the evolutionary algorithm with an optimization subroutine to optimize the parameters of the rotation gates present in the quantum circuit. In the second, the algorithm solely relies on evolutionary operations (i.e., mutations and crossover). We approach the problem from two sides: (1) constructing circuits from the ground up by starting with random initializations and (2) initializing individuals with a target circuit in order to optimize it further according to the fitness function. We run experiments on random circuits with 4 and 6 qubits varying in circuit depth. Our results show that the proposed methods are able to significantly reduce the depth of circuits while still retaining a high fidelity to the target state.

While exploration in single-agent reinforcement learning has been studied extensively in recent years, consid-erably less work has focused on its counterpart in multi-agent reinforcement learning. To address this issue, this work proposes a peer-incentivized reward function inspired by previous research on intrinsic curiosity and influence-based rewards. The PIMAEX reward, short for Peer-Incentivized Multi-Agent Exploration, aims to improve exploration in the multi-agent setting by encouraging agents to exert influence over each other to increase the likelihood of encountering novel states. We evaluate the PIMAEX reward in conjunction with PIMAEX-Communication, a multi-agent training algorithm that employs a communication channel for agents to influence one another. The evaluation is conducted in the Consume/Explore environment, a partially observable environment with deceptive rewards, specifically designed to challenge the exploration vs. exploitation dilemma and the credit-assignm ent problem. The results empirically demonstrate that agents using the PI-MAEX reward with PIMAEX-Communication outperform those that do not.

Optimizing the architecture of variational quantum circuits (VQCs) is crucial for advancing quantum computing (QC) towards practical applications. Current methods range from static ansatz design and evolutionary methods to machine learned VQC optimization, but are either slow, sample inefficient or require infeasible circuit depth to realize advantages. Quality diversity (QD) search methods combine diversity-driven optimization with user-specified features that offer insight into the optimization quality of circuit solution candidates. However, the choice of quality measures and the representational modeling of the circuits to allow for optimization with the current state-of-the-art QD methods like covariance matrix adaptation (CMA), is currently still an open problem. In this work we introduce a directly matrix-based circuit engineering, that can be readily optimized with QD-CMA methods and evaluate heuristic circuit quality properties like expressivity and gate-diversity as quality measures. We empirically show superior circuit optimization of our QD optimization w.r.t. speed and solution score against a set of robust benchmark algorithms from the literature on a selection of NP-hard combinatorial optimization problems.

Abstract We propose discriminative reward co-training (DIRECT) as an extension to deep reinforcement learning algorithms. Building upon the concept of self-imitation learning (SIL), we introduce an imitation buffer to store beneficial trajectories generated by the policy, determined by their return. A discriminator network is trained concurrently to the policy to distinguish between trajectories generated by the current policy and beneficial trajectories generated by previous policies. The discriminator’s verdict is used to construct a reward signal for optimizing the policy. By interpolating prior experience, DIRECT is able to act as a reward surrogate, steering policy optimization toward more valuable regions of the reward landscape, thus, toward learning an optimal policy. In this article, we formally introduce the additional components, their intended purpose and parameterization, and define a unified training procedure. To reveal insights into the mechanics of the proposed architecture, we provide evaluations of the introduced hyperparameters. Further benchmark evaluations in various discrete and continuous control environments provide evidence that DIRECT is especially beneficial in environments possessing sparse rewards, hard exploration tasks, and shifting circumstances. Our results show that DIRECT outperforms state-of-the-art algorithms in those challenging scenarios by providing a surrogate reward to the policy and direct the optimization toward valuable areas.

324

Surrogate models are ubiquitously used in industry and academia to efficiently approximate black box functions. As state-of-the-art methods from classical machine learning frequently struggle to solve this problem accurately for the often scarce and noisy data sets in practical applications, investigating novel approaches is of great interest. Motivated by recent theoretical results indicating that quantum neural networks (QNNs) have the potential to outperform their classical analogs in the presence of scarce and noisy data, we benchmark their qualitative performance for this scenario empirically. Our contribution displays the first application-centered approach of using QNNs as surrogate models on higher dimensional, real world data. When compared to a classical artificial neural network with a similar number of parameters, our QNN demonstrates significantly better results for noisy and scarce data, and thus motivates future work to explore this potential quantum advantage. Finally, we demonstrate the performance of current NISQ hardware experimentally and estimate the gate fidelities necessary to replicate our simulation results.

Variational Quantum Algorithms are one of the most promising candidates to yield the first industrially relevant quantum advantage.Being capable of arbitrary function approximation, they are often referred to as Quantum Neural Networks (QNNs) when being used in analog settings as classical Artificial Neural Networks (ANNs).Similar to the early stages of classical machine learning, known schemes for efficient architectures of these networks are scarce.Exploring beyond existing design patterns, we propose a reduced-width circuit ansatz design, which is motivated by recent results gained in the analysis of dropout regularization in QNNs.More precisely, this exploits the insight, that the gates of overparameterized QNNs can be pruned substantially until their expressibility decreases.The results of our case study show, that the proposed design pattern can significantly reduce training time while maintaining the same result quality as the standard "full-width" design in the presence of noise.

A central challenge in quantum machine learning is the design and training of parameterized quantum circuits (PQCs).Similar to deep learning, vanishing gradients pose immense problems in the trainability of PQCs, which have been shown to arise from a multitude of sources.One such cause are non-local loss functions, that demand the measurement of a large subset of involved qubits.To facilitate the parameter training for quantum applications using global loss functions, we propose a Sequential Hamiltonian Assembly, which iteratively approximates the loss function using local components.Aiming for a prove of principle, we evaluate our approach using Graph Coloring problem with a Varational Quantum Eigensolver (VQE).Simulation results show, that our approach outperforms conventional parameter training by 29.99% and the empirical state of the art, Layerwise Learning, by 5.12% in the mean accuracy.This paves the way towards locality-aware learning techniques, allowing to evade vanishing gradients for a large class of practically relevant problems.

Anomaly detection in Endpoint Detection and Response (EDR) is a critical task in cybersecurity programs of large companies.With rapidly growing amounts of data and the omnipresence of zero-day attacks, manual and rule-based detection techniques are no longer eligible in practice.While classical machine learning approaches to this problem exist, they frequently show unsatisfactory performance in differentiating malicious from benign anomalies.A promising approach to attain superior generalization than currently employed machine learning techniques are quantum generative models.Allowing for the largest representation of data on available quantum hardware, we investigate Quantum Annealing based Quantum Boltzmann Machines (QBMs) for the given problem.We contribute the first fully unsupervised approach for the problem of anomaly detection using QBMs and evaluate its performance on an EDR inspired synthetic dataset.Our results indicate that QBMs can outperform their classical analog (i.e., Restricted Boltzmann Machines) in terms of result quality and training steps in special cases.When employing Quantum Annealers from D-Wave Systems, we conclude that either more accurate classical simulators or substantially more QPU time is needed to conduct the necessary hyperparameter optimization allowing to replicate our simulation results on quantum hardware.

Abstract Peer incentivization (PI) is a recent approach where all agents learn to reward or penalize each other in a distributed fashion, which often leads to emergent cooperation. Current PI mechanisms implicitly assume a flawless communication channel in order to exchange rewards. These rewards are directly incorporated into the learning process without any chance to respond with feedback. Furthermore, most PI approaches rely on global information, which limits scalability and applicability to real-world scenarios where only local information is accessible. In this paper, we propose Mutual Acknowledgment Token Exchange (MATE), a PI approach defined by a two-phase communication protocol to exchange acknowledgment tokens as incentives to shape individual rewards mutually. All agents condition their token transmissions on the locally estimated quality of their own situations based on environmental rewards and received tokens. MATE is completely decentralized and only requires local communication and information. We evaluate MATE in three social dilemma domains. Our results show that MATE is able to achieve and maintain significantly higher levels of cooperation than previous PI approaches. In addition, we evaluate the robustness of MATE in more realistic scenarios, where agents can deviate from the protocol and communication failures can occur. We also evaluate the sensitivity of MATE w.r.t. the choice of token values.

Applying new computing paradigms like quantum computing to the field of machine learning has recently gained attention.However, as high-dimensional real-world applications are not yet feasible to be solved using purely quantum hardware, hybrid methods using both classical and quantum machine learning paradigms have been proposed.For instance, transfer learning methods have been shown to be successfully applicable to hybrid image classification tasks.Nevertheless, beneficial circuit architectures still need to be explored.Therefore, tracing the impact of the chosen circuit architecture and parameterization is crucial for the development of beneficially applicable hybrid methods.However, current methods include processes where both parts are trained concurrently, therefore not allowing for a strict separability of classical and quantum impact.Thus, those architectures might produce models that yield a superior prediction accuracy whilst employing the least possible quantum impact.To tackle this issue, we propose Sequential Quantum Enhanced Training (SE-QUENT) an improved architecture and training process for the traceable application of quantum computing methods to hybrid machine learning.Furthermore, we provide formal evidence for the disadvantage of current methods and preliminary experimental results as a proof-of-concept for the applicability of SEQUENT.

Overfitting is a problem in Convolutional Neural Networks (CNN) that causes poor generalization of models on unseen data. To remediate this problem, many new and diverse data augmentation (DA) methods have been proposed to supplement or generate more training data, and thereby increase its quality. In this work, we propose a new DA algorithm: VoronoiPatches (VP). We primarily utilize non-linear re-combination of information within an image, fragmenting and occluding small information patches. Unlike other DA methods, VP uses small convex polygon-shaped patches in a random layout to transport information around within an image. In our experiments, VP outperformed current DA methods regarding model variance and overfitting tendencies. We demonstrate DA utilizing non-linear re-combination of information within images, and non-orthogonal shapes and structures improves CNN model robustness on unseen data.

In recent years, quantum machine learning has seen a substantial increase in the use of variational quantum circuits (VQCs). VQCs are inspired by artificial neural networks, which achieve extraordinary performance in a wide range of AI tasks as massively parameterized function approximators. VQCs have already demonstrated promising results, for example, in generalization and the requirement for fewer parameters to train, by utilizing the more robust algorithmic toolbox available in quantum computing. A VQCs’ trainable parameters or weights are usually used as angles in rotational gates and current gradient-based training methods do not account for that. We introduce weight re-mapping for VQCs, to unambiguously map the weights to an interval of length 2π, drawing inspiration from traditional ML, where data rescaling, or normalization techniques have demonstrated tremendous benefits in many circumstances. We employ a set of five functions and evaluate them on the Iris and Wine datasets using variational classifiers as an example. Our experiments show that weight re-mapping can improve convergence in all tested settings. Additionally, we were able to demonstrate that weight re-mapping increased test accuracy for the Wine dataset by 10% over using unmodified weights.

Stochastic partial observability poses a major challenge for decentralized coordination in multi-agent reinforcement learning but is largely neglected in state-of-the-art research due to a strong focus on state-based centralized training for decentralized execution (CTDE) and benchmarks that lack sufficient stochasticity like StarCraft Multi-Agent Challenge (SMAC). In this paper, we propose Attention-based Embeddings of Recurrence In multi-Agent Learning (AERIAL) to approximate value functions under stochastic partial observability. AERIAL replaces the true state with a learned representation of multi-agent recurrence, considering more accurate information about decentralized agent decisions than state-based CTDE. We then introduce MessySMAC, a modified version of SMAC with stochastic observations and higher variance in initial states, to provide a more general and configurable benchmark regarding stochastic partial observability. We evaluate AERIAL in Dec-Tiger as well as in a variety of SMAC and MessySMAC maps, and compare the results with state-based CTDE. Furthermore, we evaluate the robustness of AERIAL and state-based CTDE against various stochasticity configurations in MessySMAC.

The safe application of reinforcement learning (RL) requires generalization from limited training data to unseen scenarios. Yet, fulfilling tasks under changing circumstances is a key challenge in RL. Current state-of-the-art approaches for generalization apply data augmentation techniques to increase the diversity of training data. Even though this prevents overfitting to the training environment(s), it hinders policy optimization. Crafting a suitable observation, only containing crucial information, has been shown to be a challenging task itself. To improve data efficiency and generalization capabilities, we propose Compact Reshaped Observation Processing (CROP) to reduce the state information used for policy optimization. By providing only relevant information, overfitting to a specific training layout is precluded and generalization to unseen environments is improved. We formulate three CROPs that can be applied to fully observable observation- and action-spaces and provide methodical foundation. We empirically show the improvements of CROP in a distributionally shifted safety gridworld. We furthermore provide benchmark comparisons to full observability and data-augmentation in two different-sized procedurally generated mazes.

Common to all different kinds of recurrent neural networks (RNNs) is the intention to model relations between data points through time. When there is no immediate relationship between subsequent data points (like when the data points are generated at random, e.g.), we show that RNNs are still able to remember a few data points back into the sequence by memorizing them by heart using standard backpropagation. However, we also show that for classical RNNs, LSTM and GRU networks the distance of data points between recurrent calls that can be reproduced this way is highly limited (compared to even a loose connection between data points) and subject to various constraints imposed by the type and size of the RNN in question. This implies the existence of a hard limit (way below the information-theoretic one) for the distance between related data points within which RNNs are still able to recognize said relation.

Quadratic unconstrained binary optimization (QUBO) has become the standard format for optimization using quantum computers, i.e., for both the quantum approximate optimization algorithm (QAOA) and quantum annealing (QA). We present a toolkit of methods to transform almost arbitrary problems to QUBO by (i) approximating them as a polynomial and then (ii) translating any polynomial to QUBO. We showcase the usage of our approaches on two example problems (ratio cut and logistic regression).

A key element of biological structures is self-replication. Neural networks are the prime structure used for the emergent construction of complex behavior in computers. We analyze how various network types lend themselves to self-replication. Backpropagation turns out to be the natural way to navigate the space of network weights and allows non-trivial self-replicators to arise naturally. We perform an in-depth analysis to show the self-replicators' robustness to noise. We then introduce artificial chemistry environments consisting of several neural networks and examine their emergent behavior. In extension to this work's previous version (Gabor et al., 2019), we provide an extensive analysis of the occurrence of fixpoint weight configurations within the weight space and an approximation of their respective attractor basins.

We focus on resilience in cooperative multi-agent systems, where agents can change their behavior due to udpates or failures of hardware and software components. Current state-of-the-art approaches to cooperative multi-agent reinforcement learning (MARL) have either focused on idealized settings without any changes or on very specialized scenarios, where the number of changing agents is fixed, e.g., in extreme cases with only one productive agent. Therefore, we propose Resilient Adversarial value Decomposition with Antagonist-Ratios (RADAR). RADAR offers a value decomposition scheme to train competing teams of varying size for improved resilience against arbitrary agent changes. We evaluate RADAR in two cooperative multi-agent domains and show that RADAR achieves better worst case performance w.r.t. arbitrary agent changes than state-of-the-art MARL.

In this work, we present a general procedure for acoustic leak detection in water networks that satisfies multiple real-world constraints such as energy efficiency and ease of deployment. Based on recordings from seven contact microphones attached to the water supply network of a municipal suburb, we trained several shallow and deep anomaly detection models. Inspired by how human experts detect leaks using electronic sounding-sticks, we use these models to repeatedly listen for leaks over a predefined decision horizon. This way we avoid constant monitoring of the system. While we found the detection of leaks in close proximity to be a trivial task for almost all models, neural network based approaches achieve better results at the detection of distant leaks.

In industrial applications, the early detection of malfunctioning factory machinery is crucial. In this paper, we consider acoustic malfunction detection via transfer learning. Contrary to the majority of current approaches which are based on deep autoencoders, we propose to extract features using neural networks that were pretrained on the task of image classification. We then use these features to train a variety of anomaly detection models and show that this improves results compared to convolutional autoencoders in recordings of four different factory machines in noisy environments. Moreover, we find that features extracted from ResNet based networks yield better results than those from AlexNet and Squeezenet. In our setting, Gaussian Mixture Models and One-Class Support Vector Machines achieve the best anomaly detection performance.

A characteristic of reinforcement learning is the ability to develop unforeseen strategies when solving problems. While such strategies sometimes yield superior performance, they may also result in undesired or even dangerous behavior. In industrial scenarios, a system's behavior also needs to be predictable and lie within defined ranges. To enable the agents to learn (how) to align with a given specification, this paper proposes to explicitly transfer functional and non-functional requirements into shaped rewards. Experiments are carried out on the smart factory, a multi-agent environment modeling an industrial lot-size-one production facility, with up to eight agents and different multi-agent reinforcement learning algorithms. Results indicate that compliance with functional and non-functional constraints can be achieved by the proposed approach.

Some of the most relevant future applications of multi-agent systems like autonomous driving or factories as a service display mixed-motive scenarios, where agents might have conflicting goals. In these settings agents are likely to learn undesirable outcomes in terms of cooperation under independent learning, such as overly greedy behavior. Motivated from real world societies, in this work we propose to utilize market forces to provide incentives for agents to become cooperative. As demonstrated in an iterated version of the Prisoner's Dilemma, the proposed market formulation can change the dynamics of the game to consistently learn cooperative policies. Further we evaluate our approach in spatially and temporally extended settings for varying numbers of agents. We empirically find that the presence of markets can improve both the overall result and agent individual returns via their trading activities.

An anomalous sound detection (ASD) system detects substantial deviations from the norm and reports the degree of abnormality through an anomaly score. An important application scenario is the detection of malfunctions in factory machinery. Recent approaches train autoencoders on small segments of the sound's time-frequency representation and use the reconstruction error as a measure of abnormality. However, it was recently shown that this approach leads to consistently higher reconstruction errors for the edge frames of the segments. To alleviate this problem, the Interpolation Deep Neural Network (IDNN) predicts the center frame from the remaining context frames. In this work, we propose DRINK - Deep Recurrent INterpolation NetworKs, an extension of the aforementioned IDNN that enables a variable amount of center and context frames. Moreover, we use a Long-Short Term Memory network to explicitly account for the sequential nature of sound as opposed to simple feed-forward neural networks in the original work. We show that under the right setting of context and center frames, our method is able to outperform the IDNN and autoencoder baselines on a dataset of recordings from factory machinery in 13 out of 16 cases.

Building autonomous agents that are capable to cooperate with other machines is an essential step towards large scale application of AI systems. Especially systems comprised of multiple self-interested agents with general sum returns can profit from cooperative behavior as cooperation can help to increase the return from all agents simultaneously. A critical aspect that might undermine cooperation is given if agents cannot make credible threats or promises (called commitment problems). Inspired by this idea in this work we augment deep reinforcement learning agents with the capability to build agreements with one another, thereby enabling agents to autonomously learn at which time to cooperate with other agents. This approach, called distributed emergent agreement learning (DEAL), enables agents to commit to specific policies defined by the agreement. We evaluate DEAL with up to 16 agents, represented as Deep Q-Networks or instances of Proximal Policy Optimization in a factory domain and empirically show that agreements increase cooperation by improving both overall and agent individual returns.

We apply the vision transformer, a deep machine learning model build around the attention mechanism, on mel-spectrogram representations of raw audio recordings. When adding mel-based data augmentation techniques and sample-weighting, we achieve comparable performance on both (PRS and CCS challenge) tasks of ComParE21, outperforming most single model baselines. We further introduce overlapping vertical patching and evaluate the influence of parameter configurations. Index Terms: audio classification, attention, mel-spectrogram, unbalanced data-sets, computational paralinguistics

Value function factorization (VFF) is a popular approach to cooperative multi-agent reinforcement learning in order to learn local value functions from global rewards. However, state-of-the-art VFF is limited to a handful of agents in most domains. We hypothesize that this is due to the flat factorization scheme, where the VFF operator becomes a performance bottleneck with an increasing number of agents. Therefore, we propose VFF with variable agent sub-teams (VAST). VAST approximates a factorization for sub-teams which can be defined in an arbitrary way and vary over time, e.g., to adapt to different situations. The sub-team values are then linearly decomposed for all sub-team members. Thus, VAST can learn on a more focused and compact input representation of the original VFF operator. We evaluate VAST in three multi-agent domains and show that VAST can significantly outperform state-of-the-art VFF, when the number of agents is sufficiently large.

Abstract In evolutionary algorithms, the notion of diversity has been adopted from biology and is used to describe the distribution of a population of solution candidates. While it has been known that maintaining a reasonable amount of diversity often benefits the overall result of the evolutionary optimization process by adjusting the exploration/exploitation trade-off, little has been known about what diversity is optimal. We introduce the notion of productive fitness based on the effect that a specific solution candidate has some generations down the evolutionary path. We derive the notion of final productive fitness, which is the ideal target fitness for any evolutionary process. Although it is inefficient to compute, we show empirically that it allows for ana posteriorianalysis of how well a given evolutionary optimization process hit the ideal exploration/exploitation trade-off, providing insight intowhydiversity-aware evolutionary optimization often performs better.

Many problems of industrial interest are NP-complete, and quickly exhaust resources of computational devices with increasing input sizes. Quantum annealers (QA) are physical devices that aim at this class of problems by exploiting quantum mechanical properties of nature. However, they compete with efficient heuristics and probabilistic or randomised algorithms on classical machines that allow for finding approximate solutions to large NP-complete problems.

Robustness to out-of-distribution (OOD) data is an important goal in building reliable machine learning systems. Especially in autonomous systems, wrong predictions for OOD inputs can cause safety critical situations. As a first step towards a solution, we consider the problem of detecting such data in a value-based deep reinforcement learning (RL) setting. Modelling this problem as a one-class classification problem, we propose a framework for uncertainty-based OOD classification: UBOOD. It is based on the effect that an agent's epistemic uncertainty is reduced for situations encountered during training (in-distribution), and thus lower than for unencountered (OOD) situations. Being agnostic towards the approach used for estimating epistemic uncertainty, combinations with different uncertainty estimation methods, e.g. approximate Bayesian inference methods or ensembling techniques are possible. We further present a first viable solution for calculating a dynamic classification threshold, based on the uncertainty distribution of the training data. Evaluation shows that the framework produces reliable classification results when combined with ensemble-based estimators, while the combination with concrete dropout-based estimators fails to reliably detect OOD situations. In summary, UBOOD presents a viable approach for OOD classification in deep RL settings by leveraging the epistemic uncertainty of the agent's value function.

In many fields of research, labeled datasets are hard to acquire. This is where data augmentation promises to overcome the lack of training data in the context of neural network engineering and classification tasks. The idea here is to reduce model over-fitting to the feature distribution of a small under-descriptive training dataset. We try to evaluate such data augmentation techniques to gather insights in the performance boost they provide for several convolutional neural networks on mel-spectrogram representations of audio data. We show the impact of data augmentation on the binary classification task of surgical mask detection in samples of human voice (ComParE Challenge 2020). Also we consider four varying architectures to account for augmentation robustness. Results show that most of the baselines given by ComParE are outperformed.

Abstract Systems are becoming increasingly more adaptive, using techniques like machine learning to enhance their behavior on their own rather than only through human developers programming them. We analyze the impact the advent of these new techniques has on the discipline of rigorous software engineering, especially on the issue of quality assurance. To this end, we provide a general description of the processes related to machine learning and embed them into a formal framework for the analysis of adaptivity, recognizing that to test an adaptive system a new approach to adaptive testing is necessary. We introduce scenario coevolution as a design pattern describing how system and test can work as antagonists in the process of software evolution. While the general pattern applies to large-scale processes (including human developers further augmenting the system), we show all techniques on a smaller-scale example of an agent navigating a simple smart factory. We point out new aspects in software engineering for adaptive systems that may be tackled naturally using scenario coevolution. This work is a substantially extended take on Gabor et al. (International symposium on leveraging applications of formal methods, Springer, pp 137–154, 2018).

State-of-the-art approaches to partially observable planning like POMCP are based on stochastic tree search. While these approaches are computationally efficient, they may still construct search trees of considerable size, which could limit the performance due to restricted memory resources. In this paper, we propose Partially Observable Stacked Thompson Sampling (POSTS), a memory bounded approach to openloop planning in large POMDPs, which optimizes a fixed size stack of Thompson Sampling bandits. We empirically evaluate POSTS in four large benchmark problems and compare its performance with different tree-based approaches. We show that POSTS achieves competitive performance compared to tree-based open-loop planning and offers a performancememory tradeoff, making it suitable for partially observable planning with highly restricted computational and memory resources.

The extraction of 3D models represented by Constructive Solid Geometry (CSG) trees from point clouds is a common problem in reverse engineering pipelines as used by Computer Aided Design (CAD) tools. We propose three independent enhancements on state-of-the-art Genetic Algorithms (GAs) for CSG tree extraction: (1) A deterministic point cloud filtering mechanism that significantly reduces the computational effort of objective function evaluations without loss of geometric precision, (2) a graph-based partitioning scheme that divides the problem domain in smaller parts that can be solved separately and thus in parallel and (3) a 2-level improvement procedure that combines a recursive CSG tree redundancy removal technique with a local search heuristic, which significantly improves GA running times. We show in an extensive evaluation that our optimized GA-based approach provides faster running times and scales better with problem size compared to state-of-the-art GA-based approaches.

This paper considers the problem of path planning under dynamic aspects. We propose ”Neural Gas Dynamic Path Planning” (NGDPP), a novel algorithm that continuously provides a valid path between two points inside an environment that transforms in an unpredictable manner. These transformations can occur due to both, changes in the environment’s shape and moving collision objects. The algorithm incorporates several techniques: Neural Gas, a dynamic discretization method; the A* Algorithm, a path planning algorithm for graphs; and the Potential Field method, which facilitates the avoidance of collisions. We empirically evaluate the proposed algorithm under various aspects providing performance information and guidance about situations and applications benefiting from the algorithm. The evaluation reveals that NGDPP is a solid algorithm for path planning in dynamic environments. Yet, the algorithm is based on heuristic information, i.e. a optimal result in term of the path length cannot be guaranteed.

We propose an approach to general subgoal-based temporal abstraction in MCTS. Our approach approximates a set of available macro-actions locally for each state only requiring a generative model and a subgoal predicate. For that, we modify the expansion step of MCTS to automatically discover and optimize macro-actions that lead to subgoals. We empirically evaluate the effectiveness, computational efficiency and robustness of our approach w.r.t. different parameter settings in two benchmark domains and compare the results to standard MCTS without temporal abstraction.

We propose Stable Yet Memory Bounded Open-Loop (SYMBOL) planning, a general memory bounded approach to partially observable open-loop planning. SYMBOL maintains an adaptive stack of Thompson Sampling bandits, whose size is bounded by the planning horizon and can be automatically adapted according to the underlying domain without any prior domain knowledge beyond a generative model. We empirically test SYMBOL in four large POMDP benchmark problems to demonstrate its effectiveness and robustness w.r.t. the choice of hyperparameters and evaluate its adaptive memory consumption. We also compare its performance with other open-loop planning algorithms and POMCP.

Diversity is an important factor in evolutionary algorithms to prevent premature convergence towards a single local optimum. In order to maintain diversity throughout the process of evolution, various means exist in literature. We analyze approaches to diversity that (a) have an explicit and quantifiable influence on fitness at the individual level and (b) require no (or very little) additional domain knowledge such as domain-specific distance functions. We also introduce the concept of genealogical diversity in a broader study. We show that employing these approaches can help evolutionary algorithms for global optimization in many cases.

In recent years, the importance of location-based services and indoor positioning systems increased significantly for both, research and industry. Visual localization systems have the advantage of not depending on dedicated infrastructure and thus they are interesting for navigation within buildings. While there are already approaches which are using pre-recorded databases of reference images to obtain an absolute position for a given query image, suitable applications which are estimating the relative movement of pedestrians out of a first person perspective video are still missing. This paper presents a novel approach for a pedometer as well as for an activity detector using a such a first person perspective video stream of a pedestrian as input data. The system counts the number of steps and furthermore detects current activities of a user. Therefore, we analyze all video input data with the SURF algorithm in order to extract robust feature points. Especially the orientation and scaling properties of this feature points are used for an accurate measurement.

In recent years, location-based services and indoor positioning systems gained increasing importance for both, research and industry. Visual localization systems have the advantage of not being dependent on dedicated infrastructure and thus are especially interesting for navigation within buildings. While there are already approaches of using pre-recorded databases of reference images to obtain an absolute position for a given query image, suitable means to estimate the relative movement of pedestrians from an ego perspective video are still missing. This paper presents a novel visual odometry system for pedestrians. The user carries a mobile device while walking - the camera aims into the direction of walking. Using only the video stream as input, the system generates a two-dimensional trajectory, which describes the path traveled by the user. Both, the user's current heading as well as the walking direction are estimated based on the movement of visual feature points in successive video frames. In order to assess the accuracy of the system, it is evaluated in three different scenarios (indoors in an university building, in an urban area and in a city park). Not relying on reference points (for instance provided by a database, which references visual feature points with geo-data), the error accumulates with distance traveled. After a walked distance of 100 meters, the average error lies between 4.6 and 13.9 meters (depending on the scenario). Consequently, the system is a promising approach for visual odometry, which can be used in conjunction with existing absolute visual positioning systems or as a core part of a future SLAM (simultaneous localization and mapping) system.

Wi-Fi enabled devices periodically broadcast unencrypted management information which can easily be used for an involuntary tracking of users in an area of interest. However, reliable trajectory estimations on this data remain challenging, due to arbitrary and imprecise position fixes of moving targets. Probabilistic methods can help to increase the estimation accuracy significantly, but may degrade other important metrics, e.g. scalability, complexity, or robustness. In this paper, we investigate probabilistic solutions for a feasible tracking system for indoor scenarios. Beside the usage of Viterbi's algorithm and a common particle filter, we propose a novel state particle filter with a more restricted transition model based on discrete state nodes. All methods are compared and evaluated on various user traces using real Wi-Fi captures from common mobile devices at our office building. The results indicate that the proposed state particle filter performs best in terms of accuracy, and precision while using a smaller amount of particles which renders this approach scalable, and thus, feasible for indoor tracking systems.

A large-scale system hundreds of millions of people encounter every day is radio. While this “system” has a tremendous reach it also is technologically outdated. Technological constraints offer great protection of listeners' privacy but prevent radio stations from implementing modern business models like personalization and mining valuable user information, at the same time. This paper describes a fully distributed system that aims at overcoming current technological constraints in interconnecting radio stations and their listeners while retaining a comparable protection of sensitive user data. Namely, a peer-to-peer architecture with integrated data-mining-capabilities employing differential privacy is proposed. The system will offer personalizable radio programs to listeners and it will enable radio stations to gather valuable information about their listeners. Furthermore, this paper points out key challenges in deploying, bootstrapping and maintaining such a distributed system.

The knowledge of future Signal Phase and Timing information (SPaT) of traffic lights ahead enables a vast number of driving assistance functions, such as Green Light Optimal Speed Control, Red Light Duration Advisory or efficient Start-Stop Control. However, these assisting functionalities necessitate information on future signals of the traffic light ahead. One approach to obtain this information is to estimate SPaT from vehicular probe data, also called Floating Car Data. With the determination of the exact crossing time of traffic lights, it is possible to reconstruct the past signaling times. This paper addresses the question of how many recorded crossings of street intersections are needed in order to reconstruct SPaT of pre-timed (traffic lights with fixed pre-programmed settings) with a certain quality. We distinct between cycle time reconstruction and green signaling reconstruction and indicate a lower bound of necessary data under ideal conditions. We show that, albeit the amount of collected probe data is highly dependent on the intersections' complexity, the needed number of recorded crossings is within an attainable range.

Innovative driving assisting systems, such as Green Light Optimal Speed Advisory (GLOSA), efficient start-stop control or traffic light warning systems can contribute to reducing CO2emissions and traffic accidents. These systems necessitate reliable estimations on future traffic light signals on a large scale. However, due to dynamic adjustment of traffic lights' signal phasing and timing, the provision of such information on a certain quality level is a difficult task. This paper deals with the challenges of an extensive prediction of complete urban areas' traffic light networks. We introduce a real-time prediction algorithm and back-end implementation that is able to generate signaling predictions based on historical Signal Phase and Timing information (SPaT) for adaptive traffic lights of an entire urban area. Our approach is able to meet the requirement of dealing with high latency times for historical data and incomplete data sets. The proposed algorithm is able to provide predictions for 85% of the adaptive traffic lights with available historical SPaT in 65% of the time and thereby reaches an accuracy of 92% to 97%.

In this paper we propose the design and evaluation of an algorithm capable of inferring basic floor plans from motion paths generated from smartphone sensors. First we shortly discuss the applicability of modern smartphones equipped with various sensors to create motion paths through premises. Some dead reckoning approaches which are used to track and calculate the position of devices are mentioned. After that we present an algorithm which shall use motion paths for the creation of maps holding probabilistic information regarding the accessibility of specific areas within a confined structure such as a building. The algorithm thus differs from traditional SLAM and mapping approaches in that it explicitly focuses on map generation and path interpretation and does not implement any sort of sensor-specific processing. In the concluding evaluation it is shown that the presented algorithm while still prone to certain types of bad input data can calculate free space from the motion paths with a sufficient accuracy for basic mapping tasks, especially if more input samples are used. Finally, various possible improvements are discussed which may improve the algorithms processing quality and robustness.

The evaluation of novel algorithms, protocols, applications, or security attacks in context of Online Social Networks (OSN) necessitates datasets that represent a realistic snapshot of the underlying social graph. As crawling social graphs can become a time and resource consuming task, only a few anonymized datasets exist which are shared among the research community. Besides concerns about de-anonymization attacks on crawled graphs and the fact that such datasets cannot satisfy the statistical confidence in simulation results, more and more secure and privacy-preserving Peer-to-Peer (P2P) OSN architectures emerge that do not facilitate crawling of social graph data at all. In order to evaluate new metrics for OSNs in general, we need social graph models which enable the generation of synthetic datasets. In this paper we present a generic model to synthesize social interaction graphs for both centralized OSNs like Facebook and secure and privacy-preserving P2P OSNs such as Vegas. Our approach accounts for a static component which models relationships and network effects and a dynamic component which models interactions among users. A flexible parameterization schema allows our model to individually influence certain graph characteristics like node degrees, clustering coefficients, and node interactions.

Introducing the Internet Protocol to the in-car network, we need communication software for fast and robust development of future networked applications. The heterogeneity within the communication network concerning the differences in device capability and application requirements can be targeted by establishing a middleware consisting of several specifications. In this paper, we give evidence that a minimal middleware specification can be feasible even for smallest embedded Electronic Control Units but still largely interoperable with the more complex communication demands of powerful infotainment and driver assistance ones. We therefore present a prototype implementation, which is evaluated towards interoperability, resource consumption, and execution performance.

VANETS extend the driver's horizon to decrease traffic accidents by utilizing IEEE 802.11p wireless networks. In high density traffic situations an overloaded communication channel leads to significant packet loss. To overcome this problem multiple channels can be used in IEEE 802.11p. This itself leads to adjacent channel interference and impacts the performance of nearby channels. In this paper we define performance requirements with respect to packet loss and communication range. We show the impact of ACI on simultaneous channel usage by using our multi channel propagation extension for JiST/SWANS. We evaluate the performance on adjacent channels with respect to maximum channel load and variable transmit power. Our results show that the simultaneous usage of nearby channels is an serious issue when no channel access synchronization is applied.

With the increasing amount of digital multimedia content, the user is more and more overstrained. A promising solution for this problem is personalization that assists the user in selecting content with respect to the user's interest. Since the capabilities of mobile devices increased significantly in the recent years, the number of mobile multimedia services grows steadily. The mobile services are confronted with a much more changing environment than stationary services. This additional information enables even more precise personalization.This paper introduces an approach for context-aware personalization of mobile multimedia services. We developed a generic framework for application developers that can easily be configured and extended with application-specific algorithms matching content and context. In the proposed system, content selection is performed in a distributed way.

Services need to adapt themselves to their computing environment and the situation of the user to be context-aware services (CASs). For their adaptation, they are in need of specific context information, that often has to be derived from other pieces of low-level context information. The process of retrieving context information is highly dynamic since appropriate context sources are often unknown at design time and need to be discovered and understood at runtime. This paper proposes the CoCo concept (context composition) that consists of key infrastructural components supporting the process of context retrieval and context composition, and a graph-oriented language describing the steps that need to be executed to compose context information. CoCo needs an information model specifying the semantics of the information. CoCo is able to automate the process of context retrieving $even in heterogenous environments - to simplify the development and deployment of CASs and to increase their robustness.

This paper introduces a comprehensive framework that allows mobile users to access a variety of services provided by their current environment (e.g. print services). Novel to our approach is that selection and execution of services takes into account the user’s current context. Instead of being harassed by useless activities as service browsing or configuration issues, environmental services get seamlessly aligned to the user’s present task. Thus, the challenge is to develop a new service framework that fulfils these demands. The paper proposes a document-based approach; so called Context-Aware Packets (CAPs) contain context constraints and data for describing an entire service request. The core framework, Context-Aware Packets Enabling Ubiquitous Services (CAPEUS), reverts to CAPs for realising context-aware selection and execution of services.

The globalization of industrial enterprises leads to problems caused by the heterogeneity of hard- and software. To overcome these problems, distributed platforms can be used. Yet, much work remains to be done on distributed platforms before they actually can meet industrial needs. For example, there is no standardized mechanism for asynchronous data transfer and no possibility to transmit so-called container objects. The authors look at different methods of data transfer based on the Common Object Request Broker Architecture (CORBA), which are synchronous, one-way, retarded synchronous and asynchronous data transfer. Based upon the requirements of different enterprises, several experiments to realize an optimal CORBA-based data transfer are studied.