In this paper we consider the problem of how a reinforcement learning agent that is tasked with solving a sequence of reinforcement learning problems (a sequence of Markov decision processes) can use knowledge acquired early in its lifetime to improve its ability to solve new problems. We argue that previous experience with similar problems can provide an agent with information about how it should explore when facing a new but related problem. We show that the search for an optimal exploration strategy can be formulated as a reinforcement learning problem itself and demonstrate that such strategy can leverage patterns found in the structure of related problems. We conclude with experiments that show the benefits of optimizing an exploration strategy using our proposed framework.

Abstract Path planning is a fundamental problem in many areas, ranging from robotics and artificial intelligence to computer graphics and animation. Although there is extensive literature for computing optimal, collision‐free paths, there is relatively little work that explores the satisfaction of spatial constraints between objects and agents at the global navigation layer. This paper presents a planning framework that satisfies multiple spatial constraints imposed on the path. The type of constraints specified can include staying behind a building, walking along walls, or avoiding the line of sight of patrolling agents. We introduce two hybrid environment representations that balance computational efficiency and search space density to provide a minimal, yet sufficient, discretization of the search graph for constraint‐aware navigation. An extended anytime dynamic planner is used to compute constraint‐aware paths, while efficiently repairing solutions to account for varying dynamic constraints or an updating world model. We demonstrate the benefits of our method on challenging navigation problems in complex environments for dynamic agents using combinations of hard and soft, attracting and repelling constraints, defined by both static obstacles and moving obstacles. Copyright © 2014 John Wiley & Sons, Ltd.

This paper presents a GPU-based wave-front propagation technique for multi-agent path planning in extremely large, complex, dynamic environments. Our work proposes an adaptive subdivision of the environment with efficient indexing, update, and neighbor-finding operations on the GPU to address several known limitations in prior work. In particular, an adaptive environment representation reduces the device memory requirements by an order of magnitude which enables for the first time, GPU-based goal path planning in truly large-scale environments (> 2048 m2) for hundreds of agents with different targets. We compare our approach to prior work that uses an uniform grid on several challenging navigation benchmarks and report significant memory savings, and up to a 1000X computational speedup.

Path finding is a fundamental, yet computationally expensive problem in robotics navigation. Often times, it is necessary to sacrifice optimality to find a feasible plan given a time constraint due to the search complexity. Dynamic environments may further invalidate current computed plans, requiring an efficient planning strategy that can repair existing solutions. This paper presents a massively parallelized wavefront-based approach to path planning, running on the GPU, that can efficiently repair plans to accommodate world changes and agent movement, without having to restart the wavefront propagation process. In addition, we introduce a termination condition which ensures the minimum number of GPU iterations while maintaining strict optimality constraints on search graphs with non-uniform costs.

Path planning is a fundamental problem in many areas ranging from robotics and artificial intelligence to computer graphics and animation. While there is extensive literature for computing optimal, collision-free paths, there is little work that explores the satisfaction of spatial constraints between objects and agents at the global navigation layer. This paper presents a planning framework that satisfies multiple spatial constraints imposed on the path. The type of constraints specified could include staying behind a building, walking along walls, or avoiding the line of sight of patrolling agents. We introduce a hybrid environment representation that balances computational efficiency and discretization resolution, to provide a minimal, yet sufficient discretization of the search graph for constraint-aware navigation. An extended anytime-dynamic planner is used to compute constraint-aware paths, while efficiently repairing solutions to account for dynamic constraints. We demonstrate the benefits of our method on challenging navigation problems in complex environments for dynamic agents using combinations of hard and soft constraints, attracting and repelling constraints, on static obstacles and moving obstacles.

In the context of real-time detection of transient signals, a likelihood ratio (LR) test is evaluated at every sampling interval. Performing the LR tests at a lower rate reduces significantly the computational complexity of the detection algorithm. However, in general, this simplification also leads to a strong degradation of the detector performance. For example, with a small shift error, an arriving transient may be in quadrature with its model. This degradation is particularly noticeable when the signals to detect are deterministic and sampled at a frequency close to the Nyquist rate. This letter proposes a method to overcome this limitation by using locally stationary models of the signals to detect. The resulting detectors are robust to shift errors and computationally efficient.

Local stationarity of an L/sup 2/(/spl Ropf/) bandpass random process reflects in specific regions of either the frequency plane of its 2 dimensional power spectrum or the time-frequency plane of its Wigner distribution. The paper addresses the problem of estimating from data a covariance matrix that satisfies the constraint of being locally stationary. We also show, with a real-data case study, the improvement in performance achieved by using locally stationary covariance matrices in the development of low cost quadratic detectors.

This paper shows how the sampling theorem relates with the variations along time of the second order statistics of L2(ℝ) nonstationary processes. As a consequence, and mainly due to the positive semidefiniteness of autocorrelation functions, it is possible to conclude if a nonstationary process is locally stationary (i.e., if its second order statistics vary slowly along time) by the direct observation of its 2-dimension power spectrum or its Wigner distribution. A simple example illustrates how two different strategies for the estimation of autocorrelation functions from a small number of data can lead to opposite results in terms of local stationarity.

In a previous paper W. Gardner (see IEEE Trans. Inform. Theory, vol.IT-18, p.808-9, 1972) developed a sampling theorem for nonstationary random processes, under the condition that the two-dimensional (2-D) power spectrum (2DPS) of the process has compact support. In this letter, it is shown that, for L/sup 2/(R) processes, only a one-dimensional (1-D) restriction on the marginal along time of the time-frequency distribution is necessary to guarantee the compactness of the 2DPS in the 2-D plane. As a direct consequence, it is observed that under mild conditions, a nonstationary autocorrelation function of a bandpass L/sup 2/(R) process is nearly stationary in small time intervals, The influence of this result in real-time detection of nonstationary stochastic signals is discussed.

In general, finite-dimensional discrete-time representations of continuous-time Gaussian transients is not complete. Such representations typically lead to suboptimal detectors, where the compromise between computational complexity and processor performance requires optimization, specially when real-time processing is mandatory. This paper proposes a procedure for the optimization of the processor parameters, using the Bhattacharyya distance to evaluate the resemblance between the original continuous-time signal and its finite dimensional discrete representation. Two different decompositions are analyzed and compared, namely the Karhunen-Loeve decomposition (KLD) and the discrete wavelet transform (DWT). It is shown that the DWT presents serious advantages when the signals to detect have a large number of important eigenvalues, which is often the case in some applications such as passive sonar.

Vaidyanathan (1999) and Ferreira (1999) reported on nonlinear systems with exponential eigenfunctions. In this letter, we introduce a larger class of such systems, generated from a special family of exponential-input/constant-output operators that capture exponential signals from general inputs. In particular, we extend the classes of systems with exponential eigenfunctions defined in Vaidyanathan and Ferreira to include causal nonhomogeneous time-varying systems.

In underwater acoustics, the modeling of impulsive noise ambients by symmetric-/spl alpha/-stable laws is motivated by the generalized central limit theorem. However, detection of stochastic signals under such additive noise is a difficult task to implement, due to the lack of a closed-form expression of the a-posteriori probability density function. We present a suboptimal detector for Gaussian bandpass transients in impulsive noise that uses a nonlinear, memoryless prefilter followed by a discrete wavelet transform. The resulting signals present a Gaussian-like behavior and the decision is achieved by the comparison of a quadratic likelihood ratio with a threshold. The tuning of the nonlinearity parameter is performed either by looking at the receiver operating characteristic or using the Chernoff distance, that, although resulting in an approximate solution, is easier to compute. Simulation results are presented by Monte-Carlo simulation.

The paper presents a wavelet transform based frequency classifier for stochastic bandpass transient signals. Using compactly supported bases of wavelets, continuous nonstationary bandpass processes can be described by reduced dimension discrete time signal representations. We develop an optimization procedure to adapt both the sampling frequency and the discrete time wavelet filters to the continuous classes of signals involved. When compared to the classical Bayesian structures, the proposed receiver strongly reduces the required computational load. The receiver performance is accessed by Monte Carlo simulation, the probabilities of detection for each signal class, and the probability of false alarm, being computed.