For multi-robot teams with limited communication, the ability to rapidly recognize the intention of a teammate via its exhibited behavior is key to achieving effective collaboration. While current research on plan and goal recognition provide powerful tools, most of them rely on a high-level abstraction of the environment and of its dynamics. We propose online waypoint recognition (OWR) that incorporates knowledge about the dynamic models into the analysis of the observed agent behavior. Our algorithm takes the form of a Kalman filter and performs recognition of the agent's intended waypoint at high frequency. The approach is robust to uncertainties in dynamics and observations. Moreover, it does not require the agent to reach the next waypoint to perform recognition, which saves valuable time. Our empirical evaluation shows the ability of our proposed algorithm to expedite recognition of both simulated and real-world mobile robots.

Research on underwater human-robot teaming is particularly promising because of complementary sensory skills that may overcome unique challenges associated with the combination of limited connectivity and low visibility. Nevertheless, testing human-robot collaboration under water, especially in complex, real-world scenarios, poses severe safety, cost, and time constraints that significantly hinder academic research in this space. This paper presents a novel cyber-physical extended-reality (XR) testbed, referred to as Underwater Real-Time Human Autonomous Systems Collaboration (UnRealTHASC), designed to enable human-robot interactions in simulated photorealistic environments with mixed real and virtual wearables, and advanced autonomous underwater sensors in-the-loop. Novel sensor interfaces are designed to integrate real and virtual sensors for measuring physiological and cognitive human states underpinning decision-making abilities. Physics-based human and robot motion models are developed along with new sensor simulations in order to capture the couplings between underwater behaviors and perception based on measurements from optical and sonar sensors. Real-time data acquisition pipelines are created to access and share data from both real and virtual sensors and robots, such that new methods for online planning and collaboration may be tested via human-in-the-loop demonstrations.

Traditional models of motor control typically operate in the domain of continuous signals such as spike rates, forces, and kinematics. However, there is growing evidence that precise spike timings encode significant information that coordinates and causally influences motor control. Some existing neural network models incorporate spike timing precision but they neither predict motor spikes coordinated across multiple motor units nor capture sensory-driven modulation of agile locomotor control. In this paper, we propose a visual encoder and model of a sensorimotor system based on a recurrent neural network (RNN) that utilizes spike timing encoding during smooth pursuit target tracking. We use this to predict a nearly complete, spike-resolved motor program of a hawkmoth that requires coordinated millisecond precision across 10 major flight motor units. Each motor unit enervates one muscle and utilizes both rate and timing encoding. Our model includes a motion detection mechanism inspired by the hawkmoth's compound eye, a convolutional encoder that compresses the sensory input, and a simple RNN that is sufficient to sequentially predict wingstroke-to-wingstroke modulation in millisecond-precise spike timings. The two-layer output architecture of the RNN separately predicts the occurrence and timing of each spike in the motor program. The dataset includes spikes recorded from all motor units during a tethered flight where the hawkmoth attends to a moving robotic flower, with a total of roughly 7000 wingstrokes from 16 trials on 5 hawkmoth subjects. Intra-trial and same-subject inter-trial predictions on the test data show that nearly every spike can be predicted within 2 ms of its known spike timing precision values. Whereas, spike occurrence prediction accuracy is about 90%. Overall, our model can predict the precise spike timing of a nearly complete motor program for hawkmoth flight with a precision comparable to that seen in agile flying insects. Such an encoding framework that captures visually-modulated precise spike timing codes and coordination can reveal how organisms process visual cues for agile movements. It can also drive the next generation of neuromorphic controllers for navigation in complex environments.

Information-driven control can be used to develop intelligent sensors that can optimize their measurement value based on environmental feedback. In object tracking applications, sensor actions are chosen based on the expected reduction in uncertainty also known as information gain. Random finite set (RFS) theory provides a formalism for quantifying and estimating information gain in multi-object tracking problems. However, estimating information gain in these applications remains computationally challenging. This paper presents a new tractable approximation of the RFS expected information gain applicable to sensor control for multi-object search and tracking. Unlike existing RFS approaches, the information gain approximation presented in this paper considers the contributions of non-ideal noisy measurements, missed detections, false alarms, and object appearance/disappearance. The effectiveness of the information-driven sensor control is demonstrated through two multi-vehicle search-while-tracking experiments using real video data from remote terrestrial and satellite sensors.

Spike train decoding is considered one of the grand challenges in reverse-engineering neural control systems as well as in the development of neuromorphic controllers. This paper presents a novel relative-time kernel design that accounts for not only individual spike train patterns, but also the relative spike timing between neuron pairs in the population. The new relative-time-kernel-based spike train decoding method proposed in this paper allows us to map the spike trains of a population of neurons onto a lower-dimensional manifold, in which continuous-time trajectories live. The effectiveness of our novel approach is demonstrated by comparing it with existing kernel-based and rate-based decoders, including the traditional reproducing kernel Hilbert space framework. In this paper, we use the data collected in hawk moth flower tracking experiments to test the importance of relative spike timing information for neural control, and focus on the problem of uncovering the mapping from the spike trains of ten primary flight muscles to the resulting forces and torques on the moth body. We show that our new relative-time-kernel-based decoder improves the prediction of the resulting forces and torques by up to 52.1 %. Our proposed relative-time-kernel-based decoder may be used to reverse-engineer neural control systems more accurately by incorporating precise relative spike timing information in spike trains.

Traditional cameras field of view (FOV) and resolution predetermine computer vision algorithm performance. These trade-offs decide the range and performance in computer vision algorithms. We present a novel foveating camera whose viewpoint is dynamically modulated by a programmable micro-electromechanical (MEMS) mirror, resulting in a natively high-angular resolution wide-FOV camera capable of densely and simultaneously imaging multiple regions of interest in a scene. We present calibrations, novel MEMS control algorithms, a real-time prototype, and comparisons in remote eye-tracking performance against a traditional smartphone, where high-angular resolution and wide-FOV are necessary, but traditionally unavailable.

The ability to anticipate human actions is critical to many cyber-physical systems, such as robots and autonomous vehicles. Computer vision and sensing algorithms to date have focused on extracting and predicting visual features that are explicit in the scene, such as color, appearance, actions, positions, and velocities, using video and physical measurements, such as object depth and motion. Human actions, however, are intrinsically influenced and motivated by many implicit factors such as context, human roles and interactions, past experience, and inner goals or intentions. For example, in a sport team, the team strategy, player role, and dynamic circumstances driven by the behavior of the opponents, all influence the actions of each player. This article proposes a holistic framework for incorporating visual features, as well as hidden information, such as social roles, and domain knowledge. The approach, relying on a novel dynamic Markov random field (DMRF) model, infers the instantaneous team strategy and, subsequently, the players’ roles that are temporally evolving throughout the game. The results from the DMRF inference stage are then integrated with instantaneous visual features, such as individual actions and position, in order to perform holistic action anticipation using a multi-layer perceptron (MLP). The approach is demonstrated on the team sport of volleyball, by first training the DMRF and MLP offline with past videos, and, then, by applying them to new volleyball videos online. These results show that the method is able to infer the players’ roles with an average accuracy of 86.99%, and anticipate future actions over a sequence of up to 46 frames with an average accuracy of 80.50%. Additionally, the method predicts the onset and duration of each action achieving a mean relative error of 14.57% and 15.67%, respectively.

Directional sensors, such as video cameras, have become ubiquitous to many autonomous robots applications, such as monitoring and surveillance. The performance of these sensors and processing algorithms, however, may be hindered by the presence of objects that block visibility. This article presents a novel approach for planning the path of a mobile directional sensor deployed to observe multiple targets distributed in an environment populated with multiple obstacles and occlusions. Unlike existing art gallery or watchman’s route methods, the visibility theory and motion planners developed in this article account for both line-of-sight visibility and bounded field-of-view constraints, and can provide obstacle avoidance based on robot geometry and kinodynamic constraints. The computational complexity analysis and experiments on a camera-equipped drone demonstrate that, despite the challenging geometric characteristics of directional C-targets, the approach scales to real-world problems. Furthermore, when compared to algorithms inspired by traveling salesman and target coverage approaches, the directional visibility planners presented in this article are significantly more effective both at guaranteeing complete target visibility and at minimizing distance traveled.

The role of negative information is particularly important to search-detect-track problems in which the number of objects is unknown a priori, and the size of the sensor field-of-view is far smaller than that of the region of interest. This paper presents an approach for systematically incorporating knowledge of the field-of-view geometry and position and object inclusion/exclusion evidence into object state densities and random finite set multi-object cardinality distributions. The approach is derived for a representative set of multi-object distributions and demonstrated through a sensor planning problem involving a multi-Bernoulli process with up to one-hundred potential targets.

Most cameras today photograph their entire visual field. In contrast, decades of active vision research have proposed foveating camera designs, which allow for selective scene viewing. However, active vision's impact is limited by slow options for mechanical camera movement. We propose a new design, called FoveaCam, and which works by capturing reflections off a tiny, fast moving mirror. FoveaCams can obtain high resolution imagery on multiple regions of interest, even if these are at different depths and viewing directions. We first discuss our prototype and optical calibration strategies. We then outline a control algorithm for the mirror to track target pairs. Finally, we demonstrate a practical application of the full system to enable eye tracking at a distance for frontal faces.

Bayesian nonparametric models, such as the Dirichlet Process Gaussian Process (DPGP), have been shown very effective at learning models of dynamic targets exclusively from data. Previous work on batch DPGP learning and inference, however, ceases to be efficient in multi-sensor applications that require decentralized measurements to be obtained sequentially over time. Batch processing, in this case, leads to redundant computations that may hinder online applicability. This paper develops a recursive approach for DPGP learning and inference in which a novel Dirichlet Process prior based on Wasserstein metric is used for measuring the similarity between multiple Gaussian Processes (GPs). Combined with the GP recursive fusion law, the proposed recursive DPGP fusion approach enables efficient online data fusion. The problem of active sensing for recursive DPGP learning and inference is also investigated by uncertainty reduction via expected mutual information. Simulation and experimental results show that the proposed approach successfully learns the models of moving targets and outperforms existing benchmark methods.

This paper develops a decentralized approach for mapping and information-driven path planning for Very Large Scale Robotic (VLSR) systems. In this approach, obstacle mapping is performed using a continuous probabilistic representation known as a Hilbert map, which formulates the mapping problem as a binary classification task and uses kernel logistic regression to train a discriminative classifier online. A novel Hilbert map fusion method is presented that quickly and efficiently combines the information from individual robot maps. An integrated mapping and path planning algorithm is presented to determine paths of maximum information value, while simultaneously performing obstacle avoidance. Furthermore, the effect of how percentage communication failure effects the overall performance of the system is investigated. The approach is demonstrated on a VLSR system with hundreds of robots that must map obstacles collaboratively over a large region of interest using onboard range sensors and no prior obstacle information. The results show that, through fusion and decentralized processing, the entropy of the map decreases over time and robot paths remain collision-free.

Recent developments in neural stimulation and recording technologies are providing scientists with the ability of recording and controlling the activity of individual neurons in vitro or in vivo, with very high spatial and temporal resolution. Tools such as optogenetics, for example, are having a significant impact in the neuroscience field by delivering optical firing control with the precision and spatiotemporal resolution required for investigating information processing and plasticity in biological brains. While a number of training algorithms have been developed to date for spiking neural network (SNN) models of biological neuronal circuits, exiting methods rely on learning rules that adjust the synaptic strengths (or weights) directly, in order to obtain the desired network-level (or functional-level) performance. As such, they are not applicable to modifying plasticity in biological neuronal circuits, in which synaptic strengths only change as a result of pre- and post-synaptic neuron firings or biological mechanisms beyond our control. This paper presents a weight-free training algorithm that relies solely on adjusting the spatiotemporal delivery of neuron firings in order to optimize the network performance. The proposed weight-free algorithm does not require any knowledge of the SNN model or its plasticity mechanisms. As a result, this training approach is potentially realizable in vitro or in vivo via neural stimulation and recording technologies, such as optogenetics and multielectrode arrays, and could be utilized to control plasticity at multiple scales of biological neuronal circuits. The approach is demonstrated by training SNNs with hundreds of units to control a virtual insect navigating in an unknown environment.

This paper presents a model-based approach for computing real-time optimal decision strategies in the pursuit-evasion game of Ms. Pac-Man. The game of Ms. Pac-Man is an excellent benchmark problem of pursuit-evasion game with multiple, active adversaries that adapt their pursuit policies based on Ms. Pac-Man's state and decisions. In addition to evading the adversaries, the agent must pursue multiple fixed and moving targets in an obstacle-populated environment. This paper presents a novel approach by which a decision-tree representation of all possible strategies is derived from the maze geometry and the dynamic equations of the adversaries or ghosts. The proposed models of ghost dynamics and decisions are validated through extensive numerical simulations. During the game, the decision tree is updated and used to determine optimal strategies in real time based on state estimates and game predictions obtained iteratively over time. The results show that the artificial player obtained by this approach is able to achieve high game scores, and to handle high game levels in which the characters speeds and maze complexity become challenging even for human players.

This paper develops a new indirect method for distributed optimal control (DOC) that is applicable to optimal planning for very-large-scale robotic (VLSR) systems in complex environments. The method is inspired by the nested analysis and design method known as generalized reduced gradient (GRG). The computational complexity analysis presented in this paper shows that the GRG method is significantly more efficient than classical optimal control or direct DOC methods. The GRG method is demonstrated for VLSR path planning in obstacle-populated environments in which robots are subject to external forces and disturbances. The results show that the method significantly improves performance compared to the existing direct DOC and stochastic gradient methods.

This paper presents a multi-layer reproducing kernel Hilbert space (RKHS) approach for probability distribution to real and probability distribution to function regressions. The approach maps the distributions into RKHS by distribution embeddings and, then, constructs a multi-layer RKHS within which the multi-kernel distribution regression can be implemented using an existing kernel regression algorithm, such as kernel recursive least squares (KRLS). The numerical simulations on synthetic data obtained via Gaussian mixtures show that the proposed approach outperforms existing probability distribution (DR) regression algorithms by achieving smaller mean squared errors (MSEs) and requiring less training samples.

This paper presents a novel constrained integration (CINT) method for solving initial boundary value partial differential equations (PDEs). The CINT method combines classical Galerkin methods with a constrained backpropogation training approach to obtain an artificial neural network representation of the PDE solution that approximately satisfies the boundary conditions at every integration step. The advantage of CINT over existing methods is that it is readily applicable to solving PDEs on irregular domains, and requires no special modification for domains with complex geometries. Furthermore, the CINT method provides a semi-analytical solution that is infinitely differentiable. In this paper the CINT method is demonstrated on two hyperbolic and one parabolic initial boundary value problems with a known analytical solutions that can be used for performance comparison. The numerical results show that, when compared to the most efficient finite element methods, the CINT method achieves significant improvements both in terms of computational time and accuracy.

This paper presents a camera control approach for learning unknown nonlinear target dynamics by approximating information value functions using particles that represent targets' position distributions. The target dynamics are described by a non-parametric mixture model that can learn a potentially infinite number of motion patterns. Assuming that each motion pattern can be represented as a velocity field, the target behaviors can be described by a non-parametric Dirichlet process-Gaussian process (DP-GP) mixture model. The DP-GP model has been successfully applied for clustering time-invariant spatial phenomena due to its flexibility to adapt to data complexity without overfitting. A new DP-GP information value function is presented that can be used by the sensor to explore and improve the DP-GP mixture model. The optimal camera control is computed to maximize this information value function online via a computationally efficient particle-based search method. The proposed approach is demonstrated through numerical simulations and hardware experiments in the RAVEN testbed at MIT.

This paper presents a visibility-based method for planning the motion of a mobile robotic sensor with bounded field-of-view to optimally track a moving target while localizing itself. The target and robot states are estimated from online sensor measurements and a set of a priori known landmarks, using an extended Kalman filter (EKF), and thus the proposed method is applicable to robots without a global positioning system. It is shown that the problem of optimizing the target tracking and robot localization performance is equivalent to optimizing the visibility or probability of detection in the EKF framework under mild assumptions. The control law that maximizes the probability of detection for a robotic sensor with a sector-shaped field-of-view (FoV) is derived as a function of the robot heading and aperture. Simulations have been conducted on synthetic experiments and the results show that the optimized-visibility approach is effective at avoiding target loss, and outperforms a state-of-the-art potential method based on robot trailer models [1].

There is considerable precedence in the sensor tracking and estimation literature for modeling maneuvering targets by Markov motion models in order to estimate the target state from multiple, distributed sensor measurements. Although the transition probability density functions of these Markov models are routinely outputted by tracking and estimation algorithms, little work has been done to use them in sensor coordination and control algorithms. This paper presents a geometric transversals approach for representing the probability of track detection by multiple, distributed sensors, as a function of the Markov model transition probabilities. By this approach, the Markov parameters of maneuvering targets that may be detected by the sensors are represented by three-dimensional cones that are finitely generated by the sensors fields-of-view in a spatiotemporal Euclidian space. Then, the problem of deploying a sensor network for the purpose of maximizing the expected number of target detections can be formulated as a nonlinear program that can be solved numerically for the optimal sensor placement. Numerical results show that the optimal sensor placements obtained by this geometric transversals approach significantly outperform greedy, grid, or randomized sensor deployments.

This paper presents a constrained backpropagation (CPROP) methodology for solving nonlinear elliptic and parabolic partial differential equations (PDEs) adaptively, subject to changes in the PDE parameters or external forcing. Unlike existing methods based on penalty functions or Lagrange multipliers, CPROP solves the constrained optimization problem associated with training a neural network to approximate the PDE solution by means of direct elimination. As a result, CPROP reduces the dimensionality of the optimization problem, while satisfying the equality constraints associated with the boundary and initial conditions exactly, at every iteration of the algorithm. The effectiveness of this method is demonstrated through several examples, including nonlinear elliptic and parabolic PDEs with changing parameters and nonhomogeneous terms.

This paper presents an information potential method for integrated path planning and control. The method is applicable to unicycle robotic sensors deployed to classify multiple targets in an obstacle-populated environment. A new navigation function, referred to as information potential, is generated from the target conditional mutual information, and used to design a closed-loop stable switched control law. The information potential is shown to obey the properties of potential navigation functions and to enable measurements that maximize the information value over time. The information potential is also used to construct a local roadmap for escaping local minima. The properties and computational complexity of the local roadmap algorithm are analyzed. Numerical simulation results show that the method outperforms other strategies, such as rapidly exploring random trees and classical potential field methods.

MATLAB is a popular computational system and programming environment that is used in numerous engineering and science programs in the United States. One feature of MATLAB is the capability to generate 3D visualizations, which can be used to visualize scientific data or even to simulate engineering models and processes. Unfortunately, MATLAB provides only limited interactivity for these visualizations. As a solution to this problem, we have developed a software system that easily integrates with MATLAB scripts to provide the capability to view visualizations and interact with them in virtual reality (VR) systems. We call this system “ML2VR” and expect it will introduce more users to VR by enabling a large population of MATLAB programmers to easily transition to immersive systems. We will describe the system architecture of ML2VR and report on a successful case study involving the use of ML2VR.

In some virtual reality (VR) systems, OpenGL intercept methods are used to capture and render a desktop application's OpenGL calls within an immersive display. These systems often suffer from lower frame rates due to network bandwidth limitations, implementation of the intercept routine, and in some cases, the intercepted application's frame rate. To mitigate these issues and to enhance intercept-based systems in other ways, we present intercept tags, which are OpenGL geometries that are interpreted instead of rendered. We have identified and developed several uses for intercept tags, including hand-off interactions, display techniques, and visual enhancements. To demonstrate the value of intercept tags, we conducted a user study to compare a simple virtual hand technique implemented with and without intercept tags. Our results show that intercept tags significantly improve user performance and experience.

This paper investigates the comparative performance of several information-driven search strategies and decision rules using a canonical target classification problem. Five sensor models are considered: one obtained from classical estimation theory and four obtained from Bernoulli, Poisson, binomial, and mixture-of-binomial distributions. A systematic approach is presented for deriving information functions that represent the expected utility of future sensor measurements from mutual information, Rènyi divergence, Kullback-Leibler divergence, information potential, quadratic entropy, and the Cauchy-Schwarz distance. The resulting information-driven strategies are compared to direct-search, alert-confirm, task-driven (TS), and log-likelihood-ratio (LLR) search strategies. Extensive numerical simulations show that quadratic entropy typically leads to the most effective search strategy with respect to correct-classification rates. In the presence of prior information, the quadratic-entropy-driven strategy also displays the lowest rate of false alarms. However, when prior information is absent or very noisy, TS and LLR strategies achieve the lowest false-alarm rates for the Bernoulli, mixture-of-binomial, and classical sensor models.

A novel artificial-potential approach is presented for planning the minimum-exposure paths of multiple vehicles in a dynamic environment containing multiple mobile sensors, and multiple fixed obstacles. This approach presents several advantages over existing techniques, such as the ability of computing multiple minimum-exposure paths online, while avoiding mutual collisions, as well as collisions with obstacles sensed during the motion. Other important advantages include the ability of utilizing heterogenous sensor models, and of meeting multiple objectives, such as minimizing power required, and reaching a set of goal configurations. The approach is demonstrated through numerical simulations involving autonomous underwater vehicles (AUVs) deployed in a region of interest near the New Jersey coast, with ocean currents simulated using real coastal ocean dynamics applications radar (CODAR) data.

A novel artificial-potential function approach is presented for planning the paths of distributed sensor networks in a complex dynamic environment. The approach implements a novel potential function generated from a probability density function (PDF) parameterized by an adaptive Gaussian mixture that is optimized to meet network-level objectives, such as cooperative track detection. The PDF represents the goal density that would be obtained by sampling a statistically-significant number of sensors from the mixture. However, since a smaller number of sensors may be deployed, and each sensor is represented by a disk, the potential function is generated by multiplying the PDF by a likelihood update model that produces networks with disjoint fields-of-view. The approach is demonstrated through numerical simulations involving ocean sensor networks deployed in a region of interest near the New Jersey coast.

The quality of service of a network performing cooperative track detection is represented by the probability of obtaining multiple elementary detections over time along a target track. Recently, two different lines of research, namely, distributed-search theory and geometric transversals, have been used in the literature for deriving the probability of track detection as a function of random and deterministic sensors' positions, respectively. In this paper, we prove that these two approaches are equivalent under the same problem formulation. Also, we present a new performance function that is derived by extending the geometric-transversal approach to the case of random sensors' positions using Poisson flats. As a result, a unified approach for addressing track detection in both deterministic and probabilistic sensor networks is obtained. The new performance function is validated through numerical simulations and is shown to bring about considerable computational savings for both deterministic and probabilistic sensor networks.

A constrained-backpropagation (CPROP) training technique is presented to solve Partial Differential Equations (PDEs). The technique is based on constrained optimization and minimizes an error function subject to a set of equality constraints, provided by the boundary conditions of the differential problem. As a result, sigmoidal neural networks can be trained to approximate the solution of PDEs avoiding the discontinuity in the derivative of the solution, which may affect the stability of classical methods. Also, the memory provided to the network through the constrained approach may be used to solve PDEs on line when the forcing term changes over time, learning different solutions of the differential problem through a continuous nonlinear mapping. The effectiveness of this method is demonstrated by solving a nonlinear PDE on a circular domain. When the underlying process changes subject to the same boundary conditions, the CPROP network is capable of adapting online and approximate the new solution, while memory of the boundary conditions is maintained virtually intact at all times.

The advantages brought about by using classical linear control theory in conjunction with neural approximators have long been recognized in the literature. In particular, using linear controllers to obtain the starting neural control design has been shown to be a key step for the successful development and implementation of adaptive-critic neural controllers. Despite their adaptive capabilities, neural controllers are often criticized for not providing the same performance and stability guarantees as classical linear designs. Therefore, this paper develops an algebraic synthesis procedure for designing dynamic output-feedback neural controllers that are closed-loop stable and meet the same performance objectives as any classical linear design. The performance synthesis problem is addressed by deriving implicit model-following algebraic relationships between model matrices, obtained from the classical design, and the neural control parameters. Additional linear matrix inequalities (LMIs) conditions for closed-loop exponential stability of the neural controller are derived using existing integral quadratic constraints (IQCs) for operators with repeated slope-restricted nonlinearities. The approach is demonstrated by designing a recurrent neural network controller for a highly maneuverable tailfin-controlled missile that meets multiple design objectives, including pole placement for transient tuning, H(infinity) and H(2) performance in the presence of parameter uncertainty, and command-input tracking.

A methodology is developed for planning the sensing strategy of a robotic sensor deployed for the purpose of classifying multiple fixed targets located in an obstacle-populated workspace. Existing path planning techniques are not directly applicable to robots whose primary objective is to gather sensor measurements using a bounded field of view (FOV). This paper develops a novel approximate cell-decomposition method in which obstacles, targets, sensor's platform, and FOV are represented as closed and bounded subsets of an Euclidean workspace. The method constructs a connectivity graph with observation cells that is pruned and transformed into a decision tree from which an optimal sensing strategy can be computed. The effectiveness of the optimal sensing strategies obtained by this methodology is demonstrated through a mine-hunting application. Numerical experiments show that these strategies outperform shortest path, complete coverage, random, and grid search strategies, and are applicable to nonoverpass capable robots that must avoid targets as well as obstacles.

This paper presents an information-driven sensor management problem, referred to as treasure hunt, which is relevant to mobile-sensor applications such as mine hunting, monitoring, and surveillance. The objective is to infer a hidden variable or treasure by selecting a sequence of measurements associated with multiple fixed targets distributed in the sensor workspace. The workspace is represented by a connectivity graph, where each node represents a possible sensor deployment, and the arcs represent possible sensor movements. An additive conditional entropy reduction function is presented to efficiently compute the expected benefit of a measurement sequence over time. Then, the optimal treasure hunt strategy is determined by a novel label-correcting algorithm operating on the connectivity graph. The methodology is illustrated through the board game of CLUE, which is shown to be a benchmark example of the treasure hunt problem. The game results show that a computer player implementing the strategies developed in this paper outperforms players implementing Bayesian networks, Q-learning, or constraint satisfaction, as well as human players.

The detective board game of CLUEregcan be viewed as a benchmark example of the treasure hunt problem, in which a sensor path is planned based on the expected value of information gathered from targets along the path. The sensor is viewed as an information gathering agent that makes imperfect measurements or observations from the targets, and uses them to infer one or more hidden variables (such as, target features or classification). The treasure hunt problem arises in many modern surveillance systems, such as demining and reconnaissance robotic sensors. Also, it arises in the board game of CLUEreg, where pawns must visit the rooms of a mansion to gather information from which the hidden cards can be inferred. In this paper, Q-learning is used to develop an automated neural computer player that plans the path of its pawn, makes suggestions about the hidden cards, and infers the answer, often winning the game. A neural network is trained to approximate the decision-value function representing the value of information, for which there exists no general closed-form representation. Bayesian inference, test (suggestions), and action (motion) decision making are unified using an MDP framework. The resulting computer player is shown to outperform other computer players implementing Bayesian networks, or constraint satisfaction.

Spiking neural networks have been shown capable of simulating sigmoidal artificial neural networks providing promising evidence that they too are universal function approximators. Spiking neural networks offer several advantages over sigmoidal networks, because they can approximate the dynamics of biological neuronal networks, and can potentially reproduce the computational speed observed in biological brains by enabling temporal coding. On the other hand, the effectiveness of spiking neural network training algorithms is still far removed from that exhibited by backpropagating sigmoidal neural networks. This paper presents a novel algorithm based on reward-modulated spike-timing-dependent plasticity that is biologically plausible and capable of training a spiking neural network to learn the exclusive-or (XOR) computation, through rate-based coding. The results show that a spiking neural network model with twenty-three nodes is able to learn the XOR gate accurately, and performs the computation on time scales of milliseconds. Moreover, the algorithm can potentially be verified in light-sensitive neuronal networks grown in vitro by determining the spikes patterns that lead to the desired synaptic weights computed in silico when induced by blue light in vitro.

A constrained-backpropagation training technique is presented to suppress interference and preserve prior knowledge in sigmoidal neural networks, while new information is learned incrementally. The technique is based on constrained optimization, and minimizes an error function subject to a set of equality constraints derived via an algebraic training approach. As a result, sigmoidal neural networks with long term procedural memory (also known as implicit knowledge) can be obtained and trained repeatedly on line, without experiencing interference. The generality and effectiveness of this approach is demonstrated through three applications, namely, function approximation, solution of differential equations, and system identification. The results show that the long term memory is maintained virtually intact, and may lead to computational savings because the implicit knowledge provides a lasting performance baseline for the neural network.

This paper presents a new coverage formulation addressing the quality of service of sensor networks that cooperatively detect targets traversing a region of interest. The problem of track coverage consists of finding the positions of n sensors such that a Lebesgue measure on the set of tracks detected by at least k sensors is optimized. This paper studies the geometric properties of the network, addressing a deterministic track-coverage formulation and binary sensor models. It is shown that the tracks detected by a network of heterogeneous omnidirectional sensors are the geometric transversals of non-translates families of circles. A novel methodology based on cone theory is presented for representing and measuring sets of transversals in closed-form. Then, the solution of the track-coverage problem can be formulated as a nonlinear program (NLP). The numerical results show that this approach can improve track coverage by up to two orders of magnitude compared to grid and random deployments. Also, it can be used to reduce the number of sensors required to achieve a desired detection performance by up to 50%, and to optimally replenish or reposition existing sensor networks.

In this paper, a supervised neural network training technique based on constrained optimization is developed for preserving prior knowledge of an input-output mapping during repeated incremental training sessions. The prior knowledge, referred to as long-term memory (LTM), is expressed in the form of equality constraints obtained by means of an algebraic training technique. Incremental training, which may be used to learn new short-term memories (STMs) online, is then formulated as an error minimization problem subject to equality constraints. The solution of this problem is simplified by implementing an adjoined error gradient that circumvents direct substitution and exploits classical backpropagation. A target application is neural network function approximation in adaptive critic designs. For illustrative purposes, constrained training is implemented to update an adaptive critic flight controller, while preserving prior knowledge of an established performance baseline that consists of classical gain-scheduled controllers. It is shown both analytically and numerically that the LTM is accurately preserved while the controller is repeatedly trained over time to assimilate new STMs.

A constrained approximate dynamic programming (ADP) approach is presented for designing adaptive neural network (NN) controllers with closed-loop stability and performance guarantees. Prior knowledge of the linearized equations of motion is used to guarantee that the closed-loop system meets performance and stability objectives when the plant operates in a linear parameter-varying (LPV) regime. In the presence of unmodeled dynamics or failures, the NN controller adapts to optimize its performance online, whereas constrained ADP guarantees that the LPV baseline performance is preserved at all times. The effectiveness of an adaptive NN flight controller is demonstrated for simulated control failures, parameter variations, and near-stall dynamics.

An algebraic approach for representing multidimensional nonlinear functions by feedforward neural networks is presented. In this paper, the approach is implemented for the approximation of smooth batch data containing the function's input, output, and possibly, gradient information. The training set is associated to the network adjustable parameters by nonlinear weight equations. The cascade structure of these equations reveals that they can be treated as sets of linear systems. Hence, the training process and the network approximation properties can be investigated via linear algebra. Four algorithms are developed to achieve exact or approximate matching of input-output and/or gradient-based training sets. Their application to the design of forward and feedback neurocontrollers shows that algebraic training is characterized by faster execution speeds and better generalization properties than contemporary optimization techniques.