Recent advances in reinforcement learning (RL) have demonstrated impressive capabilities in complex decision-making tasks. This progress raises a natural question: how do these artificial systems compare to biological agents, which have been shaped by millions of years of evolution? To help answer this question, we undertake a comparative study of biological mice and RL agents in a predator-avoidance maze environment. Through this analysis, we identify a striking disparity: RL agents consistently demonstrate a lack of self-preservation instinct, readily risking ``death'' for marginal efficiency gains. These risk-taking strategies are in contrast to biological agents, which exhibit sophisticated risk-assessment and avoidance behaviors. Towards bridging this gap between the biological and artificial, we propose two novel mechanisms that encourage more naturalistic risk-avoidance behaviors in RL agents. Our approach leads to the emergence of naturalistic behaviors, including strategic environment assessment, cautious path planning, and predator avoidance patterns that closely mirror those observed in biological systems.

This paper presents an active search trajectory synthesis technique for autonomous mobile robots with nonlinear measurements and dynamics. The presented approach uses the ergodicity of a planned trajectory with respect to an expected information density map to close the loop during search. The ergodic control algorithm does not rely on discretization of the search or action spaces and is well posed for coverage with respect to the expected information density whether the information is diffuse or localized, thus trading off between exploration and exploitation in a single-objective function. As a demonstration, we use a robotic electrolocation platform to estimate location and size parameters describing static targets in an underwater environment. Our results demonstrate that the ergodic exploration of distributed information algorithm outperforms commonly used information-oriented controllers, particularly when distractions are present.

Tracking the position of moving objects requires tight coordination of sensing and movement, in both biological contexts such as prey pursuit and capture, and in target localization by mobile robots. Algorithms for target tracking often use a probabilistic map, or information map, of the domain to guide active search. Though it is reasonable to expect that the best approach would be to choose control actions driving the robot toward the maximum of this information map, we show improved performance in simulation by using a simple heuristic incorporating the time history of robot movement into the map. Furthermore, our results indicate that as the distribution of robot positions approaches the distribution of the density of information, the variance of the estimate is decreased and tracking improves. We conclude that control actions based solely on information maximization may under-perform in information orientated tasks, such as the estimation of moving target positions.

This paper presents an algorithm for active search where the goal is to calculate optimal trajectories for autonomous robots during data acquisition tasks. Formulating the problem as parameter estimation enables us to use Fisher information to create an explicit connection between robot dynamics and the informative regions of the search space. We use optimal control to automate design of trajectories that spend time in regions proportional to the probability of collecting informative data and use acquired data to update the probability closed-loop. Experimental and simulated results use a robotic electrosense platform to localize a feature in one-dimension. We demonstrate that this method is robust with respect to disturbances and initial conditions, and results in successful localization of the feature with a 100% experimental success rate and a 34% reduction in localization time compared to the next best tested controller.

Certain electric fish use a self-generated AC electric field to navigate and hunt. Thousands of sensors on the surface of the fish's body detect the pattern of amplitude and phase distortions of the field caused by nearby objects. Prior research has suggested that phase distortions may be especially useful for recognition of live objects. Here we present the first study of the utility of phase information in a robotic implementation of active electrosense. Using our robotic implementation, we investigated how the phase information depends on the frequency of the emitted field, the conductivity of the surrounding water, and object properties. An analytical model was developed serving as qualitative explanation of the dependency. We show that in certain situations phase information enables discrimination between two objects that are otherwise very similar in the amplitude of their electric images. We also show the utility of probing objects with multiple frequencies.

We have designed an underwater robot that uses perturbations of an emitted electric field to sense, localize, and map its environment. This system is inspired by weakly electric fish, which emit an electric field to sense objects, localize prey, and communicate. When nearby objects distort the electric field, electroreceptors (fish) or voltage sensors (robot) detect these perturbations. Further analysis of the perturbations can reveal information about the associated target, such as size, shape, and distance. One difficulty with extracting distance-to-target for our robotic electrosense platform is that the measurements are dependent on orientation of the robot with respect to the object. We solve this problem by applying techniques from range-only SLAM, with modifications for some of the ways in which electrosense differs from the sensors typically used. We use two different Bayesian filters to estimate the orientation and position separately. Using this approach, we show that our electrosense robot can accurately localize and orient itself, and improve its estimate of position and orientation using motion.

Few effective technologies exist for sensing in dark or murky underwater situations. For this reason, we have been exploring the use of a novel biologically-inspired approach to non-visual sensing based on the detection of perturbations to a self generated electric field. This is used by many species of neotropical nocturnal freshwater fish. This approach, termed active electrosense, presents unique challenges for sensing and tracking of nearby objects. We explore two methods for estimating the velocity of objects through active electrosense. The first of these methods uses a simple cross-correlation method, which depends on the uniformity of the electric field. We show some of the ramifications of making this assumption for a self-generated field around a cylindrical pod-shaped sensor in a rectangular tank. We then evaluate the use of methods developed for electrical impedance tomography (EIT) for localization and tracking. This is an unusual application of EIT in that typical applications involve surrounding the volume of interest (such as the thorax of humans) with sensor/emitters. Here, rather than this “outside in” approach, we are using EIT “inside out.” In simulation, we nonetheless find significant improvements in the accuracy of estimated velocity when using the EIT approach. Additionally, we demonstrate how EIT may be used for accurate position estimation. Under the conditions evaluated, the computation time for inversion is low enough to make its use feasible as a primary position and velocity estimator in an on- line system or as a secondary system to augment a computationally inexpensive estimator.

While there is accumulating evidence for the importance of the metabolic cost of information in sensory systems, how these costs are traded-off with movement when sensing is closely linked to movement is poorly understood. For example, if an animal needs to search a given amount of space beyond the range of its vision system, is it better to evolve a higher acuity visual system, or evolve a body movement system that can more rapidly move the body over that space? How is this trade-off dependent upon the three-dimensional shape of the field of sensory sensitivity (hereafter, sensorium)? How is it dependent upon sensorium mobility, either through rotation of the sensorium via muscles at the base of the sense organ (e.g., eye or pinna muscles) or neck rotation, or by whole body movement through space? Here we show that in an aquatic model system, the electric fish, a choice to swim in a more inefficient manner during prey search results in a higher prey encounter rate due to better sensory performance. The increase in prey encounter rate more than counterbalances the additional energy expended in swimming inefficiently. The reduction of swimming efficiency for improved sensing arises because positioning the sensory receptor surface to scan more space per unit time results in an increase in the area of the body pushing through the fluid, increasing wasteful body drag forces. We show that the improvement in sensory performance that occurs with the costly repositioning of the body depends upon having an elongated sensorium shape. Finally, we show that if the fish was able to reorient their sensorium independent of body movement, as fish with movable eyes can, there would be significant energy savings. This provides insight into the ubiquity of sensory organ mobility in animal design. This study exposes important links between the morphology of the sensorium, sensorium mobility, and behavioral strategy for maximally extracting energy from the environment. An "infomechanical" approach to complex behavior helps to elucidate how animals distribute functions across sensory systems and movement systems with their diverse energy loads.

We explore the capabilities of a robot designed to locate objects underwater through active movement of an electric field emitter and sensor apparatus. The robot is inspired by the biological phenomenon of active electrolocation, a sensing strategy found in two groups of freshwater fishes known to emit weak electric fields for target localization and communication. We characterize the performance of the robot using several types of automatic electrolocation controllers, objects, and water conditions. We demonstrate successful electrolocation both in the conditions in which it is naturally observed, in low conductivity water, as well as in conditions in which it is not observed, in water of ocean salinity. The belief of the position of the target is maintained via a particle filter and refined with each measurement

We present experimental results of thrust produced by a robotic propulsor, the design of which is inspired by the ribbon fin of the South American black ghost knifefish (Apteronotus albifrons). This remarkably nimble fish moves by oscillating its ribbon fin rays out of phase and thereby passing a traveling wave along the fin's length. Combinations of thrust from the ribbon fin and body rolls produced by the two pectoral fins enable the black ghost to swim in nearly any direction without bending its body. The fish's agile locomotor system is tightly integrated with its omnidirectional, active sensing system. The robotic ribbon fin has eight individually actuated metal rays which are linked by a thin latex sheet. The experimental results demonstrate the effect of varying the propulsive wave's frequency, amplitude and length on the robotic fin's thrust production. We found that thrust production peaks at particular combinations of the three variables and that the fin could produce steady forward thrust, despite the relatively small number of rays. The robotic ribbon fin has potential application as a propulsor for future underwater vehicles, in addition to being a valuable scientific instrument in understanding the swimming mechanics of the black ghost and similar fish