Robot end effectors with proximity detection and contact sensing capabilities can reactively position the gripper to align objects and ensure successful grasps. In this paper, we introduce SonicFinger, an acoustic aura based sensing system capable of full-surface pre-touch and contact sensing. A single piezoelectric transducer embedded within a novel 3D printed finger is excited using a monotone to create an acoustic aura encompassing the finger; this enables pre-touch sensing and gripper alignment, while changes in finger-transducer acoustic coupling indicate contact. SonicFinger is low-cost, compact, and easy to manufacture and assemble. Sensing capabilities are evaluated using a set of objects with various physical properties such as optical reflectivity, dielectric constants, mechanical properties, and acoustic absorption. A dataset with over 8,000 proximity and contact events is collected. Our system shows a pre-touch detection true positive rate (TPR) of 92.4% and a true negative rate (TNR) of 95.3%. Contact detection experiments show a TPR of 93.7% and a TNR of 98.7%. Furthermore, pretouch detection information from Sonic Finger is used to adjust the robot grippers pose to align a target object at the center of both fingers.

Contact sensing and localization capabilities that mimic human skin are highly desirable for robots. In this paper, we introduce AcouSkin, an acoustic wave based full surface contact localization system. Acoustic waves produced by piezoelectric transceivers using a monotone are coupled to surfaces turning them into an active sensor. Our system leverages information from four piezoelectric transceivers mounted on the surface of an acrylic sheet and vacuum cleaner robot bumper to localize contacts to 18 unique segments. We first characterize acoustic wave propagation based on signal and material properties and then propose hardware and software methods to realize full surface contact localization. Our results show that AcouSkin can reliably localize contact on a flat acrylic sheet with 18 uniformly spaced locations across a 54cm length with mean absolute error (MAE) of ≤ 1 locations using maximum likelihood estimator (MLE) and multilayer perceptron (MLP) models. On the vacuum cleaner robot bumper AcouSkin shows a zero MAE. Further, the system is also able to localize contacts made using forces as low as 2N (Newtons) and as high as 20N. Overall, AcouSkin provides full surface contact localization while requiring minimal instrumentation with easy deployment on real-world robots.

In this paper, we present AmbiSense, an acoustic field based sensing system that performs proximity detection and bearing estimation for safer physical human-robot interactions. A single low cost piezoelectric transducer is used to setup this novel acoustic sensing modality to create a blindspot-free sound field engulfing a robot arm. Two detection algorithms leveraging spectral information from reflected audio waves of objects entering the acoustic field are proposed to infer object presence and bearing. We also present a new receiver structure which improves signal to noise ratio (SNR). AmbiSense is paired with a collision avoidance inverse kinematic solver for real world deployment on a Kinova Gen3 robot. Validation is performed using ten test objects generating 2000 proximity and bearing estimation events in real world settings, we show that AmbiSense detects proximity with 93.8% sensitivity and 96.6 % specificity. It estimates bearing and maps it to three zones on a robot link with 100% sensitivity and specificity, while using fewer sensors than state of the art methods for similar coverage.

Humans have the ability to pour various media, both liquid and granular, to desired ends in various containers. We do this by using multiple senses simultaneously in a constant feedback loop to complete a pouring task. Combining multiple sensing modalities, similar to humans, could aid in robotic pouring control outside of a structured or industrial setting. We present a multi-sensory pouring dataset consisting of human pouring demonstrations of various granular media, coupled with two multi-sensory networks that estimate pouring rate and pouring average height. For both pouring metrics, a combined input of audio and visual data provides a lower median error than either the audio network or visual network. The multi-sensory network achieves a median error of 6.4 mm for average height estimation and 0.06 N/s for pouring rate estimation.

Perceiving obstacles and avoiding collisions is fundamental to the safe operation of a robot system, particularly when the robot must operate in highly dynamic human environments. Proximity detection using on-robot sensors can be used to avoid or mitigate impending collisions. However, existing proximity sensing methods are orientation and placement dependent, resulting in blind spots even with large numbers of sensors. In this paper, we introduce the phenomenon of the Leaky Surface Wave (LSW), a novel sensing modality, and present AuraSense, a proximity detection system using the LSW. AuraSense is the first system to realize no-dead-spot proximity sensing for robot arms. It requires only a single pair of piezoelectric transducers, and can easily be applied to off-the-shelf robots with minimal modifications. We further introduce a set of signal processing techniques and a lightweight neural network to address the unique challenges in using the LSW for proximity sensing. Finally, we demonstrate a prototype system consisting of a single piezoelectric element pair on a robot manipulator, which validates our design. We conducted several micro benchmark experiments and performed more than 2000 on-robot proximity detection trials with various potential robot arm materials, colliding objects, approach patterns, and robot movement patterns. AuraSense achieves 100% and 95.3% true positive proximity detection rates when the arm approaches static and mobile obstacles respectively, with a true negative rate over 99%, showing the real-world viability of this system.

Collision detection is critical for safe robot operation in the presence of humans. Acoustic information originating from collisions between robots and objects provides opportunities for fast collision detection and localization; however, audio information from microphones on robot manipulators needs to be robustly differentiated from motors and external noise sources. In this paper, we present Panotti, the first system to efficiently detect and localize on-robot collisions using low-cost microphones. We present a novel algorithm that can localize the source of a collision with centimeter level accuracy and is also able to reject false detections using a robust spectral filtering scheme. Our method is scalable, easy to deploy, and enables safe and efficient control for robot manipulator applications. We implement and demonstrate a prototype that consists of 8 miniature microphones on a 7 degree of freedom (DOF) manipulator to validate our design. Extensive experiments show that Panotti realizes near perfect on-robot true positive collision detection rate with almost zero false detections even in high noise environments. In terms of accuracy, it achieves an average localization error of less than 3.8 cm under various experimental settings.

This paper compares the performance of several classifier algorithms on a standard database of handwritten digits. We consider not only raw accuracy, but also training time, recognition time, and memory requirements. When available, we report measurements of the fraction of patterns that must be rejected so that the remaining patterns have misclassification rates less than a given threshold.

We compare the performance of three types of neural network-based ensemble techniques to that of a single neural network. The ensemble algorithms are two versions of boosting and committees of neural networks trained independently. For each of the four algorithms, we experimentally determine the test and training error curves in an optical character recognition (OCR) problem as both a function of training set size and computational cost using three architectures. We show that a single machine is best for small training set size while for large training set size some version of boosting is best. However, for a given computational cost, boosting is always best. Furthermore, we show a surprising result for the original boosting algorithm: namely, that as the training set size increases, the training error decreases until it asymptotes to the test error rate. This has potential implications in the search for better training algorithms.

A neural network with 136000 connections for recognition of handwritten digits has been implemented using a mixed analog/digital neural network chip. The neural network chip is capable of processing 1000 characters/s. The recognition system has essentially the same rate (5%) as a simulation of the network with 32-b floating-point precision.

Hardware architectures for character recognition are discussed, and choices for possible circuits are outlined. An advanced (and working) reconfigurable neural-net chip that mixes analog and digital processing is described. It is found that different approaches to image recognition often lead to neural-net architectures that have limited connectivity and repeated use of the same set of weights. This architecture is ideal for time-multiplexing (a combined parallel-series processing) on hardware systems that would be too small to evaluate the entire network in parallel. To make this process efficient, a chip needs to have shift registers to format the input data and additional registers to store intermediate results. Within this framework, it is possible to design chips that have broad utility, large connection capacity, and high speed. This was demonstrated by a new chip with 32000 reconfigurable connections

The ability of learning networks to generalize can be greatly enhanced by providing constraints from the task domain. This paper demonstrates how such constraints can be integrated into a backpropagation network through the architecture of the network. This approach has been successfully applied to the recognition of handwritten zip code digits provided by the U.S. Postal Service. A single network learns the entire recognition operation, going from the normalized image of the character to the final classification.