This study presents a sensing system composed of PVDF-based sensing arrays to recognize an object’s hardness and texture. The system is mounted on the fingertip of the Hannes prosthetic hand, which obtains the texture and hardness features by grasping daily-life objects. Time-domain features are extracted from the sensor’s response and evaluated through the Support Vector Machine (SVM) algorithm. Additionally, two deep learning (DL) algorithms including One-Dimensional Convolutional Neural Network (1-D CNN) and Long-Short Term Memory (LSTM) were implemented. Results demonstrated that the 1-D CNN attained the highest recognition accuracy 90% ≈ for both hardness and texture. Moreover, it demonstrated that the hardness information is integrated into the full grasp action, while the texture information can be extracted using the first contact with the object. Deploying the 1-D CNN into a microcontroller proved that the system is energy efficient and capable of extracting tactile information in real-time. This study demonstrated the effectiveness of the proposed system in recognizing object properties highlighting its robustness and suitability for developing prosthetics.

This paper presents the implementation of machine learning algorithms on GAP9 for real-time tactile data processing. The case study is object recognition based on multisensory glove data. A shallow 1D-CNN model was employed to first process the data, then deployed in the GAP SDK simulation environment (GVSoC), and various metrics were extracted to evaluate the GAP9 in this task. The results showed a latency of 85.37 μs. Additionally, memory profiling showed that no clock cycles were wasted waiting for pure I/O operations, since the model was allocated 23.98kB (20.7%) of the L1 cache and 13.96kB (1.07%) of L2 without any need for buffering between caches or external memory access operations. The GAP9, using our model, has a dynamic energy consumption of 6.76 pJ per inference. The results demonstrate the effectiveness of the proposed methodology, opening up an interesting perspective for various biomedical applications.

In this article, we propose a radar-based human action recognition system, capable of recognizing actions in real time. Range-Doppler maps extracted from a low-cost frequency-modulated continuous wave (FMCW) radar are fed into a deep neural network. The system is deployed on an edge device. The results show that the system can recognize five human actions with an accuracy of 93.2% and an inference time of 2.95 s. Raising an alarm when a harmful action happens is a crucial feature in an indoor safety application. Thus, the performance during the binary classification, i.e., fall vs nonfall actions, is also assessed, achieving an accuracy of 96.8% with a false-negative rate of 4%. To find the best tradeoff between accuracy and computational cost, the energy precision ratio of the system deployed on the edge is measured. The system achieves a 1.04 energy precision ratio value, where an ideal ratio would be close to zero.

Robotic grippers can be equipped with tactile sensing systems to extract information from a manipulated object. The real-time classification of the physical properties of a grasped object on resource-constrained devices requires efficient and effective pre-processing techniques and machine-learning (ML) algorithms. In this paper, we propose a tactile sensing system mounted on the Baxter robot for the hardness classification of objects. In particular, we pre-processed the raw data with low computational cost techniques, and we designed three ML algorithms to provide real-time, energy-efficient, and low-memory impact classification on a resource-constrained microcontroller. Results show that convolutional neural networks (CNNs) achieve the best accuracy (>98%), while the support vector machine (SVM) presents the lowest memory occupation (1576 bytes), inference time (<0.077ms), and energy consumption (<5.74μJ).

This work presents a method for tactile classification of materials for virtual reality (VR) based robot teleoperation. In our system, a human-operator uses a remotely controlled robot-manipulator with an optical fibre-based tactile and proximity sensor to scan surfaces of objects in a remote environment. Tactile and proximity data and the robot's end-effector state feedback are used for the classification of objects' materials which are then visualized in the VR reconstruction of the remote environment for each object. Machine learning techniques such as random forest, convolutional neural and multi-modal convolutional neural networks were used for material classification. The proposed system and methods were tested with five different materials and classification accuracy of 90 % and more was achieved. The results of material classification were successfully exploited for visualising the remote scene in the VR interface to provide more information to the human-operator.

Tactile object shapes are considered as important properties in robotic manipulation. Many researches have focused recently on using tactile sensing systems to enable tactile information processing in robotics. Spiking Neural Networks (SNNs) are emerging as promising methods alternative to deep learning due to their ability to process information in an event-driven manner. In this paper, we propose a SNN architecture and hardware implementation for tactile object shapes recognition. The network is fed by an array of 160 piezoresistive tactile sensors where the object shapes are applied. Results demonstrate that the proposed system is able to discriminate the tactile object shapes with 100% accuracy on unseen data having time steps up to 0.1 ms. Moreover, the network has been implemented on a Raspberry Pi platform achieving real time classification.

Robots and users of hand prosthesis could easily manipulate objects if endowed with the sense of touch. Towards this goal, information about touched objects and surfaces has to be inferred from raw data coming from the sensors. An important cue for objects discrimination is the orientation of edges, that is used both in artificial vision and touch as pre-processing stage. We present a spiking neural network, inspired on the encoding of edges in human first order tactile afferents. The network uses three layers of Leaky Integrate and Fire neurons to distinguish different edge orientations of a bar pressed on the artificial skin of the iCub robot. The architecture is successfully able to discriminate eight different orientations (from 0oto 180o), by implementing a structured model of overlapping receptive fields. We demonstrate that the network can learn the appropriate connectivity through unsupervised spike based learning, and that the number and spatial distribution of sensitive areas within the receptive fields are important in edge orientation discrimination.

Employing Machine learning algorithms in tactile sensing systems have emerged recently to recognize/classify touch patterns. The high computational complexity of the ML algorithms makes challenging the embedded implementation of tactile data processing. This paper proposes a complexity optimized tensorial-based machine learning algorithm for touch modality classification. The aim is to introduce an efficient algorithm minimizing the system complexity in terms of number of operations and memory storage which directly affect time latency and power consumption. With respect to the state of the art, the proposed approach reduces the number of operations per inference from 545 M-ops to 18 M-ops and the memory storage from 52.2 KB to 1.7 KB. Moreover, the proposed method speeds up the inference time by a factor of 43× at a cost of only 2% loss in accuracy.

We explored rate mode control for virtual reality (VR) based robot teleoperation with constant and variable mapping of the human-operator’s joystick position to the speed (rate) of the robot’s end-effector. The variable mapping depended on the visual scale of the virtual reconstruction of the remote environment to the scale of the real remote environment. We demonstrated how the rate mode control and variable scaling based on the VR reconstruction scale can be efficiently used for seated VR based robot teleoperation when the operator’s arms are supported to reduce tiredness. The experimental study with five human participants demonstrated that variable mapping allowed participants to teleoperate the robot more effectively, by adjusting the VR visual scale albeit at a cost of increased perceived workload.

This paper presents a novel hardware architecture of the Tensorial Support Vector Machine (TSVM) based on Shallow Neural Networks (NN) for the Single Value Decomposition (SVD) computation. The proposed NN achieves a comparable Mean Squared Error and Cosine Similarity to the widely used one-sided Jacobi algorithm. When implemented on an FPGA, the NN offers$324\times $faster computations than the one-sided Jacobi with reductions up to 58% and 67% in terms of hardware resources and power consumption respectively. When validated on a touch modality classification problem, the NN-based TSVM implementation has achieved a real-time operation while consuming about 88% less energy per classification than the Jacobi-based TSVM with an accuracy loss of at most 3%. Such results offer the ability to deploy intelligence on resource-limited platform for energy-constrained applications.

This work describes a virtual reality (VR) based robot teleoperation framework which relies on scene visualization from depth cameras and implements human-robot and human-scene interaction gestures. We suggest that mounting a camera on a slave robot's end-effector (an in-hand camera) allows the operator to achieve better visualization of the remote scene and improve task performance. We compared experimentally the operator's ability to understand the remote environment in different visualization modes: single external static camera, in-hand camera, in-hand and external static camera, in-hand camera with OctoMap occupancy mapping. The latter option provided the operator with a better understanding of the remote environment whilst requiring relatively small communication bandwidth. Consequently, we propose suitable grasping methods compatible with the VR based teleoperation with the in-hand camera. Video demonstration: https://youtu.be/3vZaEykMS_E.

Electronic-skin aiming to mimic human skin is becoming a reality and systems able to process data close to the sensors are required to reduce latency and power consumption. This paper presents the design and implementation of an energy efficient smart system for tactile sensing based on a RISC-V parallel ultra-low power platform (PULP). The PULP processor, called Mr. Wolf, performs the on-board classification of different touch modalities. This demonstrates the promising use of on-board classification for emerging robot and prosthetic applications. Experimental results demonstrate the effectiveness of the platform on improving the energy efficiency of the online classification. In our experiments, Mr. Wolf runs 3.6 times faster than an ARM Cortex M4F (STM32F40), consuming only 28 mW. The proposed platform achieves 15× better energy efficiency, than the classification done on the STM32F40, consuming only 81mJ per classification.

The lack of tactile sensation in current upper limb prostheses is the key limitation to enable more intuitive use of the prosthesis and restore the functionality of the natural limb. Electronic skin in the form of tactile sensing arrays can be integrated onto the upper limb prosthetic device to record information about touch, given back to the amputee as a sensory feedback. This contribution focusses on the development of tactile sensing arrays based on PVDF-TrFE piezoelectric polymers. Both piezoelectric polymer sensors and metal contacts have been screen-printed on a transparent plastic foil. The geometric layout and size of the 4×4 array is suitable for prosthesis fingertips. Preliminary experimental validation of the skin patches is reported in this paper.

Closing the prosthesis control loop by providing tactile sensory feedback to the user is a key point in research on active prosthetics as well as an often cited requirement of the prosthesis users. The live demo system includes: 1) electronic skin (e -skin) including a matrix of 64 sensing elements (taxels), 2) interface electronics for signal conditioning and data acquisition, 3) fully programmable multichannel electrotactile stimulator connected to flexible electrode matrices, and 4) host laptop PC which runs the online control loop implemented in Matlab.

Developing portable autonomous systems is highly requested for numerous application domains such as Internet of Things (IoT), wearable devices, and biomedical applications. Portable systems usually contain autonomous and networked sensors; each sensor hosts multiple input channels (e.g. tactile) closely coupled to embedded computing unit and power supply. The embedded computing unit should locally extract meaningful information by employing sophisticated methods. This imposes challenges on real time operation and adds a burden regarding power consumption. Approximate or inexact computing represents a promising solution for energy efficient data processing; it tunes the accuracy of computation on the specific application requirements in order to reduce power consumption. In this paper, inexact arithmetic circuits have been employed to improve the energy efficiency for sensors digital data processing. The proposed inexact circuits achieve up to 80% power saving when compared to the exact one, and similar solutions presented in literature with a maximum loss of 1.39 dB in output SNR when applied to FIR filters.

Electronic skin can be integrated into a prosthetic device to endow the prosthesis with artificial cutaneous sensing, thereby partially restoring the sensory information lost due to an amputation. Non-invasive cutaneous electrostimulation transmits the tactile information sensed by the electronic skin on the prosthetic hand to the human brain, through the amputee's afferent nervous system. In this paper, our current benchtop prototype of a distributed sensing-stimulation system is presented, together with the envisaged high-fidelity solution which will be integrated into a real prosthetic hand.

We present an event-driven tactile sensing element that encodes both the absolute value of the input force and its variation over time. It is based on the POSFET device and Leaky-Integrate and Fire neurons, connected by a transconductance amplifier; the proposed circuit exploits the advantages of the POSFET device, such as high integration scale, fast response, wide bandwidth and force sensitivity, as well as the advantages of event-driven encoding, such as low latency, low power dissipation, and high temporal resolution, coupled with redundancy reduction.

In this work, we report a novel circuit architecture to implement event-driven tactile sensing using the POSFET tactile device. The proposed circuit matches advantages of the POSFET device (integration of sensing and electronics on the same die, high electromechanical transduction bandwidth, etc.) with the ones of the event-driven approach. In the proposed circuit, the POSFET device is interfaced with a spiking neuron, of the type integrate and fire: the input mechanical stimulus is translated into digital pulses. The proposed approach paves the way for the implementation of neuromorphic integrated tactile sensing systems based on POSFET devices.

Tactile sensing is considered a key technology for implementing complex robot interaction tasks. The contribution of this article is two-fold: (i) we propose a general-purpose algorithm for the reconstruction of deformation and force distributions for capacitance-based skin-like systems; (ii) real-time performance can be tuned according to available computational resources, which leads to an any-time formulation. Experiments (both in simulation and with real robot skin) provide a quantitative analysis of results.

The two major components of a robotic tactile-sensing system are the tactile-sensing hardware at the lower level and the computational/software tools at the higher level. Focusing on the latter, this research assesses the suitability of computational-intelligence (CI) tools for tactile-data processing. In this context, this paper addresses the classification of sensed object material from the recorded raw tactile data. For this purpose, three CI paradigms, namely, the support-vector machine (SVM), regularized least square (RLS), and regularized extreme learning machine (RELM), have been employed, and their performance is compared for the said task. The comparative analysis shows that SVM provides the best tradeoff between classification accuracy and computational complexity of the classification algorithm. Experimental results indicate that the CI tools are effective in dealing with the challenging problem of material classification.

This work presents and experimentally evaluates novel POSFET (Piezoelectric Oxide Semiconductor Field Effect Transistor) devices based tactile sensing arrays. The arrays, primarily developed for the robotic applications, consist of 5 × 5 POSFET touch sensing devices or taxels. The POSFET touch sensing devices are developed by spin coating piezoelectric polymer P(VDF-TrFE) film on the gate area of MOS devices and polarizing the film in situ. To detect contact events, the taxels utilize the contact forces induced change in the polarization level (and hence change in the induced channel current) of piezoelectric polymer. Both, individual taxels and the array are designed to match spatio-temporal performance of the human fingertips. Experimental results demonstrate that the POSFET tactile sensing arrays presented here are able to detect complex dynamic contact events such as rolling of an object.

This work presents the development of novel POSFET (Piezoelectric Oxide Semiconductor Field Effect Transistor) devices based tactile sensing system. The tactile sensing system, primarily developed for the robotic applications, consists of 5×5 POSFET touch sensing array and the associated read out and data acquisition system. The POSFET touch sensing devices are obtained by spin coating piezoelectric polymer P(VDF-TrFE), poly(vinylidene fluoride - trifluoroethylene), film on the gate area of MOS (Metal Oxide Semiconductor) devices and polarizing the film in situ. To detect contact events, the taxels utilize the contact forces induced change in the polarization level (and hence change in the induced channel current) of piezoelectric polymer. Both, individual taxels and the array are designed to match spatio-temporal performance of the human fingertips. The data acquisition system is implemented with off-the-shelf electronic components and its design takes into account both the application related requirements as well as the constraints posed by existing hardware on the humanoid robot `iCub'. The biasing scheme for using POSFET devices and the problems thereof are also been discussed.

Starting from human ¿sense of touch,¿ this paper reviews the state of tactile sensing in robotics. The physiology, coding, and transferring tactile data and perceptual importance of the ¿sense of touch¿ in humans are discussed. Following this, a number of design hints derived for robotic tactile sensing are presented. Various technologies and transduction methods used to improve the touch sense capability of robots are presented. Tactile sensing, focused to fingertips and hands until past decade or so, has now been extended to whole body, even though many issues remain open. Trend and methods to develop tactile sensing arrays for various body sites are presented. Finally, various system issues that keep tactile sensing away from widespread utility are discussed.

Wireless sensor networks (WSNs) simulation is essential for the development and optimization of such kind of networks. In this paper we present modifications made to prowler, a WSNs simulator, making its MAC protocol model compliant with the crossbow MICAz mote running on Tiny OS. Moreover, we added radio propagation models to the simulator and, most important, we implemented an energy consumption estimation model of the CC2420 radio chip. Then, the simulator is addressed to the development of WSNs applications with a particular emphasis on energy consumption optimization (and consequently battery lifetime). We present as case study the comparison between simulated results of two different physical deployments, showing how real operating conditions can affect and modify the system behavior in comparison with the results predicted by theoretical analysis.

Probability density function (PDF) estimation is a very critical task in many applications of data analysis. For example in the Bayesian framework decisions are taken according to Bayes' rule, which directly involves the evaluation of the PDF. Many methods are available to this aim, but there is no consensus in the literature about which to use, nor about the pros and cons of each of them. In this paper, we present a thorough and extensive experimental comparison between two of the most popular methods: Parzen window and finite Gaussian mixture. Extended experimental results and application development guidelines are reported

We present and discuss the major results of our research activity aimed to the analog VLSI implementation of on-chip learning neural networks. In particular we present the SLANP (self learning neural processor) chip results. The SLANP architecture implements an on-chip learning multilayer perceptron network. The learning algorithm is based on the back propagation but it exhibits increased capabilities due to the local learning rate management. A prototype chip has been designed and fabricated in a CMOS 0.7 /spl mu/m minimum channel length technology. The experimental results confirm the functionality of the chip and the soundness of the approach. The SLANP performance compares favorably with that reported in the literature.

In this paper we present the analog on-chip learning architecture of a gradient descent learning algorithm: the weight perturbation learning algorithm. From the circuit implementation point of view our approach is based on current mode and translinear operated circuits. The proposed architecture is very efficient in terms of speed, size, precision and power consumption; moreover it exhibits also high scalability and modularity.

Gradient descent learning algorithms (namely backpropagation and weight perturbation) can significantly increase their classification performances by adopting a local and adaptive learning rate management approach. We present the results of the comparison of the classification performance of the two algorithms in a tough application: quality control analysis in the steel industry. The feedforward network is hierarchically organized (i.e. tree of multilayer perceptrons). The comparison has been performed starting from the same operating conditions (i.e. network topology, stopping criterion, etc.): the results show that the probability of correct classification is significantly better for the weight perturbation algorithm.

In this paper we present the analog on-chip learning architecture of a gradient descent learning algorithm: the Weight Perturbation learning algorithm. From the circuit implementation point of view our approach is based on current mode and translinear operated circuits. The proposed architecture is very efficient in terms of speed, size, precision and power consumption; moreover it exhibits also high scalability and modularity.

The CMOS circuit implementation of the feedforward neural primitives of a generic Multi Layer Perceptron network is presented. Basically our approach is based on current mode computation and is aimed at a low power/low voltage circuit implementation; moreover, it is easily scalable to implement networks of any size. Experimental results are reported.

An architectural configuration for the back-propagation (BP) algorithm is illustrated. The circuit solution for the basic blocks is presented, and the effect of weight discretization on the BP algorithm is analyzed. It is demonstrated, through simulations, how the BP algorithm can be operated successfully with discretized weights. In particular, better performances can be achieved with an exponential discretization, i.e. the strength of weights varies exponentially with the controlling variable (voltage). The discretized voltage values differ by a quantity high enough that the neural network can be backed up with a refresh technique in combination with a multilevel dynamic memory that entails a particularly low wiring cost. A quasi-analog adaptive architecture is devised, properly matching the BP algorithm, and its CMOS circuit implementation is detailed. The mechanism controlling weight changes is simple enough to be reproduced locally at each synapsis site, thus meeting one of the requirements for an efficient storage technology for analog VLSI