Human employs different touch patterns to convey diverse social messages; for example, a stroke is an encouragement, whereas a hit is an offense. Various tactile sensors have been developed to grant an intuitive physical interaction with a robotic system, yet many encountered limitations in achieving broad sensibility or fabricating into a large skin. This paper presents a robotic skin with multimodal tactile sensing modules to achieve broad spatiotemporal sensibility with a few sensing elements. The multimodal module is composed of a microphone and a vented screw installed on a conductive sensory domain. A multilayered fabric with a textured surface covers the sensory domain and forms a piezoresistive structure. High and low temporal components of touch elicit a micro-vibration and a conductivity change on the skin, where both are measured with multimodal modules. The measurements are each processed with short-time Fourier transform (STFT) and electrical resistance tomography (ERT) to encode two spatiotemporal feature maps, which are classified into ten touch classes using a convolutional neural network. Due to a sensibility to both high and low temporal components of touch, the skin classifies touches with an accuracy of 97.0 %, whereas only 84.7 % and 90.6 % are achieved when one type of feature map is used. Also, the skin is robust and beneficial in power consumption and fabrication since the multimodal modules are not exposed to an external stimulus and are sparsely distributed.

A human can socially interact in a non-verbal manner by understanding the intention behind a tactile stimulus. Patting on one’s back is one of tactile communications, which is considered as a sign of encouragement in most cultures. The majority of such tactile communication is carried out by a dynamic tactile on large passive body parts and differently interpreted by how and where on the body is touched. Thus any robotic system that physically interacts with a human requires a dynamic tactile sensor for further social interaction. This paper presents a large dynamic tactile sensor that could cover a robot’s passive body parts using a few sparsely distributed microphones to cover a large area in an efficient manner. A porous structured mesh, neoprene, and loop fabric are used to form a sensor’s skin that could well generate and transfer a signal to distributed microphones when a touch is introduced. TDOA source localisation algorithms are implemented to find the touch point locating in between the distributed microphones, and a simple convolutional neural network is trained to classify a type of the touch. A localising performance is qualitatively achieved in a testbed of the sensor and applied to a mannequin’s back to show the applicability, which classified a touch into six classes with an accuracy of 88 %.

Recognizing and predicting the movement and intention of the wearer in control of an exoskeleton robot is very challenging. It is difficult for exoskeleton robots, which measure and drive human movements, to interact with humans. Therefore, many different types of sensors are needed. When using various sensors, a data design is needed for effective sensing. An electromyographic (EMG) signal can be used to identify intended motion before the actual movement, and the delay time can be shortened via control of the exoskeleton robot. Before using a lower limb exoskeleton to help in walking, the aim of this work is to distinguish the walking environment and gait period using various sensors, including the surface electromyography (sEMG) sensor. For this purpose, a gait experiment was performed on four subjects using the ground reaction force, human-robot interaction force, and position sensors with sEMG sensors. The purpose of this paper is to show progress with the use of sEMG when recognizing walking environments and the gait period with other sensors. For effective data design, we used a combination of sensor types, sEMG sensor locations, and sEMG features. The results obtained using an individual mechanical sensor together with sEMG showed improvement compared to the case of using an individual sensor, and the combination of sEMG and position information showed the best performance in the same number of combinations of three sensors. When four sensor combinations were used, the environment classification accuracy was 96.1%, and the gait period classification accuracy was 97.8%. Vastus medialis (VM) and gastrocnemius (GAS) were the most effective combinations of two muscle types among the five sEMG sensor locations on the legs, and the results were 74.4% in pre-heel contact (preHC) and 71.7% in pre-toe-off (preTO) for environment classification, and 68.0% for gait period classification, when using only the sEMG sensor. The two effective sEMG feature combinations were “mean absolute value (MAV), zero crossings (ZC)” and “MAV, waveform length (WL)”, and the “MAV, ZC” results were 80.0%, 77.1%, and 75.5%. These results suggest that the sEMG signal can be effectively used to control an exoskeleton robot.