The stringent timing and reliability requirements in mission-critical applications require a detailed statistical characterization of end-to-end latency. Teleoperation is a representative use case, in which a human operator (HO) remotely controls a robot by exchanging command and feedback signals. We present a framework to analyze the latency of a closed-loop teleoperation system consisting of three entities: an HO, a robot located in remote environment, and a Base Station (BS) with Mobile edge Computing (MEC) capabilities. A model of each component is used to analyze the closed-loop latency and optimize the compression strategy. The closed-form expression of the distribution of the closed-loop latency is difficult to estimate, such that suitable upper and lower bounds are obtained. We formulate a non-convex optimization problem to minimize the closed-loop latency. Using the obtained upper and lower bound on the closed-loop latency, a computationally efficient procedure to optimize the closed-loop latency is presented. The simulation results reveal that compression of sensing data is not always beneficial, while system design based on average performance leads to under-provisioning and may cause performance degradation. The applicability of the proposed analysis is much wider than teleoperation, including a large class of systems whose latency budget consists of many components.

We present a framework to analyse the latency budget in wireless systems with Mobile Edge Computing (MEC). Our focus is on teleoperation and telerobotics, as use cases that are representative of mission-critical uplink-intensive IoT systems with requirements on low latency and high reliability. The study is motivated by a general question: What is the optimal compression strategy in reliability and latency constrained systems? We address this question by studying the latency of an uplink connection from a multi-sensor IoT device to the base station. This is a critical link tasked with a timely and reliable transfer of potentially significant amount of data from the multitude of sensors. We introduce a comprehensive model for the latency budget, incorporating data compression and data transmission. The uplink latency is a random variable whose distribution depends on the computational capabilities of the device and on the properties of the wireless link. We formulate two optimization problems corresponding to two transmission strategies: (1) Outage-constrained, and (2) Latency-constrained. We derive the optimal system parameters under a reliability criterion. We show that the obtained results are superior compared to the ones based on the optimization of the expected latency.

Humans systematically adapt their strategies for closed-loop control based on visual feedback according to the dynamics of the system. Tactile feedback is a key element in many human–machine interfaces, but it is not known if and how well human control adapts to changes in system dynamics when information about the system state is provided using this type of feedback. In this study, 11 participants tracked a pseudorandom trajectory with a virtual, position- or velocity-controlled plant using a joystick. Visual or electrotactile feedback provided the instantaneous error between the target and generated trajectory. Frequency-domain system identification indicated that human control adapted in similar ways to the different control modes (i.e., position/velocity control) for both feedback modalities. For the plant dynamics modeled as gain and integrator, the human controller behaved as a low-pass filter and gain, respectively (under the assumption of quasi-linear behavior). However, while tracking quality was largely similar for both control modes with visual feedback, velocity control enabled substantially worse control with electrotactile feedback compared to position control. Furthermore, for both control modes, the crossover frequency of open-loop transfer functions was lower for electrotactile feedback (0.9 and 1.1 rad/s) than for visual feedback (1.5 and 1.7 rad/s) indicating limited control bandwidth. To summarize, closed-loop control based on electrotactile feedback enables natural adaptations in human control strategy, which is encouraging for tactile feedback-controlled human–machine interfaces, but the lower control bandwidth and lower tracking quality with velocity control may impose functional limitations.

The performance of a classifier in a brain-computer interface (BCI) system is highly dependent on the quality and quantity of training data. Typically, the training data are collected in a laboratory where the users perform tasks in a controlled environment. However, users' attention may be diverted in real-life BCI applications and this may decrease the performance of the classifier. To improve the robustness of the classifier, additional data can be acquired in such conditions, but it is not practical to record electroencephalogram (EEG) data over several long calibration sessions. A potentially time- and cost-efficient solution is artificial data generation. Hence, in this study, we proposed a framework based on the deep convolutional generative adversarial networks (DCGANs) for generating artificial EEG to augment the training set in order to improve the performance of a BCI classifier. To make a comparative investigation, we designed a motor task experiment with diverted and focused attention conditions. We used an end-to-end deep convolutional neural network for classification between movement intention and rest using the data from 14 subjects. The results from the leave-one subject-out (LOO) classification yielded baseline accuracies of 73.04% for diverted attention and 80.09% for focused attention without data augmentation. Using the proposed DCGANs-based framework for augmentation, the results yielded a significant improvement of 7.32% for diverted attention ( ) and 5.45% for focused attention ( ). In addition, we implemented the method on the data set IVa from BCI competition III to distinguish different motor imagery tasks. The proposed method increased the accuracy by 3.57% ( ). This study shows that using GANs for EEG augmentation can significantly improve BCI performance, especially in real-life applications, whereby users' attention may be diverted.

Dexterous control of upper limb prostheses with multiarticulated wrists/hands is still a challenge due to the limitations of myoelectric man-machine interfaces. Multiple factors limit the overall performance and usability of these interfaces, such as the need to control degrees of freedom sequentially and not concurrently, and the inaccuracies in decoding the user intent from weak or fatigued muscles. In this article, we developed a novel man-machine interface that endows a myoelectric prosthesis (MYO) with artificial perception, estimation of user intention, and intelligent control (MYO-PACE) to continuously support the user with automation while preparing the prosthesis for grasping. We compared the MYO-PACE against state-of-the-art myoelectric control (pattern recognition) in laboratory and clinical tests. For this purpose, eight able-bodied and two amputee individuals performed a standard clinical test consisting of a series of manipulation tasks (portion of the SHAP test), as well as a more complex sequence of transfer tasks in a cluttered scene. In all tests, the subjects not only completed the trials faster using the MYO-PACE but also achieved more efficient myoelectric control. These results demonstrate that the implementation of advanced perception, context interpretation, and autonomous decision-making into active prostheses improves control dexterity. Moreover, it also effectively supports the user by speeding up the preshaping phase of the movement and decreasing muscle use.

Closing the prosthesis control loop by providing tactile sensory feedback to the user is a key point in research on active prosthetics as well as an often cited requirement of the prosthesis users. The live demo system includes: 1) electronic skin (e -skin) including a matrix of 64 sensing elements (taxels), 2) interface electronics for signal conditioning and data acquisition, 3) fully programmable multichannel electrotactile stimulator connected to flexible electrode matrices, and 4) host laptop PC which runs the online control loop implemented in Matlab.

Electronic skin can be integrated into a prosthetic device to endow the prosthesis with artificial cutaneous sensing, thereby partially restoring the sensory information lost due to an amputation. Non-invasive cutaneous electrostimulation transmits the tactile information sensed by the electronic skin on the prosthetic hand to the human brain, through the amputee's afferent nervous system. In this paper, our current benchtop prototype of a distributed sensing-stimulation system is presented, together with the envisaged high-fidelity solution which will be integrated into a real prosthetic hand.

Tactile feedback is pivotal for grasping and manipulation in humans. Providing functionally effective sensory feedback to prostheses users is an open challenge. Past paradigms were mostly based on vibro- or electrotactile stimulations. However, the tactile sensitivity on the targeted body parts (usually the forearm) is greatly less than that of the hand/fingertips, restricting the amount of information that can be provided through this channel. Visual feedback is the most investigated technique in motor learning studies, where it showed positive effects in learning both simple and complex tasks; however, it was not exploited in prosthetics due to technological limitations. Here, we investigated if visual information provided in the form of augmented reality (AR) feedback can be integrated by able-bodied participants in their sensorimotor control of a pick-and-lift task while controlling a robotic hand. For this purpose, we provided visual continuous feedback related to grip force and hand closure to the participants. Each variable was mapped to the length of one of the two ellipse axes visualized on the screen of wearable single-eye display AR glasses. We observed changes in behavior when subtle (i.e., not announced to the participants) manipulation of the AR feedback was introduced, which indicated that the participants integrated the artificial feedback within the sensorimotor control of the task. These results demonstrate that it is possible to deliver effective information through AR feedback in a compact and wearable fashion. This feedback modality may be exploited for delivering sensory feedback to amputees in a clinical scenario.