A perceptual deadzone of a reference force, quantified in terms of the just noticeable difference (JND), is a region where any change in the force with respect to the reference force is not perceived. This perceptual deadzone is employed for efficient encoding of the haptic data through an adaptive sampling mechanism. However, for multi-dimensional force stimuli, the effect of the magnitude and direction of the stimuli on the JND has not been investigated yet, and thus, the structure of the deadzone is yet to be determined. This article aims to investigate the perceptual deadzone for 2D haptic force. To achieve this, we designed a psychophysical experiment in which a subject experiences a 2D kinesthetic force stimulus via a force-feedback device and is asked to respond to the change. We employ a machine learning classification-based approach to analyze the \(\textrm{2D}\) perceptual deadzone on collected haptic responses (perceived and non-perceived) of 12 subjects. Our hypothesis is that a classifier optimized on the collected responses, which yields the least prediction error, will define the structure of the deadzone. The characteristics of the optimal deadzone structure have been analyzed in terms of shape, effect of force direction, and symmetry. The results demonstrate that the deadzone is direction variant and radially asymmetric. Thus, both the force magnitude and direction affect force perception. These findings are significant for the fundamental understanding of multi-dimensional force perception by humans, which can be crucial for enhancing haptic force-feedback control systems in virtual reality or robotic controls.

In the literature, perceptual deadband approach based on the Weber’s law of perception has been employed to reduce haptic data (i.e., force) rate for a typical teleoperation application. The approach selects only those samples for transmission which lie outside the perceptual deadband. The existing structure of the deadband has linear decision boundaries and assume that the just noticeable differences (JNDs) for increasing and decreasing change in a reference force stimulus are similar. This paper questions this assumption and searches for an asymmetric perceptual deadband using a data-driven approach. For the purpose, we design an experimental setup for collecting human haptic responses (perceived and non-perceived) of several users for a force range [3, 5] N. A machine learning classifier inspired from the Weber’s law is trained to predict the labels of the responses and define a generalized linear perceptual deadband for each user. The results show that the generalized deadband do provide different increasing and decreasing JND (i.e., pointing towards asymmetry in force perception), but fails to improve the data reduction significantly as compared to the existing symmetric one.

The IEEE “Tactile Internet” (TI) Standards working group (WG), designated the numbering IEEE 1918.1, undertakes pioneering work on the development of standards for the TI. This paper describes the WG, its intentions, and its developing baseline standard and the associated reasoning behind that and touches on a further standard already initiated under its scope: IEEE 1918.1.1 on “Haptic Codecs for the TI.” IEEE 1918.1 and its baseline standard aim to set the framework and act as the foundations for the TI, thereby also serving as a basis for further standards developed on TI within the WG. This paper discusses the aspects of the framework such as its created TI architecture, including the elements, functions, interfaces, and other considerations therein, as well as the novel aspects and differentiating factors compared with, e.g., 5G Ultra-Reliable Low-Latency Communication, where it is noted that the TI will likely operate as an overlay on other networks or combinations of networks. Key foundations of the WG and its baseline standard are also highlighted, including the intended use cases and associated requirements that the standard must serve, and the TI's fundamental definition and assumptions as understood by the WG, among other aspects.

The Tactile Internet will enable users to physically explore remote environments and to make their skills available across distances. An important technological aspect in this context is the acquisition, compression, transmission, and display of haptic information. In this paper, we present the fundamentals and state of the art in haptic codec design for the Tactile Internet. The discussion covers both kinesthetic data reduction and tactile signal compression approaches. We put a special focus on how limitations of the human haptic perception system can be exploited for efficient perceptual coding of kinesthetic and tactile information. Further aspects addressed in this paper are the multiplexing of audio and video with haptic information and the quality evaluation of haptic communication solutions. Finally, we describe the current status of the ongoing IEEE standardization activity P1918.1.1 which has the ambition to standardize the first set of codecs for kinesthetic and tactile information exchange across communication networks.

Media technology, in particular video recording and playback, keeps improving to provide users with high-quality real and virtual visual content. In recent years, increasing the temporal sampling rate of videos and the refresh rate of displays has become one focus of technical innovation. This raises the question, how high the sampling and refresh rates should be? To answer this question, we determine the minimum temporal sampling rate at which a video should be presented to make temporal sampling imperceptible to viewers. Through a psychophysical study, we find that this minimum sampling rate depends on both the speed of the objects in the image plane and the exposure time of the recording camera. We propose a model to compute the required minimum sampling rate based on these two parameters. In addition, state-of-the-art video codecs employ motion vectors from which the local object movement speed can be inferred. Therefore, we present a procedure to compute the minimum sampling rate given an encoded video and camera exposure time. Since the object motion speed in a video may vary, the corresponding minimum frame rate is also varying. This is why the results of this paper are particularly applicable when used together with adaptive frame rate computer generated graphics or novel video communication solutions that drop insignificant frames. In our experiments, we show that videos played back at the minimum adaptive frame rate achieve an average bit rate reduction of 26% compared to constant frame rate playback, while perceptually no difference can be observed.

Assessing grasp stability is essential to prevent the failure of robotic manipulation tasks due to sensory data and object uncertainties. Learning-based approaches are widely deployed to infer the success of a grasp. Typically, the underlying model used to estimate the grasp stability is trained for a specific task, such as lifting, hand-over, or pouring. Since every task has individual stability demands, it is important to adapt the trained model to new manipulation actions. If the same trained model is directly applied to a new task, unnecessary grasp adaptations might be triggered, or in the worst case, the manipulation might fail. To address this issue, we divide the manipulation task used for training into seven sub-tasks, defined as modular tasks. We deploy a learning-based approach and assess the stability for each modular task separately. We further propose analytical features to reduce the dimensionality and the redundancy of the tactile sensor readings. A main task can thereby be represented as a sequence of relevant modular tasks. The stability prediction of the main task is computed based on the inferred success labels of the modular tasks. Our experimental evaluation shows that the proposed feature set lowers the prediction error up to 5.69% compared to other sets used in state-of-the-art methods. Robotic experiments demonstrate that our modular task-oriented stability assessment avoids unnecessary grasp force adaptations and regrasps for various manipulation tasks.

This paper presents a method that automatically transforms a virtual environment (VE) according to the mood of input music. We use machine learning to extract a mood from the music. We then select images exhibiting the mood and transfer their styles to the textures of objects in the VE photorealistically or artistically. Our user study results indicate that our method is effective in transferring valence-related aspects, but not arousal-related ones. Our method can still provide novel experiences in virtual reality and speed up the production of VEs by automating its procedure.

Perceptually adaptive sampling based on Weber's law has been used to reduce the haptic packet rate for efficient haptic data transmission over Internet. In a teleoperation, perceptually significant force samples are transmitted from a robot to a human operator. If the time spacing between two consecutive perceptually sampled kinesthetic force stimuli is less than the minimum time spacing (temporal resolution Tr) required in perceiving the jump discontinuity, then the second force stimulus will not be perceived even if it is well above the just noticeable difference. Hence, there is no need to transmit the second force sample to the operator. Thus, for the transmission in a teleoperation, the temporal resolution Trneeds also to be considered while effecting perceptually adaptive sampling. To the best of our knowledge, it has not been studied in the literature. In this work, we propose a statistical method to estimate the temporal resolution Tr. In order to achieve this, we design an appropriate experimental set up, and record the haptic responses for several users extensively. We also study the effect of perceptual fatigue on the temporal resolution Tr, and validate all results using the classical psychometric approach.

In this article, we identify adaptive sampling strategies for haptic signals. Our approach relies on experiments wherein we record the response of several users to haptic stimuli. We then learn different classifiers to predict the user response based on a variety of causal signal features. The classifiers that have good prediction accuracy serve as candidates to be used in adaptive sampling. We compare the resultant adaptive samplers based on their rate-distortion tradeoff using synthetic as well as natural data. For our experiments, we use a haptic device with a maximum force level of 3 N and 10 users. Each user is subjected to several piecewise constant haptic signals and is required to click a button whenever he perceives a change in the signal. For classification, we not only use classifiers based on level crossings and Weber’s law but also random forests using a variety of causal signal features. The random forest typically yields the best prediction accuracy and a study of the importance of variables suggests that the level crossings and Weber’s classifier features are most dominant. The classifiers based on level crossings and Weber’s law have good accuracy (more than 90%) and are only marginally inferior to random forests. The level crossings classifier consistently outperforms the one based on Weber’s law even though the gap is small. Given their simple parametric form, the level crossings and Weber’s law--based classifiers are good candidates to be used for adaptive sampling. We study their rate-distortion performance and find that the level crossing sampler is superior. For example, for haptic signals obtained while exploring various rendered objects, for an average sampling rate of 10 samples per second, the level crossings adaptive sampler has a mean square error about 3dB less than the Weber sampler.