Vibrotactile signals contain rich haptic information about textured surfaces but their large data volume makes it a challenging task to transmit such signals to remote locations to create immersive and realistic user experiences. Inspired by the recent success of deep neural network (DNN) based autoencoder, we make the first attempt to apply autoencoder for lossy compression of haptic vibrotactile signals, where a convolutional neural network (CNN) and a rate-distortion (RD) function are used as the transform and cost functions, respectively. Performance comparisons with state-of-the-art methods using both peak signal-to-noise ratio (PSNR) and perceptually motivated spectral temporal similarity (ST-SIM) measures show that the proposed end-to-end vibrotactile autoencoder (EVA) is highly competitive at preserving signal quality while keeping the data rate low.

Developing a signal compression technique that is able to achieve a low bit rate while maintaining high perceptual signal quality is a classical signal processing problem vigorously studied for audio, speech, image, and video type of signals. Yet, until recently, there has been limited effort directed toward the compression of vibrotactile signals, which represent a crucial element of rich touch (haptic) information. A vibrotactile signal; produced when stroking a textured surface with a tool-tip or bare-finger; like other signals contains a great deal of redundant and imperceptible information that can be exploited for efficient compression. This paper presents PVC-SLP, a vibrotactile perceptual coding approach. PVC-SLP employs a model of tactile sensitivity; called ASF (Acceleration Sensitivity Function); for perceptual coding. The ASF is inspired by the four channels model that mediate the perception of vibrotactile stimuli in the glabrous skin. The compression algorithm introduces sparsity constraints in a linear prediction scheme both on the residual and the predictor coefficients. The perceptual quantization of the residual is developed through the use of ASF. The quantization parameters of the residual and the predictor coefficients were jointly optimized; by means of both squared error and perceptual quality measures; to find the sweet spot of the rate-distortion curve. PVC-SLP coding performance is evaluated using two publicly available databases that collectively comprise 1281 vibrotactile signals covering 193 material classes. Furthermore, we compare PVC-SLP with a recent vibrotactile compression method and show that PVC-SLP perceptually outperforms existing method by a sizable margin. Most recently, PVC-SLP has been selected to become part of the haptic codec standard currently under preparation by IEEE P1918.1.1, aka Haptic Codecs for the Tactile Internet.

The Tactile Internet will enable users to physically explore remote environments and to make their skills available across distances. An important technological aspect in this context is the acquisition, compression, transmission, and display of haptic information. In this paper, we present the fundamentals and state of the art in haptic codec design for the Tactile Internet. The discussion covers both kinesthetic data reduction and tactile signal compression approaches. We put a special focus on how limitations of the human haptic perception system can be exploited for efficient perceptual coding of kinesthetic and tactile information. Further aspects addressed in this paper are the multiplexing of audio and video with haptic information and the quality evaluation of haptic communication solutions. Finally, we describe the current status of the ongoing IEEE standardization activity P1918.1.1 which has the ambition to standardize the first set of codecs for kinesthetic and tactile information exchange across communication networks.

Recent advances in haptic communication cast light onto the promise of full immersion into remote real or virtual environments. The quality of compressed haptic signals is crucial to fulfill this promise. Traditionally, the quality of haptic signals is evaluated through a series of subjective experiments. So far, only very limited attention was directed toward developing objective quality measures for haptic communication. As touch applications continue to grow, the need for efficient haptic quality assessment (HQA) measures is becoming more pronounced. In this work, we attempt to pragmatically approach this problem inspired by recent advances in the visual domain. More specifically, we design a HQA measure based on our knowledge of human haptic perception represented by Stevens power law and a similarity comparison between a reference and a compressed haptic force-feedback signal represented by the SSIM index as a full-reference quality measure. The proposed HSSIM measure provides clear advantages over existing approaches. We validate the performance of the proposed HQA measure with subjective experiments which show high correlation with human quality assessment results.

There has been a growing interest in image fusion technologies, but how to objectively evaluate the quality of fused images has not been fully understood. Here, we propose a method for objective quality assessment of multiexposure multifocus image fusion based on the evaluation of three key factors of fused image quality: 1) contrast preservation; 2) sharpness; and 3) structure preservation. Subjective experiments are conducted to create an image fusion database, based on which, performance evaluation shows that the proposed fusion quality index correlates well with subjective scores, and gives a significant improvement over the existing fusion quality measures.

Sharpness is an important determinant in visual assessment of image quality. The human visual system is able to effortlessly detect blur and evaluate sharpness of visual images, but the underlying mechanism is not fully understood. Existing blur/sharpness evaluation algorithms are mostly based on edge width, local gradient, or energy reduction of global/local high frequency content. Here we understand the subject from a different perspective, where sharpness is identified as strong local phase coherence (LPC) near distinctive image features evaluated in the complex wavelet transform domain. Previous LPC computation is restricted to be applied to complex coefficients spread in three consecutive dyadic scales in the scale-space. Here we propose a flexible framework that allows for LPC computation in arbitrary fractional scales. We then develop a new sharpness assessment algorithm without referencing the original image. We use four subject-rated publicly available image databases to test the proposed algorithm, which demonstrates competitive performance when compared with state-of-the-art algorithms.

Sharpness is one of the most determining factors in the perceptual assessment of image quality. Objective image sharpness measures may play important roles in the design and optimization of visual perception-based auto-focus systems and image enhancement, restoration and compression algorithms. Here we propose a new sharpness measure where sharpness is identified as strong local phase coherence evaluated in the complex wavelet transform domain. Our test using the LIVE blur database shows that the proposed algorithm correlates well with subjective quality evaluations. An additional advantage of our approach is that other image distortions such as compression, median filtering and noise contamination that may affect perceptual sharpness can also be detected.

The major challenges in automatic multi-sensor image registration are the inconsistency in intensity or contrast patterns, and the existence of partial or missing information between images. Here we propose a novel image registration method based on local phase coherence features, which are insensitive to changes in intensity or contrast. Furthermore, a new objective function based on weighted mutual information is proposed, where less weight is given to the objects that have no correspondence between images. The proposed method has been tested on both synthetic and medical images and evaluated based on registration accuracy. Our experiments demonstrate good performance of the proposed approach with missing or partial data, with significant changes in contrast, and with the presence of noise.