When dealing with the haptic teleoperation of multi-limbed mobile manipulators, the problem of mitigating the destabilizing effects arising from the communication link between the haptic device and the remote robot has not been properly addressed. In this work, we propose a passive control architecture to haptically teleoperate a legged mobile manipulator, while remaining stable in the presence of time delays and frequency mismatches in the master and slave controllers. At the master side, a discrete-time energy modulation of the control input is proposed. At the slave side, passivity constraints are included in an optimization-based whole-body controller to satisfy the energy limitations. A hybrid teleoperation scheme allows the human operator to remotely operate the robot’s end- effector while in stance mode, and its base velocity in locomotion mode. The resulting control architecture is demonstrated on a quadrupedal robot with an artificial delay added to the network.

During robot-assisted therapy of hemiplegic patients, interaction with the patient must be intrinsically safe. Straight-forward collision avoidance solutions can provide this safety requirement with conservative margins. These margins heavily reduce the robot's workspace and make interaction with the patient's unguided body parts impossible. However, interaction with the own body is highly beneficial from a therapeutic point of view. We tackle this problem by combining haptic rendering techniques with classical computer vision methods. Our proposed solution consists of a pipeline that builds collision objects from point clouds in real-time and a controller that renders haptic interaction. The raw sensor data is processed to overcome noise and occlusion problems. Our proposed approach is validated on the 6 DoF exoskeleton ANYexo for direct impacts, sliding scenarios, and dynamic collision surfaces. The results show that this method has the potential to successfully prevent collisions and allow haptic interaction for highly dynamic environments. We believe that this work significantly adds to the usability of current exoskeletons by enabling virtual haptic interaction with the patient's body parts in human-robot therapy.

This article proposes an application-layer multiplexing scheme for teleoperation systems with multimodal feedback (video, audio, and haptics). The available transmission resources are carefully allocated to avoid delay-jitter for the haptic signal potentially caused by the size and arrival time of the video and audio data. The multiplexing scheme gives high priority to the haptic signal and applies a preemptive-resume scheduling strategy to stream the audio and video data. The proposed approach estimates the available transmission rate in real time and adapts the video bitrate, data throughput, and force buffer size accordingly. Furthermore, the proposed scheme detects sudden transmission rate drops and applies congestion control to avoid abrupt delay increases and converge promptly to the altered transmission rate. The performance of the proposed scheme is measured objectively in terms of end-to-end signal latencies, packet rates, and peak signal-to-noise ratio (PSNR) for visual quality. Moreover, peak-delay and convergence time measurements are carried out to investigate the performance of the congestion control mode of the system.

We propose a perceptual haptic data reduction approach for teleoperation systems which use the time domain passivity approach (TDPA) as their control architecture for dealing with time-varying communication delay. Our goal is to reduce the packet rate over the communication network while preserving system stability in the presence of time-varying and unknown delays. Compared to the existing wave variable-based (WV-based) haptic data reduction approaches, our proposed scheme leads to smaller distortion in the force signals and robustly deals with time-varying delays. Experiments show that our proposed approach can reduce the average packet rate by up to 80%, without introducing significant distortion. In addition, the proposed approach outperforms the existing WV-based approaches in both packet rate reduction and subjective preference for the tested communication delays.

In this paper, we propose a macro-block (MB) level rate control algorithm for low delay H.264/AVC video communication based on the ρ domain rate model. In the proposed algorithm, an exponential model is used to characterize the relation between ρ and the quantization step (Qstep) at the MB level, with which the quantization parameter (QP) for a MB can be obtained. Furthermore, a switched QP calculation scheme is introduced to obtain the QP for each MB to avoid large deviation of the actual frame size from the target bit budget. Compared with the original ρ domain rate control, the proposed method can achieve better video quality and improved bit-rate accuracy. Meanwhile, the computational complexity is also significantly reduced.

Video synchronization is an essential processing step in many multimedia applications, and various methods have been proposed in the literature each of which addresses the problem from a different point of vantage. In this article, we present an information theoretic approach to video synchronization, based on the state-of-the-art in hybrid video coding. Time series derived from the videos' instantaneous bitrate demand are correlated in a robust manner employing the recently published ConCor algorithm. We enhance ConCor with integrated normalization capabilities in order to improve its shape-oriented matching performance. Furthermore, we present a mathematical framework to derive the most suitable ConCor parameters given a specific class of input videos. In an extensive experimental analysis, we give an insight into the representation of synchronization-relevant scene changes with bitrate data, and examine the influence of encoding parameters on the synchronization performance. Experiments on diverse video input substantiate the reliable performance of our easy to implement, yet effective video synchronization algorithm which distinguishes itself in that it operates largely without manual intervention.

Vibrotactile signals convey the touch-based characteristics of object surfaces felt through a tool. They particularly enhance the quality of human-machine interactions by providing realistic haptic perception of textures. In this paper, inspired by the similarities observed between vibrotactile texture signals and speech signals, we present a novel vibrotactile texture codec for bilateral teleoperation, based on well-known speech coding techniques. The proposed low bitrate, high quality codec preserves not only the spectral signature vital to the general feel of the texture, but also important temporal features of the texture signal. We report a compression ratio of 8:1 (12.5 %) with a constant output bitrate of 4 kbps, and we validate the perceptual transparency of the codec via rigorous subjective tests and analyses.

Consensus-based Cross-correlation (ConCor) is a recently presented algorithm for robust synchronization of noisy and corrupted signals. ConCor has a number of interdependent parameters that need to be set correctly to guarantee good performance. In this paper we analyse the effects of the individual parameters on ConCor's behaviour and performance. As a second contribution, we show that a parameter sweep with subsequent majority voting can be used to boost ConCor's performance and produce a trustworthy confidence measure. As a final contribution we show how the proposed extension also allows performing multi-modal (joint audio-video) synchronization of casual multi-perspective video recordings enabling superior matching performance.

In H.264 standard, coding of motion vectors constitutes a significant portion of total bitrate especially at low bitrate regimes. This is because differential coding of motion vectors is inefficient when the bit budget is very low. In this paper, we propose a novel estimation and coding algorithm for motion vectors of B-frames at very low bitrates. In this method, the encoder selects the optimal motion vector from a limited set of candidate vectors that are determined at the decoder side using true motion estimation. Since these candidate vector sets are fixed by the decoder for each macroblock, there is no need for explicit coding of motion information, which reduces the bitrate required for coding. Also, true motion vector estimates are used for improved direct mode coding in B-frames. The algorithm provides an average of 0.68 dB PSNR gain for B-frames when compared to the reference H.264 results at the same bitrates. Simulation results also indicate significant improvement in visual quality of the compressed B-frames.