Accurate cognitive load (CL) detection during virtual reality (VR) locomotion is critical for enhancing user experience and improving interaction design. Traditional CL assessment methods, such as self-reports and physiological measures, face challenges in VR environments. Eye tracking has shown potential as a reliable indicator of CL across various human-computer interaction (HCI) tasks. It offers significant promise as a discriminative feature for predictive models in VR. This study explores the feasibility of detecting CL induced by VR locomotion using an explainable machine-learning approach along with eye-tracking techniques. A comparative user study employing a within-subjects design evaluated five unique gait-free locomotion techniques. Statistical analysis revealed distinct CL levels across these locomotion techniques. Several machine learning models were developed for CL detection using eye-tracking data, with the Light Gradient Boosting Machine (LightGBM) achieving the highest accuracy of 0.78. The SHAP approach was employed to analyze the importance of features to provide interpretability, offering insights into the machine learning model's decision-making process. Our findings highlight the potential of using eye-tracking-based machine learning techniques as a practical approach for cognitive load detection in VR, contributing to the growing research in multimedia analytics, human perception, and user intent within immersive environments. Additionally, our work demonstrates how eye-tracking data can be leveraged to improve user interactions and optimize immersive multimedia experiences based on cognitive load analysis.

We introduce a model-free approach to predict the future state of the ball and learn the appropriate stroke accordingly for robotic table tennis. Based on the gated recurrent unit (GRU) and the encoder-decoder (ED), a GRU-ED approach is developed for predicting the future state (position, velocity and acceleration) of the ball when observing a partial trajectory. By taking as input the predicted state at hitting time, we learn an appropriate stroke movement with a model-free reinforcement learning (RL) approach. The experimental results show that the proposed approach outperforms others in trajectory prediction. Acceleration and spin of the ball provide an equivalent effect in learning an accurate stroke motion. An additional experiment conducted with a real table tennis robot shows that the robot can accurately hit the ball and return the ball to the desired target with a pretrained RL model.

Reinforcement learning (RL) has achieved some impressive recent successes in various computer games and simulations. Most of these successes are based on having large numbers of episodes from which the agent can learn. In typical robotic applications, however, the number of feasible attempts is very limited. In this paper we present a sample-efficient RL algorithm applied to the example of a table tennis robot. In table tennis every stroke is different, with varying placement, speed and spin. An accurate return therefore has to be found depending on a high-dimensional continuous state space. To make learning in few trials possible the method is embedded into our robot system. In this way we can use a one-step environment. The state space depends on the ball at hitting time (position, velocity, spin) and the action is the racket state (orientation, velocity) at hitting. An actor-critic based deterministic policy gradient algorithm was developed for accelerated learning. Our approach performs competitively both in a simulation and on the real robot in a number of challenging scenarios. Accurate results are obtained without pre-training in under 200 episodes of training. The video presenting our experiments is available at https://youtu.be/uRAtdoL6Wpw.

In table tennis, the rotation (spin) of the ball plays a crucial role. A table tennis match will feature a variety of strokes. Each generates different amounts and types of spin. To develop a robot that can compete with a human player, the robot needs to detect spin, so it can plan an appropriate return stroke. In this paper we compare three methods to estimate spin. The first two approaches use a high-speed camera that captures the ball in flight at a frame rate of 380 Hz. This camera allows the movement of the circular brand logo printed on the ball to be seen. The first approach uses background difference to determine the position of the logo. In a second alternative, we train a CNN to predict the orientation of the logo. The third method evaluates the trajectory of the ball and derives the rotation from the effect of the Magnus force. This method gives the highest accuracy and is used for a demonstration. Our robot successfully copes with different spin types in a real table tennis rally against a human opponent.