The integration of affective computing with virtual reality (VR) often uses machine learning to analyze users’ emotional responses through physiological and behavioral signals, enabling personalized interactions within VR environments. However, current research in this field is characterized by inconsistent experimental protocols, which hinders comprehensive conclusions and cross-study comparisons. To address this gap, a systematic review was conducted following the PRISMA guidelines, identifying 24 studies that used physiological measures and machine learning to predict emotions in VR settings. The review covers five key areas: experimental protocols, VR environments, implicit measurements, emotion models, and machine learning approaches. In addition, it provides guidelines for standardizing data collection, biosignal processing, and emotion modeling. These proposed guidelines aim to establish consistent reporting practices and experimental protocols, thus improving the comparability and reproducibility of future VR affective computing research.

This study investigated how integrating sentiment analysis derived from verbal responses to open-ended questions with physiological signals into support vector machine (SVM) models can enhance the prediction of arousal and valence in virtual reality (VR) gaming. Three VR Pong scenarios–slow, fast, and lag-inducing–were tested to observe the impact of game-play speed and responsiveness on sentiment scores. Participant responses were measured using the Self-Assessment Manikin for arousal and valence, and sentiment was analyzed from open-ended responses. Physiological measures, such as electrocardio-grams, galvanic skin responses, and electromyogram activity in forearm muscles, were also recorded. The results indicated that gameplay speed and responsiveness significantly affects player emotions, with faster games leading to higher sentiment scores. Additionally, incorporating these sentiment scores into SVM models improved the accuracy of valence and arousal predictions. This highlights the role of gameplay speed and responsiveness in influencing both emotional and physiological responses in VR and suggests that using diverse data types, including audio-derived sentiment analysis, could enhance the accuracy of affective computing models.

Human-robot collaboration has significant potential in recycling due to the wide variation in the composition of recyclable products. Six participants performed a recyclable item sorting task collaborating with a robot arm equipped with a vision system. The effect of three different levels of human in-volvement or assistance to the robot (Level 1- occlusion removal; Level 2- optimal spacing; Level 3- optimal grip) on performance metrics such as robot accuracy, task time and subjective fluency were assessed. Results showed that human involvement had a remarkable impact on the robot's accuracy, which increased with human involvement level. Mean accuracy values were 33.3% for Level1, 69% for Level 2 and 100% for Level 3. The results imply that for sorting processes involving diverse materials that vary in size, shape, and composition, human assistance could improve the robot's accuracy to a significant extent while also being cost-effective.

Meal-assistance robots (MARs) help feed meals to users with disabilities or who need help consuming their meals. Although such MARs have been introduced in various fields, guidelines for evaluating MARs are limited. This study aims to develop comprehensive guidelines to assess the user experience (UX) of MAR designs considering its social interaction characteristics between humans and robots. Participants from three groups (patients, doctors, and caregivers) with different perspectives on MARs were recruited and a focus group interview was conducted to collect their UXs with MARs. The three groups showed different UXs with MARs in user interface design, robot arm motion, and safety and mobility. In addition, based on the literature review, eight UX features (usability, emotion, value, naturalness, assistance, acceptance, personality, and culture) are proposed to evaluate MAR interfaces. The proposed comprehensive design guideline will be particularly useful in evaluating and designing the UX of MARs.

We solve active target tracking, one of the essential tasks in autonomous systems, using a deep reinforcement learning (RL) approach. In this problem, an autonomous agent is tasked with acquiring information about targets of interests using its on-board sensors. The classical challenges in this problem are system model dependence and the difficulty of computing information-theoretic cost functions for a long planning horizon. RL provides solutions for these challenges as the length of its effective planning horizon does not affect the computational complexity, and it drops the strong dependency of an algorithm on system models. In particular, we introduce Active Tracking Target Network (ATTN), a unified deep RL policy that is capable of solving major sub-tasks of active target tracking – in-sight tracking, navigation, and exploration. The policy shows robust behavior for tracking agile and anomalous targets with a partially known target model. Additionally, the same policy is able to navigate in obstacle environments to reach distant targets as well as explore the environment when targets are positioned in unexpected locations.

We develop a Multi-Agent Reinforcement Learning (MARL) method to learn scalable control policies for target tracking. Our method can handle an arbitrary number of pursuers and targets; we show results for tasks consisting up to 1000 pursuers tracking 1000 targets. We use a decentralized, partially-observable Markov Decision Process framework to model pursuers as agents receiving partial observations (range and bearing) about targets which move using fixed, unknown policies. An attention mechanism is used to parameterize the value function of the agents; this mechanism allows us to handle an arbitrary number of targets. Entropy-regularized off-policy RL methods are used to train a stochastic policy, and we discuss how it enables a hedging behavior between pursuers that leads to a weak form of cooperation in spite of completely decentralized control execution. We further develop a masking heuristic that allows training on smaller problems with few pursuers-targets and execution on much larger problems. Thorough simulation experiments and comparisons to state of the art algorithms are performed to study the scalability of the approach and robustness of performance to varying numbers of agents and targets.

While off-policy temporal difference (TD) methods have widely been used in reinforcement learning due to their efficiency and simple implementation, their Bayesian counterparts have not been utilized as frequently. One reason is that the non-linear max operation in the Bellman optimality equation makes it difficult to define conjugate distributions over the value functions. In this paper, we introduce a novel Bayesian approach to off-policy TD methods, called as ADFQ, which updates beliefs on state-action values, Q, through an online Bayesian inference method known as Assumed Density Filtering. We formulate an efficient closed-form solution for the value update by approximately estimating analytic parameters of the posterior of the Q-beliefs. Uncertainty measures in the beliefs not only are used in exploration but also provide a natural regularization for the value update considering all next available actions. ADFQ converges to Q-learning as the uncertainty measures of the Q-beliefs decrease and improves common drawbacks of other Bayesian RL algorithms such as computational complexity. We extend ADFQ with a neural network. Our empirical results demonstrate that ADFQ outperforms comparable algorithms on various Atari 2600 games, with drastic improvements in highly stochastic domains or domains with a large action space.

In this paper, we propose a novel Reinforcement Learning approach for solving the Active Information Acquisition problem, which requires an agent to choose a sequence of actions in order to acquire information about a process of interest using on-board sensors. The classic challenges in the information acquisition problem are the dependence of a planning algorithm on known models and the difficulty of computing information-theoretic cost functions over arbitrary distributions. In contrast, the proposed framework of reinforcement learning does not require any knowledge on models and alleviates the problems during an extended training stage. It results in policies that are efficient to execute online and applicable for real-time control of robotic systems. Furthermore, the state-of-the-art planning methods are typically restricted to short horizons, which may become problematic with local minima. Reinforcement learning naturally handles the issue of planning horizon in information problems as it maximizes a discounted sum of rewards over a long finite or infinite time horizon. We discuss the potential benefits of the proposed framework and compare the performance of the novel algorithm to an existing information acquisition method for multi-target tracking scenarios.

This article presents a computational model that predicts finger-drag gesture performance on touchscreen devices, by integrating the queueing network (QN) cognitive architecture and motion tracking. Specifically, the QN-based model was developed to predict two execution times: the finger movement time of drag-gesture (i.e., only the motion time of the finger touched and dragged on the surface of touchscreen) and the comprehensive process time of drag-gesture (i.e., the entire process time to complete the finger-drag task, including visual attention shift, memory storage and retrieval, and hand-finger movements). To develop predictive models for the finger movement time of drag-gesture, 11 participants’ motion data were collected and a regression analysis with parameters of hand-finger anthropometric data and eight angular directions was conducted. Human subject data from our previous study (Jeong & Liu, 2017a) were used to evaluate the QN-based model, generating similar outputs (R2 was more than 80% and root-mean square was less than 300 msec) for both execution times.

In order for humanoid robots to complete various assigned tasks without any human assistance, they must have the ability to stand up on their own. In this abstract, we introduce complex stand-up motion of humanoid robots learned by using Reinforcement Learning.

Standing up after falling is an essential ability for humanoid robots in order to resume their tasks without help from humans. Although many humanoid robots, especially small-size humanoid robots, have their own stand-up motions, there has not been a generalized method to automatically learn flexible stand-up motions for humanoid robots which can be applied to various fallen positions. In this research, we propose a method for learning stand-up motions for humanoid robots using Q-learning making use of their bilateral symmetry. We implemented this method on DarwIn-OP humanoid robots and learned an optimal policy in simulation. We compared the resulting stand-up motion with manually designed stand-up motions and with stand-up motions learned without considering bilateral symmetry. Both in simulation and on the real robot, the new stand-up motion was successful in most trials while other motions took longer or were not as robust.

Major rule changes for the RoboCup Humanoid League in 2015 pose significant vision and locomotion challenges for disambiguating similarly colored objects and navigating soft terrain. These significant changes highlight the need for applying general purpose humanoid robotics approaches that can handle abrupt environment modifications, and we utilize the general purpose THOR (Tactical Hazardous Operations Robot) series of robot from the recent DARPA Robotics Challenge (DRC). Specific techniques for vision, kicking and autonomy complement software developed for robust deployments in the DRC. In this paper, we present these soccer playing techniques, which were validated in the Humanoid AdultSize league in Hefei. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Humanoid robots can be a highly desirable substitute for humans when it performs various tasks using tools and equipment designed for humans. One of such possible applications is controlling a vehicle. A humanoid robot can sit in the pilot's seat and command the vehicle using the control columns or steering wheels, pedals, switches, levers, and dials. In this paper, we propose a framework of automating a vehicle, an airplane in particular, with a humanoid. In order to perform various tasks of flying an unmodified airplane, the robot needs to perform three levels of tasks - recognition, decision, and action. The robot should collect information of the vehicle by using its own sensors and from various instruments in the cockpit, in addition to possible data link, a privilege of a robot. The robot then decides how to operate the flight control equipment in order to follow a given flight plan. Finally, it directly manipulates the control input equipment by computing the kinematic variables in the presence of various constraints from the surroundings. In order to validate the proposed framework, a piloting robot system is developed using a small low-cost humanoid and a flight simulation equipment designed for humans. The robot showed adequate performance to fly the airplane from cold start to landing to a stop on the runway.