Cross-domain sequential recommendation leverages a user’s historical interaction behavior across multiple domains to forecast the subsequent item with which they are likely to engage. Despite the efforts made by previous methods in representing users across domains, we believe they still have three inherent limitations: 1) Ignoring the importance of the relations between items among various domains, leading to suboptimal performance. 2) Most existing cross-domain sequential recommendation models are incapable of making full use of timestamp information, merely representing the sequence of user-item interaction sequence through positional embeddings. 3) Existing cross-domain recommendation methods consider only individual users, failing to recognize that social relations may influence user preferences. To overcome these problems, we propose an innovative approach called Social-Augmented Time-Aware Cross-Domain Sequential Recommendation (STRec). The method leverages the integration of multi-domain data with temporal information, further enriched by social influence, to afford a holistic understanding of user preferences. STRec consists of three main components: Initially, embedding modules are employed to convert timestamps and interaction information into embedding vectors respectively. Subsequently, graph neural network modules to uncover collaborative characteristics among inter-sequence items, while sequence attention encoders extract sequential features within item sequences and uncover the potential dependencies of temporal information in different contexts. Conclusively, a social aware attention network is designed to consider users’ social neighbor information and capture the dynamic patterns of social relationships. Through extensive experimentation on real-world datasets across multiple scenarios, we have demonstrated that STRec surpasses several state-of-the-art approaches.

The need to design for appropriate human-autonomy teaming has become increasingly important as systems grow in complexity, especially those that require time-pressured interactions like in unmanned aerial vehicle (UAV) operations. However, it is not always clear whether operators develop effective strategies for computer-based technologies. When operators are given such tools, their performances can be statistically compared but often such assessments only provide summative information. These comparisons do not indicate how and why technology influenced people's strategies and actions. To fill this gap, we utilized Hidden Markov Models (HMMs) to represent strategies employed by operators in first-person control of UAVs for inspection tasks. The resulting models captured differences in strategies for people who both succeeded and crashed, as well as those who were overconfident in their self-assessments, and those who were not. People who were not overconfident exhibited less risky strategies and were more successful. These findings were further strengthened by a quantitative state similarity metric, which indicated where and for who possible interventions could improve outcomes. This application of HMMs to operator strategy representation could help to identify effective operator strategy development in the use of computer-based technologies, and what kind of interventions could be the most effective in improving outcomes.

Situation awareness (SA) is generally considered as the perception, understanding, and projection of objects’ properties and positions. We believe if the system can sense drivers’ SA, it can appropriately provide warnings for objects that drivers are not aware of. To investigate drivers’ awareness, in this study, a human-subject experiment of driving simulation was conducted for data collection. While a previous predictive model for drivers’ situation awareness utilized drivers’ gaze movement only, this work utilizes object properties, characteristics of human visual sensory and memory mechanism. As a result, the proposed driver SA prediction model achieves over 70% accuracy and outperforms the baselines.

Human-automation collaborations, like automated driving assistance and piloting drones, have become prevalent as these technologies become more commonplace. Designers need tools that help them understand how and why design interventions may change the strategies of operators in such complex human supervisory control systems. To this end, we demonstrate that when the divergence metric is applied to Hidden Markov Model (HMM) comparisons, it can accurately capture statistical differences between operator strategies for interfaces that embody different tasks. However, the use of such an approach is problematic when used to compare HMM strategy models with non-equivalent observations. To address this limitation, we developed an observation reduction approach and conducted a sensitivity analysis to assess the impact of this approach. Our results show that when comparing two non-equivalent interfaces, our observation reduction approach does not fundamentally change the divergence metric, thus allowing for direct model comparison. The results further show that HMMs from different interfaces produce a much higher divergence metric than model comparison from the same people who repeatedly use the same interface. Future work will examine if this method can detect differences in models with different tasks or modified interfaces.

In this work, we synthesize collaboration protocols for human-unmanned aerial vehicle (H-UAV) command and control systems, where the human operator aids in securing the UAV by intermittently performing geolocation tasks to confirm its reported location. We first present a stochastic game-based model for the system that accounts for both the operator and an adversary capable of launching stealthy false-data injection attacks, causing the UAV to deviate from its path. We also describe a synthesis challenge due to the UAV's hidden-information constraint. Next, we perform human experiments using a developed RESCHU-SA testbed to recognize the geolocation strategies that operators adopt. Furthermore, we deploy machine learning techniques on the collected experimental data to predict the correctness of a geolocation task at a given location based on its geographical features. By representing the model as a delayed-action game and formalizing the system objectives, we utilize off-the-shelf model checkers to synthesize protocols for the human-UAV coalition that satisfy these objectives. Finally, we demonstrate the usefulness of the H-UAV protocol synthesis through a case study where the protocols are experimentally analyzed and further evaluated by human operators.

Human supervisory control (HSC) is a widely used knowledge-based control scheme, in which human operators are in charge of planning and making high-level decisions for systems with embedded autonomy. With the variability of operators' behaviors in such systems, the stability of an operator modeling technique, i.e., that a modeling approach produces similar results across repeated applications, is critical to the extensibility and utility of such a model. Using an unmanned vehicle simulation testbed where such vehicles can be hacked, we compared two operator behavioral models from two different experiments using a hidden Markov modeling (HMM) approach. The resulting HMM models revealed operators' dominant strategies when conducting hacking detection tasks. The similarity between these two models was measured via multiple aspects, including model structure, state distribution, divergence distance, and co-emission probability distance. The similarity measure results demonstrate the stability of modeling human operators in HSC scenarios using HMM models. These results indicate that even when operators perform differently on specific tasks, such an approach can reliably detect whether strategies change across different experiments.

An increasingly relevant security issue for unmanned aerial vehicles (UAVs, also known as drones) is the possibility of a global positioning system (GPS) spoofing attack. Given the existing problems in current GPS spoofing detection techniques and human visual advantages in searching and localizing targets, we propose a human-autonomy collaborative approach of human geolocation to assist UAV control systems in detecting GPS spoofing attacks. An interactive testbed and experiment were designed and used to evaluate this approach, which demonstrated that human-autonomy collaborative hacking detection is a viable concept. Using the hidden Markov model (HMM) approach, operator behavior patterns and strategies from the experiment were modeled via hidden states and transitions among them. These models revealed two dominant hacking detection strategies. Statistical results and expert performer evaluations show no significant difference between different hacking detection strategies in terms of correct detection. The detection strategy model suggests areas of future research in decision support tool design for UAV hacking detection. Also, the development of HMMs presents the feasibility of quantitatively investigating operator behavior patterns and strategies in human supervisory control scenarios.

Unmanned Aerial Vehicles (UAVs), also known as drones, have extensive applications in civilian rescue and military surveillance realms. A common drone control scheme among such applications is human supervisory control, in which human operators remotely navigate drones and direct them to conduct high-level tasks. However, different levels of autonomy in the control system and different operator training processes may affect operators’ performance in task success rate and efficiency. An experiment was designed and conducted to investigate such potential impacts. The results showed us that a dedicated supervisory drone control interface tended toward increased operator successful task completion as compared to an enhanced teleoperation control interface, although this difference was not statistically significant. In addition, using Hidden Markov Models, operator behavior models were developed to further study the impact of operators’ drone control strategies as a function of differing levels of autonomy. These models revealed that people with both supervisory and enhanced teleoperation control training were not able to determine the right control action at the right time to the same degree that people with just training in the supervisory control mode. Future work is needed to determine how trust plays a role in such settings.