This paper advances research on social networks, extended reality, and the metaverse by bringing together innovations from two different communities – AI and cognitive science – to develop LLM-based agents with not only fluent responses but also realistic opinion dynamics that capture a variety of human biases, imperfections, and general departures from rationality. This avenue of investigation can empower applications from social simulation of human opinions in geopolitical hotspots to realistic non-player character interactions in metaverse games. Recent advances in AI have made remarkable progress toward general intelligence with the introduction of large language models (LLMs). They also enabled grounding LLM responses in specialized information stored externally using retrieval-augmented generation (RAG). In a separate line of research, studies on human cognition have produced cognitive architectures that emulate human departures from rationality, such as biases and imperfections, which are crucial to understanding a wide range of social phenomena and human preferences. A critical mechanism in cognitive architectures is the modulation of retrieval weights from (human) memory; we are biased in what we remember. Combining RAG with cognitive model-inspired computation of information retrieval weights, we develop the Irrational LLM – one that weighs information retrieval in RAG systems according to cognitive models, thereby accurately emulating human opinion formation. We implement the novel human cognition-inspired RAG framework (CogRAG) and use it to emulate option developments on different sides of a conflict regarding debated issues. Responses generated by CogRAG (on posts withheld from training data) show close correspondence with real responses posted on social media, suggesting the viability of this approach in approximating biased human opinions. We hope this study paves the way to new directions in AI, social networks, metaverse computing, and human-in-the-loop modeling that better represent diverse human opinions in geopolitical, entertainment, and socio-technical contexts.

This article investigates the cost function learning in social information networks, wherein human memory and cognitive bias are explicitly taken into account. We first propose a model for social information-diffusion dynamics, with a focus on the systematic modeling of asymmetric cognitive bias represented by confirmation bias and novelty bias. Building on the dynamics model, we then propose the M3IRL—a memorized model and maximum-entropy-based inverse reinforcement learning—for learning cost functions. Compared with the existing model-free IRLs, the characteristics of M3IRL are significantly different here: no dependence on the Markov decision process principle, the need for only a single finite-time trajectory sample, and bounded decision variables. Finally, the effectiveness of the proposed social information-diffusion model and the M3IRL algorithm is validated by the online social media data.

Masking strategies for cyberdefense (i.e., disguising network attributes to hide the real state of the network) are predicted to be effective in simulated experiments. However, it is unclear how effective they are against human attackers. We address three factors that challenge the effectiveness of the masking strategies in practice: (1) we relax the assumption of rationality of the attackers made by Game Theory/Machine Learning defense algorithms; (2) we provide a cognitive model of human attackers that can inform these defense algorithms; and (3) we provide a way to generate data on attacker’s decisions through simulation with a cognitive model. Two masking strategies of defense were generated using Game Theory and Machine Learning (ML) algorithms. The effectiveness of these two masking strategies of defense, risk averse and rational, are compared in an experiment with human attackers. We collected attacker’s decisions against the two masking strategies. With the limited human participant’s data, the results indicate that the risk averse strategy can reduce the defense losses compared to the rational masking strategy. We also propose a cognitive model based on Instance-Based Learning Theory that accurately represents and predicts the attacker’s decisions in this task. We demonstrate the model’s process by generating simulated data and comparing it to the attacker’s actual actions in the experiment. The model is able to capture the data at the aggregate and at the individual levels of attackers making decisions in both rational and risk averse defense algorithms. We propose that this model can be used to inform game theoretic defense algorithms and to produce synthetic data that can be used by ML algorithms to generate new defense strategies.

We present a method to validate a cognitive model, based on the cognitive architecture ACT-R, in dynamic human-automation systems with discrete human input. We are inspired by the general problem of K-choice games as a proxy for many decision making applications in dynamical systems. We model the human as a Markovian controller based on gathered experimental data, that is, a non-deterministic control input with known likelihoods of control actions associated with certain configurations of the state-space. We use reachability analysis to predict the outcome of the resulting discrete-time stochastic hybrid system, in which the outcome is defined as a function of the system trajectory. We suggest that the resulting expected outcomes can be used to validate the cognitive model against actual human subject data. We apply our method to a two-choice game in which the human is tasked with maximizing net coverage of a robotic swarm that can operate under rendezvous or deployment dynamics. We validate the corresponding ACTR cognitive model generated with the data from eight human subjects. The novelty of this work is (1) a method to compute expected outcome in a hybrid dynamical system with a Markov chain model of the human's discrete choice, and (2) application of this method to validation of cognitive models with a database of actual human subject data.

Robots are increasingly becoming key players in human-robot teams. To become effective teammates, robots must possess profound understanding of an environment, be able to reason about the desired commands and goals within a specific context, and be able to communicate with human teammates in a clear and natural way. To address these challenges, we have developed an intelligence architecture that combines cognitive components to carry out high-level cognitive tasks, semantic perception to label regions in the world, and a natural language component to reason about the command and its relationship to the objects in the world. This paper describes recent developments using this architecture on a fielded mobile robot platform operating in unknown urban environments. We report a summary of extensive outdoor experiments; the results suggest that a multidisciplinary approach to robotics has the potential to create competent human-robot teams.

Identifying the tasks a given piece of malware was designed to perform (e.g. logging keystrokes, recording video, establishing remote access, etc.) is a difficult and time-consuming operation that is largely human-driven in practice. In this paper, we present an automated method to identify malware tasks. Using two different malware collections, we explore various circumstances for each - including cases where the training data differs significantly from test; where the malware being evaluated employs packing to thwart analytical techniques; and conditions with sparse training data. We find that this approach consistently out-performs the current state-of-the art software for malware task identification as well as standard machine learning approaches - often achieving an unbiased F1 score of over 0.9. In the near future, we look to deploy our approach for use by analysts in an operational cyber-security environment.

Generating future states of the world is an essential component of high-level cognitive tasks such as planning. We explore the notion that such future-state generation is more widespread and forms an integral part of cognition. We call these generated states expectations, and propose that cognitive systems constantly generate expectations, match them to observed behavior and react when a difference exists between the two. We describe an ACT-R model that performs expectation-driven cognition on two tasks – pedestrian tracking and behavior classification. The model generates expectations of pedestrian movements to track them. The model also uses differences in expectations to identify distinctive features that differentiate these tracks. During learning, the model learns the association between these features and the various behaviors. During testing, it classifies pedestrian tracks by recalling the behavior associated with the features of each track. We tested the model on both single and multiple behavior datasets and compared the results against a k-NN classifier. The k-NN classifier outperformed the model in correct classifications, but the model had fewer incorrect classifications in the multiple behavior case, and both systems had about equal incorrect classifications in the single behavior case.

Two information decay methods are examined that help multi-agent systems cope with dynamic environments. The agents in this simulation have human-like memory and a mechanism to moderate their communications: they forget internally stored information via temporal decay, and they forget distributed information by filtering it as it passes through a communication network. The agents play a foraging game, in which performance depends on communicating facts and requests and on storing facts in internal memory. Parameters of the game and agent models are tuned to human data. Agent groups with moderated communication in small-world networks achieve optimal performance for typical human memory decay values, while non-adaptive agents benefit from stronger memory decay. The decay and filtering strategies interact with the properties of the network graph in ways suggestive of an evolutionary co-optimization between the human cognitive system and an external social structure.

We discuss an experiment involving visual path planning for multiple, remote robots in a partially visible building, with a partial 2D map available. Participants in the experiment defined waypoints for each robot to circumnavigate obstacles and explore the building. A cognitively plausible model of visual planning is evaluated using a normalized metric of the fit between model and subject itineraries. We discuss variation in the data and model fit, indicating individual differences in strategies to cope with task demands.

The SAL cognitive architecture is a synthesis of two well-established constituents: ACT-R, a hybrid symbolic-subsymbolic cognitive architecture, and Leabra, a neural architecture. These component architectures have vastly different origins yet suggest a surprisingly convergent view of the brain, the mind and behaviour. Furthermore, both of these architectures are internally pluralistic, recognising that models at a single level of abstraction cannot capture the required richness of behaviour. In this article, we offer a brief principled defence of epistemological pluralism in cognitive science and artificial intelligence, and elaborate on the SAL architecture as an example of how pluralism can be highly effective as an approach to research in cognitive science.

As in political debate, the commentaries in this issue mostly tend towards the two ends of a similar spectrum: ‘one true path’ versus ‘anything goes’. We hope to persuade swing voters of a more mod...

The ACT-R system is a general system for modeling a wide range of higher level cognitive processes. Recently, it has been embellished with a theory of how its higher level processes interact with a visual interface. This includes a theory of how visual attention can move across the screen, encoding information into a form that can be processed by ACT-R. This system is applied to modeling several classic phenomena in the literature that depend on the speed and selectivity with which visual attention can move across a visual display. ACT-R is capable of interacting with the same computer screens that subjects do and, as such, is well suited to provide a model for tasks involving human-computer interaction. In this article, we discuss a demonstration of ACT-R's application to menu selection and show that the ACT-R theory makes unique predictions, without estimating any parameters, about the time to search a menu. These predictions are confirmed.