Large language models (LLMs) are increasingly used in the creation of online content, creating feedback loops as subsequent generations of models will be trained on this synthetic data.Such loops were shown to lead to distribution shifts -models misrepresenting the true underlying distributions of human data (also called model collapse).However, how human data properties affect such shifts remains poorly understood.In this paper, we provide the first empirical examination of the effect of such properties on the outcome of recursive training.We first confirm that using different human datasets leads to distribution shifts of different magnitudes.Through exhaustive manipulation of dataset properties combined with regression analyses, we then identify a set of properties associated with distribution shift magnitudes.Lexical diversity is found to amplify these shifts, while semantic diversity and data quality mitigate them.Furthermore, we find that these influences are highly modular: data scrapped from a given internet domain has little influence on the content generated for another domain.Finally, experiments on political bias reveal that human data properties affect whether the initial bias will be amplified or reduced.Overall, our results portray a novel view, where different parts of internet may undergo different types of distribution shift.

One of the initial motivations for reservoir computing was the effort to understand how recurrent connectivity could explain observations of neural activity patterns in the prefrontal cortex of behaving primates. Recurrent connections provide for the projections of inputs into a high dimensional space. In individual reservoir units, this results in activation patterns that display non-linear mixtures of inputs and abstract internal states, referred to as mixed selectivity, which has been identified as a key component of reservoir activity. Interestingly it is also a key element in primate brain activity. An equally prominent characteristic of primate brain activity is a temporal processing gradient, with fast input driven responses in sensory areas, and progressively prolonged time constants in increasingly associative cortical areas. Recent research has explained a temporal integration hierarchy as a function of local connectivity within structured reservoirs. As in the primate brain, areas that receive sensory input have fast integration. Via local connections, this input driven activation flows through local connections to progressively distant areas, thus physically implementing the temporal integration gradient. In the current research we test the hypothesis that this physical hierarchy will also produce a gradient in mixed selectivity in the structured reservoirs. Indeed, simulations demonstrate that reservoirs constrained by a connection distance rule produce a gradient of mixed selectivity, with mixed selectivity progressively increasing from input-driven to more distant associative areas. This allows us to predict that the same kind of gradient for mixed selectivity should be observed in the human cortex. In order to test this prediction, we exploited human brain activation data that included a form of multidimensional narrative structure that was well suited for characterizing mixed selectivity. Applying the same analysis from the reservoir analysis, we observe the presence of mixed selectivity in human cortex, and evidence for a gradient from lower-level sensorimotor areas, to higher level integrative areas. This research contributes to the characterization of the principals of computation in anatomically structured reservoirs and the human brain.

Computational models of language are having an increasing impact in understanding the neural bases of language processing in humans. A recent model of cortical dynamics based on reservoir computing was able to account for temporal aspects of human narrative processing as revealed by fMRI. In this context the current research introduces a form of structured reservoir computing, where network dynamics are further constrained by the connectivity architecture in order to begin to explain large scale hierarchical network properties of human cortical activity during narrative comprehension. Cortical processing takes place at different time scales depending on the position in a “hierarchy” from posterior sensory input areas to higher level associative frontal cortical areas. This phenomena is likely related to the cortical connectivity architecture. Recent studies have identified heterogeneity in this posterior-anterior hierarchy, with certain frontal associative areas displaying a faster narrative integration response than much more posterior areas. We hypothesize that these discontinuities can be due to white matter connectivity that would create shortcuts from fast sensory areas to distant frontal areas. To test this hypothesis, we analysed the white matter connectivity of these areas and discovered clear connectivity patterns in accord with our hypotheses. Based on these observations we performed simulations using reservoir networks with connectivity patterns structured with an exponential distance rule, yielding the sensory-associative hierarchy. We then introduce connectivity short-cuts corresponding to those observed in human anatomy, resulting in frontal areas with unusually fast narrative processing. Using structured reservoir computing we confirmed the hypothesis that topographic position in a cortical hierarchy can be dominated by long distance connections that can bring frontal areas closer to the sensory periphery.

Recent research has revealed that during continuous perception of movies or stories, humans display cortical activity patterns that reveal hierarchical segmentation of event structure. Thus, sensory areas like auditory cortex display high frequency segmentation related to the stimulus, while semantic areas like posterior middle cortex display a lower frequency segmentation related to transitions between events. These hierarchical levels of segmentation are associated with different time constants for processing. Likewise, when two groups of participants heard the same sentence in a narrative, preceded by different contexts, neural responses for the groups were initially different and then gradually aligned. The time constant for alignment followed the segmentation hierarchy: sensory cortices aligned most quickly, followed by mid-level regions, while some higher-order cortical regions took more than 10 seconds to align. These hierarchical segmentation phenomena can be considered in the context of processing related to comprehension. In a recently described model of discourse comprehension word meanings are modeled by a language model pre-trained on a billion word corpus. During discourse comprehension, word meanings are continuously integrated in a recurrent cortical network. The model demonstrates novel discourse and inference processing, in part because of two fundamental characteristics: real-world event semantics are represented in the word embeddings, and these are integrated in a reservoir network which has an inherent gradient of functional time constants due to the recurrent connections. Here we demonstrate how this model displays hierarchical narrative event segmentation properties beyond the embeddings alone, or their linear integration. The reservoir produces activation patterns that are segmented by a hidden Markov model (HMM) in a manner that is comparable to that of humans. Context construction displays a continuum of time constants across reservoir neuron subsets, while context forgetting has a fixed time constant across these subsets. Importantly, virtual areas formed by subgroups of reservoir neurons with faster time constants segmented with shorter events, while those with longer time constants preferred longer events. This neurocomputational recurrent neural network simulates narrative event processing as revealed by the fMRI event segmentation algorithm provides a novel explanation of the asymmetry in narrative forgetting and construction. The model extends the characterization of online integration processes in discourse to more extended narrative, and demonstrates how reservoir computing provides a useful model of cortical processing of narrative structure.

People are becoming increasingly comfortable using Digital Assistants (DAs) to interact with services or connected objects. However, for non-programming users, the available possibilities for customizing their DA are limited and do not include the possibility of teaching the assistant new tasks. To make the most of the potential of DAs, users should be able to customize assistants by instructing them through Natural Language (NL). To provide such functionalities, NL interpretation in traditional assistants should be improved: (1) The intent identification system should be able to recognize new forms of known intents, and to acquire new intents as they are expressed by the user. (2) In order to be adaptive to novel intents, the Natural Language Understanding module should be sample efficient, and should not rely on a pretrained model. Rather, the system should continuously collect the training data as it learns new intents from the user. In this work, we propose AidMe (Adaptive Intent Detection in Multi-Domain Environments), a user-in-the-loop adaptive intent detection framework that allows the assistant to adapt to its user by learning his intents as their interaction progresses. AidMe builds its repertoire of intents and collects data to train a model of semantic similarity evaluation that can discriminate between the learned intents and autonomously discover new forms of known intents. AidMe addresses two major issues - intent learning and user adaptation - for instructable digital assistants. We demonstrate the capabilities of AidMe as a standalone system by comparing it with a one-shot learning system and a pretrained NLU module through simulations of interactions with a user. We also show how AidMe can smoothly integrate to an existing instructable digital assistant.

Developmental machine learning studies how artificial agents can model the way children learn open-ended repertoires of skills. Such agents need to create and represent goals, select which ones to pursue and learn to achieve them. Recent approaches have considered goal spaces that were either fixed and hand-defined or learned using generative models of states. This limited agents to sample goals within the distribution of known effects. We argue that the ability to imagine out-of-distribution goals is key to enable creative discoveries and open-ended learning. Children do so by leveraging the compositionality of language as a tool to imagine descriptions of outcomes they never experienced before, targeting them as goals during play. We introduce IMAGINE, an intrinsically motivated deep reinforcement learning architecture that models this ability. Such imaginative agents, like children, benefit from the guidance of a social peer who provides language descriptions. To take advantage of goal imagination, agents must be able to leverage these descriptions to interpret their imagined out-of-distribution goals. This generalization is made possible by modularity: a decomposition between learned goal-achievement reward function and policy relying on deep sets, gated attention and object-centered representations. We introduce the Playground environment and study how this form of goal imagination improves generalization and exploration over agents lacking this capacity. In addition, we identify the properties of goal imagination that enable these results and study the impacts of modularity and social interactions.

The following topics are dealt with: human-robot interaction; mobile robots; learning (artificial intelligence); control engineering computing; humanoid robots; service robots; medical robotics; robot vision; computer aided instruction; motion control.

As rats learn to search for multiple sources of food or water in a complex environment, they generate increasingly efficient trajectories between reward sites. Such spatial navigation capacity involves the replay of hippocampal place-cells during awake states, generating small sequences of spatially related place-cell activity that we call "snippets". These snippets occur primarily during sharp-wave-ripples (SWRs). Here we focus on the role of such replay events, as the animal is learning a traveling salesperson task (TSP) across multiple trials. We hypothesize that snippet replay generates synthetic data that can substantially expand and restructure the experience available and make learning more optimal. We developed a model of snippet generation that is modulated by reward, propagated in the forward and reverse directions. This implements a form of spatial credit assignment for reinforcement learning. We use a biologically motivated computational framework known as 'reservoir computing' to model prefrontal cortex (PFC) in sequence learning, in which large pools of prewired neural elements process information dynamically through reverberations. This PFC model consolidates snippets into larger spatial sequences that may be later recalled by subsets of the original sequences. Our simulation experiments provide neurophysiological explanations for two pertinent observations related to navigation. Reward modulation allows the system to reject non-optimal segments of experienced trajectories, and reverse replay allows the system to "learn" trajectories that it has not physically experienced, both of which significantly contribute to the TSP behavior.

A central component of the human self is the narrative history of shared interactions with others, which provides the foundation for social relations that develop over extended time. The loss of this narrative self progressively becomes catastrophic for aging subjects with degenerative disease of the memory system. A prosthetic device for narrative memory can provide an at least temporary solution to this problem. We identify requirements for a narrative memory capability that can allow individuals with diminished memory to continue to interact socially with partners with whom they have shared experiences. A memory prosthetic should provide access to past memories of the subject, and should accompany the subject in the formation, organization and retrieval of new memories. Based on these requirements, we have implemented the V1.0 narrative memory companion on the Pepper humanoid robot using the native Choregraph and NAOqi system capabilities. We exploit principals developed in our research in autobiographical memory and the organization of experience in cooperative humanoid robots, and the mapping of narrative structure onto this experience. In the narrative companion, past memories are first collected from the subject or members of their entourage via a template-based interview, and a small number of photographs that illustrate important people and events in the subject's past. New memories constructed via interaction with Pepper, and by simple narratives told by the human partner, are stored in the Autobiographical Memory (ABM) implemented in the ALKnolwedge base of the NAOqi system. Memories are then recalled and shared by narrative. Results from a naïve case study are presented, and future applications for improved quality of life are discussed.

Primates display a remarkable ability to adapt to novel situations. Determining what is most pertinent in these situations is not always possible based only on the current sensory inputs, and often also depends on recent inputs and behavioral outputs that contribute to internal states. Thus, one can ask how cortical dynamics generate representations of these complex situations. It has been observed that mixed selectivity in cortical neurons contributes to represent diverse situations defined by a combination of the current stimuli, and that mixed selectivity is readily obtained in randomly connected recurrent networks. In this context, these reservoir networks reproduce the highly recurrent nature of local cortical connectivity. Recombining present and past inputs, random recurrent networks from the reservoir computing framework generate mixed selectivity which provides pre-coded representations of an essentially universal set of contexts. These representations can then be selectively amplified through learning to solve the task at hand. We thus explored their representational power and dynamical properties after training a reservoir to perform a complex cognitive task initially developed for monkeys. The reservoir model inherently displayed a dynamic form of mixed selectivity, key to the representation of the behavioral context over time. The pre-coded representation of context was amplified by training a feedback neuron to explicitly represent this context, thereby reproducing the effect of learning and allowing the model to perform more robustly. This second version of the model demonstrates how a hybrid dynamical regime combining spatio-temporal processing of reservoirs, and input driven attracting dynamics generated by the feedback neuron, can be used to solve a complex cognitive task. We compared reservoir activity to neural activity of dorsal anterior cingulate cortex of monkeys which revealed similar network dynamics. We argue that reservoir computing is a pertinent framework to model local cortical dynamics and their contribution to higher cognitive function.

With the Robonaut-2 humanoid robot now permanently flying on the ISS, the potential role for robots participating in cooperative activity in space is becoming a reality. Recent research has demonstrated that cooperation in the joint achievement of shared goals is a promising framework for human interaction with robots, with application in space. Perhaps more importantly, with the turn-over of crew members, robots could play an important role in maintaining and transferring expertise between outgoing and incoming crews. In this context, the current research builds on our experience in systems for cooperative human-robot interaction, introducing novel interface and interaction modalities that exploit the long‐term experience of the robot. We implement a system where the human agent can teach the Nao humanoid new actions by physical demonstration, visual imitation, and spoken command. These actions can then be composed into joint action plans that coordinate the cooperation between agent and human. We also implement algorithms for an Autobiographical Memory (ABM) that provides access to of all of the robots interaction experience. These functions are assembled in a novel interaction paradigm for the capture, maintenance and transfer of knowledge in a five-tiered structure. The five tiers allow the robot to 1) learn simple behaviors, 2) learn shared plans composed from the learned behaviors, 3) execute the learned shared plans efficiently, 4) teach shared plans to new humans, and 5) answer questions from the human to better understand the origin of the shared plan. Our results demonstrate the feasibility of this system and indicate that such humanoid robot systems will provide a potential mechanism for the accumulation and transfer of knowledge, between humans who are not co-present. Applications to space flight operations as a target scenario are discussed.

A new research platform has been developed to study human-robot interaction and communication. In this setup, a humanoid robot is used as a proxy between two humans involved in dyadic interactions. An experimenter is bound with a humanoid robot. He can control in real-time and sensor free the eye and face/head movements performed by a humanoid robot with his own movements. The experimenter can perceive the scene as if he was the robot. Manipulations can be applied in real-time to any movement leaving the rest of the dynamics untouched. For instance, we have started investigating the effect of damping head movements during dyadic interaction. Preliminary results show that naive subjects' head nods increase when attenuation was applied on the robot's head movements.

Cooperation1is at the core of human social life. In this context, two major challenges face research on humanrobot interaction: the first is to understand the underlying structure of cooperation, and the second is to build, based on this understanding, artificial agents that can successfully and safely interact with humans. Here we take a psychologically grounded and human-centered approach that addresses these two challenges. We test the hypothesis that optimal cooperation between a naïve human and a robot requires that the robot can acquire and execute a joint plan, and that it communicates this joint plan through ecologically valid modalities including spoken language, gesture and gaze. We developed a cognitive system that comprises the human-like control of social actions, the ability to acquire and express shared plans and a spoken language stage. In order to test the psychological validity of our approach we tested 12 naïve subjects in a cooperative task with the robot. We experimentally manipulated the presence of a joint plan (vs. a solo plan), the use of task-oriented gaze and gestures, and the use of language accompanying the unfolding plan. The quality of cooperation was analyzed in terms of proper turn taking, collisions and cognitive errors. Results showed that while successful turn taking could take place in the absence of the explicit use of a joint plan, its presence yielded significantly greater success. One advantage of the solo plan was that the robot would always be ready to generate actions, and could thus adapt if the human intervened at the wrong time, whereas in the joint plan the robot expected the human to take his/her turn. Interestingly, when the robot represented the action as involving a joint plan, gaze provided a highly potent nonverbal cue that facilitated successful collaboration and reduced errors in the absence of verbal communication. These results support the cooperative stance in human social cognition, and suggest that cooperative robots should employ joint plans, fully communicate them in order to sustain effective collaboration while being ready to adapt if the human makes a midstream mistake.

As elaborate human-robot interaction capabilities continue to develop, humans will increasingly be in proximity with robots, and the management of the ongoing control in case of breakdown becomes increasingly important: taking care of what happens when cooperation goes wrong. The current research addresses three categories of breakdowns where cooperation can go wrong. In the first category, the human detects some type of problem and generates a self-issued stop signal, with a physical palm up posture. In the second category, the human becomes distracted, and physically changes his orientation away from the shared space of cooperation. In the final category that we investigate, the human becomes physically close to the robot such that safety limits are reached and detected by the robot. In each of these three cases, the robot cognitive system detects the failure via the perception of distinct physical states from motion capture: the hand up posture; change in head orientation; and physical distance reaching a minimum threshold. In each case the robot immediately halts the current action. Then, the system should recover appropriately. Each error type returns a specific code, allowing the Supervisor system to handle the specific type of error. Our cognitive system allows the robot to learn composite actions, as a sequence of atomic actions. These composite actions can then be composed into higher level plans. When a plan fails at the level of a composite action, the recovery method is not trivial: should recovery take place at the level of the composite action, or the actual atomic action which physically failed? As the best recovery may depend on the physical context, we expand the plan into atomic actions, and recover at this level, allowing the user to specify whether the action should be skipped or retried. We demonstrate that this system allows graceful recovery from three principal categories of interaction breakdown, and provides an invaluable mechanism for preserving the integrity of cooperative HRI.

If robots are to cooperate with humans in an increasingly human-like manner, then significant progress must be made in their abilities to observe and learn to perform novel goal directed actions in a flexible and adaptive manner. The current research addresses this challenge. In CHRIS.I [1], we developed a platform-independent perceptual system that learns from observation to recognize human actions in a way which abstracted from the specifics of the robotic platform, learning actions including “put X on Y” and “take X”. In the current research, we extend this system from action perception to execution, consistent with current developmental research in human understanding of goal directed action and teleological reasoning. We demonstrate the platform independence with experiments on three different robots. In Experiments 1 and 2 we complete our previous study of perception of actions “put” and “take” demonstrating how the system learns to execute these same actions, along with new related actions “cover” and “uncover” based on the composition of action primitives “grasp X” and “release X at Y”. Significantly, these compositional action execution specifications learned on one iCub robot are then executed on another, based on the abstraction layer of motor primitives. Experiment 3 further validates the platform-independence of the system, as a new action that is learned on the iCub in Lyon is then executed on the Jido robot in Toulouse. In Experiment 4 we extended the definition of action perception to include the notion of agency, again inspired by developmental studies of agency attribution, exploiting the Kinect motion capture system for tracking human motion. Finally in Experiment 5 we demonstrate how the combined representation of action in terms of perception and execution provides the basis for imitation. This provides the basis for an open ended cooperation capability where new actions can be learned and integrated into shared plans for cooperation. Part of the novelty of this research is the robots' use of spoken language understanding and visual perception to generate action representations in a platform independent manner based on physical state changes. This provides a flexible capability for goal-directed action imitation.

One of the long term objectives of robotics and artificial cognitive systems is that robots will increasingly be capable of interacting in a cooperative and adaptive manner with their human counterparts in open-ended tasks that can change in real-time. In such situations, an important aspect of the robot behavior will be the ability to acquire new knowledge of the cooperative tasks by observing humans. At least two significant challenges can be identified in this context. The first challenge concerns development of methods to allow the characterization of human actions such that robotic systems can observe and learn new actions, and more complex behaviors made up of those actions. The second challenge is associated with the immense heterogeneity and diversity of robots and their perceptual and motor systems. The associated question is whether the identified methods for action perception can be generalized across the different perceptual systems inherent to distinct robot platforms. The current research addresses these two challenges. We present results from a cooperative human-robot interaction system that has been specifically developed for portability between different humanoid platforms. Within this architecture, the physical details of the perceptual system (e.g. video camera vs IR video with reflecting markers) are encapsulated at the lowest level. Actions are then automatically characterized in terms of perceptual primitives related to motion, contact and visibility. The resulting system is demonstrated to perform robust object and action learning and recognition on two distinct robotic platforms. Perhaps most interestingly, we demonstrate that knowledge acquired about action recognition with one robot can be directly imported and successfully used on a second distinct robot platform for action recognition. This will have interesting implications for the accumulation of shared knowledge between distinct heterogeneous robotic systems.

Robot platforms have now reached a level of technical development wherein they are becoming physically capable of useful interaction with humans, while ensuring safety and reasonable cost. The current challenge is for cognitive systems science to provide these robots with the necessary capabilities so that they can interact and cooperate with humans in a natural manner. We are addressing this problem by exploiting two central ideas derived from the human psychological sciences. The first idea is that the human conceptual system is based on situated simulations that are instantiated in the same systems that are used for perception and action, referred to as embodied cognition. The second idea is that human cooperation relies on the cooperating agents sharing a common representation of their shared plan, which involves the actions of both agents. This representation allows them to cooperate, to trade roles, and to help one another if necessary. We have implemented these concepts on multiple robot platforms including the HRP2 humanoid, and the Cooperator and Cooperator II visually guided robot manipulators. This paper will present the motivation for this system and results, and will then outline what we consider to be the crucial issues for human-like cognitive systems.

The current research analyses and demonstrates how spoken language can be used by human users to communicate with the HRP-2 humanoid to program the robot's behavior in a cooperative task. The task involves the humans and the HRP-2 working together to assemble a piece of furniture. The objectives of the system are to 1) Allow the human to impart knowledge of how to accomplish a cooperative task to the robot, i.e. to program the robot, in the form of a sensory-motor action plan. 2) To do this in a semi-natural and real-time manner using spoken language. In this framework, a system for spoken language programming (SLP) is presented, and experimental results are presented from this prototype system. In Experiment 1, the human programs the robot to assist in assembling a small table. In Experiment 2, the generalization of the system is demonstrated as the user programs the robot to assist in taking the table apart. The SLP is evaluated in terms of the changes in efficiency as revealed by task completion time and number of command operations required to accomplish the tasks with and without SLP. Lessons learned are discussed, along with plans for improving the system, including developing a richer base of robot action and perception predicates that will allow the use of richer language. We thus demonstrate - for the first time - the capability for a human user to tell a humanoid what to do in a cooperative task so that in real time, the robot performs the task, and acquires new skills that significantly facilitate the cooperative human-robot interaction.

This research takes grammatical constructions (sentence form-to-meaning mappings) as an alternative to abstract generative grammars in the context of understanding the emergence of language. A model of sentence processing based on this construction grammar approach is presented, and then a series of neuropsychological and neurophysiological studies are reviewed that attempt to validate the model and to establish its neurophysiological underpinnings. The resulting model is demonstrated to provide insight into a developmental and evolutionary passage from unitary idiom-like holophrases to progressively more abstract grammatical constructions. The model is then functionally validated by its insertion into a perceptually grounded system that allows spoken language interaction with a human interlocutor. The potential utility of this emergence approach in understanding language is discussed.