Large language models (LLMs) can sometimes report the strategies they actually use to solve tasks, yet at other times seem unable to recognize those strategies that govern their behavior. This suggests a limited degree of metacognition --- the capacity to monitor one's own cognitive processes for subsequent reporting and self-control. Metacognition enhances LLMs' capabilities in solving complex tasks but also raises safety concerns, as models may obfuscate their internal processes to evade neural-activation-based oversight (e.g., safety detector). Given society's increased reliance on these models, it is critical that we understand their metacognitive abilities. To address this, we introduce a neuroscience-inspired \emph{neurofeedback} paradigm that uses in-context learning to quantify metacognitive abilities of LLMs to \textit{report} and \textit{control} their activation patterns. We demonstrate that their abilities depend on several factors: the number of in-context examples provided, the semantic interpretability of the neural activation direction (to be reported/controlled), and the variance explained by that direction. These directions span a ``metacognitive space'' with dimensionality much lower than the model's neural space, suggesting LLMs can monitor only a small subset of their neural activations. Our paradigm provides empirical evidence to quantify metacognition in LLMs, with significant implications for AI safety (e.g., adversarial attack and defense).

Understanding connections between artificial and biological intelligent systems can reveal fundamental principles of general intelligence. While many artificial intelligence models have a neuroscience counterpart, such connections are largely missing in Transformer models and the self-attention mechanism. Here, we examine the relationship between interacting attention heads and human episodic memory. We focus on induction heads, which contribute to in-context learning in Transformer-based large language models (LLMs). We demonstrate that induction heads are behaviorally, functionally, and mechanistically similar to the contextual maintenance and retrieval (CMR) model of human episodic memory. Our analyses of LLMs pre-trained on extensive text data show that CMR-like heads often emerge in the intermediate and late layers, qualitatively mirroring human memory biases. The ablation of CMR-like heads suggests their causal role in in-context learning. Our findings uncover a parallel between the computational mechanisms of LLMs and human memory, offering valuable insights into both research fields.

One of the most remarkable features of the human brain is its ability to adapt rapidly and efficiently to external task demands. Novel and non-routine tasks, for example, are implemented faster than structural connections can be formed. The neural underpinnings of these dynamics are far from understood. Here we develop and apply novel methods in network science to quantify how patterns of functional connectivity between brain regions reconfigure as human subjects perform 64 different tasks. By applying dynamic community detection algorithms, we identify groups of brain regions that form putative functional communities, and we uncover changes in these groups across the 64-task battery. We summarize these reconfiguration patterns by quantifying the probability that two brain regions engage in the same network community (or putative functional module) across tasks. These tools enable us to demonstrate that classically defined cognitive systems-including visual, sensorimotor, auditory, default mode, fronto-parietal, cingulo-opercular and salience systems-engage dynamically in cohesive network communities across tasks. We define the network role that a cognitive system plays in these dynamics along the following two dimensions: (i) stability vs. flexibility and (ii) connected vs. isolated. The role of each system is therefore summarized by how stably that system is recruited over the 64 tasks, and how consistently that system interacts with other systems. Using this cartography, classically defined cognitive systems can be categorized as ephemeral integrators, stable loners, and anything in between. Our results provide a new conceptual framework for understanding the dynamic integration and recruitment of cognitive systems in enabling behavioral adaptability across both task and rest conditions. This work has important implications for understanding cognitive network reconfiguration during different task sets and its relationship to cognitive effort, individual variation in cognitive performance, and fatigue.