In this work, we review research studies that combine Reinforcement Learning (RL) and Large Language Models (LLMs), two areas that owe their momentum to the development of Deep Neural Networks (DNNs). We propose a novel taxonomy of three main classes based on the way that the two model types interact with each other. The first class, RL4LLM, includes studies where RL is leveraged to improve the performance of LLMs on tasks related to Natural Language Processing (NLP). RL4LLM is divided into two sub-categories depending on whether RL is used to directly fine-tune an existing LLM or to improve the prompt of the LLM. In the second class, LLM4RL, an LLM assists the training of an RL model that performs a task that is not inherently related to natural language. We further break down LLM4RL based on the component of the RL training framework that the LLM assists or replaces, namely reward shaping, goal generation, and policy function. Finally, in the third class, RL+LLM, an LLM and an RL agent are embedded in a common planning framework without either of them contributing to training or fine-tuning of the other. We further branch this class to distinguish between studies with and without natural language feedback. We use this taxonomy to explore the motivations behind the synergy of LLMs and RL and explain the reasons for its success, while pinpointing potential shortcomings and areas where further research is needed, as well as alternative methodologies that serve the same goal.

Interactive spherical displays offer numerous opportunities for engagement and education in public settings. Prior work established that users' touch-gesture patterns on spherical displays differ from those on flatscreen tabletops, and speculated that these differences stem from dissimilarity in how users conceptualize interactions with these two form factors. We analyzed think-aloud data collected during a gesture elicitation study to understand adults' and children's (ages 7 to 11) conceptual models of interaction with spherical displays and compared them to conceptual models of interaction with tabletop displays from prior work. Our findings confirm that the form factor strongly influenced users' mental models of interaction with the sphere. For example, participants conceptualized that the spherical display would respond to gestures in a similar way as real-world spherical objects like physical globes. Our work contributes new understanding of how users draw upon the perceived affordances of the sphere as well as prior touchscreen experience during their interactions.