As chatbots continue to evolve toward humanlike, real-world, interactions, multimodality remains an active area of research and exploration.So far, efforts to integrate multimodality into chatbots have primarily focused on image-centric tasks, such as visual dialogue and image-based instructions, placing emphasis on the "eyes" of human perception while neglecting the "ears", namely auditory aspects.Moreover, these studies often center around static interactions that focus on discussing the modality rather than naturally incorporating it into the conversation, which limits the richness of simultaneous, dynamic engagement.Furthermore, while multimodality has been explored in multi-party and multi-session conversations, task-specific constraints have hindered its seamless integration into dynamic, natural conversations.To address these challenges, this study aims to equip chatbots with "eyes and ears" capable of more immersive interactions with humans.As part of this effort, we introduce a new multimodal conversation dataset, Multimodal Multi-Session Multi-Party Conversation (M 3 C), and propose a novel multimodal conversation model featuring multimodal memory retrieval.Our model, trained on the M 3 C, demonstrates the ability to seamlessly engage in long-term conversations with multiple speakers in complex, real-world-like settings, effectively processing visual and auditory inputs to understand and respond appropriately.Human evaluations highlight the model's strong performance in maintaining coherent and dynamic interactions, demonstrating its potential for advanced multimodal conversational agents. 1

In the field of natural language processing, open-domain chatbots have emerged as an important research topic.However, a major limitation of existing open-domain chatbot research is its singular focus on short single-session dialogue, neglecting the potential need for understanding contextual information in multiple consecutive sessions that precede an ongoing dialogue.Among the elements that compose the context in multi-session conversation settings, the time intervals between sessions and the relationships between speakers would be particularly important.Despite their importance, current research efforts have not sufficiently addressed these dialogical components.In this paper, we introduce a new 1M multisession dialogue dataset, called CONVERSA-TION CHRONICLES, for implementing a longterm conversation setup in which time intervals and fine-grained speaker relationships are incorporated.Following recent works, we exploit a large language model to produce the data.The extensive human evaluation shows that dialogue episodes in CONVERSATION CHRONI-CLES reflect those properties while maintaining coherent and consistent interactions across all the sessions.We also propose a dialogue model, called REBOT, which consists of chronological summarization and dialogue generation modules using only around 630M parameters.When trained on CONVERSATION CHRONI-CLES, REBOT demonstrates long-term context understanding with a high human engagement score. 1