Tu Anh Dinh, Carlos Mullov, Leonard Bärmann, Zhaolin Li, Danni Liu, Simon Reiß, Jueun Lee, Nathan Lerzer, Jianfeng Gao, Fabian Peller-Konrad, Tobias Röddiger, Alexander Waibel, Tamim Asfour, Michael Beigl, Rainer Stiefelhagen, Carsten Dachsbacher, Klemens Böhm, Jan Niehues. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

Artificial cognitive architectures traditionally rely on complex memory models to encode, store, and retrieve information. However, the conventional practice of transferring all data from working memory (WM) to long-term memory (LTM) leads to high data volumes and challenges in efficient information processing and access. Deciding what information to retain or discard within a robot’s LTM is particularly challenging since knowledge about future data utilization is absent. Drawing inspiration from human forgetting this paper implements and evaluates novel forgetting techniques that allow consolidation in the robot’s LTM only when new information is encountered. The proposed approach combines fast filtering during data transfer to the robot’s LTM with slower yet more precise forgetting mechanisms that are periodically evaluated for offline data deletion inside the LTM. We compare different mechanisms, utilizing metrics such as data similarity, data age, and consolidation frequency. The efficacy of forgetting techniques is evaluated by comparing their performance in a task where two ARMAR robots search through their LTM for past object locations in episodic ego-centric images and robot state data. Experimental results show that our forgetting techniques significantly reduce the space requirements of a robot’s LTM while maintaining its capacity to successfully perform tasks relying on LTM information. Notably, similarity-based forgetting methods outperform frequency- and time-based approaches. The combination of online frequency-based, online similarity-based, offline similarity-based, and time-based decay methods shows superior performance compared to using individual forgetting strategies.

To perform versatile mobile manipulation tasks in human-centered environments, the ability to efficiently transfer learned skills, knowledge, and experiences from one robot to another or across different environments is critical. In this paper, we present MAkEable, a versatile uni- and multi-manual mobile manipulation framework that facilitates the transfer of capabilities and knowledge across different tasks, environments, and robots. Our framework integrates an affordance-based task description into the memory-centric cognitive architecture of the ARMAR humanoid robot family, which supports the sharing of experiences and demonstrations for transferring mobile manipulation skills. By representing mobile manipulation actions through affordances, i. e., interaction possibilities of the robot with its environment, we provide a unifying framework for the autonomous uni- and multi-manual manipulation of known and unknown objects in various environments. We demonstrate MAkEable’s applicability in real-world experiments for multiple robots, tasks, and environments. This includes grasping known and unknown objects, object placing, bimanual object grasping, memory-enabled skill transfer in a drawer opening scenario across two different humanoid robots, and a pouring task learned from human demonstration. Code is available through our project page1.

Humanoid robots will be able to assist humans in their daily life, in particular due to their versatile action capabilities. However, while these robots need a certain degree of autonomy to learn and explore, they also should respect various constraints, for access control and beyond. We explore the novel field of incorporating privacy, security, and access control constraints with robot task planning approaches. We report preliminary results on the classical symbolic approach, deep-learned neural networks, and modern ideas using large language models as knowledge base. From analyzing their trade-offs, we conclude that a hybrid approach is necessary, and thereby present a new use case for the emerging field of neuro-symbolic artificial intelligence.

Humanoid robots will be able to assist humans in their daily life, in particular due to their versatile action capabilities. However, while these robots need a certain degree of autonomy to learn and explore, they also should respect various constraints, for access control and beyond. We explore incorporating privacy and security constraints (Activity-Centric Access Control and Deep Learning Based Access Control) with robot task planning approaches (classical symbolic planning and end-to-end learning-based planning). We report preliminary results on their respective trade-offs and conclude that a hybrid approach will most likely be the method of choice.

In the not too distant future, assistive humanoid robots will provide versatile assistance for coping with everyday life. In their interactions with humans, not only safety, but also security and privacy issues need to be considered. In this Blue Sky paper, we therefore argue that it is time to bring task planning and execution as a well-established field of robotics with access and usage control in the field of security and privacy closer together. In particular, the recently proposed activity-based view on access and usage control provides a promising approach to bridge the gap between these two perspectives. We argue that humanoid robots provide for specific challenges due to their task-universality and their use in both, private and public spaces. Furthermore, they are socially connected to various parties and require policy creation at runtime due to learning. We contribute first attempts on the architecture and enforcement layer as well as on joint modeling, and discuss challenges and a research roadmap also for the policy and objectives layer. We conclude that the underlying combination of decentralized systems' and smart environments' research aspects provides for a rich source of challenges that need to be addressed on the road to deployment.