As robots start to execute complex manipulation tasks, they are expected to improve their skill set over time as humans do. A prominent approach to accomplish this is having robots to keep models of their actions based on their experiences in order to improve their action executions in the future. In this paper, we present such a methodology where robots start to execute some actions with random parameters and record their generic execution logs with semantic annotations in a symbolic knowledge base for robots. Using the data inside logs, multivariate Gaussian mixture models are fitted to the high-level action parameters for later executions. These affordance models are being updated whenever a new execution is carried on. In essence, robots can use these continuously-updated probabilistic model for improving their actions To prove the applicability we demonstrate opening-a-fridge-door experiments with a PR2 robot.

To enable robots to perform human-level tasks flexibly in varying conditions, we need a mechanism that allows them to exchange knowledge between themselves for crowd-sourcing the knowledge gap problem. One approach to achieve this is to equip a cloud application with a range of encyclopedic knowledge (i.e. ontologies) and execution logs of different robots performing the same tasks in different environments. In this paper, we show how knowledge exchange between robots can be done using OPENEASE as the cloud application. We equipped OPENEASE with ontologies about the kitchen domain, execution logs of three robots operating in two different kitchens, and semantic descriptions of both environments. By addressing two different use cases, we show that two PR2 robots and one Fetch robot can successfully adapt each other's plan parameters and sub symbolic data to the experiments that they are conducting.

To enable robots to work on real home environments, they have to not only consider common knowledge in the global society, but also be aware of existing rules there. Since such “local rules” are not describable beforehand, robot agents must acquire them through their lives after deployment. To achieve this, we developed a framework that a) lets robots record long-term episodic memories in their deployed environments, b) autonomously builds probabilistic object localization map as structurization of logged data and c) make adapted task plans based on the map. We equipped our framework on PR2 and Fetch robots operating and recording episodic memory for 41 days with semantic common knowledge of the environment. We also conducted demonstrations in which a PR2 robot tidied up a room, showing that the robot agent can successfully plan and execute local-rule-aware home assistive tasks by using our proposed framework.