Affordances - i.e. possibilities for action that an environment or objects in it provide - are important for robots operating in human environments to perceive. Existing approaches train such capabilities on annotated static images or shapes. This work presents a novel dataset for affordance learning of common household tasks. Unlike previous approaches, our dataset consists of video sequences demonstrating the tasks from first- and third-person perspectives, along with metadata about the affordances that are manifested in the task, and is aimed towards training perception systems to recognize affordance manifestations. The demonstrations were collected from several participants and in total record about seven hours of human activity. The variety of task performances also allows studying preparatory maneuvers that people may perform for a task, such as how they arrange their task space, which is also relevant for collaborative service robots.

Situationally-aware artificial agents operating with competence in natural environments face several challenges: spatial awareness, object affordance detection, dynamic changes and unpredictability. A critical challenge is the agent’s ability to identify and monitor environmental elements pertinent to its objectives. Our research introduces a neurosymbolic modular architecture for reactive robotics. Our system combines a neural component performing object recognition over the environment and image processing techniques such as optical flow, with symbolic representation and reasoning. The reasoning system is grounded in the embodied cognition paradigm, via integrating image schematic knowledge in an ontological structure. The ontology is operatively used to create queries for the perception system, decide on actions, and infer entities’ capabilities derived from perceptual data. The combination of reasoning and image processing allows the agent to focus its perception for normal operation as well as discover new concepts for parts of objects involved in particular interactions. The discovered concepts allow the robot to autonomously acquire training data and adjust its subsymbolic perception to recognize the parts, as well as making planning for more complex tasks feasible by focusing search on those relevant object parts. We demonstrate our approach in a simulated world, in which an agent learns to recognize parts of objects involved in support relations. While the agent has no concept of handle initially, by observing examples of supported objects hanging from a hook it learns to recognize the parts involved in establishing support and becomes able to plan the establishment/destruction of the support relation. This underscores the agent’s capability to expand its knowledge through observation in a systematic way, and illustrates the potential of combining deep reasoning with reactive robotics in dynamic settings.

Abstract Human beings and other biological agents appear driven by curiosity to explore the affordances of their environments. Such exploration is its own reward – children have fun when playing – but it probably also serves the practical purpose of learning theories with which to predict outcomes of actions. Cognitive robots however have yet to match the performance of human beings at learning and reusing manipulation skills. In this paper, we implement a method that emulates the curiosity drive and uses it as a heuristic to guide (simulated) exploration of a particular task – pouring liquids. The result of this exploration is a collection of symbolic rules linking qualitative descriptions of object arrangements and the pouring action with qualitative descriptions of likely outcomes. The manner in which qualitative descriptions of object arrangements and actions are converted to numerical descriptions for the purpose of simulation parametrization is via probability distributions, which themselves are adjusted in the process of simulated exploration. This allows the grounding of the symbolic descriptions to attempt to adapt itself to the task. The resulting symbolic rules form a theory that, together with the probability distributions that ground it in numerical parametrizations, is intended to be used to predict qualitative outcomes or select manners of pouring towards achieving a goal.

When considering two concepts in terms of extensional logic, their combination will often be trivial, returning an empty extension. Consider e.g. “a Fish Vehicle”, i.e., “a Vehicle which is also a Fish”. Still, people use sophisticated strategies to produce new, non-empty concepts. All these strategies involve the human ability to mend the conflicting attributes of the input concepts and to create new properties of the combination. We focus in particular on the case where a Head concept has superior ‘asymmetric’ control over steering the resulting combination (or hybridisation) with a Modifier concept. Specifically, we propose a dialogical model of the cognitive and logical mechanics of this asymmetric form of hybridisation. Its implementation is then evaluated using a combination of example ontologies.

When people combine concepts these are often characterised as “hybrid”, “impossible”, or “humorous”. However, when simply considering them in terms of extensional logic, the novel concepts understood as a conjunctive concept will often lack meaning having an empty extension (consider “a tooth that is a chair”, “a pet flower”, etc.). Still, people use different strategies to produce new non-empty concepts: additive or integrative combination of features, alignment of features, instantiation, etc. All these strategies involve the ability to deal with conflicting attributes and the creation of new (combinations of) properties. We here consider in particular the case where a Head concept has superior ‘asymmetric’ control over steering the resulting concept combination (or hybridisation) with a Modifier concept. Specifically, we propose a dialogical approach to concept combination and discuss an implementation based on axiom weakening, which models the cognitive and logical mechanics of this asymmetric form of hybridisation.

Autonomous robots struggle with plan adaption in uncertain and changing environments. Although modern robots can make popcorn and pancakes, they are incapable of performing such tasks in unknown settings and unable to adapt action plans if ingredients or tools are missing. Humans are continuously aware of their surroundings. For robotic agents, real-time state updating is time-consuming and other methods for failure handling are required. Taking inspiration from human cognition, we propose a plan adaption method based on event segmentation of the image-schematic states of subtasks within action descriptors. For this, we reuse action plans of the robotic architecture CRAM and ontologically model the involved objects and image-schematic states of the action descriptor cutting. Our evaluation uses a robot simulation of the task of cutting bread and demonstrates that the system can reason about possible solutions to unexpected failures regarding tool use.

In recent years, there has been a growth in the use of reference conceptual models to capture information about complex and critical domains. However, as the complexity of domain increases, so does the size and complexity of the models that represent them. Over the years, different techniques for complexity management in large conceptual models have been developed. In particular, several authors have proposed different techniques for model abstraction. In this paper, we leverage on the ontologically well-founded semantics of the modeling language OntoUML to propose a novel approach for model abstraction in conceptual models. We provide a precise definition for a set of Graph-Rewriting rules that can automatically produce much-reduced versions of OntoUML models that concentrate the models' information content around the ontologically essential types in that domain, i.e., the so-called Kinds. The approach has been implemented using a model-based editor and tested over a repository of OntoUML models.

We discuss steps towards a formalisation of the principles of an agentive naïve proto-physics, designed to match a level of abstraction that reflects the pre-linguistic conceptualisations and elementary notions of agency, as they develop during early human cognitive development. To this end, we present an agentive extension of the multi-dimensional image schema logic ISL based on variants of STIT theory, thus replacing the temporal dimension of ISL with an action-agnostic theory of agency. To begin grasping the notion of ‘animate agent’, we apply the newly defined logic to model the image schematic notion of ‘self movement’ as a means to distinguish the agentive capabilities of a mouse from those of a ball. Finally, we outline the prospects for employing the theory in cognitive robotics.

In recent years, there has been a growth in the use of reference conceptual models, in general, and domain ontologies, in particular, to capture information about complex and critical domains. These models play a fundamental role in different types of critical semantic interoperability tasks. Therefore, it is essential that domain experts are able to understand and reason using the models' content. In other words, it is important that conceptual models are cognitively tractable. However, it is unavoidable that when the information of the represented domain grows, so does the size and complexity of the artifacts and models that represent them. For this reason, more sophisticated techniques for complexity management in ontology-driven conceptual models, need to be developed. Some approaches are based on the notion of model modularization. In this paper, we follow the work on model modularization to present an approach for view extraction for the ontology-driven conceptual modeling language OntoUML. We provide a formal definition for ontological views over OntoUML conceptual models that completely leverages on the ontologically well-grounded real-world semantics of that language. Moreover, we present a plug-in tool, particularly developed for an OntoUML model-based editor that implements this formal view structure in terms of queries defined over the OntoUML metamodel embedded in that tool.