This article presents an integrated robot system capable of interactive learning in dialogue with a human. Such a system needs to have several competencies and must be able to process different types of representations. In this article, we describe a collection of mechanisms that enable integration of heterogeneous competencies in a principled way. Central to our design is the creation of beliefs from visual and linguistic information, and the use of these beliefs for planning system behaviour to satisfy internal drives. The system is able to detect gaps in its knowledge and to plan and execute actions that provide information needed to fill these gaps. We propose a hierarchy of mechanisms which are capable of engaging in different kinds of learning interactions, e.g. those initiated by a tutor or by the system itself. We present the theory these mechanisms are build upon and an instantiation of this theory in the form of an integrated robot system. We demonstrate the operation of the system in the case of learning conceptual models of objects and their visual properties.

Model fitting is a fundamental component in computer vision for salient data selection, feature extraction and data parameterization. Conventional approaches such as the RANSAC family show limitations when dealing with data containing multiple models, high percentage of outliers or sample selection bias, commonly encountered in computer vision applications. In this paper, we present a novel model evaluation function based on Gaussian-weighted Jensen–Shannon divergence, and integrate into a particle swarm optimization (PSO) framework using ring topology. We avoid two problems from which most regression algorithms suffer, namely the requirements to specify inlier noise scale and the number of models. The novel evaluation method is generic and does not require any estimation of inlier noise. The continuous and meta-heuristic exploration facilitates estimation of each individual model while delivering the number of models automatically. Tests on datasets comprised of inlier noise and a large percentage of outliers (more than 90 % of the data) demonstrate that the proposed framework can efficiently estimate multiple models without prior information. Superior performance in terms of processing time and robustness to inlier noise is also demonstrated with respect to state of the art methods.

As an important information resource, visual perception has been widely employed for various indoor mobile robots. The common-sense knowledge about object locality (CSOL), e.g. a cup is usually located on the table top rather than on the floor and vice versa for a trash bin, is a very helpful context information for a robotic visual search task. In this paper, we propose an online knowledge acquisition mechanism for discovering CSOL, thereby facilitating a more efficient and robust robotic visual search. The proposed mechanism is able to create conceptual knowledge with the information acquired from the largest and the most diverse medium - the Internet. Experiments using an indoor mobile robot demonstrate the efficiency of our approach as well as reliability of goal-directed robot behaviour.

In this paper we present representations and mechanisms that facilitate continuous learning of visual concepts in dialogue with a tutor and show the implemented robot system. We present how beliefs about the world are created by processing visual and linguistic information and show how they are used for planning system behaviour with the aim at satisfying its internal drive - to extend its knowledge. The system facilitates different kinds of learning initiated by the human tutor or by the system itself. We demonstrate these principles in the case of learning about object colours and basic shapes.

Attention operators based on 2D image cues (such as color, texture) are well known and discussed extensively in the vision literature but are not ideally suited for robotic applications. In such contexts it is the 3D structure of scene elements that makes them interesting or not. We show how a bottom-up exploration mechanism that fuses 2D saliency-based conspicuity with spatial abstraction resulting from the coherent plane estimation and stereo line detection is well suited for typical indoor robotics tasks. This spatial abstraction is performed by a joint probabilistic model which takes the interaction of stereo line detection and 3D supporting plane estimation into consideration. By maximizing the probability of the joint model, our method facilitates reduction of false-positive stereo line detection and refines the estimation of supporting surface simultaneously. Experiments demonstrate that our approach provides more accurate and plausible attention.

Traffic psychology presents interesting avenues towards the development of Intelligent Transportation Systems (ITS). Analysis of driver state, emotion and behavior are important components of traffic psychology. While being comprehensive in terms of theoretical frameworks, these analyses lack neurobiological computational models for evaluation. In this paper, we develop computational models for driver state and behavior, also known as Mental State Inference (MSI) based on the Mirror Neuron System (MNS) architecture. The integrated system combines neurobiological models with computer vision techniques for traffic monitoring from surveillance video leading to MSI and event recognition. Evaluation of the system is carried out in terms of actual, psychophysical as well as neurobiological criteria on both simulated and real data. Results demonstrate event and mental state recognition convergence within 0.5 normalized time units for the designed event models on synthetic and real data. The model is also robust to perturbations and is aligned to behavior expected at the psychophysical level.