Service robots are expected to be more autonomous and work effectively in human-centric environments. This implies that robots should have special capabilities, such as learning from past experiences and real-time object category recognition. This paper proposes an open-ended 3D object recognition system which concurrently learns both the object categories and the statistical features for encoding objects. In particular, we propose an extension of Latent Dirichlet Allocation to learn structural semantic features (i.e., visual topics), from low-level feature co-occurrences, for each category independently. Moreover, topics in each category are discovered in an unsupervised fashion and are updated incrementally using new object views. In this way, the advantages of both the (hand-crafted) local features and the (learned) structural semantic features have been considered and combined in an efficient way. An extensive set of experiments has been performed to assess the performance of the proposed Local-LDA in terms of descriptiveness, scalability, and computation time. Experimental results show that the overall classification performance obtained with Local-LDA is clearly better than the best performances obtained with the state-of-the-art approaches. Moreover, the best scalability, in terms of number of learned categories, was obtained with the proposed Local-LDA approach, closely followed by a Bag-of-Words (BoW) approach. Concerning computation time, the best result was obtained with BoW, immediately followed by the Local-LDA approach.

Service robots are expected to autonomously and efficiently work in human-centric environments. For this type of robots, object perception and manipulation are challenging tasks due to need for accurate and real-time response. This paper presents an interactive open-ended learning approach to recognize multiple objects and their grasp affordances concurrently. This is an important contribution in the field of service robots since no matter how extensive the training data used for batch learning, a robot might always be confronted with an unknown object when operating in human-centric environments. The paper describes the system architecture and the learning and recognition capabilities. Grasp learning associates grasp configurations (i.e., end-effector positions and orientations) to grasp affordance categories. The grasp affordance category and the grasp configuration are taught through verbal and kinesthetic teaching, respectively. A Bayesian approach is adopted for learning and recognition of object categories and an instance-based approach is used for learning and recognition of affordance categories. An extensive set of experiments has been performed to assess the performance of the proposed approach regarding recognition accuracy, scalability and grasp success rate on challenging datasets and real-world scenarios.

Experience-based planning domains (EBPDs) have been proposed to improve problem solving by learning from experience. They rely on acquiring and using task knowledge, i.e., activity schemata, for generating solutions to problem instances in a class of tasks. Using Three-Valued Logic Analysis (TVLA), we extend our previous work to generate a set of conditions that determine the scope of applicability of an activity schema. The inferred scope is an abstract representation of a potentially unbounded set of problems, in the form of a 3-valued logical structure, which is used to test the applicability of the respective activity schema for solving different task problems. We validate this work on two classical planning domains and a simulated PR2 in Gazebo.

There is growing need for robots that can interact with people in everyday situations. For service robots, it is not reasonable to assume that one can pre-program all object categories. Instead, apart from learning from a batch of labelled training data, robots should continuously update and learn new object categories while working in the environment. This paper proposes a cognitive architecture designed to create a concurrent 3D object category learning and recognition in an interactive and open-ended manner. In particular, this cognitive architecture provides automatic perception capabilities that will allow robots to detect objects in highly crowded scenes and learn new object categories from the set of accumulated experiences in an incremental and open-ended way. Moreover, it supports constructing the full model of an unknown object in an on-line manner and predicting next best view for improving object detection and manipulation performance. We provide extensive experimental results demonstrating system performance in terms of recognition, scalability, next-best-view prediction and real-world robotic applications.

To deploy a robot in a human-centric environment, it is important that the robot is able to continuously acquire and update object categories while working in the environment. Therefore, autonomous robots must have the ability to continuously execute learning and recognition in a concurrent or interleaved fashion. One of the main challenges in unconstrained human environments is to cope with the effects of context change. This paper presents two main contributions: (i) an approach for evaluating open-ended object category learning and recognition methods in multi-context scenarios; (ii) evaluation of different object category learning and recognition approaches regarding their ability to cope with the effects of context change. Off-line evaluation approaches such as cross-validation do not comply with the simultaneous nature of learning and recognition. A teaching protocol, supporting context change, was therefore designed and used in this work for experimental evaluation. Seven learning and recognition approaches were evaluated and compared using the protocol. The best performance, in terms of number of learned categories, was obtained with a recently proposed local variant of Latent Dirichlet Allocation (LDA), closely followed by a Bag-of-Words (BoW) approach. In terms of adaptability, i.e. coping with context change, the best result was obtained with BoW, immediately followed by the local LDA variant.

This paper addresses robot task model learning and planning with loops. By detecting and modeling loops in solved tasks it is possible to learn and solve wider classes of problems. We extend our previous work on experience-based planning domains in robotics to detect, represent and generate loops in action sequences. This approach provides methods for, (i) conceptualizing robot experiences possibly containing loops and learning high-level robot activity schemata with loops; and (ii) instantiating schemata with loops for solving problem instances of the same task with varying sets of objects. Demonstrations of this system in both real and simulated environments prove its potentialities.

Object representation is one of the most challenging tasks in robotics because it must provide reliable information in real-time to enable the robot to physically interact with the objects in its environment. To ensure reliability, a global object descriptor must be computed based on a unique and repeatable object reference frame. Moreover, the descriptor should contain enough information enabling to recognize the same or similar objects seen from different perspectives. This paper presents a new object descriptor named Global Orthographic Object Descriptor (GOOD) designed to be robust, descriptive and efficient to compute and use. The performance of the proposed object descriptor is compared with the main state-of-the-art descriptors. Experimental results show that the overall classification performance obtained with GOOD is comparable to the best performances obtained with the state-of-the-art descriptors. Concerning memory and computation time, GOOD clearly outperforms the other descriptors. Therefore, GOOD is especially suited for real-time applications.

Robots are still not able to grasp all unforeseen objects. Finding a proper grasp configuration, i.e. the position and orientation of the arm relative to the object, is still challenging. One approach for grasping unforeseen objects is to recognize an appropriate grasp configuration from previous grasp demonstrations. The underlying assumption in this approach is that new objects that are similar to known ones (i.e. they are familiar) can be grasped in a similar way. However finding a grasp representation and a grasp similarity metric is still the main challenge in developing an approach for grasping familiar objects. In this paper, interactive object view learning and recognition capabilities are integrated in the process of learning and recognizing grasps. The object view recognition module uses an interactive incremental learning approach to recognize object view labels. The grasp pose learning approach uses local and global visual features of a demonstrated grasp to learn a grasp template associated with the recognized object view. A grasp distance measure based on Mahalanobis distance is used in a grasp template matching approach to recognize an appropriate grasp pose. The experimental results demonstrate the high reliability of the developed template matching approach in recognizing the grasp poses. Experimental results also show how the robot can incrementally improve its performance in grasping familiar objects.

Most robots lack the ability to learn new objects from past experiences. To migrate a robot to a new environment one must often completely re-generate the knowledge- base that it is running with. Since in open-ended domains the set of categories to be learned is not predefined, it is not feasible to assume that one can pre-program all object categories required by robots. Therefore, autonomous robots must have the ability to continuously execute learning and recognition in a concurrent and interleaved fashion. This paper proposes an open-ended 3D object recognition system which concurrently learns both the object categories and the statistical features for encoding objects. In particular, we propose an extension of Latent Dirichlet Allocation to learn structural semantic features (i.e. topics) from low-level feature co-occurrences for each category independently. Moreover, topics in each category are discovered in an unsupervised fashion and are updated incrementally using new object views. The approach contains similarities with the organization of the visual cortex and builds a hierarchy of increasingly sophisticated representations. Results show the fulfilling performance of this approach on different types of objects. Moreover, this system demonstrates the capability of learning from few training examples and competes with state-of-the-art systems.

In open-ended domains, robots must continuously learn new object categories. When the training sets are created offline, it is not possible to ensure their representativeness with respect to the object categories and features the system will find when operating online. In the Bag of Words model, visual codebooks are usually constructed from training sets created offline. This might lead to non-discriminative visual words and, as a consequence, to poor recognition performance. This paper proposes a visual object recognition system which concurrently learns in an incremental and online fashion both the visual object category representations as well as the codebook words used to encode them. The codebook is defined using Gaussian Mixture Models which are updated using new object views. The approach contains similarities with the human visual object recognition system: evidence suggests that the development of recognition capabilities occurs on multiple levels and is sustained over large periods of time. Results show that the proposed system with concurrent learning of object categories and codebooks is capable of learning more categories, requiring less examples, and with similar accuracies, when compared to the classical Bag of Words approach using codebooks constructed offline.

Heart disorders are one of the most problematic issues of human health. There are currently many efforts to reduce the time for first assistance based on electronic systems that continuously records the electric heart activity (ECG), for further inspection and anomalies detection. However, an efficient automatic detection of heart problems still being a big challenge mainly due to difficulty of accessing some specific cardiac problems, signal variations between different patients and the level of uncertainty caused by the absence of significant data or medical elements from patient. This research explores the use of several health condition signals and indicators for heart monitoring and injuries detection. This paper presents the first approach of a portable solution running decision support algorithms to produce medical diagnosis based on electrical and sound heart signals, ambient conditions, patient mobility, patient personal data and clinical history.

This paper addresses the problem of grounding semantic representations in intelligent service robots. In particular, this work contributes to addressing two important aspects, namely the anchoring of object symbols into the perception of the objects and the grounding of object category symbols into the perception of known instances of the categories. The paper discusses memory requirements for storing both semantic and perceptual data and, based on the analysis of these requirements, proposes an approach based on two memory components, namely a semantic memory and a perceptual memory. The perception, memory, learning and interaction capabilities, and the perceptual memory, are the main focus of the paper. Three main design options address the key computational issues involved in processing and storing perception data: a lightweight, NoSQL database, is used to implement the perceptual memory; a thread-based approach with zero copy transport of messages is used in implementing the modules; and a multiplexing scheme, for the processing of the different objects in the scene, enables parallelization. The system is designed to acquire new object categories in an incremental and open-ended way based on user-mediated experiences. The system is fully integrated in a broader robot system comprising low-level control and reactivity to high-level reasoning and learning.

Intelligent service robots should be able to improve their knowledge from accumulated experiences through continuous interaction with the environment, and in particular with humans. A human user may guide the process of experience acquisition, teaching new concepts, or correcting insufficient or erroneous concepts through interaction. This paper reports on work towards interactive learning of objects and robot activities in an incremental and open-ended way. In particular, this paper addresses human-robot interaction and experience gathering. The robot's ontology is extended with concepts for representing human-robot interactions as well as the experiences of the robot. The human-robot interaction ontology includes not only instructor teaching activities but also robot activities to support appropriate feedback from the robot. Two simplified interfaces are implemented for the different types of instructions including the teach instruction, which triggers the robot to extract experiences. These experiences, both in the robot activity domain and in the perceptual domain, are extracted and stored in memory, and they are used as input for learning methods. The functionalities described above are completely integrated in a robot architecture, and are demonstrated in a PR2 robot.

Heart disorders are one of the most problematic issues of human health. There are currently many efforts to reduce the time for first assistance based on electronic systems that continuously records the electric heart activity for further inspection and anomalies detection. The most popular are portable monitoring systems based on the Electrocardiogram (ECG) signal. However, an efficient detection of heart problems still being a big challenge mainly due to the difficulty of accessing some specific cardiac problems and signal variations between different patients. A different technique for heart diagnosing is based on spatial recording of electrical heart activity, commonly known as Vectorcardiogram (VCG). VCG is pointed by several authors as a more efficient tool than ECG for heart inspection and problem detection. This research explores the possibility of using VCG signals in a portable device for constant heart monitoring and injuries detection. It is presented a portable solution with VCG recording and digital signal processing for automatic diagnosis. This paper covers the aspects of the VCG signal and its ability to be converted into a 12-lead ECG signal. It is also presented a system for automatic diagnosis based on VCG which is based on a multichannel portable hardware platform to support noise cancellation, parameter extraction and decision making algorithms.

This paper presents an evaluation of the dialogue manager (DM) used on Carl, a prototype of an intelligent service robot, designed and developed having in mind hosting tasks in a building or event. The developed DM, based on the “Information State” approach, is described. In an experimental evaluation, in which 10 participants attempted to complete several interaction tasks with the robot, 81% of tasks were performed successfully. The results of an usability evaluation are also presented and discussed.

Knowledge plays significant roles in real environments to successfully complete service tasks. Almost all knowledge inference mechanisms assume complete and correct knowledge about environments. However, these environments are said to be partially observable and uncertain with respect to service robots. Thus, a robot cannot often complete the given service tasks. To cope with such a difficulty, we propose a novel weighted action-coupled semantic network (wASN), which represents the world by symbolic relationships and their weights, and its association mechanism using bi-directional spreading activation model which recommends goal-oriented actions from action-coupled knowledge. The improvisational actions provide more opportunities for a service robot to achieve goals.

The coordination methodologies of CAMBADA, a robotic soccer team designed to participate in the RoboCup middle-size league (MSL), are presented. The approach, which relies on information sharing and integration within the team, is based on formations, flexible positionings and dynamic role and positioning assignment. Role/positioning assignment follows a new priority-based algorithm that maintains a competitive formation, covering the most important roles/positionings when malfunctions lead to a reduction of the team size. Coordinated procedures for passing and setplays have also been implemented. With this design, CAMBADA reached the 1st place in the RoboCup'2008 world championship. Competition results and performance measures computed from logs and videos of real competition games are presented and discussed.

Interest on using mobile autonomous agents has been growing, recently, due to their capacity to cooperate for diverse purposes, from rescue to demining and security. However, such cooperation requires the exchange of state data that is time sensitive while achieving timeliness with RF communication is intrinsically difficult due to the openess of the medium. This paper describes a communication layer that improves the timeliness of periodic data exchanges among the team reducing the chances of lost packets caused by collisions between team members. In particular, the paper extends a previous proposal for an adaptive TDMA protocol with new self-configuration capabilities according to the current number of active team members. This feature further reduces the likelyhood of collisions within the team. Several experimental results with an actual system implementation show the effectiveness of the proposed solution.

Language sets humans apart from other animals. In the context of robotics, language is a fast and surprisingly efficient channel to communicate thoughts; it is used to instruct and teach and is arg...

Motivated by the need to support language-based communication between robots and their human users, as well as grounded symbolic reasoning, this paper presents a learning architecture that can be used by robotic agents for long-term and open-ended category acquisition. To be more adaptive and to improve learning performance as well as memory usage, this learning architecture includes a metacognitive processing component. Multiple object representations and multiple classifiers and classifier combinations are used. At the object level, the main similarity measure is based on a multi-resolution matching algorithm. Categories are represented as sets of known instances. In this instance-based approach, storing and forgetting rules optimise memory usage. Classifier combinations are based on majority voting and the Dempster–Shafer evidence theory. All learning computations are carried out during the normal execution of the agent, which allows continuous monitoring of the performance of the different classifiers. The measured classification successes of the individual classifiers support an attentional selection mechanism, through which classifier combinations are dynamically reconfigured and a specific classifier is chosen to predict the category of a new unseen object. A simple physical agent, incorporating these learning capabilities, is used to test the approach. A long-term experiment was carried out having in mind the open-ended nature of category learning. With the help of a human mediator, the agent incrementally learned 68 categories of real-world objects visually perceivable through an inexpensive camera. Various aspects of the approach are evaluated through systematic experiments.

The paper focuses on an Information State (IS) based dialogue manager developed for Carl, an intelligent mobile robot. It uses a Knowledge Acquisition and Management (KAM) module that integrates information obtained from various interlocutors. This mixed-initiative dialogue manager (DM) handles pronoun resolution, is capable of performing different kinds of clarification/confirmation questions and generates observations based on the current knowledge acquired. An evaluation of the DM on knowledge acquisition tasks is shown.

The recent evolution of Carl, an intelligent mobile robot, is presented. The paper focuses on robust spoken language understanding (SLU) and on knowledge representation and reasoning (KRR). Robustness in SLU is achieved through the combination of deep and shallow parsing, tolerating non-grammatical utterances. The KRR module supports the integration of information coming from different interlocutors and is capable of handling contradictory facts. The knowledge representation language is based on semantic networks. Question answering is based on deductive as well as inductive inference. A preliminary evaluation of the efficiency of the SLU/KRR system, for the purpose of knowledge acquisition, is presented.

This paper describes the architecture and current capabilities of Carl, a prototype of an intelligent service robot, designed having in mind such tasks as serving food in a reception or acting as a host in an organization. The approach that has been followed in the design of Carl is based on an explicit concern with the integration of the major dimensions of intelligence, namely communication, action, reasoning and learning. The paper focuses on the multi-modal human-robot communication capabilities of Carl, since these have been significantly improved during the last year.

The development of robots capable of accepting instructions in terms of familiar concepts to the user is still a challenge. For these robots to emerge it’s essential the development of natural language interfaces, since this is regarded as the only interface acceptable for a machine which expected to have a high level of interactivity with Man. Our group has been involved for several years in the development of a mobile intelligent robot, named Carl, designed having in mind such tasks as serving food in a reception or acting as a host in an organization. The approach that has been followed in the design of Carl is based on an ex-plicit concern with the integration of the major dimensions of intelligence, namely Communication, Action, Reasoning and Learning. This paper focuses on the multi-modal human-robot language communication capabilities of Carl, since these have been significantly improved during the last year. 1.

Carl is a prototype of an intelligent service robot, designed having in mind for such tasks as serving food in a reception or acting as a host in an organization. The approach that has been followed in the design of Carl is based on an explicit concern with the integration of the major dimensions of intelligence, namely communication, action, reasoning and learning. Although different communities have thoroughly studied these dimensions in the past, their integration has seldom been attempted in a systematic way. This paper describes the software architecture of Carl as well as the main modules, from sensor fusion and navigation to language-level communication and learning.

The development of robots that are able to accept instructions, via a friendly interface, in terms of concepts that are familiar to a human user remains a challenge. It is argued that designing and building such intelligent robots can be seen as the problem of integrating four main dimensions: human-robot communication, sensory motor skills and perception, decision-making capabilities, and learning. Although these dimensions have been thoroughly studied in the past, their integration has seldom been attempted in a systematic way. It is further argued that, for the common user, the only sufficiently practical interface is spoken language. The "body and soul" of CARL (Communication, Action, Reasoning and Learning), a robot currently under construction in our laboratory, are presented. The spoken-language interface is given particular attention.

The paper presents the architecture and algorithms developed for Dom Dinis, a simple compass-guided robot built by the authors. This includes environment exploration, task planning and task execution. Environment exploration, based on repeating a reactive goal search, enables a progressive construction of a grid based map. Based on the (possibly incomplete) map, the robot is able to plan its tasks. The execution capabilities of the robot include exception handling. Essential to all these capabilities is the knowledge of the robot's position in the world. The position is computed based on tracking traversed distances and followed orientations. Orientation is given by a compass. Dom Dinis does not use wheel encoders at all.

A framework for planning and supervision of robotized assembly tasks is initially presented, with emphasis on failure recovery. The approach to the integration services and the modeling of tasks, resources and environment is briefly described. A planning strategy and domain knowledge for nominal plan execution is presented. Through the use of machine learning techniques, the supervision architecture will be given capabilities for improving its performance over time. In particular, an approach for memorizing failure recovery episodes, based on abstraction, deductive generalization and feature construction, is presented. Recovery planning consists of adapting plan skeletons from similar episodes previously occurred.

A generic architecture for evolutive supervision of robotized assembly tasks, in a context of integrated manufacturing systems, is presented. This architecture provides, at different levels of abstraction, functions for dispatching actions, monitoring their execution, and diagnosing and recovering from failures. The problem of integration of legacy systems is discussed and an implementation approach described. Modeling execution failures through taxonomies and causal relations plays a central role in diagnosis and recovery. Through the use of machine learning techniques, the supervision architecture will be given capabilities for improving its performance over time. Particular attention is given to the inductive generation of structured classification knowledge for diagnosis. Methodologies used, performed experiments, and obtained results are described in detail.

A generic architecture for evolutive supervision of robotized assembly tasks is presented. This architecture provides, at different levels of abstraction, functions for dispatching actions, monitoring their execution, and diagnosing and recovering from failures. Modeling execution failures through taxonomies and causal networks plays a central role in diagnosis and recovery. Through the use of machine learning techniques, the supervision architecture will be given capabilities for improving its performance over time. Particular attention is given to the inductive generation of structured classification knowledge for diagnosis. The applied methodologies, performed experiments and obtained results are described in detail.

Research results concerning error detection and recovery in robotized assembly systems, key components of flexible manufacturing systems, are presented. A planning strategy and domain knowledge for nominal plan execution and for error recovery is described. A supervision architecture provides, at different levels of abstraction, functions for dispatching actions, monitoring their execution, and diagnosing and recovering from failures. Through the use of machine learning techniques, the supervision architecture will be given capabilities for improving its performance over time. Particular attention is given to the inductive generation of structured classification knowledge for diagnosis.

A generic architecture for execution supervision of robotic assembly tasks is presented. This architecture provides, at different levels of abstraction, functions for dispatching actions, monitoring their execution, and diagnosing and recovering from failures. Modeling execution failures through taxonomies and causal networks plays a central role in diagnosis and recovery. A discussion on the process of acquisition of such monitoring knowledge is made. Through the use of machine learning techniques, the supervision architecture will be given capabilities for improving its performance over time. Preliminary results of applying machine learning in this area are presented and planned extensions discussed.>