Performing large step-ups is a challenging task for a humanoid robot. It requires the robot to perform motions at the limit of its reachable workspace while straining to move its body upon the obstacle. This paper presents a non-linear trajectory optimization method for generating step-up motions. We adopt a simplified model of the centroidal dynamics to generate feasible Center of Mass trajectories aimed at reducing the torques required for the step-up motion. The activation and deactivation of contacts at both feet are considered explicitly. The output of the planner is a Center of Mass trajectory plus an optimal duration for each walking phase. These desired values are stabilized by a whole-body controller that determines a set of desired joint torques. We experimentally demonstrate that by using trajectory optimization techniques, the maximum torque required to the full-size humanoid robot Atlas can be reduced up to 20% when performing a step-up motion.

This talk covers two main research directions based on the iCub humanoid robot. The iCub is a humanoid robot designed to support research in embodied AI. At 104 cm tall, the iCub has the size of a five-year-old child. It can crawl on all fours, walk, and sit up to manipulate objects. Its hands have been designed to support sophisticate manipulation skills. The iCub is distributed as Open Source following the GPL license and can now count on a worldwide community of enthusiastic developers. There are more than 40 robots available in laboratories across the globe. The iCub sensory system allows seeing, hearing and feeling physical contacts. It is one of the few platforms in the world with a sensitive full-body skin. The iCub is being used at the Italian Institute of Technology as a model platform to develop the technology of future interactive service robots. In particular, I will describe our work in the field of physical and social interaction. For example, through extensive use of machine learning, we developed algorithms to interpret and use external contact information in a variety of tasks as well as contactless cues - vision, sound - to ease interaction between the user and the robot.

Robotic arms can perform grasping actions thanks to their “dexteorus” part, i.e. the gripper. Among the various categories, nowadays pneumatic grippers became the most employed in industry, as they have low cost and little bulkiness. Despite their simplicity, controlling the force applied by these grippers is not straightforward due to the dependence of such a force on the air pressure in the gripper chambers. As a result, it is still tricky to implement closed-loop force control for pneumatic grippers. This paper intends to deliver a control scheme relying on the force measurement to control pneumatic grippers. The force might be measured through a commercial sensor (e.g. a load cell) and fed back to close the control loop. This includes a calibration which maps the force-pressure relation taking into account both desired force and length of the gripper fingers. The control scheme exploits two different pressure regulators to precisely adjust the air pressure inside the gripper chambers (i.e. opening and closing chambers). To this aim, a quadratic programming algorithm is employed. The control scheme performance revealed to be good: results will be shown in terms of gripper response to sinusoidal and step inputs, along with the pressure-force characterization.

With robots leaving factories and entering less controlled domains, possibly sharing the space with humans, safety is paramount and multimodal awareness of the body surface and the surrounding environment is fundamental. Taking inspiration from peripersonal space representations in humans, we present a framework on a humanoid robot that dynamically maintains such a protective safety zone, composed of the following main components: (i) a human 2D keypoints estimation pipeline employing a deep learning based algorithm, extended here into 3D using disparity; (ii) a distributed peripersonal space representation around the robot»s body parts; (iii) a reaching controller that incorporates all obstacles entering the robot»s safety zone on the fly into the task. Pilot experiments demonstrate that an effective safety margin between the robot»s and the human»s body parts is kept. The proposed solution is flexible and versatile since the safety zone around individual robot and human body parts can be selectively modulated---here we demonstrate stronger avoidance of the human head compared to rest of the body. Our system works in real time and is self-contained, with no external sensory equipment and use of onboard cameras only.

A common approach to the generation of walking patterns for humanoid robots consists in adopting a layered control architecture. This paper proposes an architecture composed of three nested control loops. The outer loop exploits a robot kinematic model to plan the footstep positions. In the mid layer, a predictive controller generates a Center of Mass trajectory according to the well-known table-cart model. Through a whole-body inverse kinematics algorithm, we can define joint references for position controlled walking. The outcomes of these two loops are then interpreted as inputs of a stack-of-task QP-based torque controller, which represents the inner loop of the presented control architecture. This resulting architecture allows the robot to walk also in torque control, guaranteeing higher level of compliance. Real world experiments have been carried on the humanoid robot iCub.

The present study highlights the benefits of using well-controlled experimental designs, grounded in experimental psychology research and objective neuroscientific methods, for generating progress in human-robot interaction (HRI) research. More specifically, we aimed at implementing a well-studied paradigm of attentional cueing through gaze (the so-called “joint attention” or “gaze cueing”) in an HRI protocol involving the iCub robot. Similarly to documented results in gaze-cueing research, we found faster response times and enhanced event-related potentials of the EEG signal for discrimination of cued, relative to uncued, targets. These results are informative for the robotics community by showing that a humanoid robot with mechanistic eyes and human-like characteristics of the face is in fact capable of engaging a human in joint attention to a similar extent as another human would do. More generally, we propose that the methodology of combining neuroscience methods with an HRI protocol, contributes to understanding mechanisms of human social cognition in interactions with robots and to improving robot design, thanks to systematic and well-controlled experimentation tapping onto specific cognitive mechanisms of the human, such as joint attention.

Hand-eye coordination is a requirement for many manipulation tasks including grasping and reaching. However, accurate hand-eye coordination has shown to be especially difficult to achieve in complex robots like the iCub humanoid. In this work, we solve the hand-eye coordination task using a visuomotor deep neural network predictor that estimates the arm's joint configuration given a stereo image pair of the arm and the underlying head configuration. As there are various unavoidable sources of sensing error on the physical robot, we train the predictor on images obtained from simulation. The images from simulation were modified to look realistic using an image-to-image translation approach. In various experiments, we first show that the visuomotor predictor provides accurate joint estimates of the iCub's hand in simulation. We then show that the predictor can be used to obtain the systematic error of the robot's joint measurements on the physical iCub robot. We demonstrate that a calibrator can be designed to automatically compensate this error. Finally, we validate that this enables accurate reaching of objects while circumventing manual fine-calibration of the robot.

We consider object recognition in the context of lifelong learning, where a robotic agent learns to discriminate between a growing number of object classes as it accumulates experience about the environment. We propose an incremental variant of the Regularized Least Squares for Classification (RLSC) algorithm, and exploit its structure to seamlessly add new classes to the learned model. The presented algorithm addresses the problem of having an unbalanced proportion of training examples per class, which occurs when new objects are presented to the system for the first time. We evaluate our algorithm on both a machine learning benchmark dataset and two challenging object recognition tasks in a robotic setting. Empirical evidence shows that our approach achieves comparable or higher classification performance than its batch counterpart when classes are unbalanced, while being significantly faster.

Future humanoid robots will be expected to carry out a wide range of tasks for which they had not been originally equipped by learning new skills and adapting to their environment. A crucial requirement towards that goal is to be able to take advantage of external elements as tools to perform tasks for which their own manipulators are insufficient; the ability to autonomously learn how to use tools will render robots far more versatile and simpler to design. Motivated by this prospect, this paper proposes and evaluates an approach to allow robots to learn tool affordances based on their 3D geometry. To this end, we apply tool-pose descriptors to represent tools combined with the way in which they are grasped, and affordance vectors to represent the effect tool-poses achieve in function of the action performed. This way, tool affordance learning consists in determining the mapping between these 2 representations, which is achieved in 2 steps. First, the dimensionality of both representations is reduced by unsupervisedly mapping them onto respective Self-Organizing Maps (SOMs). Then, the mapping between the neurons in the tool-pose SOM and the neurons in the affordance SOM for pairs of tool-poses and their corresponding affordance vectors, respectively, is learned with a neural based regression model. This method enables the robot to accurately predict the effect of its actions using tools, and thus to select the best action for a given goal, even with tools not seen on the learning phase.

The torso of a humanoid robot is a fundamental part of its kinematic structure because it defines the reachable workspace, supports the entire upper-body and can be used to control the position of the center of mass. The majority of the torso joints are designed exploiting serial or differential mechanisms, while parallel kinematic structures are less used mainly because of their greater design complexity. This paper describes the design and construction of a 4 degrees of freedom (DoF) torso for our new humanoid robot. Three degrees of freedom, namely roll, pitch and heave, have been implemented using a 3 DoF parallel kinematic structure, while the fourth DoF, namely yaw, has been implemented with a rotational joint on top of the parallel structure. The design has been optimized to reduce the cost and the volume of the system. A first prototype of the torso has been constructed and validated with respect to our design requirements. Eventually, experimental tests have been conducted to assess the functionality of the proposed system.

In recent years the robotics field has witnessed an interesting new trend. Several companies started the production of service robots whose aim is to cooperate with humans. The robots developed so far are either rather expensive or unsuitable for manipulation tasks. This article presents the result of a project which wishes to demonstrate the feasibility of an affordable humanoid robot. R1 is able to navigate, and interact with the environment (grasping and carrying objects, operating switches, opening doors etc). The robot is also equipped with a speaker, microphones and it mounts a display in the head to support interaction using natural channels like speech or (simulated) eye movements. The final cost of the robot is expected to range around that of a family car, possibly, when produced in large quantities, even significantly lower. This goal was tackled along three synergistic directions: use of polymeric materials, light-weight design and implementation of novel actuation solutions. These lines, as well as the robot with its main features, are described hereafter.

This paper presents a novel approach for incremental semiparametric inverse dynamics learning. In particular, we consider the mixture of two approaches: Parametric modeling based on rigid body dynamics equations and nonparametric modeling based on incremental kernel methods, with no prior information on the mechanical properties of the system. The result is an incremental semiparametric approach, leveraging the advantages of both the parametric and nonparametric models. We validate the proposed technique learning the dynamics of one arm of the iCub humanoid robot.

The ability to learn about and efficiently use tools constitutes a desirable property for general purpose humanoid robots, as it allows them to extend their capabilities beyond the limitations of their own body. Yet, it is a topic that has only recently been tackled from the robotics community. Most of the studies published so far make use of tool representations that allow their models to generalize the knowledge among similar tools in a very limited way. Moreover, most studies assume that the tool is always grasped in its common or canonical grasp position, thus not considering the influence of the grasp configuration in the outcome of the actions performed with them. In the current paper we present a method that tackles both issues simultaneously by using an extended set of functional features and a novel representation of the effect of the tool use. Together, they implicitly account for the grasping configuration and allow the iCub to generalize among tools based on their geometry. Moreover, learning happens in a self-supervised manner: First, the robot autonomously discovers the affordance categories of the tools by clustering the effect of their usage. These categories are subsequently used as a teaching signal to associate visually obtained functional features to the expected tool's affordance. In the experiments, we show how this technique can be effectively used to select, given a tool, the best action to achieve a desired effect.

Tactile sensing is of fundamental importance for object manipulation and perception. Several sensors for hands have been proposed in the literature, however, only a few of them can be fully integrated with robotic hands. Typical problems preventing integration include the need for deformable sensors that can be deployed on curved surfaces, and wiring complexity. In this paper we describe a fingertip for the hands of the iCub robot, each fingertip consists of 12 sensors. Our approach builds on previous work on the iCub tactile system. The sensing elements of the fingertip are capacitive sensors made from a flexible PCB, and a multi-layer fabric that includes the dielectric material and the conductive layer. The novelty the proposed sensor lies in incorporating the multi-layer fabric technology into a small fingertip sensor that can be attached to the hands of a humanoid robot. The new sensors are more robust. The manufacturing is easier and relies on industrial techniques for the fabrication of the components, which results in higher repeatability. We performed experimental characterization of the sensor. We show that the sensor is able to detect forces as low as 0.05 N with no cross-talk between the taxels. We identified some hysteresis in the response of the sensor which must be taken into account if the robot exerts large forces for a long period of time. The taxels have spatially overlapping receptive fields, this has been demonstrated to be a useful property that allows hyperacuity.

Application domains of robotic systems are growing in complexity. It seems therefore plausible that robotic software will continue to be designed to be executed on distributed computer architectures interconnected through a network. It is a common practice today to rely on best-effort performance and assume that the latter are adequate given enough computational and networking resources. This approach however does not make best use of the available resources and, maybe more importantly, does not guarantee that performance remain constant over time. Real-time and Quality of Service become therefore important aspects in the software architecture of a robot. This article describes an approach for introducing these concepts in a publish-subscribe software middleware. The key contribution of our approach is that it leverages on the services provided by the operating system (scheduling priority and packet QoS) and abstracts them in a set of levels of priority that can be assigned dynamically, and with the granularity of individual communication channels. We implemented our approach on the YARP middleware and performed an experimental evaluation that demonstrates its benefit for increasing determinism and reducing latency in data communication. We further demonstrate this in a real-robot experiment that shows increased performance in a closed-loop scenario.

With robots leaving factory environments and entering less controlled domains, possibly sharing living space with humans, safety needs to be guaranteed. To this end, some form of awareness of their body surface and the space surrounding it is desirable. In this work, we present a unique method that lets a robot learn a distributed representation of space around its body (or peripersonal space) by exploiting a whole-body artificial skin and through physical contact with the environment. Every taxel (tactile element) has a visual receptive field anchored to it. Starting from an initially blank state, the distance of every object entering this receptive field is visually perceived and recorded, together with information whether the object has eventually contacted the particular skin area or not. This gives rise to a set of probabilities that are updated incrementally and that carry information about the likelihood of particular events in the environment contacting a particular set of taxels. The learned representation naturally serves the purpose of predicting contacts with the whole body of the robot, which is of clear behavioral relevance. Furthermore, we devised a simple avoidance controller that is triggered by this representation, thus endowing a robot with a “margin of safety” around its body. Finally, simply reversing the sign in the controller we used gives rise to simple “reaching” for objects in the robot's vicinity, which automatically proceeds with the most activated (closest) body part.

This paper proposes a learning from demonstration system based on a motion feature, called phase transfer sequence. The system aims to synthesize the knowledge on humanoid whole body motions learned during teacher-supported interactions, and apply this knowledge during different physical interactions between a robot and its surroundings. The phase transfer sequence represents the temporal order of the changing points in multiple time sequences. It encodes the dynamical aspects of the sequences so as to absorb the gaps in timing and amplitude derived from interaction changes. The phase transfer sequence was evaluated in reinforcement learning of sitting-up and walking motions conducted by a real humanoid robot and compatible simulator. In both tasks, the robotic motions were less dependent on physical interactions when learned by the proposed feature than by conventional similarity measurements. Phase transfer sequence also enhanced the convergence speed of motion learning. Our proposed feature is original primarily because it absorbs the gaps caused by changes of the originally acquired physical interactions, thereby enhancing the learning speed in subsequent interactions.

Motivated by the impact of superresolution methods for imaging, we undertake a detailed and systematic analysis of localization acuity for a biomimetic fingertip and a flat region of tactile skin. We identify three key factors underlying superresolution that enable the perceptual acuity to surpass the sensor resolution: 1) the sensor is constructed with multiple overlapping, broad but sensitive receptive fields; 2) the tactile perception method interpolates between receptors (taxels) to attain subtaxel acuity; and 3) active perception ensures robustness to unknown initial contact location. All factors follow from active Bayesian perception applied to biomimetic tactile sensors with an elastomeric covering that spreads the contact over multiple taxels. In consequence, we attain extreme superresolution with a 35-fold improvement of localization acuity (0.12 mm) over sensor resolution (4 mm). We envisage that these principles will enable cheap high-acuity tactile sensors that are highly customizable to suit their robotic use. Practical applications encompass any scenario where an end-effector must be placed accurately via the sense of touch.

In this paper we propose a weighted supervised pooling method for visual recognition systems. We combine a standard Spatial Pyramid Representation which is commonly adopted to encode spatial information, with an appropriate Feature Space Representation favoring semantic information in an appropriate feature space. For the latter, we propose a weighted pooling strategy exploiting data supervision to weigh each local descriptor coherently with its likelihood to belong to a given object class. The two representations are then combined adaptively with Multiple Kernel Learning. Experiments on common benchmarks (Caltech-256 and PASCAL VOC-2007) show that our image representation improves the current visual recognition pipeline and it is competitive with similar state-of-art pooling methods. We also evaluate our method on a real Human-Robot Interaction setting, where the pure Spatial Pyramid Representation does not provide sufficient discriminative power, obtaining a remarkable improvement.

In this paper we propose an auto encoder-based method for the unsupervised identification of subword units. We experiment with different types and architectures of auto encoders to assess what auto encoder properties are most important for this task. We first show that the encoded representation of speech produced by standard auto encoders is more effective than Gaussian posteriorgrams in a spoken query classification task. Finally we evaluate the subword inventories produced by the proposed method both in terms of classification accuracy in a word classification task (with lexicon size up to 263 words) and in terms of consistency between subword transcription of different word examples of a same word type. The evaluation is carried out on Italian and American English datasets.

We present a novel method for three-finger precision grasp and its implementation in a complete grasping tool-chain. We start from binocular vision to recover the partial 3D structure of unknown objects. We then process the incomplete 3D point clouds searching for good triplets according to a function that accounts for both the feasibility and the stability of the solution. In particular, while stability is determined using the classical force-closure approach, feasibility is evaluated according to a new measure that includes information about the possible configuration shapes of the hand as well as the hand's inverse kinematics. We finally extensively assess the proposed method using the stereo vision and the kinematics of the iCub robot.

This paper presents a new technique to control highly redundant mechanical systems, such as humanoid robots. We take inspiration from two approaches. Prioritized control is a widespread multi-task technique in robotics and animation: tasks have strict priorities and they are satisfied only as long as they do not conflict with any higher-priority task. Optimal control instead formulates an optimization problem whose solution is either a feedback control policy or a feedforward trajectory of control inputs. We introduce strict priorities in multi-task optimal control problems, as an alternative to weighting task errors proportionally to their importance. This ensures the respect of the specified priorities, while avoiding numerical conditioning issues. We compared our approach with both prioritized control and optimal control with tests on a simulated robot with 11 degrees of freedom.

Calibration continues to receive significant attention in robotics because of its key impact on performance and cost associated with the operation of complex robots. Calibration of kinematic parameters is typically the first mandatory step. To this end, a variety of metrology systems and corresponding algorithms have been described in the literature relying on measurements of the pose of the end-effector using a camera or laser tracking system, or, exploiting constraints arising from contacts of the end-effector with the environment. In this work, we take inspiration from the behavior of infants and certain animals, who are believed to use self-stimulation or self-touch to “calibrate” their body representations, and present a new solution to this problem by letting the robot close the kinematic chain by touching its own body. The robot considered in this paper is sensorized with tactile arrays for a total of about 4200 sensing points. The correspondence between the predicted contact point from existing forward kinematics and the actual position on the robot's `skin' provides sample data that allows refining the kinematic representation (DH parameters). The data collection procedure is automated - self-touch is autonomously executed by the robot - and can be repeated at any time, providing a compact self-calibration system that does not require an external measurement apparatus.

In this paper we tackle the problem of estimating the local compliance of tactile arrays exploiting global measurements from a single force and torque sensor. The proposed procedure exploits a transformation matrix (describing the relative position between the local tactile elements and the global force/torque measurements) to define a linear regression problem on the unknown local stiffness. Experiments have been conducted on the foot of the iCub robot, sensorized with a single force/torque sensor and a tactile array of 250 tactile elements (taxels) on the foot sole. Results show that a simple calibration procedure can be employed to estimate the stiffness parameters of virtual springs over a tactile array and to use these model to predict normal forces exerted on the array based only on the tactile feedback. Leveraging on previous works [1] the proposed procedure does not necessarily need a-priori information on the transformation matrix of the taxels which can be directly estimated from available measurements.

Systematically developing high-quality reusable software components is a difficult task and requires careful design to find a proper balance between potential reuse, functionalities and ease of implementation. Extendibility is an important property for software which helps to reduce cost of development and significantly boosts its reusability. This work introduces an approach to enhance components reusability by extending their functionalities using plug-ins at the level of the connection points (ports). Application-dependent functionalities such as data monitoring and arbitration can be implemented using a conventional scripting language and plugged into the ports of components. The main advantage of our approach is that it avoids to introduce application-dependent modifications to existing components, thus reducing development time and fostering the development of simpler and therefore more reusable components. Another advantage of our approach is that it reduces communication and deployment overheads as extra functionalities can be added without introducing additional modules. The details of the plug-in system is described in the paper and its advantages for the development of robotics applications are demonstrated by developing a step-by-step example on the iCub humanoid robot.

Robotic technology has made significant progresses in the past years. Robots are now common in large manufacturing plants and other industrial settings, safely confined in closed work cells. But to be even more helpful, robots need the capability of interacting physically with humans, and with unstructured environments. This poses new challenges in the design of safe robotic systems. In this article we addressed this problem by proposing a novel design for the joints of the iCub robot. The new design provides the robot with an overload protection mechanism. The overload protection acts as a “passive” torque saturator, which is intrinsically safe. We constructed a prototype of a robotic joint that implements this approach. We first show that our solution is effective in a typical impact scenario. We then evaluate the possible problems arising when the device is controlled with a position control loop. We show that a conventional feedback control loop can trigger positive feedback and instability. Operating the actuator in these conditions is dangerous and can lead to severe failures. We therefore propose the implementation of a relatively simple control strategy that allows to avoid this situation by monitoring slippage, without additional sensors. The quantitative evaluations in the paper demonstrate that our approach is effective and can improve the robustness and safety of complex robotic systems. Indeed these aspects are particularly critical in the case of humaniod robots that are systems prone to severe whole-body impacts in unstructured environments (e.g. falling).

Legged robots are typically in rigid contact with the environment at multiple locations, which add a degree of complexity to their control. We present a method to control the motion and a subset of the contact forces of a floating-base robot. We derive a new formulation of the lexicographic optimization problem typically arising in multi-task motion/force control frameworks. The structure of the constraints of the problem (i.e. the dynamics of the robot) allows us to find a sparse analytical solution. This leads to an equivalent optimization with reduced computational complexity, comparable to inverse-dynamics based approaches. At the same time, our method preserves the flexibility of optimization based control frameworks. Simulations were carried out to achieve different multi-contact behaviors on a 23-degree-of-freedom humanoid robot, validating the presented approach. A comparison with another state-of-the-art control technique with similar computational complexity shows the benefits of our controller, which can eliminate force/torque discontinuities.

This paper argues in favour of using formal methods to ensure safety of deployed stochastic policies learned by robots in unstructured environments. It has been demonstrated that multi-objective learning alone is not sufficient to ensure globally safe behaviours in such robots, whereas learning-specific methods yield deterministic policies which are less flexible or effective in practice. Under certain restrictions on state-space, modelling safety using probabilistic computational tree logic and ensuring such safety via automated repair can overcome these shortcomings. Promising results are obtained on a realistic setup and pros and cons of such method are discussed.

This paper describes a developmental framework for action-driven perception in anthropomorphic robots. The key idea of the framework is that action generation develops the agent's perception of its own body and actions. Action-driven development is critical for identifying changing body parts and understanding the effects of actions in unknown or nonstationary environments. We embedded minimal knowledge into the robot's cognitive system in the form of motor synergies and actions to allow motor exploration. The robot voluntarily generates actions and develops the ability to perceive its own body and the effect that it generates on the environment. The robot, in addition, can compose this kind of learned primitives to perform complex actions and characterize them in terms of their sensory effects. After learning, the robot can recognize manipulative human behaviors with cross-modal anticipation for recovery of unavailable sensory modality, and reproduce the recognized actions afterward. We evaluated the proposed framework in the experiments with a real robot. In the experiments, we achieved autonomous body identification, learning of fixation, reaching and grasping actions, and developmental recognition of human actions as well as their reproduction.

For any robot, the ability to recognize and manipulate unknown objects is crucial to successfully work in natural environments. Object recognition and categorization is a very challenging problem, as 3-D objects often give rise to ambiguous, 2-D views. Here, we present a perception-driven exploration and recognition scheme for in-hand object recognition implemented on the iCub humanoid robot. In this setup, the robot actively seeks out object views to optimize the exploration sequence. This is achieved by regarding the object recognition problem as a localization problem. We search for the most likely viewpoint position on the viewsphere of all objects. This problem can be solved efficiently using a particle filter that fuses visual cues with associated motor actions. Based on the state of the filter, we can predict the next best viewpoint after each recognition step by searching for the action that leads to the highest expected information gain. We conduct extensive evaluations of the proposed system in simulation as well as on the actual robot and show the benefit of perception-driven exploration over passive, vision-only processes at discriminating between highly similar objects. We demonstrate that objects are recognized faster and at the same time with a higher accuracy.

This work aims to augment the capacities for haptic perception in the iCub robot to generate a controller for surface exploration. The main task involves moving the hand over an irregular surface with uncertain slope, by concurrently regulating the pressure of the contact. Providing this ability will enable the autonomous extraction of important haptic features, such as texture and shape. We propose a hand controller whose operational space is defined over the surface of contact. The surface is estimated using a robust probabilistic estimator, which is then used for path planning. The motor commands are generated using a feedback controller, taking advantage of the kinematic information available by proprioception. Finally, the effectiveness of this controller is extended using a cerebellar-like adapter that generates reliable pressure tracking over the finger and results in a trajectory with less vulnerability to perturbations. The results of this work are consistent with insights about the role of the cerebellum on haptic perception in humans.

In this work, we present an active tactile perception approach for contour following based on a probabilistic framework. Tactile data were collected using a biomimetic fingertip sensor. We propose a control architecture that implements a perception-action cycle for the exploratory procedure, which allows the fingertip to react to tactile contact whilst regulating the applied contact force. In addition' the fingertip is actively repositioned to an optimal position to ensure accurate perception. The method is trained off-line and then the testing performed on-line based on contour following around several different test shapes. We then implement object recognition based on the extracted shapes. Our active approach is compared with a passive approach, demonstrating that active perception is necessary for successful contour following and hence shape recognition.

The paper aims at building a computer vision system for automatic image labeling in robotics scenarios. We show that the weak supervision provided by a human demonstrator, through the exploitation of the independent motion, enables a realistic Human-Robot Interaction (HRI) and achieves an automatic image labeling. We start by reviewing the underlying principles of our previous method for egomotion compensation [1], then we extend our approach removing the dependency on a known kinematics in order to provide a general method for a wide range of devices. From sparse salient features we predict the egomotion of the camera through a heteroscedastic learning method. Subsequently we use an object recognition framework for testing the automatic image labeling process: we rely on the State of the Art method from Yang et al. [2], employing local features augmented through a sparse coding stage and classified with linear SVMs. The application has been implemented and validated on the iCub humanoid robot and experiments are presented to show the effectiveness of the proposed approach. The contribution of the paper is twofold: first we overcome the dependency on the kinematics in the independent motion detection method, secondly we present a practical application for automatic image labeling through a natural HRI.

The paper describes a developmental framework of action-driven perception in anthropomorphic robots. The key idea of the framework is that action develops the agent's perception of the own body and its action. In this framework, a robot voluntarily generates movements, and then develops the ability to perceive its own body and the effects of action primitives. The robot, moreover, demonstrates manipulative actions composed of the learned primitives, and characterizes the actions based on their sensory effects. After learning, the robot can predictively recognize humans' manipulative actions with cross-modal recovery of unavailable sensory information and reproduce the recognized actions. We evaluated the proposed framework in experiments with a real robot. In the experiments, we achieved developmental recognition of human actions as well as their reproduction.

We present a strategy for learning Deep-Neural-Network (DNN)-based Acoustic-to-Articulatory Mapping (AAM) functions where the contribution of an articulatory feature (AF) to the global reconstruction error is weighted by its relevance. We first empirically show that when an articulator is more crucial for the production of a given phone it is less variable, confirming previous findings. We then compute the relevance of an articulatory feature as a function of its frame-wise variance dependent on the acoustic evidence which is estimated through a Mixture Density Network (MDN). Finally we combine acoustic and recovered articulatory features in a hybrid DNN-HMM phone recognizer. Tested on the MOCHA-TIMIT corpus, articulatory features reconstructed by a standardly trained DNN lead to a 8.4% relative phone error reduction (w.r.t. a recognizer that only uses MFCCs), whereas when the articulatory features are reconstructed taking into account their relevance the relative phone error reduction increased to 10.9%

Recent developments in learning sophisticated, hierarchical image representations have led to remarkable progress in the context of visual recognition. While these methods are becoming standard in modern computer vision systems, they are rarely adopted in robotics. The question arises of whether solutions, which have been primarily developed for image retrieval, can perform well in more dynamic and unstructured scenarios. In this paper we tackle this question performing an extensive evaluation of state of the art methods for visual recognition on a iCub robot. We consider the problem of classifying 15 different objects shown by a human demonstrator in a challenging Human-Robot Interaction scenario. The classification performance of hierarchical learning approaches are shown to outperform benchmark solutions based on local descriptors and template matching. Our results show that hierarchical learning systems are computationally efficient and can be used for real-time training and recognition of objects.

Cooperation1is at the core of human social life. In this context, two major challenges face research on humanrobot interaction: the first is to understand the underlying structure of cooperation, and the second is to build, based on this understanding, artificial agents that can successfully and safely interact with humans. Here we take a psychologically grounded and human-centered approach that addresses these two challenges. We test the hypothesis that optimal cooperation between a naïve human and a robot requires that the robot can acquire and execute a joint plan, and that it communicates this joint plan through ecologically valid modalities including spoken language, gesture and gaze. We developed a cognitive system that comprises the human-like control of social actions, the ability to acquire and express shared plans and a spoken language stage. In order to test the psychological validity of our approach we tested 12 naïve subjects in a cooperative task with the robot. We experimentally manipulated the presence of a joint plan (vs. a solo plan), the use of task-oriented gaze and gestures, and the use of language accompanying the unfolding plan. The quality of cooperation was analyzed in terms of proper turn taking, collisions and cognitive errors. Results showed that while successful turn taking could take place in the absence of the explicit use of a joint plan, its presence yielded significantly greater success. One advantage of the solo plan was that the robot would always be ready to generate actions, and could thus adapt if the human intervened at the wrong time, whereas in the joint plan the robot expected the human to take his/her turn. Interestingly, when the robot represented the action as involving a joint plan, gaze provided a highly potent nonverbal cue that facilitated successful collaboration and reduced errors in the absence of verbal communication. These results support the cooperative stance in human social cognition, and suggest that cooperative robots should employ joint plans, fully communicate them in order to sustain effective collaboration while being ready to adapt if the human makes a midstream mistake.

In this paper we describe the mathematical model of a tendon driven eye. We focus on describing its movements, posing a specific attention on cyclotorsion, that is the rotation around the eye optical axis. This study aims at understanding the cause of a cyclotorsion effect that has been noticed on a real setup. The paper starts from a qualitative analysis of the effect. Then, it proposes two different models for its motion. Finally both models are validated by comparing their predictions with the outcomes on the real robot. Given the complexity of the system and of its motion results are for the moment just qualitative and quantitative comparison will be the goal of our future works.

Given a stochastic policy learned by reinforcement, we wish to ensure that it can be deployed on a robot with demonstrably low probability of unsafe behavior. Our case study is about learning to reach target objects positioned close to obstacles, and ensuring a reasonably low collision probability. Learning is carried out in a simulator to avoid physical damage in the trial-and-error phase. Once a policy is learned, we analyze it with probabilistic model checking tools to identify and correct potential unsafe behaviors. The whole process is automated and, in principle, it can be integrated step-by-step with routine task-learning. As our results demonstrate, automated fixing of policies is both feasible and highly effective in bounding the probability of unsafe behaviors.

In this paper, we are presenting a human robot interaction study, which is targeting the question, how the attention system of the iCub robot is performing in comparison with infants. To answer this, we have studied a task presentation, towards infants, the iCub simulator and the iCub robot. We compared the gazing behavior of the recipients. In developmental robotics, as well as in tutoring situations, the gazing behavior of the recipient plays an important role for the following interaction.

The problem of "providing robots with the sense of touch" is fundamental in order to develop the next generations of robots capable of interacting with humans in different contexts: in daily housekeeping activities, as working partners or as caregivers, just to name a few.

Interaction with its environment is a key requisite for a humanoid robot. Especially the ability to recognize and manipulate unknown objects is crucial to successfully work in natural environments. Visual object recognition, however, still remains a challenging problem, as three-dimensional objects often give rise to ambiguous, two-dimensional views. Here, we propose a perception-driven, multisensory exploration and recognition scheme to actively resolve ambiguities that emerge at certain viewpoints. We define an efficient method to acquire two-dimensional views in an object-centered task space and sample characteristic views on a view sphere. Information is accumulated during the recognition process and used to select actions expected to be most beneficial in discriminating similar objects. Besides visual information we take into account proprioceptive information to create more reliable hypotheses. Simulation and real-world results clearly demonstrate the efficiency of active, multisensory exploration over passive, visiononly recognition methods.

We present an original method for independent motion detection in dynamic scenes. The algorithm is designed for robotics real-time applications and it overcomes the short-comings of current approaches for the egomotion estimation in presence of many outliers, occlusions and cluttered background. The method relies on a stereo system which performs the reprojection of a sparse set of features following the camera displacement. We assume that noisy prior knowledge of the motion is available (i.e. a robot's kinematic model). Since this estimation leads to a heteroscedastic regression problem due to input-dependent noise, we employ a simple, but computationally efficient approach in order to accurately determine the latent egomotion subspace spanned by the Degrees of Freedom (DOFs) of the robot. The algorithm has been implemented and validated on the iCub humanoid robot. Qualitative and quantitative experiments are presented to show the effectiveness of the proposed approach. The contribution of the paper is a modular framework for independent motion detection naturally extendable to any architecture featuring a visual sensor that can be directly controllable.

Recent advances in modeling animal perception has motivated an approach of Bayesian perception applied to biomimetic robots. This study presents an initial application of Bayesian perception on an iCub fingertip sensor mounted on a dedicated positioning robot. We systematically probed the test system with five cylindrical stimuli offset by a range of positions relative to the fingertip. Testing the real-time speed and accuracy of shape and position discrimination, we achieved sub-millimeter accuracy with just a few taps. This result is apparently the first explicit demonstration of perceptual hyperacuity in robot touch, in that object positions are perceived more accurately than the taxel spacing. We also found substantial performance gains when the fingertip can reposition itself to avoid poor perceptual locations, which indicates that improved robot perception could mimic active perception in animals.

This paper investigates the role of precise estimation of contact points in force control. This analysis is motivated by scenarios in which robots make contacts, either voluntarily or accidentally, with different parts of their body. Control paradigms that are usually implemented in robots with no tactile system, make the hypothesis that contacts occur at the end-effectors only. In this paper we try to investigate what happens when this assumption is not verified. First we consider a simple feedforward force control law, and then we extend it by introducing a proportional feedback term. For both controllers we find the error in the resulting contact force, that is induced by a hypothetic error in the estimation of the contact point. We show that, depending on the geometry of the contact, incorrect estimation of contact points can induce undesired joint accelerations. We validate the presented analysis with tests on a simulated robot arm. Moreover we consider a complex real world scenario, where most of the assumptions that we make in our analytical derivation do not hold. Through tests on the iCub humanoid robot we see how errors in contact localization affect the performance of a parallel force/position controller. In order to estimate contact points and contact forces on the forearm of the iCub we do not use any model of the environment, but we exploit its 6-axis force/torque sensor and its sensorized skin.

In this paper we experiment with methods based on Deep Belief Networks (DBNs) to recover measured articulatory data from speech acoustics. Our acoustic-to-articulatory mapping (AAM) processes go through multi-layered and hierarchical (i.e., deep) representations of the acoustic and the articulatory domains obtained through unsupervised learning of DBNs. The unsupervised learning of DBNs can serve two purposes: (i) pre-training of the Multi-layer Perceptrons that perform AAM; (ii) transformation of the articulatory domain that is recovered from acoustics through AAM. The recovered articulatory features are combined with MFCCs to compute phone posteriors for phone recognition. Tested on the MOCHA-TIMIT corpus, the recovered articulatory features, when combined with MFCCs, lead to up to a remarkable 16.6% relative phone error reduction w.r.t. a phone recognizer that only uses MFCCs.

Analytical models for robot dynamics often perform suboptimally in practice, due to various non-linearities and the difficulty of accurately estimating the dynamic parameters. Machine learning techniques are less sensitive to these problems and therefore an interesting alternative for modeling robot dynamics. We propose a learning method that combines a least squares algorithm with a non-linear feature mapping and an efficient update rule. Using data from five different robots, we show that the method can accurately model manipulator dynamics, either when trained in batch or incrementally. Furthermore, the update time and memory usage of the method are bounded, therefore allowing use in real-time control loops.

We investigated the use of optical flow to predict the time and location of impact of an incoming object. By examining local patterns of optical flow, we make predictions on an object's trajectory with respect to a stationary observer, and its time-to-contact with the observer's (assumed planar) body. Such a cue may serve as inputs to motor responses for reactive threat avoidance, or object interception (such as catching a ball). The presented approach is based on the observed behaviour of neurons in the F4 region of the pre-motor cortex of primates and it is part of a larger project which aims at modelling multi-sensory neurons and their contribution to reaching behaviour on the iCub humanoid platform. We present preliminary experimental results of a computation model of F4's visual responses using real image sequences acquired from the robot, and demonstrate their application in a real-time threat detection/prediction system.

The paper describes a constructive approach on active perception for anthropomorphic robots. The key idea is that a robot tries to identify a human's action as an own action based on the observation of action effects for objects. In the proposed framework, the active perception is decomposed into the three phases; First, a robot voluntarily generates actions to discover the own body and objects. Second, the robot characterizes its own action with the effect for the objects. Third, the robot identifies the human action with the own action. The mirrored perception of the own action and the human's action allows the robot to share the goal-directed behavior with humans. The proposed framework of active perception was experimentally validated with the integrated sensory modalities of vision, proprioception and touch.

This paper describes a self-protective whole-body control method for humanoid robots. A set of postural reactions are used to create whole-body movements. A set of reactions is merged to cope with a general falling down direction, while allowing the upper limbs to contact safely with obstacles. The collision detection is achieved by force sensing. We verified that our method generates the self-protective motion in real time, and reduced the impact energy in multiple situations by simulator. We also verified that our systems works adequately in real-robot.

Visual motion is a simple yet powerful cue widely used by biological systems to improve their perception and adaptation to the environment. Examples of tasks that greatly benefit from the ability to detect movement are object segmentation, 3D scene reconstruction and control of attention. In computer vision several algorithms for computing visual motion and optic flow exist. However their application in robotics is not straightforward as in these platforms visual motion is often dominated by (self) motion produced by the movement of the robot (egomotion) making it difficult to disambiguate between motion induced by the scene dynamics or by the own actions of the robot. Independent motion detection is an active field in computer vision and robotics, however approaches in this area typically require that some models of both the environment and the robot visual system are available and are hardly suitable for real-time control. In this paper we describe the motionCUT, a derivation of the Lucas-Kanade optical flow algorithm that allows detecting moving objects, irrespectively of the egomotion produced by the robot. Our method is purely visual and does not require information other than the images coming from the cameras. As such it can be easily adapted to any robotic platform. The system was tested on a stereo tracking task on the iCub humanoid robot, demonstrating that the algorithm performs well and can easily execute in real-time.

We propose an algorithm for the visual detection and localisation of the hand of a humanoid robot. This algorithm imposes low requirements on the type of supervision required to achieve good performance. In particular the system performs feature selection and adaptation using images that are only labelled as containing the hand or not, without any explicit segmentation. Our algorithm is an online variant of Multiple Instance Learning based on boosting. Experiments in real-world conditions on the iCub humanoid robot confirm that the algorithm can learn the visual appearance of the hand, reaching an accuracy comparable with its off-line version. This remains true when supervision is generated by the robot itself in a completely autonomous fashion. Algorithms with weak supervision requirements like the one we describe are useful for autonomous robots that learn and adapt online to a changing environment. The algorithm is not hand-specific and could be easily applied to wide range of problems involving visual recognition of generic objects.

If robots are to cooperate with humans in an increasingly human-like manner, then significant progress must be made in their abilities to observe and learn to perform novel goal directed actions in a flexible and adaptive manner. The current research addresses this challenge. In CHRIS.I [1], we developed a platform-independent perceptual system that learns from observation to recognize human actions in a way which abstracted from the specifics of the robotic platform, learning actions including “put X on Y” and “take X”. In the current research, we extend this system from action perception to execution, consistent with current developmental research in human understanding of goal directed action and teleological reasoning. We demonstrate the platform independence with experiments on three different robots. In Experiments 1 and 2 we complete our previous study of perception of actions “put” and “take” demonstrating how the system learns to execute these same actions, along with new related actions “cover” and “uncover” based on the composition of action primitives “grasp X” and “release X at Y”. Significantly, these compositional action execution specifications learned on one iCub robot are then executed on another, based on the abstraction layer of motor primitives. Experiment 3 further validates the platform-independence of the system, as a new action that is learned on the iCub in Lyon is then executed on the Jido robot in Toulouse. In Experiment 4 we extended the definition of action perception to include the notion of agency, again inspired by developmental studies of agency attribution, exploiting the Kinect motion capture system for tracking human motion. Finally in Experiment 5 we demonstrate how the combined representation of action in terms of perception and execution provides the basis for imitation. This provides the basis for an open ended cooperation capability where new actions can be learned and integrated into shared plans for cooperation. Part of the novelty of this research is the robots' use of spoken language understanding and visual perception to generate action representations in a platform independent manner based on physical state changes. This provides a flexible capability for goal-directed action imitation.

This paper deals with the problem of estimating the position of tactile elements (i.e. taxels) that are mounted on a robot body part. This problem arises with the adoption of tactile systems with a large number of sensors, and it is particularly critical in those cases in which the system is made of flexible material that is deployed on a curved surface. In this scenario the location of each taxel is partially unknown and difficult to determine manually. Placing the device is in fact an inaccurate procedure that is affected by displacements in both position and orientation. Our approach is based on the idea that it is possible to automatically infer the position of the taxels by measuring the interaction forces exchanged between the sensorized part and the environment. The location of the contact is estimated through force/torque (F/T) measures gathered by a sensor mounted on the kinematic chain of the robot. Our method requires few hypotheses and can be effectively implemented on a real platform, as demonstrated by the experiments with the iCub humanoid robot.

When a robot is required to safely interact with a physical environment, two approaches are typically reported in literature: using a force/torque sensor to regulate the interaction forces at the end effector, or integrating sensors in each robot joint to regulate their torques. In this paper we want to discuss the benefits and the disadvantages of the two approaches, showing a direct comparison between the information which can be obtained from the two categories of sensors. Results obtained on the new iCub arm, which integrates torque sensing capabilities at joint level will be presented and discussed.

Even though the sense of touch is crucial for humans, most humanoid robots lack tactile sensing. While a large number of sensing technologies exist, it is not trivial to incorporate them into a robot. We have developed a compliant “skin” for humanoids that integrates a distributed pressure sensor based on capacitive technology. The skin is modular and can be deployed on nonflat surfaces. Each module scans locally a limited number of tactile-sensing elements and sends the data through a serial bus. This is a critical advantage as it reduces the number of wires. The resulting system is compact and has been successfully integrated into three different humanoid robots. We have performed tests that show that the sensor has favorable characteristics and implemented algorithms to compensate the hysteresis and drift of the sensor. Experiments with the humanoid robot iCub prove that the sensors can be used to grasp unmodeled, fragile objects.

This paper describes the control of a human-like robotic neck actuated with tendons. The controller regulates the length of the tendons to achieve a desired orientation of the neck and at the same time it maintains the tension of the tendons within certain limits. The solution we propose does not use any model of the system, but it relies on online learning of the different Jacobian mappings required by the controller. Learning, data acquisition and control are simultaneous; thus learning is completely autonomous, and purely online. We show that after enough iterations the controller produces straight trajectories in the task space and is able to maintain the tension of the tendons within safe limits.

In this paper we consider the problem of ensuring that a multi-agent robot control system is both safe and effective in the presence of learning components. Safety, i.e., proving that a potentially dangerous configuration is never reached in the control system, usually competes with effectiveness, i.e., ensuring that tasks are performed at an acceptable level of quality. In particular, we focus on a robot playing the air hockey game against a human opponent, where the robot has to learn how to minimize opponent's goals (defense play). This setup is paradigmatic since the robot must see, decide and move fastly, but, at the same time, it must learn and guarantee that the control system is safe throughout the process. We attack this problem using automata-theoretic formalisms and associated verification tools, showing experimentally that our approach can yield safety without heavily compromising effectiveness.

During the last decades, interaction (with humans and with the environment) has become an increasingly interesting topic of research within the field of robotics. At the basis of interaction, a fundamental role is played by the ability to actively regulate the interaction forces. In this paper we propose a technique for controlling the interaction forces exploiting a proximal six axes force/torque sensor. The major assumption is the knowledge of the point where external forces are applied. The proposed approach is tested and validated on the four limbs of the iCub, a humanoid robot designed for research in embodied cognition. Remarkably, the proposed approach can be used to implement active compliance in other non passively back-drivable manipulators by simply inserting one or more force/torque sensor anywhere along the kinematic chain.

One of the major challenges in developing autonomous systems is to make them able to recognize and categorize objects robustly. However, the appearance-based algorithms that are widely employed for robot perception do not explore the functionality of objects, described in terms of their affordances. These affordances (e.g., manipulation, grasping) are discriminative for object categories and are important cues for reliable robot performance in everyday environments. In this paper, we propose a strategy for object recognition that integrates both visual appearance and grasp affordance features. Following previous work, we hypothesize that additional grasp information improves object recognition, even if we reconstruct the grasp modality from visual features using a mapping function. We considered two different representations for the grasp modality: (1) motor information of the hand posture while grasping and (2) a more general grasp affordance descriptor. Using a multi-modal classifier we show that having real grasp information significantly boost object recognition. This improvement is preserved, although to a lesser extent, if the grasp modality is reconstructed using the mapping function.

Online optimal planning of robotic arm movement is addressed. Optimality is inspired by computational models, where a “cost function” is used to describe limb motions according to different criteria. A method is proposed to implement optimal planning in Cartesian space, minimizing some cost function, by means of numerical approximation to a generalized nonlinear model predictive control problem. The Extended RItz Method is applied as a functional approximation technique. Differently from other approaches, the proposed technique can be applied on platforms with strict control temporal constraints and limited processing capability, since the computational burden is completely concentrated in an off-line phase. The trajectory generation on-line is therefore computationally efficient. Task to joint space conversion is implemented on-line by a closed loop inverse kinematics algorithm, taking into account the robot's physical limits. Experimental results, where a 4DOF arm moves according to a particular nonlinear cost, show the effectiveness of the proposed approach, and suggest interesting future developments.

In this paper we describe the design of a Cartesian Controller for a generic robot manipulator. We address some of the challenges that are typically encountered in the field of humanoid robotics. The solution we propose deals with a large number of degrees of freedom, produce smooth, human-like motion and is able to compute the trajectory on-line. In this paper we support the idea that to produce significant advancements in the field of robotics it is important to compare different approaches not only at the theoretical level but also at the implementation level. For this reason we test our software on the iCub platform and compare its performance against other available solutions.

This work proposes a plausible approach for a humanoid robot to define its own body parts based on the correlation of two different sensory signals: vision and proprioception. The high correlation between the motions in vision and proprioception informs the robot that the visually attractive object is related to the motor function of its own body. When the robot finds the highly motor-correlated object during head-arm movements, the visuomotor cues such as the body posture and visual features are stored in a visuomotor memory. Then, developmentally, the robot defines the motor-correlated objects as the own-body parts without prior knowledge on the body appearances and kinematics. It is also adaptable to extended body parts such as a grasped tool. The body movements are generated by stochastic motor babbling. The visuomotor memory biases the babbling to keep the own-body parts in sight. This memory-based bias towards the own-body parts helps the robot explore the large head-arm joint space. The acquired visuomotor memory is also used to anticipate the own-body image from the motor commands in advance of the body movement. The proposed approach was evaluated with two humanoid platforms; iCub and James.

In order to successfully perform object manipulation, humanoid robots must be equipped with tactile sensors. However, the limited space that is available in robotic fingers imposes severe design constraints. In [1] we presented a small prototype fingertip which incorporates a capacitive pressure system. This paper shows an improved version, which has been integrated on the hand of the humanoid robot iCub. The fingertip is 14.5 mm long and 13 mm wide. The capacitive pressure sensor system has 12 sensitive zones and includes the electronics to send the 12 measurements over a serial bus with only 4 wires. Each synthetic fingertip is shaped approximately like a human fingertip. Furthermore, an integral part of the capacitive sensor is soft silicone foam, and therefore the fingertip is compliant. We describe the structure of the fingertip, their integration on the humanoid robot iCub and present test results to show the characteristics of the sensor.

This work presents and experimentally evaluates novel POSFET (Piezoelectric Oxide Semiconductor Field Effect Transistor) devices based tactile sensing arrays. The arrays, primarily developed for the robotic applications, consist of 5 × 5 POSFET touch sensing devices or taxels. The POSFET touch sensing devices are developed by spin coating piezoelectric polymer P(VDF-TrFE) film on the gate area of MOS devices and polarizing the film in situ. To detect contact events, the taxels utilize the contact forces induced change in the polarization level (and hence change in the induced channel current) of piezoelectric polymer. Both, individual taxels and the array are designed to match spatio-temporal performance of the human fingertips. Experimental results demonstrate that the POSFET tactile sensing arrays presented here are able to detect complex dynamic contact events such as rolling of an object.

This work presents the development of novel POSFET (Piezoelectric Oxide Semiconductor Field Effect Transistor) devices based tactile sensing system. The tactile sensing system, primarily developed for the robotic applications, consists of 5×5 POSFET touch sensing array and the associated read out and data acquisition system. The POSFET touch sensing devices are obtained by spin coating piezoelectric polymer P(VDF-TrFE), poly(vinylidene fluoride - trifluoroethylene), film on the gate area of MOS (Metal Oxide Semiconductor) devices and polarizing the film in situ. To detect contact events, the taxels utilize the contact forces induced change in the polarization level (and hence change in the induced channel current) of piezoelectric polymer. Both, individual taxels and the array are designed to match spatio-temporal performance of the human fingertips. The data acquisition system is implemented with off-the-shelf electronic components and its design takes into account both the application related requirements as well as the constraints posed by existing hardware on the humanoid robot `iCub'. The biasing scheme for using POSFET devices and the problems thereof are also been discussed.

The sense of touch is of major importance for object handling. Nevertheless, adequate cutaneous sensors for humanoid robot hands are still missing. Designing such sensors is challenging, because they should not only give reliable measurements and integrate many sensing points into little space, but they should also be compliant and should not obstruct the other functions of the robot. This paper presents a capacitive pressure sensor system with 108 sensitive zones for the hands of the humanoid robot iCub. In particular, the palm has 48 taxels and each of the five fingertips has 12 taxels. The size and the shape of the hand are similar to that of a human child. When designing the sensors, we paid special attention to the integration on the robot. Also the ease and speed of production was an important design factor. Furthermore, the sensor incorporates silicone foam and is therefore compliant. We show the working principle of the sensor, how it has been integrated into the hands, and describe experiments that have been performed to show the characteristics of the sensor.

Starting from human ¿sense of touch,¿ this paper reviews the state of tactile sensing in robotics. The physiology, coding, and transferring tactile data and perceptual importance of the ¿sense of touch¿ in humans are discussed. Following this, a number of design hints derived for robotic tactile sensing are presented. Various technologies and transduction methods used to improve the touch sense capability of robots are presented. Tactile sensing, focused to fingertips and hands until past decade or so, has now been extended to whole body, even though many issues remain open. Trend and methods to develop tactile sensing arrays for various body sites are presented. Finally, various system issues that keep tactile sensing away from widespread utility are discussed.

The subjective ease with which we move gracefully in constraint filled uncertain environments often masks the enormously complex integrative apparatus needed to spell synergy among the thousands of sensors, joints, musculo-skeletal units and neuronal populations that contribute to any act's planning and execution. In this paper, we apply the computational framework of passive motion paradigm for task specific composition and coordination of the movements of a limb, network of limbs (e.g. left arm-waist-right arm) or networks of external objects coupled to the body of the 53 degrees of freedom humanoid robot `iCub'. The basic PMP model is further extended by formulation of a pair of branching nodes that allow compositionality and transfer of force fields from one relaxation network to another. The generality of the proposed approach is further illustrated using simulations of whole body reaching (WBR) tasks from a quiet standing posture that recruits virtually all the joints of the upper limbs, lower limbs, and trunk, binding together a large number of degrees of freedom into a functional unit that combines a focal task (reaching a target with the hand) and a postural task (keeping the projection of the center of mass within the bipedal support area). Preliminary comparisons of the solutions generated by the computational model with the movements of human subjects performing similar WBR tasks are presented.

For a complex autonomous robotic system such as a humanoid robot, motor-babbling-based sensorimotor learning is considered an effective method to develop an internal model of the self-body and the environment autonomously. However, learning process requires much time for exploration and computation. In this paper, we propose a method of sensorimotor learning which explores the learning domain actively. Our approach discovers that the embodied learning system can design its own learning process actively, which is different from the conventional passive data-access machine learning. The proposed model is characterized by a function we call the ¿confidence¿, and is a measure of the reliability of state control. The confidence for the state can be a good measure to bias the exploration strategy of data sampling, and to direct its attention to areas of learning interest. We consider the confidence function to be a first step toward an active behavior design for autonomous environment adaptation. The approach was experimentally validated in typical sensorimotor coordination such as arm reaching and object fixation, using the humanoid robot James and the iCub simulator.

In this paper, we describe the implementation of a precise reaching controller on an upper-torso humanoid robot. The solution we propose does not rely on prior models of the kinematic structure of either the arm or the head. A learning strategy enables the robot to acquire the required sensory-motor transformations. After learning the robot is able to precisely reach for a visually identified object in the 3-dimensional space. In this technique we use the fixation point (represented in the head joints motor space) as a reference frame to code the position of the object and to represent the eye-to-hand Jacobian matrix. This strategy successfully deals with the kinematic redundancy of the structure and constraints the dimensionality of the problem.

This survey presents an overview of the autonomous development of mental capabilities in computational agents. It does so based on a characterization of cognitive systems as systems which exhibit adaptive, anticipatory, and purposive goal-directed behavior. We present a broad survey of the various paradigms of cognition, addressing cognitivist (physical symbol systems) approaches, emergent systems approaches, encompassing connectionist, dynamical, and enactive systems, and also efforts to combine the two in hybrid systems. We then review several cognitive architectures drawn from these paradigms. In each of these areas, we highlight the implications and attendant problems of adopting a developmental approach, both from phylogenetic and ontogenetic points of view. We conclude with a summary of the key architectural features that systems capable of autonomous development of mental capabilities should exhibit

This paper describes the design of a robot head, developed in the framework of the RobotCub project. This project goals consists on the design and construction of a humanoid robotic platform, the iCub, for studying human cognition. The final platform would be approximately 90 cm tall, with 23 kg and with a total number of 53 degrees of freedom. For its size, the iCub is the most complete humanoid robot currently being designed, in terms of kinematic complexity. The eyes can also move, as opposed to similarly sized humanoid platforms. Specifications are made based on biological anatomical and behavioral data, as well as tasks constraints. Different concepts for the neck design (flexible, parallel and serial solutions) are analyzed and compared with respect to the specifications. The eye structure and the proprioceptive sensors are presented, together with some discussion of preliminary work on the face design

This work discusses the design and control approach of a 3 DOF waist joint for the "iCub" humanoid robot. "iCub" is a child like crawling robot that resembles a 2 and a half year old child. The ultimate goal of this project is to provide the cognition research community with an open human like platform for understanding of cognitive systems through the study of cognitive development. The designs of the mechanisms adopted for the waist joint are discussed. This is accompanied by discussion of the control scheme design and presentation of experimental results showing the performance of the mechanism

Within the field of Neuro Robotics we are driven primarily by the desire to understand how humans and animals live and grow and solve every day's problems. To this aim we adopted a "learn by doing" approach by building artificial systems, e.g. robots that not only look like human beings but also represent a model of some brain process. They should, ideally, behave and interact like human beings (being situated). The main emphasis in robotics has been on systems that act as a reaction to an external stimulus (e.g. tracking, reaching), rather than as a result of an internal drive to explore or "understand" the environment. We think it is now appropriate to try to move from acting, in the sense explained above, to "understanding". As a starting point we addressed the problem of learning about the effects and consequences of self-generated actions. How does the robot learn how to pull an object toward itself or to push it away? How does the robot learn that spherical objects roll while a cube only slides if pushed? Interacting with objects is important because it implicitly explores object representation, event understanding, and can provide definition of object-hood that could not be grasped with a mere passive observation of the world. Further, learning to understand what one's own body can do is an essential step toward learning by imitation. In this view two actions are similar not only if their kinematics and dynamics are similar but rather if the effects on the external world are the same. Along this line of research we discuss some recent experiments performed at the AI-Lab at MIT and at the LIRA-Lab at the University of Genova on COG and Babybot respectively. We show how the humanoid robots can learn how to poke and prod objects to obtain a consistently repeatable effect (e.g. sliding in a given direction), to help visual segmentation, and to interpret a poking action performed by a human manipulator.

Summary form only given. Vision and manipulation are inextricably intertwined in the primate brain. Tantalizing results from neuroscience are shedding light on the mixed motor and sensory representations used by the brain during reaching, grasping, and object recognition. We now know a great deal about what happens in the brain during these activities, but not necessarily why. Is the integration we see functionally important, or just a reflection of evolution's lack of enthusiasm for sharp modularity? We wish to instantiate these results in robotic form to probe their technical advantages and to find any lacunae in existing models. We believe it would be missing the point to investigate this on a platform where dextrous manipulation and sophisticated machine vision are already implemented in their mature form, and instead follow a developmental approach from simpler primitives. We begin with a precursor to manipulation, simple poking and prodding, and show how it facilitates object segmentation, a long-standing problem in machine vision. The robot can familiarize itself with the objects in its environment by acting upon them. It can then recognize other actors (such as humans) in the environment through their effect on the objects it has learned about. We argue that following causal chains of events out from the robot's body into the environment allows for a very natural developmental progression of visual competence, and relate this idea to results in neuroscience.

Developmental robotics is an emerging field located at the intersection of robotics, cognitive science and developmental sciences. This paper elucidates the main reasons and key motivations behind the convergence of fields with seemingly disparate interests, and shows why developmental robotics might prove to be beneficial for all fields involved. The methodology advocated is synthetic and two-pronged: on the one hand, it employs robots to instantiate models originating from developmental sciences; on the other hand, it aims to develop better robotic systems by exploiting insights gained from studies on ontogenetic development. This paper gives a survey of the relevant research issues and points to some future research directions.

For the purposes of manipulation, we would like to know what parts of the environment are physically coherent ensembles, that is, which parts will move together, and which are more or less independent. It takes a great deal of experience before this judgement can be made from purely visual information. This paper develops active strategies for acquiring that experience through experimental manipulation, using tight correlations between arm motion and optic flow to detect both the arm itself and the boundaries of objects with which it comes into contact.

This paper describes a novel network model, which is able to control its growth on the basis of the approximation requests. Two classes of self-tuning neural models are considered; namely Growing Neural Gas (GNG) and SoftMax function networks. We combined the two models into a new one: hence the name GNG-Soft networks. The resulting model is characterized by the effectiveness of the GNG in distributing the units within the input space and the approximation properties of SoftMax functions. We devised a method to estimate the approximation error in an incremental fashion. This measure has been used to tune the network growth rate. Results showing the performance of the network in a real-world robotic experiment are reported.

In the classical neural-gas method, the neurons are located in the input space according to the input density. But if we want to use this kind of approach as a function approximator, it can be interesting to change the spatial distribution of the neurons, so that they concentrate in regions where the unknown function appeared to be more complex. To achieve this goal, we modified the update rule of the neurons so that they move towards regions where the estimated local error is high. In this article, we first show how to estimate this error locally to each neuron, and then we detail the modification of the algorithm. Finally, we present some simulations results that allow us to compare the modified approach with the classical one.

We propose a new neural approach for approximating function using a reinforcement-type learning: each time the network generates an output, the environment responds with the scalar distance between the delivered output and the expected one. Thus, this distance is the only information the network can use to modify the estimation of the multi-dimensional output. This reinforcement feature is embedded in a neural-gas method, taking advantages of the different facilities it offers. We detail the global algorithm and we present some simulation results in order to show the behaviour of the developed method.

The aim of the paper is to propose that the adoption of a framework of biological development is suitable for the construction of artificial systems. We argue that a developmental approach does provide unique insights on how to build highly complex and adaptable artificial systems. In turn, it might also aid neuroscientists in a better understanding of the human brain functions involved in sensori-motor control. To illustrate our point, we use as an example the acquisition of goal-directed reaching in human infants, and demonstrate the underlying mechanisms of biological development. In a second part, we outline a) how mechanisms of biological development can be adapted to the artificial world, and b) how this artificial development differs from traditional engineering approaches to robotics. The experimental part is based on a set-up composed of a monocular robot head with two degrees of freedom and a two degrees of freedom arm. The motor control of both the head and the arm is based on the so-called force field approach described in the biological literature as a mechanism implemented at the level of the spinal cord to control ballistic motion of the arm. The visual part is, at this stage, limited to a simple target tracking procedure based on color information. The goal of the system is to reach with the arm the target fixated by the eye. Initially the new-born system is capable of performing a limited number of motor actions coded as rough visuo-motor maps and initiated by the appearance of the target in the field of view. During the developmental phase the system, through repetitive reaching of different points in the arm workspace, refines the visuo-motor maps without explicit knowledge of the system's kinematic parameters.

This paper describes a novel approach to the problem of reaching an object in space under visual guidance. The approach is characterized by a great robustness to calibration errors, such that virtually no calibration is required. Servoing is based on binocular vision: a continuous measure of the end-effector motion field, derived from real-time computation of the binocular optical flow over the stereo images, is compared with the actual position of the target and the relative error in the end-effector trajectory is continuously corrected. The paper outlines the general framework of the approach, shows how visual measures are obtained and discusses the synthesis of the controller along with its stability analysis. Real-time experiments are presented to show the applicability of the approach in real 3-D applications.

The role of vision in two-arm manipulation is investigated in order to suggest a qualitative approach for the solution of coordinated manipulative actions. The visuomotor behavior proposed, although partially supported by experimental results, differs from more geometrical approaches based on explicit 4D measures and trajectory estimation in that it is based on direct visual information, such as stereo disparity or the 2D image motion field. The experiment is limited to analysis of the optical flow information extracted from manipulative actions performed by a human operator. These experiments demonstrate that the qualitative approach proposed is not in contrast with human behavior and that the two-arm manipulation task proposed can be seen as a testbed for qualitative and geometrical approaches.>