Offline Reinforcement Learning (RL) provides a promising avenue for training policies from pre-collected datasets when gathering additional interaction data is infeasible. However, existing offline RL methods often assume stationarity or only consider synthetic perturbations at test time, assumptions that often fail in real-world scenarios characterized by abrupt, time-varying offsets. These offsets can lead to partial observability, causing agents to misperceive their true state and degrade performance. To overcome this challenge, we introduce Forecasting in Non-stationary Offline RL (FORL), a framework that unifies (i) conditional diffusion-based candidate state generation, trained without presupposing any specific pattern of future non-stationarity, and (ii) zero-shot time-series foundation models. FORL targets environments prone to unexpected, potentially non-Markovian offsets, requiring robust agent performance from the onset of each episode. Empirical evaluations on offline RL benchmarks, augmented with real-world time-series data to simulate realistic non-stationarity, demonstrate that FORL consistently improves performance compared to competitive baselines. By integrating zero-shot forecasting with the agent's experience, we aim to bridge the gap between offline RL and the complexities of real-world, non-stationary environments.

Tools and other objects offer agents a range of potential actions, commonly referred to as affordances. Each tool is typically designed with a primary purpose in mind-like a hammer’s function to drive nails. However, tools can also serve purposes beyond their original design. These alternative uses represent secondary affordances, extending the tool’s utility beyond its primary intended function. While prior robotics research on affordance perception and learning has primarily focused on primary affordances, our work addresses the less-explored area of learning secondary tool affordances from human partners. Using the iCub robot equipped with three cameras, we observed humans performing actions on twenty objects using four different tools in ways that deviate from their primary purposes. For example, the iCub observed humans using rulers not for measuring but to push, pull, and move objects. In this setting, we constructed a dataset by taking pictures of objects before and after each action is executed. To model secondary affordance learning, we trained three neural networks (ResNet-18, ResNet-50, and ResNet-101) on three prediction tasks using these raw images as input: (1) identifying which tool was used to move an object, (2) predicting the tool with additional action category information, and (3) jointly predicting both the tool and action performed. Our results demonstrate that deep learning architectures enable the iCub robot to successfully predict secondary tool affordances, thereby paving the road for human-robot collaborative object manipulation involving complex affordances. Code and data from this study are available at https://github.com/BosongDing/second_affordance

Our study explores how ecological aspects of motor learning enhance survival by improving movement efficiency and mitigating injury risks during task failures. Traditional motor control theories mainly address isolated body movements and often overlook these ecological factors. We introduce a novel computational motor control approach, incorporating ecological fitness and a strategy that alternates between success-driven movement efficiency and failure-driven safety, akin to win-stay/lose-shift tactics. In our experiments, participants performed squat-to-stand movements under novel force perturbations. They adapted effectively through various adaptive motor control mechanisms to avoid falls, reducing failure rates rapidly. The results indicate a high-level ecological controller in human motor learning that switches objectives between safety and movement efficiency, depending on failure or success. This approach is supported by policy learning, internal model adaptation, and adaptive feedback control. Our findings offer a comprehensive perspective on human motor control, integrating risk management in a hierarchical reinforcement learning framework for real-world environments.

Reservoir computing provides an attractive alternative for time series data representation due to their gradient-free learning and energy efficient operation. In a ‘reservoir computer’ (RC), the reservoir is a random recurrent neural network that forms a high dimensional non-linear dynamical system which can be mapped to a desired sequence or dynamical system through linear regression. Although, the non-linearity of the reservoir gives its power, there is no guarantee that it is the most suitable for the task at hand. To this end, in this study, we propose to amplify the effectiveness of reservoirs by augmenting them with higher order terms computed based on reservoir unit activations. We name this model as Higher-order-augmented Reservoir Computer (ha-RC). We test the efficacy of ha-RCs by using two types of tasks: time series representation and long-term prediction, i.e. learning a dynamical system. Our experiments show that with the proposed model, a given learning accuracy can be achieved with significantly less memory resource compared to the baseline of standard Reservoir Computer (s-RC). This becomes possible as ha-RC can handle complex learning problems with much smaller reservoirs compared to s-RC models. This memory efficiency directly translates to energy efficiency as less number of arithmetic operations are needed with ha-RCs to reach the same level of accuracy compared to s-RCs. These points make our proposed model ideal for hardware implementation and edge computing.

Learning from demonstration (LfD) is an effective way of generating robot behaviors by transferring human demonstrated movements to robots. One common method to accomplish LfD is ‘Behavior Cloning’ (BC) with human-in-the-loop control, where the data obtained by human teleoperation of the robot is used to construct a non-linear controller by learning the state-to-action mapping. In this study, we propose a novel BC system where the learning architecture parallels a feedback controller that is tasked with producing the demonstrated motor output data. The motivation behind this design is that encoding such structure in the learning mechanism is expected to endow our system with prior bias to outperform controller-agnostic BC systems, especially in resource scarce situations. The current report presents the developed novel BC model, and gives its implementation for a two degrees-of-freedom robotic system as a proof of concept. To evaluate the performance of the system, systematic experiments are run in comparison with a controller-agnostic BC system. The results show that the proposed model performs significantly better than the baseline, indicating that it is a strong candidate for LfD tasks when the demonstrated data can be assumed to be generated through a feedback controller.

Abstract Learning sensorimotor trajectories through flexible neural representations is fundamental for robots as it facilitates the building of motor skills as well as equipping them with the ability to represent the world as predictable temporal events. Recent advances in deep learning led to the development of powerful learning from demonstration (LfD) systems such as Conditional Neural Movement Primitives (CNMPs). CNMPs can robustly represent skills as movement distributions and allow them to be ‘recalled’ by conditioning the movement on a few observation points. In this study, we focus on improving CNMPs to achieve a higher resource economy by adopting a divide-and-conquer approach. We propose a novel neural architecture called Coupled CNMP (C-CNMP), that couples the latent spaces of a pair of CNMPs that splits a given sensorimotor trajectory into segments whose learning is undertaken by smaller sub-networks. Therefore, each sub-network needs to deal with a less complex trajectory making the learning less resource-hungry. With systematic simulations on a controlled trajectory data set, we show that the overhead brought by the coupling introduced in our model is well offset by the resource and performance gain obtained. To be specific, with CNMP model as the baseline, it is shown that the proposed model is able to learn to generate trajectories in the data set with a lower trajectory error measured as the mean absolute difference between the generated trajectory and the ground truth. Importantly, our model can perform well with relatively limited resources, i.e., with less number of neural network parameters compared to the baseline. To show that the findings from the controlled data set well-transfer to robot data, we use robot joint data in an LfD setting and compare the learning performance of the proposed model with the baseline model at equal complexity levels. The simulation experiments show that with also the robot joint data, the proposed model, C-CNMP, learns to generate the joint trajectories with significantly less error than the baseline model. Overall, our study improves the state of the art in sensorimotor trajectory learning and exemplifies how divide-and-conquer approaches can benefit deep learning architectures for resource economy.

Learning from demonstration (LfD) with behavior cloning is attractive for its simplicity; however, compounding errors in long and complex skills can be a hindrance. Considering a target skill as a sequence of motor primitives is helpful in this respect. Then the requirement that a motor primitive ends in a state that allows the successful execution of the subsequent primitive must be met. In this study, we focus on this problem by proposing to learn an explicit correction policy when the expected transition state between primitives is not achieved. The correction policy is learned via behavior cloning by the use of Conditional Neural Motor Primitives (CNMPs) that can generate correction trajectories in a context-dependent way. The advantage of the proposed system over learning the complete task as a single action is shown with a table-top setup in simulation, where an object has to be pushed through a corridor in two steps. Then, the applicability of the proposed method to bi-manual knotting in the real world is shown by equipping an upper-body humanoid robot with the skill of making knots over a bar in 3D space.

Reservoir Computing, in particular Echo State Networks (ESNs) offer a lightweight solution for time series representation and prediction. An ESN is based on a discrete time random dynamical system that is used to output a desired time series with the application of a learned linear readout weight vector. The simplicity of the learning suggests that an ESN can be used as a lightweight alternative for movement primitive representation in robotics. In this study, we explore this possibility and develop Context-based Echo State Networks (CESNs), and demonstrate their applicability to robot movement generation. The CESNs are designed for generating joint or Cartesian trajectories based on a user definable context input. The context modulates the dynamics represented by the ESN involved. The linear read-out weights then can pick up the context-dependent dynamics for generating different movement patterns for different contexts. To achieve robust movement execution and generalization over unseen contexts, we introduce a novel data augmentation mechanism for ESN training. We show the effectiveness of our approach in a learning from demonstration setting. To be concrete, we teach the robot reaching and obstacle avoidance tasks in simulation and in real-world, which shows that the developed system, CESN provides a lightweight movement primitive representation system that facilitate robust task execution with generalization ability for unseen seen contexts, including extrapolated ones.

Trust is an essential concept for human-human and human-robot interactions. Yet only a few studies have addressed this concept from a robot perspective -that is, forming robot trust in interaction partners. Our previous robot trust model relies on assessing the trustworthiness of the interaction partners based on the computational cognitive load incurred during the interactive task [1]. However, this model does not take into account the social markers indicative of trustworthiness, such as the gestures displayed by a human partner. In this study, we make a step toward this point by extending the model by integrating a social cue processing module to achieve social human-robot interaction. This new model serves as a novel social cognitive trust framework to enable the Pepper robot to evaluate the trustworthiness of its interaction partners based on both cognitive load (i.e., the cost of perceptual processing) and social cues (i.e., their gestures). For evaluating the efficacy of the framework, the Pepper robot with the developed model is put to interact with human partners who may take the roles of a reliable, unreliable, deceptive, or random suggestion providing partner. Overall, the results indicate that the proposed framework allows the Pepper robot to differentiate the guiding strategies of the partners by detecting deceptive partners and thus select a trustworthy partner in case of a free choice to perform the next task.

Boolean functions (BFs) can be uniquely represented with polynomial functions by representing True and False with ±1.With the 'sign-representation' framework, i.e., when the sign of the polynomials is used instead of the exact ±1, the representation is not unique anymore, and several measures of sign-representation become the target of research.One such measure is the polynomial threshold function density (PTF density), i.e., the minimum number of monomials that suffices to sign-represent a given BF.Several algorithms can find sign-representations with a low number of monomials; however, to find a representation with the minimum number of monomials possible is a combinatorial search problem.The recent success of reinforcement learning (RL) algorithms in solving combinatorial search problems poses the question of whether RL can perform well in finding sign-representations with a low number of monomials.To address this question, we focused on Deep Q-Networks (DQN) and explored its applicability to the sign-representation problem.To be concrete, we present our work on modeling RL agents for solving the signrepresentation problem and give our results on the application of DQN to BFs with a low number of variables (n = 4).Our results indicate that the trained DQN agent generalizes well and exploits intrinsic structure of BFs, such as their equivalence in terms of certain equivalence relations.

In this study, we extend our robot trust model into a multimodal setting in which the Nao robot leverages audio-visual data to perform a sequential multimodal pattern recalling task while interacting with a human partner who has different guiding strategies: reliable, unreliable, and random. Here, the humanoid robot is equipped with a multimodal auto-associative memory module to process audio-visual patterns to extract cognitive load (i.e., computational cost) and an internal reward module to perform cost-guided reinforcement learning. After interactive experiments, the robot associates a low cognitive load (i.e., high cumulative reward) yielded during the interaction with high trustworthiness of the guiding strategy of the partner. At the end of the experiment, we provide a free choice to the robot to select a trustworthy instructor. We show that the robot forms trust in a reliable partner. In the second setting of the same experiment, we endow the robot with an additional simple theory of mind module to assess the efficacy of the instructor in helping the robot perform the task. Our results show that the performance of the robot is improved when the robot bases its action decisions on factoring in the instructor assessment.

Symbolic planning and reasoning are powerful tools for robots tackling complex tasks. However, the need to manually design the symbols restrict their applicability, especially for robots that are expected to act in open-ended environments. Therefore symbol formation and rule extraction should be considered part of robot learning, which, when done properly, will offer scalability, flexibility, and robustness. Towards this goal, we propose a novel general method that finds action-grounded, discrete object and effect categories and builds probabilistic rules over them for non-trivial action planning. Our robot interacts with objects using an initial action repertoire that is assumed to be acquired earlier and observes the effects it can create in the environment. To form action-grounded object, effect, and relational categories, we employ a binary bottleneck layer in a predictive, deep encoderdecoder network that takes the image of the scene and the action applied as input, and generates the resulting effects in the scene in pixel coordinates. After learning, the binary latent vector represents action-driven object categories based on the interaction experience of the robot. To distill the knowledge represented by the neural network into rules useful for symbolic reasoning, a decision tree is trained to reproduce its decoder function. Probabilistic rules are extracted from the decision paths of the tree and are represented in the Probabilistic Planning Domain Definition Language (PPDDL), allowing off-the-shelf planners to operate on the knowledge extracted from the sensorimotor experience of the robot. The deployment of the proposed approach for a simulated robotic manipulator enabled the discovery of discrete representations of object properties such as ‘rollable’ and ‘insertable’. In turn, the use of these representations as symbols allowed the generation of effective plans for achieving goals, such as building towers of the desired height, demonstrating the effectiveness of the approach for multi-step object manipulation. Finally, we demonstrate that the system is not only restricted to the robotics domain by assessing its applicability to the MNIST 8-puzzle domain in which learned symbols allow for the generation of plans that move the empty tile into any given position.

Learning to interact with the environment not only empowers the agent with manipulation capability but also generates information to facilitate building of action understanding and imitation capabilities. This seems to be a strategy adopted by biological systems, in particular primates, as evidenced by the existence of mirror neurons that seem to be involved in multi-modal action understanding. How to benefit from the interaction experience of the robots to enable understanding actions and goals of other agents is still a challenging question. In this study, we propose a novel method, deep modality blending networks (DMBN), that creates a common latent space from multi-modal experience of a robot by blending multi-modal signals with a stochastic weighting mechanism. We show for the first time that deep learning, when combined with a novel modality blending scheme, can facilitate action recognition and produce structures to sustain anatomical and effect-based imitation capabilities. Our proposed system, which is based on conditional neural processes, can be conditioned on any desired sensory/motor value at any time step, and can generate a complete multi-modal trajectory consistent with the desired conditioning in one-shot by querying the network for all the sampled time points in parallel avoiding the accumulation of prediction errors. Based on simulation experiments with an arm-gripper robot and an RGB camera, we showed that DMBN could make accurate predictions about any missing modality (camera or joint angles) given the available ones outperforming recent multimodal variational autoencoder models in terms of long-horizon high-dimensional trajectory predictions. We further showed that given desired images from different perspectives, i.e. images generated by the observation of other robots placed on different sides of the table, our system could generate image and joint angle sequences that correspond to either anatomical or effect-based imitation behavior. To achieve this mirror-like behavior, our system does not perform a pixel-based template matching but rather benefits from and relies on the common latent space constructed by using both joint and image modalities, as shown by additional experiments. Moreover, we showed that mirror learning (in our system) does not only depend on visual experience and cannot be achieved without proprioceptive experience. Our experiments showed that out of ten training scenarios with different initial configurations, the proposed DMBN model could achieve mirror learning in all of the cases where the model that only uses visual information failed in half of them. Overall, the proposed DMBN architecture not only serves as a computational model for sustaining mirror neuron-like capabilities, but also stands as a powerful machine learning architecture for high-dimensional multi-modal temporal data with robust retrieval capabilities operating with partial information in one or multiple modalities.

This study focuses on inferring cost functions of obtained movement data using reward parameter search and pol-icy gradient based Reinforcement Learning (RL). The behavior data for this task is obtained through a series of squat-to-stand movements of human participants under dynamic perturbations. The key parameter searched in the cost function is the weight of total torque used in performing the squat-to-stand action. An approximate model is used to learn squat-to-stand movements via a policy gradient method, namely Proximal Policy Optimization(PPO). A behavioral similarity metric based on Center of Mass(COM) is used to find the most likely weight parameter. The stochasticity in the training result of PPO is dealt with multiple runs, and as a result, a reasonable and a stable Inverse Reinforcement Learning(IRL) algorithm is obtained in terms of performance. The results indicate that for some participants, the reward function parameters of the experts were inferred successfully.

Forming trust in a biological or artificial interaction partner that provides reliable strategies and employing the learned strategies to scaffold another agent are critical problems that are often addressed separately in human-robot and robot-robot interaction studies. In this paper, we provide a unified approach to address these issues in robot-robot interaction settings. To be concrete, we present a trust-based affective computational account of scaffolding while performing a sequential visual recalling task. In that, we endow the Pepper humanoid robot with cognitive modules of auto-associative memory and internal reward generation to implement the trust model. The former module is an instance of a cognitive function with an associated neural cost determining the cognitive load of performing visual memory recall. The latter module uses this cost to generate an internal reward signal to facilitate neural cost-based reinforcement learning (RL) in an interactive scenario involving online instructors with different guiding strategies: reliable, less-reliable, and random. These cognitive modules allow the Pepper robot to assess the instructors based on the average cumulative reward it can collect and choose the instructor that helps reduce its cognitive load most as the trustworthy one. After determining the trustworthy instructor, the Pepper robot is recruited to be a caregiver robot to guide a perceptually limited infant robot (i.e., the Nao robot) that performs the same task. In this setting, we equip the Pepper robot with a simple theory of mind module that learns the state-action-reward associations by observing the infant robot’s behavior and guides the learning of the infant robot, similar to when it went through the online agent-robot interactions. The experiment results on this robot-robot interaction scenario indicate that the Pepper robot as a caregiver leverages the decision-making policies – obtained by interacting with the trustworthy instructor– to guide the infant robot to perform the same task efficiently. Overall, this study suggests how robotic-trust can be grounded in human-robot or robot-robot interactions based on cognitive load, and be used as a mechanism to choose the right scaffolding agent for effective knowledge transfer.

In this paper, a method to control a manipulator using force-induced trajectory is proposed. The trajectory is learned from an operator doing the polishing task using a tool attached to the robot's end effector. The learning process is performed by a deep neural network which is designed and trained to generate a force profile according to the states (joints' positions and velocities). The admittance control technique is utilized to make the manipulator compliant to the operator movements in the teaching mode. Spring-Damper system along with Inertia-Damper system have been studied to impose the relationship between the operator's applied force and the reaction of the manipulator. The universal robot (UR5) aside with a force sensor (OptoForce) are used to run the experiment. Robot Operation System (ROS) is used to accomplish the task in real time. The polishing task is learned and achieved by the robot itself, and the force trajectories are better followed using the Inertia-Damper system as the admittance controlling scheme.

Human-in-the-loop robot control systems naturally provide the means for synergistic human-robot collaboration through control sharing. The expectation in such a system is that the strengths of each partner are combined to achieve a task performance higher than that can be achieved by the individual partners alone. However, there is no general established rule to ensure a synergistic partnership. In particular, it is not well studied how humans adapt to a nonstationary robot partner whose behavior may change in response to human actions. If the human is not given the choice to turn on or off the control sharing, the robot-human system can even be unstable depending on how the shared control is implemented. In this paper, we instantiate a human-robot shared control system with the “ball balancing task,” where a ball must be brought to a desired position on a tray held by the robot partner. The experimental setup is used to assess the effectiveness of the system and to find out the differences in human sensorimotor learning when the robot is a control sharing partner, as opposed to being a passive teleoperated robot. The results of the four-day 20-subject experiments conducted show that 1) after a short human learning phase, task execution performance is significantly improved when both human and robot are in charge. Moreover, 2) even though the subjects are not instructed about the role of the robot, they do learn faster despite the nonstationary behavior of the robot caused by the goal estimation mechanism built in.

We propose a novel method that arbitrates the control between the human and the robot actors in a teaching-by-demonstration setting to form synergy between the two and facilitate effective skill synthesis on the robot. We employed the human-in-the-loop teaching paradigm to teleoperate and demonstrate a complex task execution to the robot in real-time. As the human guides the robot to perform the task, the robot obtains the skill online during the demonstration. To encode the robotic skill we employed Locally Weighted Regression that fits local models to specific state region of the task based on the human demonstration. If the robot is in the state region where no local models exist, the control over the robotic mechanism is given to the human to perform the teaching. When local models are gradually obtained in that region, the control is given to the robot so that the human can examine its performance already during the demonstration stage, and take actions accordingly. This enables a co-adaptation between the agents and contributes to a faster and more efficient teaching. As a proof-of-concept, we realised the proposed robot teaching system on a haptic robot with the task of generation of a desired vertical force on a horizontal plane with unknown stiffness properties.

Dynamic Movement Primitives (DMPs)-originally a method for movement trajectory generation [1] has been also used for recognition tasks [2, 3]. However there has not been a systematic comparison between other recognition methods and DMPs using human movement data. This paper presents a comparison of commonly used Hidden Markov Model (HMM) based recognition with DMP based recognition using human generated letter trajectories. As the working principles of these two methods are very different, in addition to the performance, the numbers of adaptable parameters that are used in each method and, process time were compared. The results, indicate that HMM gives better results than DMP, with possible noise robustness advantage in DMPs for human movement.

Boolean functions (BFs) are central in many fields of engineering and mathematics, such as cryptography, circuit design, and combinatorics. Moreover, they provide a simple framework for studying neural computation mechanisms of the brain. Many representation schemes for BFs exist to satisfy the needs of the domain they are used in. In neural computation, it is of interest to know how many input lines a neuron would need to represent a given BF. A common BF representation to study this is the so-called polynomial sign representation where [Formula: see text] and 1 are associated with true and false, respectively. The polynomial is treated as a real-valued function and evaluated at its parameters, and the sign of the polynomial is then taken as the function value. The number of input lines for the modeled neuron is exactly the number of terms in the polynomial. This letter investigates the minimum number of terms, that is, the minimum threshold density, that is sufficient to represent a given BF and more generally aims to find the maximum over this quantity for all BFs in a given dimension. With this work, for the first time exact results for four- and five-variable BFs are obtained, and strong bounds for six-variable BFs are derived. In addition, some connections between the sign representation framework and bent functions are derived, which are generally studied for their desirable cryptographic properties.

The direct perception of actions allows a robot to predict the afforded actions of observed objects. In this paper, we present a non-parametric approach to representing the affordance-bearing subparts of objects. This representation forms the basis of a kernel function for computing the similarity between different subparts. Using this kernel function, together with motor primitive actions, the robot can learn the required mappings to perform direct action perception. The proposed approach was successfully implemented on a real robot, which could then quickly learn to generalize grasping and pouring actions to novel objects.

Human infants practice their initial, seemingly random arm movements for transforming them into voluntary reaching and grasping actions. With the developing perceptual abilities, infants further explore their environment using the behavior repertoire they have developed, and learn causality relations in the form of affordances, which they use for goal satisfaction and motor planning. This study proposes and implements a developmental progression on a robotic system mimicking the aforementioned infant development stages: An anthropomorphic robot hand with one basic action of swing-hand and the palmar reflex (i.e. the enclosure of the fingers upon contact) at its disposal, executes swing-hand action targeted to a salient object with different hand speeds. During the executions, it monitors the changes in its sensors, automatically forming behavior primitives such as `grasp', `hit', `carry-object' and `drop' by segmenting and differentiating the initial swing-hand action. The study then focuses on one of these behaviors, namely grasping, and shows how further practice allows the robot to learn affordances of more complex objects, which can be further used to make plans to achieve desired goals using the discovered behavior repertoire.

In this paper, we propose a method that enables a robot to learn not only the existence of affordances provided by objects, but also the behavioral parameters required to actualize them, and the prediction of effects generated on the objects in an unsupervised way. In a previous study, it was shown that through self-interaction and self-observation, analogous to an infant, an anthropomorphic robot can learn object affordances in a completely unsupervised way, and use this knowledge to make plans in its perceptual space. This paper extends the affordances model proposed in that study by using parametric behaviors and including the behavior parameters into affordance learning and goal-oriented plan generation. Furthermore, for handling complex behaviors and complex objects (such as execution of precision grasp on a mug), the perceptual processing is improved by using a combination of local and global features. Finally, a hierarchical clustering algorithm is used to discover the affordances in non-homogenous feature space. In short, object affordances for object manipulation are discovered together with behavior parameters based on the monitored effects.

In this paper, we use the notion of affordances, proposed in cognitive science, as a framework to propose a developmental method that would enable a robot to ground symbolic planning mechanisms in the continuous sensory-motor experiences of a robot. We propose a method that allows a robot to learn the symbolic relations that pertain to its interactions with the world and show that they can be used in planning. Specifically, the robot interacts with the objects in its environment using a pre-coded repertoire of behaviors and records its interactions in a triple that consist of the initial percept of the object, the behavior applied and its effect, defined as the difference between the initial and the final percept. The method allows the robot to learn object affordance relations which can be used to predict the change in the percept of the object when a certain behavior is applied. These relations can then be used to develop plans using forward chaining. The method is implemented and evaluated on a mobile robot system with limited object manipulation capabilities. We have shown that the robot is able to learn the physical affordances of objects from range images and use them to build symbols and relations that can be used in making multi-step predictions about the affordances of objects and achieve complex goals.

Abstract—Many complex robot motor skills can be represented using elementary movements, and there exist efficient techniques for learning parametrized motor plans using demonstrations and self-improvement. However, in many cases, the robot currently needs to learn a new elementary movement even if a parametrized motor plan exists that covers a similar, related situation. Clearly, a method is needed that modulates the elementary movement through the meta-parameters of its representation. In this paper, we show how to learn such mappings from circumstances to meta-parameters using reinforcement learning. We introduce an appropriate reinforcement learning algorithm based on a kernelized version of the reward-weighted regression. We compare this algorithm to several previous methods on a toy example and show that it performs well in comparison to standard algorithms. Subsequently, we show two robot applications of the presented setup; i.e., the generalization of throwing movements in darts, and of hitting movements in table tennis. We show that both tasks can be learned successfully using simulated and real robots. I.

Transferring human skills to dextrous robots in an easy, fast and robust way is one of the key challenges that still have to be tackled in order to bring robots to our every-day life. However, many problems remain unsolved. In particular, researchers are seeking new paradigms along with efficient and robust task representations that facilitate adaptation to new contexts and provide a means to appropriately react to unforeseen situations. In this paper, we present a new method for robot behaviour synthesis, where intrinsic characteristics of `Structured UKR manifolds' [13] are used to derive a closed-loop controller based on motion data obtained by the `Robot Skill Synthesis via Human Learning' paradigm [10]. We apply the method to the task of swapping Chinese health balls with a real 16 DOF robotic hand. Our results indicate that the marriage of `Structured UKR manifolds' with the `Robot Skill Synthesis via Human Learning' paradigm yields an efficient way of realising a dexterous manipulation capability on real robots.

We propose a framework for skill synthesis for robots that exploits the human capacity to learn novel control tasks. The conceptual idea is to incorporate the target robotic platform into the experimenter's body schema so that it can be controlled effortlessly as if the robot were a part of the body. Once this stage is achieved, the dexterity on a task exhibited with the new external limb -the robot- can be used for designing controllers for the task under consideration. This article exemplifies the proposed framework by showing the derivation of an effective open-loop controller that can manipulate two balls with the fingers of a 16-DOF robotic hand.

A cognitive robot system has to acquire and efficiently store vast knowledge about the world it operates in. To cope with every day tasks, a robot needs to learn, classify and recognize a manifold of different objects. Our work focuses on an object representation scheme that allows storing perceived objects in a compact way. This will enable the system to store extensive information about the world and will ease complex recognition tasks. The human visual system deploys several mechanisms to reduce the amount of information. Our goal is to develop an artificial system that mimics these mechanisms to create representations that can be used in cognitive tasks. In particular, in this paper we will present an approach that exploits similarities among different views of objects. The proposed representation scheme allows for reduction of storage required for the representation of objects and preserves the information about the similarity among objects. This is achieved by selecting 'important views' of objects, depending on their stability. Furthermore, by extending the same approach to multiple objects, we are able to exploit similarities between objects to find a common representation and to further reduce the storage requirements.

It is known that any dichotomy of {-1, 1}n can be learned (separated) with a higher-order neuron (polynomial function) with 2n inputs (monomials). In general, less than 2n monomials are sufficient to solve a given dichotomy. In spite of the efforts to develop algorithms for finding solutions with fewer monomials, there have been relatively fewer studies investigating maximum density (Pi(n)), the minimum number of monomials that would suffice to separate an arbitrary dichotomy of {-1, 1}n . This article derives a theoretical (upper) bound for this quantity, superseding previously known bounds. The main theorem here states that for any binary classification problem in {-1, 1}n (n > 1), one can always find a polynomial function solution with 2n -2n/4 or fewer monomials. In particular, any dichotomy of {-1, 1}n can be learned by a higher-order neuron with a fan-in of 2n -2n/4 or less. With this result, for the first time, a deterministic ratio bound independent of n is established as Pi(n)/2n < or = 0 75. The main theorem is constructive, so it provides a deterministic algorithm for achieving the theoretical result. The study presented provides the basic mathematical tools and forms the basis for further analyses that may have implications for neural computation mechanisms employed in the cerebral cortex.

This paper describes a new framework, called repulsive attractive (RA) network for baseline extraction on document images. The RA network is a self organizing feature detector which interacts with the document text image through the attractive and repulsive forces defined among the network components and the document image. Experimental results indicate that the network can successfully extract the baselines under heavy noise and with overlaps between the ascending and descending portions of the characters of adjacent lines. The proposed method is also applicable to a wide range of image processing applications, such as curve fitting, segmentation and thinning.