In Programming by Demonstration, a flexible representation of manipulation motions is necessary to learn and generalize from human demonstrations. In contrast to subsymbolic representations of trajectories, e.g. based on a Gaussian Mixture Model, a partially symbolic representation of manipulation strategies based on a temporal satisfaction problem with domain constraints is developed. By using constrained motion planning and a geometric constraint representation, generalization to different robot systems and new environments is achieved. In order to plan learned manipulation strategies the RRT-based algorithm by Stilman et al. is extended to consider, that multiple sets of constraints are possible during the extension of the search tree.

The planning of grasping motions is demanding due to the complexity of modern robot systems. In Programming by Demonstration, the observation of a human teacher allows to draw additional information about grasping strategies. Rosell showed, that the motion planning problem can be simplified by globally restricting the set of valid configurations to a learned subspace. In this work, the transformation of a humanoid grasping strategy to an anthropomorphic robot system is described by a probabilistic model, called variation model, in order to account for modeling and transformation errors. The variation model resembles a soft preference for grasping motions similar to the demonstration and therefore induces a non-uniform sampling distribution on the configuration space. The sampling distribution is used in a standard probabilistic motion planner to plan grasping motions efficiently for new objects in new environments.

In this paper we propose a process which is able to generate abstract service robot mission representations, utilized during execution for autonomous, probabilistic decision making, by observing human demonstrations. The observation process is based on the same perceptive components as used by the robot during execution, recording dialog between humans, human motion as well as objects poses. This leads to a natural, practical learning process, avoiding extra demonstration centers or kinesthetic teaching. By generating mission models for probabilistic decision making as Partially observable Markov decision processes, the robot is able to deal with uncertain and dynamic environments, as encountered in real world settings during execution. Service robot missions in a cafeteria setting, including the modalities of mobility, natural human-robot interaction and object grasping, have been learned and executed by this system.

This paper presents a technique to learn flexible action selection in autonomous, multi-modal human-robot interaction (HRI) from observing multi-modal human-human interaction (HHI). A model is generated using the proposed technique with symbolic states and actions, representing the scope of the observed mission. Variations in human behavior can be learned as stochastic action effects while execution time perception noise is taken into account, using likelihood models. During execution, the model is used for dynamic action selection in HRI situations. The model as well as the evaluation system integrate the interaction elements of spoken dialog, human body configuration and exchanged objects. The technique is evaluated on a multi-modal service robot which is both able to observe the demonstration of two humans as well as execute the generated mission autonomously.

This paper presents a reasoning system for a multi-modal service robot with human-robot interaction. The reasoning system uses partially observable Markov decision processes (POMDPs) for decision making and an intermediate level for bridging the gap of abstraction between multi-modal real world sensors and actuators on the one hand and POMDP reasoning on the other. A filter system handles the abstraction of multi-modal perception while preserving uncertainty and model-soundness. A command sequencer is utilized to control the execution of symbolic POMDP decisions on multiple actuator components. By using POMDP reasoning, the robot is able to deal with uncertainty in both observation and prediction of human behavior and can balance risk and opportunity. The system has been implemented on a multi-modal service robot and is able to let the robot act autonomously in modeled human-robot interaction scenarios. Experiments evaluate the characteristics of the proposed algorithms and architecture.

Human activity recognition is an essential ability for service robots and other robotic systems which interact with human beings. To be proactive, the system must be able to evaluate the current state of the user it is dealing with. Also future surveillance systems will benefit from robust activity recognition if real time constraints are met, allowing to automate tasks that have to be fulfilled by humans yet. In this paper, a novel approach for the integration of a feature selection in human motion recognition is proposed. Typically, the features are chosen with respect to the relevance of the features for the classification of the activity which shall be recognized. Our new approach extends this process by involving background knowledge about the features and active user engagement. Using taxonomies built on the complete feature set, users can be provided with an interface to guide and refine the selection process. Thereby, certain problems can be avoided which are common if noisy or small amounts of training data are used to train the system.

This paper presents an approach to model multi-modal human-robot interaction as partially observable Markov decision processes (POMDPs) for a service robot in realistic settings. Interaction modalities include spoken dialog and non-verbal human activities like gestures and general body postures. By using POMDPs which can model uncertainties in robot perception as well as human behavior, robustness and flexibility concerning autonomous decision making are improved in real world settings. This paper presents strategies to express perception uncertainties, stochastic human behavior and typical mission objectives in explicit POMDP models. Additionally, a system is presented to compile models from more compact representations. Finally, models are actually evaluated on a physical, autonomous service robot, controlled by POMDP decision making and compared to a classical baseline controller in typical domestic missions.

Human activity recognition is an essential ability for service robots and other robotic systems which are in interaction with human beings. To be proactive, the system must be able to evaluate the current state of the user it is dealing with. Also future surveillance systems will benefit from robust activity recognition if realtime constraints are met, allowing to automate tasks that have to be fulfilled by humans yet. In this paper, a thorough analysis of features and classifiers aimed at human activity recognition is presented. Based on a set of 10 activities, the use of different feature selection algorithms is evaluated, as well as the results different classifiers (SVMs, Neural Networks, Bayesian Classifiers) provide in this context. Also the interdependency between feature selection method and chosen classifier is investigated. Furthermore, the optimal number of features to be used for an activity is examined.