Giving an advice to a mobile robot still requires classical user interfaces. A more intuitive way of commanding can be provided by verbal or gesture commands. In this article, we present new approaches and enhancements for established methods that are in use in our laboratory. Our aim is to direct a robot with simple dynamic gestures. We focus on visual gesture recognition. Based on skin color segmentation algorithms for tracking the user's hand, hidden Markov models are used for gesture type recognition. The filters applied to the recorded trajectory strongly compress the input data. They also mark start and end point of a possible gesture. The hidden Markov models have been enhanced by a threshold model in order to wipe out insignificant movements. Pre-classification of the reference gestures serves for keeping computational effort low.

Service robots require interactive programming interfaces that allow users without programming experience to easily instruct the robots. Systems following the programming-by-demonstration (PbD) paradigm are getting closer to this goal. Visual observation of the user and environment is one important aspect for reasoning about goals and actions. With respect to the automatic generation of adaptive programs, a PbD-System should detect, classify and determine the pose of manipulable objects in a fast and stable way. This paper presents a comparison of four established methods proposing a new object classification approach that combines these methods. This gives a means for setting up a world model of a manipulations scene automatically and initializes active contour parameters in order to trade motions of these objects.

Since programming by demonstration (PbD) approaches have reached prime importance in interactive robot programming, sensor technology for tracking user actions and user behavior have become more and more important. However, traditional methods based on single sensor system input are already at their limits, since PbD is an application area where real-time requirement do play an important role. Thus, a sensor fusion approach is proposed which serves input data for a finite state automaton. The input sources are on the one hand a data-glove which classifies different grips and on the other hand a movable camera head which tracks the movements of the data glove as well as estimates object positions. Both sensor sources use time efficient algorithms, since sensory data must be processed in real-time. The efficiency of this approach is proven in a PbD environment were flexible infusion bags are handled and respective actions and object positions are recognized.