We describe an algorithm for detection of electrical outlets in images obtained by a monocular camera. We provide a method for calculating 3D coordinates of outlet holes with accuracy high enough for a robot to plug in without visual servoing. The paper proposes a novel algorithm for accurate pose estimation of a small planar object. We use the plane normal obtained from stereo data as a hard constraint for a planar PnP problem. A different cost function is proposed together with a closed-form solution for the object pose. This system is used for continuous operation of a PR2 robot enabling it to run for 2 weeks without human supervision.

We describe an autonomous robotic system capable of navigating through an office environment, opening doors along the way, and plugging itself into electrical outlets to recharge as needed. We demonstrate through extensive experimentation that our robot executes these tasks reliably, without requiring any modification to the environment. We present robust detection algorithms for doors, door handles, and electrical plugs and sockets, combining vision and laser sensors. We show how to overcome the unavoidable shortcoming of perception by integrating compliant control into manipulation motions. We present a visual-differencing approach to high-precision plug-insertion that avoids the need for high-precision hand-eye calibration.

The stereo correspondence problem is considered. A new fast and stable algorithm for matching between two rectified images is proposed. The algorithm is based on grouping of image pixels into intervals in image columns and subsequent using dynamic programming technique to produce a dense disparity map. The algorithm enables creating fully automatic - nonparametric stereovision systems. The algorithm can be used in real time stereovision applications, as the sample implementation achieves 15-25fps processing speed in a typical environment. The proposed algorithm enables user to control ratio of speed-quality.

This paper describes a hidden Markov model-based static and dynamic 3D gesture recognition system. The shape and position of the hands, segmented and tracked using a novel 3D statistical model for the upper body in stereo sequences, are used as observation vectors. The upper body model allows for accurate 3D localization of the hands in the presence of partial occlusions, self occlusions and different illumination conditions. The accuracy of our approach is reflected by the performance of our 3D gesture based editing system, that reaches 96% over 12 dynamic gestures and four static gestures.