A picture of a scene is used to extract information for an adaptive control algorithm. The object of interest is first located by means of a classifier. The position and orientation of the object are determined from a binary picture. The desired path which the gripper of the manipulator is to follow is specified by discrete points, first is the Cartesian and then in the joint coordinate system. The adaptive self-tuning controller is outlined. The controller design is based on time series difference equation model, in which the parameters are estimated on-line. The gains of the controller are tuned so that a quadratic performance criteria is minimized. The performance of the system designed is then tested experimentally. Results of these experiments are presented.