A novel 3D CAD-based vision system by using sparse structured light images is presented in this paper. By using an eye-on-hand structured light sensor, sparse range images are collected from several different positions based upon the requirements in the process of object recognition and localization. In order to recognize object with minimal sensing activities, a novel concept of maximum expected rate of hypothesis reduction (MERHR) for sensor planning is proposed its implementational procedure is carefully designed. By pre-computing the sensor's optimal configuration and storing it into a lookup table, the heavy computation burden for sensor planning during on-line recognition phase can be avoided.