Real-world objects exhibit rich physical interaction behaviours on contact. Such behaviours depend on how heavy and hard it is when held, how its surface feels when touched, how it deforms on contact, etc. Recently, there are thus growing needs for haptic exploration to estimate and extract such physical object properties as mass, friction, elasticity, relational constraints etc.. In this paper, we propose a novel paradigm, we call haptic vision,which is a vision-based haptic exploration approach toward an automatic construction of reality-based virtual space simulator, by augmenting active vision with active touch. We apply this technique to mass, relational constraints estimation and elasticity, and use these results to construct virtual object manipulation simulator. Experimental results show that feasibility and validity of the proposed approach.