This paper describes an augmented reality system that incorporates a real-time dense stereo vision system. Analysis of range and intensity data is used to perform two functions: 1) 3D detection and tracking of the user's fingertip or a pen to provide natural 3D pointing gestures, and 2) computation of the 3D position and orientation of the user's viewpoint without the need for fiducial mark calibration procedures, or manual initialization. The paper describes the stereo depth camera, the algorithms developed for pointer tracking and camera pose tracking, and demonstrates their use within an application in the field of oil and gas exploration.