Efficient robotic exploration of an unknown, sensor limited, global-information-deficient environment poses a unique challenge to path planning algorithms because no deterministic guarantees on path completion and mission success can be made. Integrated Exploration (IE), which strives to combine localization and exploration, must be solved in order to create an autonomous robotic system capable of long term operation in new and challenging environments. This paper formulates a probabilistic framework which allows the creation of exploration algorithms providing probabilistic guarantees of success. A novel connection is made between the Hamiltonian Path Problem and exploration. The Guaranteed Probabilistic Information Explorer (G-PIE) is developed for the IE problem, providing a probabilistic guarantee on path completion, and asymptotic optimality of exploration.