This paper presents a navigation network based deep reinforcement learning framework for autonomous indoor robot exploration. The presented method features a pattern cognitive non-myopic exploration strategy that can better reflect universal preferences for structure. We propose the Extendable Navigation Network (ENN) to encode the partially observed high-dimensional indoor Euclidean space to a sparse graph representation. The robot’s motion is generated by a learned Q-network whose input is the ENN. The proposed framework is applied to a robot equipped with a 2D LIDAR sensor in the GAZEBO simulation where floor plans of real buildings are implemented. The experiments demonstrate the efficiency of the framework in terms of exploration time.