Partially Observable Markov Decision Processes (POMDPs) offer a powerful mathematical framework for making optimal action choices in noisy and/or uncertain environments, in particular, allowing us to merge localization and decision-making for mobile robots. While advancements in POMDP techniques have allowed the use of much larger models, POMDPs for robot navigation are still limited by large state space requirements for even small maps. In this work, we propose a method to automatically generate a POMDP representation of an environment. By using variable resolution decomposition techniques, we can take advantage of characteristics of the environment to minimize the number of states required, while maintaining the level of detail required to find a robust and efficient policy. This is accomplished by automatically adjusting the level of detail required for planning at a given region, with few states representing large open areas, and many smaller states near objects. We validate this algorithm in POMDP simulations, a robot simulator as well as an autonomous robot.