Most of magnetic localization and orientation systems use single dipole models to calculate magnetic field, which, due to the fundamental limitation of the dipole, become inaccurate as the sensors approach the surface of the magnet. Moreover, they are unable to account for geometry, magnetization and any physical imperfections of the magnetic source. This paper presents a novel method of modeling the magnetic field of axisymmetric permanent magnets with artificial neural networks (ANNs), which permits accurate field modeling even at close proximity to the magnet. ANN based field models used to characterize experimental field data of solid and annular cylindrical magnets were found to be on average at least 10 times more accurate than that of dipole based models. Using model-based localization, tracking results from following a predetermined figure `8' path were also promising, with an average error of 0.43 mm in XY plane and 0.93 mm in XZ plane from only three sensor inputs.