We present a tangible user interface (TUI) called Tangible Graph Builder, that has been designed to allow visually impaired users to access graph and chart-based data. We describe the current paper-based materials used to allow independent graph construction and browsing, before discussing how researchers have applied virtual haptic and non-speech audio techniques to provide more flexible access. We discuss why, although these technologies overcome many of the problems of non-visual graph access, they also introduce new issues and why the application of TUIs is important. An evaluation of Tangible Graph Builder with 12 participants (8 sight deprived, 4 blind) revealed key design requirements for non-visual TUIs, including phicon design and handling marker detection failure. We finish by presenting future work and improvements to our system.