With few exceptions, most previous approaches to the structure from motion problem have been based on a decoupling between shape and motion recovery, usually via discrete or differential versions of the epipolar constraint. This paper offers a differential geometric framework for the simultaneous shape and motion recovery problem. We first pose the simultaneous shape and motion recovery problem as one of fitting a parameterized differentiable manifold to a finite set of points in a larger ambient manifold endowed with a distance metric; this framework includes epipolar constraint-based approaches as a special case. Based on this framework, we then examine a class of least-squares and total least squares fitting criteria, and the physical implications of these criteria with respect to both noise models and choice of distance metric on the relevant manifolds. We show that these criteria lead to linear objective functions on SO(3) that admit analytic solutions. We also derive a set of cyclic coordinate descent (CCD) optimization algorithms and show that simple analytic formulas can be obtained for each iteration. Simulation results on accuracy and noise sensitivity of these algorithms are also presented.