Ignoring actuator dynamics in control of rigid manipulators can in practice result in performance degradation or loss of system stability. However, consideration of actuator dynamics usually requires measurement of robot joint torques. This paper addresses motion tracking control of an n-DOF rigid robot by taking into account its actuator dynamics. Joint torque measurement is avoided by using an adaptive observer. The backstepping technique is adopted to develop a dynamically smooth adaptive nonlinear controller dealing with uncertainties in manipulator and actuator dynamics. Semi-global convergence of motion tracking errors as well as torque estimation error are proven without any persistency of excitation condition. Simulation examples demonstrate low noise sensitivity of the proposed method in comparison with those using torque measurement.