This paper presents a novel adaptive controller for image-based visual servoing of a small autonomous helicopter to cope with uncalibrated camera parameters and unknown 3-D geometry of the feature points. The controller is based on the backstepping technique but differs from the existing backstepping-based methods because the controller maps the image errors onto the actuator space via a depth-independent interaction matrix to avoid estimation the depth of the feature points. The new design method makes it possible to linearly parameterize the closed-loop dynamics by the unknown camera parameters and coordinates of the feature points in the three dimensional space so that an adaptive algorithm can be developed to estimate the unknown parameters and coordinates on-line. Two potential functions are introduced in the controller to guarantee convergence of the image errors and to avoid trivial solutions of the estimated parameters. The Lyapunov method is used to prove the asymptotic stability of the proposed controller based on the nonlinear dynamics of the helicopter. Simulations have been also conducted to demonstrate the performance of the proposed method.