In the field of aerial robotics, one of the key challenges is to enable aerial manipulators to exert substantial forces on the environment. Enabling this will allow the technology to perform meaningful tasks airborne, such as cleaning or grinding surfaces. While in contact and applying a large, continuous force, control of the UAV's attitude is a challenge. In this work, we show that a regular (PID-based) attitude controller is incapable of stabilizing aerial manipulators that apply physical contact forces on the environment that are comparable to the UAV's weight. We present a novel control algorithm that uses an LQR-optimized state feedback on the roll and yaw angle while in contact. Experiments on a UAV of 1.5 kg show that the proposed controller is capable of applying a contact force of over 15 N - equal to the UAV's weight - sustained for several minutes.