This paper proposes a safe emergency stop control for collision avoidance of gantry cranes. The crane mechanism is widely used for handling heavy load. A big problem is that the suspended load swings easily. This leads to possibilities of collision of load with obstacle during the crane work. Emergency stop for collision avoidance is, however, also dangerous. Because residual sways after the stop is easy to be caused by the stop. Therefore, this paper presents an anti-sway control method when the emergency stop. The method is based on inverse dynamics of the crane system. Thus the inverse dynamics problem is also discussed. Using the inverse dynamics calculation and a real time obstacle detection system, a feed-forward based emergency stop control method is proposed. Experiments of emergency stop control using the method are shown.