Aiming at the problem of real-time and low accuracy of automatic recognition of human abnormal behaviour in a public area surveillance video, a simple multi-scale human anomaly behaviour detection algorithm based on video was proposed. Firstly, the binary image sequence of human body in surveillance video is acquired by background modelling method based on visual background extraction (ViBe). Then, the simple multi-scale algorithm is constructed by combining the aspect ratio, motion trajectory and video continuous interframe motion acceleration of the minimum circumscribed rectangle of the binarised image. The human target behaviour is judged, and then the normal behaviour of the human body - standing, walking, jogging, and abnormal behaviour - shouting for help, falling, punching, wandering, and sudden running are identified. The experimental results show that the human body moving target recognition by ViBe combined with simple multi-scale algorithm for abnormal behaviour detection has good real-time performance and high accuracy.