Face recognition has been extensively employed in practice, such as attendance system and public security. Linear discriminant analysis (LDA) algorithm is one of the most significant ones in the field of face recognition, but it is very difficult for many clients to employ it in their resource-constrained devices (e.g., smartphones and notebook computers). Outsourcing computation provides a promising method for clients to perform heavy tasks with limited computing power. In this paper, we design a protocol of outsourcing LDA-based face recognition to an untrusted cloud, which can help the client to complete the operations of matrix inversion (MI), matrix multiplication (MM) and eigenvalue decomposition (ED) simultaneously. The proposed outsourcing protocol can hide the private data of the client from the cloud. More importantly, the client can verify whether the outsourcing results are correct or not with probability one and so it is impossible for the server to deceive the client. In addition, the proposed protocol greatly decreases the computational complexity of the client thus enabling the client to complete LDA algorithm efficiently. Finally, we implement the protocol and give a comprehensive evaluation. The experimental results demonstrate that the client obtain great computing savings and the face recognition accuracy in the proposed protocol is almost identical to the original LDA algorithm.