In 5G wireless communication network, massive machine type communication (mMTC) is an emerging research topic. For mMTC, non-orthogonal multiple access (NOMA) has been proposed to support its large-scale connectivity. Due to the sparsity of mMTC, compressed sensing based algorithms can be used to identify the active users and recover the sparse channel state information (CSI) vector. In this paper, we propose a Bayesian message passing algorithm based on expectation propagation (EP) for joint active user detection (AUD) and channel estimation (CE) in NOMA. The proposed method approximates the complicated target distribution with a Gaussian distribution to achieve linear complexity. Simulations demonstrate that the EP-based algorithm achieves better performance in joint AUD and CE than the exiting algorithms, especially in the low SNR regime.