Recently, magnetic tunnel junction with in-plane magnetization (i-MTJ) has been exploited to behave as a binary stochastic synapse. However, it suffers from its limited level of synaptic weight, resulting in an inaccurate learning. In this work, a compound synapse that employs multiple perpendicular MTJs (p-MTJs) in series is proposed. It possesses an analog-like synaptic weight under weak programming conditions, which leads to a stochastic learning rule and low power consumption per synaptic event. By performing system-level simulations on the MNIST database, it has been demonstrated that such compound spin synapses can realize stochastic neuromorphic computation with high accuracy and low energy consumption.