To address growing concerns about data privacy on mobile devices, the federated learning (FL) paradigm enables clients to collaboratively train models while sharing only local model updates. However, privacy risks remain in FL, as adversaries can still infer sensitive information from these updates. To enhance secure aggregation in FL, various protection mechanisms combining encryption and multi-party computation (MPC) have been proposed. These approaches, however, often introduce substantial communication and computational overhead, making secure aggregation impractical on resource-constrained devices, e.g., smart phones. To tackle these efficiency challenges, we are among the first to propose the integration of differential privacy (DP) with encryption and MPC for secure aggregation. Our proposed protocol, Federated Learning with Noise-based Secure Aggregation (FedNSA), injects noise through DP to obfuscate individual model updates. Encryption is employed to correlate the noise across different clients, while MPC ensures perfect noise cancellation at the server side. Finally, we theoretically analyze its advantages and conduct extensive experiments on public datasets to demonstrate the superiority of our approach across multiple dimensions in comparison with the state-of-the-art baselines.