Shared mesh protection (SMP) is a protection mechanism that recovers traffic delivery against failures on a working path as rapidly as linear protection. In addition, it allows for resource sharing among the protection paths with different endpoints in a mesh network by coordinating the use of shared resources when multiple protection paths compete for these resources. Owing to limited protection resources and the preemption in consequence of priority comparison, the SMP introduces a new optimization problem whose objective is to find the optimal priority assignment that maximizes the number of protected services against any failure case for a given SMP network. In this paper, we recognize that the problem of finding the optimal solution is NP-hard and propose min-degree and capacity-density approximation algorithms, whose approximation ratios are presented with the maximum degree of services and the maximum capacity of segments of an SMP network. Moreover, we propose a depth-based algorithm that always finds the optimal solution for a hypertree-shaped SMP network. We compare the performance of the proposed algorithms in terms of the protection ratio, which we define in this paper.