To mitigate the rising concern on privacy infringement in recommenders, federated recommender (FRec) boosted with local differential privacy (LDP) has been proposed, in which each user privately retains his (or her) dataset, and only exposes model parameters distorted by LDP to a parameter server (PS). Although such kind of solution can largely preserve data privacy, it overlooks the potential unfairness issue. For example, a user can become a free-rider by setting an extremely small privacy budget to completely distort his (or her) model such that the model training is mainly contributed by other users. To guarantee fairness, we design a fairness-aware federated recommender framework called FFRec when users adopt heterogeneous privacy budgets. In FFRec, fairness implies that clients with higher utilities will have more opportunities to participate in training. We integrate fairness into FFRec by taking both sample size and privacy requirements into account. Specifically, a user is selected by the PS to participate in model training based on his (or her) sample size and potential noise influence by LDP and cumulative participation counts. We formulate the fairness-aware client selection problem as an integer programming problem and then reduce it into a submodular problem which has been proven to be NP-hard. To resolve this problem, we employ an approximation algorithm with an approximation ratio of 1 − e−12. Extensive experiments with popular field-measured datasets (i.e., Movielens and Netflix) demonstrate that FFRec can improve fairness by 10.3%-38.5% while guaranteeing high recommendation accuracy