We consider a relay-aided massive multi-input multi-output (mMIMO) system where a base station (BS) serves various users via multiple relays by employing non-orthogonal multiple access (NOMA). We practically model this system by considering spatially-correlated Rician-faded channels, and channel estimation errors both at the BS and at the users, which consequently perform imperfect successive interference cancellation (SIC). We consider these two artifacts, and derive a lower bound on the spectral efficiency (SE) of this multi-relay NOMA system, which is valid for arbitrary number of BS antennas. We crucially show that i) Rician-faded channels are immune to the errors caused by imperfect SIC at users; ii) high spatial correlation improves the spatial diversity among relays located close to BS and achieves better sum SE than its uncorrelated counterpart.

We consider the downlink of a relay-aided multi-cell massive multi-input multi-output (mMIMO) system, where in each cell the base station (BS) serves its users via multiple relays by employing non-orthogonal multiple access (NOMA). We model this system by considering spatially-correlated Rician-faded channels and their estimation errors. The users, consequently, perform imperfect successive interference cancellation (SIC). We derive a lower bound on the spectral efficiency (SE) of this system. We then optimize the non-convex global energy efficiency (GEE) metric, which is a fractional function of the optimization variables. We solve this problem by considering a low-complexity alternating minimization maximization approach, which splits a complex joint problem into multiple simpler convex surrogate sub-problems. We propose a novel surrogate function to exploit this framework, and analytically show that it satisfies the desirable properties of a valid surrogate function. We numerically show that 1) reusing the pilots in each cell, when the channel has sufficiently hardened, provides higher SE than using orthogonal pilots in all cells and 2) the proposed GEE algorithm provides similar GEE as that of an existing joint optimization framework, but with much less complexity.