A Maximum Likelihood receiver for binary phase-shift keyed signals is derived for the case when the channel includes a bandpass nonlinearity preceded and followed by sources of additive noise. This optimum receiver is then approximated for the case of a limitertype of channel and is seen to be a nonlinear function of both pre- and post-limiter signal to noise ratios and a normalized limiter softness factor λ*. The probability of error for this receiver is evaluated for the case of a large time-bandwidth productWTusing a central limit theorem argument.

An analytical approach is developed to investigate the performance of coherent phase-shift keying (CPSK) systems transmitted through a nonlinear satellite transponder. A novel quadrature model of the satellite nonlinearity is introduced, and is found to yield an excellent fit to the actual traveling wave tube (TWT) characteristics as commonly used in a satellite transponder, with the advantage of requiring the choosing of only a few parameter values. Based on this quadrature model an expression for the bit error rate is derived in the form of an infinite series where the leading term corresponds to the error-rate for a linear channel and the remaining terms are correction terms accounting for nonlinear effects. The performance of a binary PSK system is computed and presented as a function of the up-link and down-link carrier-to-noise-power ratios (CNR) and a bandwidth spread factor.