In this paper, we address the problem of receiving a digitally modulated signal in the presence of another identically modulated co-channel interference (CCI) signal. We propose a joint MLSE algorithm that uses a dynamic channel description (J-MLSE/DCD). This algorithm deals with CCI more effectively while keeping the overall complexity low. Our approach is to adaptively truncate the channels of both the signal and the interference according to their individual power, thereby greatly reducing the complexity. Analytic BER lower bounds for the J-MLSE algorithms in the time-varying specular multipath Rayleigh-fading channels are derived. Simulation results for the GMSK signals used in the GSM system are presented.

In this paper, we address the problem of receiving a digitally modulated signal in the presence of another identically modulated cochannel interfering (CCI) signal. We propose a joint MLSE algorithm that uses a dynamic channel description (J-MLSE/DCD). This algorithm deals with CCI more effectively while keeping the overall complexity low. In the proposed algorithm, we adaptively truncate the channels of both the desired signal and the CCI according to their individual power, thereby greatly reducing the complexity. We also describe the channel by using only a small number of channel parameters to reduce the channel mismatch before the Viterbi algorithm is applied. Analytic bit-error rate (BER) lower bounds for the J-MLSE algorithms in the time-varying specular multipath Rayleigh-fading channels are derived. Simulation results of the GMSK modulated signals used in the GSM system with path delay spread up to three symbols are presented. We find that the proposed algorithm performs significantly better than the spatial-whitened MLSE and is comparable to the high-complexity joint MLSE.