In this paper we present formulas for the computation of error probability in the presence of quadrature-channel or adjacentchannel interference in addition to intersymbol interference in a minimum shift keying system. The filters in the receiver and transmitter are arbitrary but with a finite number of poles. The effect of phase jitter in the main channel, phase and symbol timing misalignment in the interfering channels, and sampling time jitter is taken into account. The probability of error is averaged over the phase and symbol timing misalignment. Numerical results are presented for Butterworth filters in the receiver and transmitter with two, three, and four poles. Curves of error probability as a function of various variables (signal-to-noise ratio, bandwidth of receiver and transmitter filters, number of poles, channel frequency separation, phase jitter, sampling time, and symbol timing and phase misalignment) are presented. The method of this paper can readily be applied to other filters; hence, it can be used in the design and prediction of the performance of digital communication systems.