In bit-interleaved coded modulation (BICM) systems, the conventional demodulation assumes equal a priori probabilities for all constellation points, inherently leading to performance degradation. To enhance the performance of BICM, BICM with iterative decoding (BICM-ID) was developed. We aim to apply neural networks to BICM to achieve joint modulation and decoding and overcome the performance degradation, with the goal of surpassing the performance of BICM-ID. In this paper, we propose a neural demodulator that incorporates an additional probability layer to mitigate performance degradation in BICM systems. Furthermore, we introduce Joint Model-1, which integrates this neural demodulator with a belief propagation (BP) decoder for 16-QAM polar-coded BICM. To further improve performance, we use Joint Model-1 as a pre-trained model and extend it by adding an additional dense layer, resulting in Joint Model-2. Experimental results show that Joint Model-2 surpasses both BICM and BICM-ID systems under Ungerboeck labeling.

In recent years, neural network decoding of polar codes, such as neural belief propagation (BP), has been intro-duced. These methods use deep learning to transform the factor graph into a neural network model by unfolding the decoding iterations, thereby enhancing the accuracy of traditional decoding processes. However, current prevalent methodologies for loss function calculation only take into account the output of the final layer. In our analysis, we found that when calculating the loss function using only the output from the last layer, the convergence speed of the decoder significantly decreases, especially when the number of unfolded iterations is higher. In this paper, we incorporate the output of all iterations into the loss function in the original neural BP structure. Additionally, we optimize the loss function by assigning different weights to losses at different iterations. As a result, the weighted loss function not only provides a lower Bit Error Rate (BER) compared to the original neural BP decoder at lower SNR, but also accelerates convergence speed.