Determining inconsistent software defect predictions in machine learning-based systems poses a significant challenge. To address this issue, we propose the utilization of Metamorphic Testing (MT) incorporating the “symmetry” metamorphic relation pattern (MRP) to transform the training datasets for training follow-up systems. In contrast, original datasets are employed to train source systems. By comparing the occurrence of inconsistent predictions between source and follow-up systems and analysing the efficacy of this approach, we aim to shed light on its effectiveness. Additionally, Explainable Artificial Intelligence (XAI) is employed to explain the inconsistencies observed. The results demonstrate that the “symmetry” MRP can induce inconsistent predictions, and XAI techniques can effectively elucidate such inconsistencies. Moreover, we find that the ordering of small-sized and imbalanced datasets can contribute to inconsistencies when using the KMeans, Random Forests or Convolutional Neural Network algorithm for software defect prediction systems. To further advance this research, future studies can extend the proposed approach by incorporating additional MRPs in domains that utilize machine learning algorithms to identify and explain inconsistencies. Another promising research avenue involves investigating the relationship between data imbalance, dataset size, and MRPs to enhance the identification of inconsistencies and derive more robust MRs.