Container reloading in international railway transport involves transferring containers to trains compatible with the destination’s railway gauge. The duration depends on factors like port facilities, staff proficiency, and customs clearance. Accurate forecasting is crucial for efficient planning and railway efficiency, including predicting train travel times and informing consignment customers about arrivals. However, current prediction methods cannot handle fluctuating nonlinear container reloading times and are too subjective in forest learner selection. Therefore, this paper proposes a novel structure-optimized deep learning model named the intelligent Bayesian deep forest (IBDF) model, which combines the deep forest and Bayesian optimization methods to handle complex datasets in predicting the railway port container reloading times. In this model, a combinatorial optimization algorithm incorporating an integer programming model and the Hungarian algorithm is first proposed to identify the optimal types of forest learners. Then, hyperparameter optimization technique is designed to obtain the optimal number of forest learners per type selected. Based on the container reloading time data from the Alataw Pass border station of China Railway Express (Chengdu-Europe) from 2017 to 2020, the performance of the IBDF model is systematically compared with ten benchmark models. This model averagely reduces the values of MSE, RMSE, MAE, and MAPE by 68.62%, 51.90%, 67.41%, and 59.52%, respectively. The prediction results show that the IBDF model provides high quality predictions, which can support the container reloading operations at the Alataw Pass border station, compressing the customs clearance time, and boosting the quality and speed of China Railway Express trains.