In this work, we conceive novel robust hybrid beamformer design schemes for millimeter-wave (mmWave) multi-cell multi-user (MCMU) systems in the presence of channel state information (CSI) uncertainty, that relies on base station (BS) coordination and minimization of total transmit power while ensuring compliance to practical signal-to-interference-noise ratio (SINR) constraints for each user. We consider a scenario where some of the users are located close to the cell boundary and thus desire to receive the signal of interest transmitted by multiple BSs while ensuring the quality-of-service (QoS) constraint. Initially, a Bayesian learning (BL) framework is developed for estimating the sparse mmWave channel of each user in the system. Next, a semidefinite relaxation (SDR) based technique has been proposed for a centralized MCMU system toward designing the fully digital beamformer (FDBF) in the presence of stochastic uncertainty in the estimated channel. Subsequently, a BL technique is employed to split the FDBF into its analog and digital constituents toward obtaining a hybrid transmit precoder (TPC). However, the centralized TPC design requires global CSI, resulting in a high signaling overhead. Next, a distributed coordinated hybrid TPC utilizing the alternating direction method of multipliers (ADMM) algorithm is developed for the mmWave MCMU system in the presence of cell-edge (CE) users. The distributed TPC design solely relies on CSI and only requires a limited exchange of information between the BSs, consequently eliminating the need for the high signaling overheads that come along with the centralized method. Our simulation results illustrate the superior performance of the proposed centralized and distributed robust TPC design methods in comparison to non-coordinated systems.