Representation learning of road networks is essential for various downstream traffic-related tasks, as road network contain multi-modal data with rich information, and the learned embeddings can be directly used in machine learning models. However, due to the dynamic changes in road networks with respect to topology and associated data, as well as the local and long-range dependency caused by complex mobility semantics, learning robust and effective representations remains challenging. To this end, we exploit the properties of the road network and the mobility semantics embedded in trajectories, and propose a novel S emantic- E nhanced G raph C ontrastive L earning (SE-GCL) framework, for learning general-purpose embeddings of road networks. Specifically, in this framework, we propose (1) a multi-modal feature embedding module to capture both the attribute and visual information of road segments, (2) a semantic-enhanced graph augmentation strategy to simulate topological changes and data missing in the road network, and (3) a semantic-enhanced contrastive optimization module that leverages geo-locality and mobility semantics to guide representation learning. Extensive experiments are conducted on two real-world road networks with three representative downstream tasks. The result demonstrate that SE-GCL yields more robust and effective representations, outperforming the state-of-the-art baselines. The source code is available at https://github.com/csjiezhao/SE-GCL .