This paper explores the evolution of wireless communication networks from utilizing the sub-6 GHz spectrum and the millimeter wave frequency band to incorporating extremely high frequencies like optical and terahertz for 6G and beyond. While these higher frequencies offer broader bandwidths and extreme data rate capabilities, the transition from single-band and heterogeneous networks to multi-band networks (MBNs), where various frequency bands coexist introduces novel challenges in channel modeling, transceiver and antenna design, programmable simulation platforms, standardization, and resource allocation. This paper provides a tutorial overview from the communication design perspective of the various frequency bands, elaborating on the above issues. Then, we introduce and examine typical MBN architectures for future networks and provide a detailed overview of state-of-the-art resource allocation problems for existing MBNs that typically operate on two frequency bands. The considered resource allocation optimization problems and solution techniques are discussed comprehensively. We then identify key performance metrics and constraint sets that should be considered for resource allocation optimization in future MBNs and provide numerical results to depict how various system parameters and user behaviors can influence their performance. Finally, we present several potential research issues as future work for the design and performance optimization of MBNs.