This paper presents new learning-based, energy-delay-aware power control strategies for the uplink of dynamic cell-free (CF) massive multiple-input multiple-output (MIMO) networks. We first propose a new algorithm to adaptively select an appropriate set of multi-antenna access points (APs) to serve each user at any time instant. A multi-level hidden Markov model (HMM) is constructed to predict the trajectory of a user and forecast the changing set of serving APs for the user over time. Given the selected, time-changing sets of serving APs, we also formulate a new multi-objective power control problem to minimize a weighted sum of the energy and delay costs of the users. By employing Bernoulli bandit learning (BBL) and Gaussian bandit learning (GBL), two new energy-delay-aware power control strategies are developed to adapt unpredictable channel dynamics, minimize the energy-delay costs online, and remain effective over a long time even when sharp changes occur in the channels. Numerical results demonstrate that the new predictive selection of serving APs and the new learning-based energy-delay-aware power control policies can significantly improve the reliability of uplink transmissions in dynamic CF massive MIMO networks, as compared to state-of-the-art static oracle solutions.