Robots in the real world experience wear and tear, leading to changing system dynamics. This challenge is particularly exacerbated for non-rigid systems such as soft robots or robotic systems made of metamaterials with hysteresis. This setting results in a challenging problem for most learning-based controllers that typically rely on the assumption that the system dynamics remain fixed over time. In the absence of explicit mechanisms to account for this change in dynamics, learning-based control algorithms show considerable degradation in performance over time. In this work, we consider a particular class of dynamics shift in under-actuated systems, that is localized to the dynamics of the fully actuated robot itself, while independently leaving the dynamics of the environment unchanged. This captures real-world phenomena such as fatigue or hysteresis in robotic systems. In this setting, we propose an efficient algorithm that can account for dynamics shift. Using a simple calibration procedure, we propose a technique for learning a non-linear “action-translation” model that can capture the localized shift in dynamics. This enables continual learning and transfer despite considerable dynamics shift during the learning process. We demonstrate the efficacy of this procedure on several tasks in simulation, as well as a real-world robotic system - a 4 DoF electrically driven handed shearing auxetic (HSA) platform.