Many sports video games contain elements such as running or throwing that are based on real-world physical activities, but the translation of these activities to game controllers means that the original physicality is lost. This results in games where players have limited opportunity to improve their physical skills, where there is little differentiation in people's physical abilities, and where skills do not change over the course of a game. To explore ways of adding these elements back into sports games, we developed two games with small-scale physical controls for running and throwing -- one game was a simple running race, and one was a team-based handball-style game called Jelly Polo. In two studies (three track-and-field tournaments for the running game, and a four-week league for Jelly Polo), we observed the effects of physical controls on gameplay. Our studies showed that the physical controls enabled substantial individual differences in running and passing skill, allowed people to increase their expertise over time, and led to fatigue-based changes in performance during a game. Physical controls increased the games' challenge, complexity, and unpredictability, and dramatically improved player interest, expressiveness, and enjoyment. Our work shows that game designers should consider the idea of "exertion in the small" as a way to improve play experience in games based on physical activities.