It is shown that a large class of nonstationary discrete parameter stochastic processes possess novel ergodic properties, which are referred to as {\em cycloergodic} properties. Specifically, it is shown that periodic components of time-varying probabilistic parameters can be consistently estimated from time averages on one sample path. The cycloergodic theory developed herein extends and generalizes existing ergodic theory for asymptotically mean stationary andN-stationary (cyclostationary) processes, and is presented in both wide-sense and strict-sense contexts.