We describe a recursive data structure for the uniform handling of permutation groups and matrix groups. This data structure allows the switching between permutation and matrix representations of segments of the input group, and has wide-ranging applications. It provides a framework to process theoretical algorithms which were considered too complicated for implementation such as the asymptotically fastest algorithms for the basic handling of large-base permutation groups and for Sylow subgroup computations in arbitrary permutation groups. It also facilitates the basic handling of matrix groups. The data structure is general enough for the easy incorporation of any matrix group or permutation group algorithm code; in particular, the library functions of the GAP computer algebra system dealing with permutation groups and matrix groups work with a minimal modification.