We propose a novel penalty force to enforce contacts with accurate Coulomb friction. The force is compatible with fully-implicit time integration and the use of optimization-based integration. The contact force is quite general. In addition to processing collisions between deformable objects, the force can be used to couple rigid bodies to deformable objects or the material point method. The force naturally leads to stable stacking without drift over time, even when solvers are not run to convergence. The force leads to an asymmetrical system, and we provide a practical solution for handling these.