In this paper, we explain how the abstract notion of a differential bundle in a tangent category provides a new way of thinking about the category of modules over a commutative ring and its opposite category. MacAdam previously showed that differential bundles in the tangent category of smooth manifolds are precisely smooth vector bundles. Here we provide characterizations of differential bundles in the tangent categories of commutative rings and (affine) schemes. For commutative rings, the category of differential bundles over a commutative ring is equivalent to the category of modules over that ring. For affine schemes, the category of differential bundles over the Spec of a commutative ring is equivalent to the opposite category of modules over said ring. Finally, for schemes, the category of differential bundles over a scheme is equivalent to the opposite category of quasi-coherent sheaves of modules over that scheme.

Abstract We propose a categorical semantics of gradient-based machine learning algorithms in terms of lenses, parametric maps, and reverse derivative categories. This foundation provides a powerful explanatory and unifying framework: it encompasses a variety of gradient descent algorithms such as ADAM, AdaGrad, and Nesterov momentum, as well as a variety of loss functions such as MSE and Softmax cross-entropy, shedding new light on their similarities and differences. Our approach to gradient-based learning has examples generalising beyond the familiar continuous domains (modelled in categories of smooth maps) and can be realized in the discrete setting of boolean circuits. Finally, we demonstrate the practical significance of our framework with an implementation in Python.

Abstract Cartesian reverse differential categories (CRDCs) are a recently defined structure which categorically model the reverse differentiation operations used in supervised learning. Here, we define a related structure called a monoidal reverse differential category, prove important results about its relationship to CRDCs, and provide examples of both structures, including examples coming from models of quantum computation.

The reverse derivative is a fundamental operation in machine learning and automatic differentiation. This paper gives a direct axiomatization of a category with a reverse derivative operation, in a similar style to that given by Cartesian differential categories for a forward derivative. Intriguingly, a category with a reverse derivative also has a forward derivative, but the converse is not true. In fact, we show explicitly what a forward derivative is missing: a reverse derivative is equivalent to a forward derivative with a dagger structure on its subcategory of linear maps. Furthermore, we show that these linear maps form an additively enriched category with dagger biproducts.

We revisit the definition of Cartesian differential categories, showing that a slightly more general version is useful for a number of reasons. As one application, we show that these general differential categories are comonadic over categories with finite products, so that every category with finite products has an associated cofree differential category. We also work out the corresponding results when the categories involved have restriction structure, and show that these categories are closed under splitting restriction idempotents.