Algorithmic differentiation (AD) is a set of techniques that provide partial derivatives of computer-implemented functions. Such functions can be supplied to state-of-the-art AD tools via their source code , or via intermediate representations produced while compiling their source code. We present the novel AD tool Derivgrind, which augments the machine code of compiled programs with forward-mode AD logic. Derivgrind leverages the Valgrind instrumentation framework for structured access to the machine code, and a shadow memory tool to store dot values. Access to the source code is required at most for the files in which input and output variables are defined. Derivgrind’s versatility mainly comes at the price of reduced run-time performance. According to our extensive regression test suite, Derivgrind produces correct results on GCC- and Clang-compiled programs, including a Python interpreter, with a small number of exceptions. We provide a list of “bit-tricks” that Derivgrind does not handle correctly, some of which actually appear in highly optimized math libraries. As long as differentiating those is avoided, Derivgrind enables black-box forward-mode AD for an unprecedentedly wide range of cross-language software with little integration efforts.

We present the new software OpDiLib, a universal add-on for classical operator overloading AD tools that enables the automatic differentiation (AD) of OpenMP parallelized code. With it, we establish support for OpenMP features in a reverse mode operator overloading AD tool to an extent that was previously only reported on in source transformation tools. We achieve this with an event-based implementation ansatz that is unprecedented in AD. Combined with modern OpenMP features around OMPT, we demonstrate how it can be used to achieve differentiation without any additional modifications of the source code; neither do we impose a priori restrictions on the data access patterns, which makes OpDiLib highly applicable. For further performance optimizations, restrictions like atomic updates on adjoint variables can be lifted in a fine-grained manner. OpDiLib can also be applied in a semi-automatic fashion via a macro interface, which supports compilers that do not implement OMPT. We demonstrate the applicability of OpDiLib for a pure operator overloading approach in a hybrid parallel environment. We quantify the cost of atomic updates on adjoint variables and showcase the speedup and scaling that can be achieved with the different configurations of OpDiLib in both the forward and the reverse pass.

There are several AD tools available that all implement different strategies for the reverse mode of AD. The most common strategies are primal value taping (implemented e.g. by ADOL-C) and Jacobian taping (implemented e.g. by Adept and dco/c++). Particulary for Jacobian taping, recent advances using expression templates make it very attractive for large scale software. However, the current implementations are either closed source or miss essential features and flexibility. Therefore, we present the new AD tool CoDiPack (Code Differentiation Package) in this paper. It is specifically designed for minimal memory consumption and optimal runtime, such that it can be used for the differentiation of large scale software. An essential part of the design of CoDiPack is the modular layout and the recursive data structures which not only allow the efficient implementation of the Jacobian taping approach but will also enable other approaches like the primal value taping or new research ideas. We will finally present the performance values of CoDiPack on a generic PDE example and on the SU2 code.

Compared to invasive Brain-Computer Interfaces (BCI), non-invasive BCI systems based on Electroencephalogram (EEG) signals have not been applied successfully for complex control tasks. In the present study, however, we demonstrate this is possible and report on the interaction of a human subject with a complex real device: a pinball machine. First results in this single subject study clearly show that fast and well-timed control well beyond chance level is possible, even though the environment is extremely rich and requires complex predictive behavior. Using machine learning methods for mental state decoding, BCI-based pinball control is possible within the first session without the necessity to employ lengthy subject training. While the current study is still of anecdotal nature, it clearly shows that very compelling control with excellent timing and dynamics is possible for a non-invasive BCI.