Efficiently solving nonlinear equations underpins numerous scientific and engineering disciplines, yet scaling these solutions for challenging system models remains a challenge. This article presents NonlinearSolve.jl —a suite of high-performance open source nonlinear equation solvers implemented natively in the Julia programming language. NonlinearSolve.jl distinguishes itself by offering a unified API that accommodates a diverse range of solver specifications alongside features such as automatic algorithm selection based on runtime analysis, support for static array kernels for improved GPU computation on smaller problems, and the utilization of sparse automatic differentiation and Jacobian-free Krylov methods for large-scale problem-solving. Through rigorous comparison with established tools such as PETSc SNES , Sundials KINSOL , and MINPACK , NonlinearSolve.jl demonstrates robustness and efficiency, achieving significant advancements in solving nonlinear equations while being implemented in a high-level programming language. The capabilities of NonlinearSolve.jl unlock new potentials in modeling and simulation across various domains, making it a valuable addition to the computational toolkit of researchers and practitioners alike.

Automatic differentiation (AD), a technique for constructing new programs which compute the derivative of an original program, has become ubiquitous throughout scientific computing and deep learning due to the improved performance afforded by gradient-based optimization. However, AD systems have been restricted to the subset of programs that have a continuous dependence on parameters. Programs that have discrete stochastic behaviors governed by distribution parameters, such as flipping a coin with probability $p$ of being heads, pose a challenge to these systems because the connection between the result (heads vs tails) and the parameters ($p$) is fundamentally discrete. In this paper we develop a new reparameterization-based methodology that allows for generating programs whose expectation is the derivative of the expectation of the original program. We showcase how this method gives an unbiased and low-variance estimator which is as automated as traditional AD mechanisms. We demonstrate unbiased forward-mode AD of discrete-time Markov chains, agent-based models such as Conway's Game of Life, and unbiased reverse-mode AD of a particle filter. Our code package is available at https://github.com/gaurav-arya/StochasticAD.jl.

To achieve high performance for MIMO wireless systems with multiple transmit and receive antennas a detailed understanding of the characteristics of the eigenspace of the channel is essential. It represents the spatial characteristics of the propagation scenario in general and can be expressed by different covariance matrices of the channel coefficients. In many MIMO-algorithms the knowledge of the eigenvectors and eigenvalues is required. As typical for the wireless channel, the spatial characteristics continuously change. Therefore, an efficient method for tracking the eigenspace with moderate computational complexity is required. Tracking of the eigenspace using incremental Jacobi rotations is presented and compared to other known algorithms for subspace tracking. Eigenbeamforming serves as an example, where such a tracking algorithm can be applied.

Due to the permantly increasing computing power eigenvalue decompositions in real time embedded systems become more and more feasible. Applying a series of orthogonal similarity transforms, so called Jacobi rotations, provides one approach of solving the symmetric eigenvalue problem. Jacobi rotations are suitable for hardware implementation because of their regularity and their inherent parallelism. However, the calculation of the rotation parameters requires numerical operations which increase the hardware effort and processing time dramatically. We propose a new algorithm with considerably less hardware requirements but still having good numerically properties. Our new algorithm only needs one division with reduced precision and a small look-up table. Square roots, further divisions and other operations all with full precision are now unnecessary. Especially in highly parallel systems like systolic arrays a considerable amount of calculation time can be saved while decreasing the hardware effort.