We extend the standard reinforcement learning framework to random time horizons. While the classical setting typically assumes finite and deterministic or infinite runtimes of trajectories, we argue that multiple real-world applications naturally exhibit random (potentially trajectory-dependent) stopping times. Since those stopping times typically depend on the policy, their randomness has an effect on policy gradient formulas, which we (mostly for the first time) derive rigorously in this work both for stochastic and deterministic policies. We present two complementary perspectives, trajectory or state-space based, and establish connections to optimal control theory. Our numerical experiments demonstrate that using the proposed formulas can significantly improve optimization convergence compared to traditional approaches.

Healthcare guided by semantic segmentation has the potential to improve our quality of life through early and accurate disease detection. Convolutional Neural Networks, especially the U-Net-based architectures, are currently the state-of-the-art learning-based segmentation methods and have given unprecedented performances. However, their decision-making processes are still an active field of research. In order to reliably utilize such methods in healthcare, explainability of how the segmentation was performed is mandated. To date, explainability is studied and applied heavily in classification tasks. In this work, we propose the Chimeric U-Net, a U-Net architecture with an invertible decoder unit, that inherently brings explainability into semantic segmentation tasks. We find that having the restriction of an invertible decoder does not hinder the performance of the segmentation task. However, the invertible decoder helps to disentangle the class information in the latent space embedding and to construct meaningful saliency maps. Furthermore, we found that with a simple k-Nearest-Neighbours classifier, we could predict the Intersection over Union scores of unseen data, demonstrating that the latent space, constructed by the Chimeric U-Net, encodes an interpretable representation of the segmentation quality. Explainability is an emerging field, and in this work, we propose an alternative approach, that is, rather than building tools for explaining a generic architecture, we propose constraints on the architecture which induce explainability. With this approach, we could peer into the architecture to reveal its class correlations and local contextual dependencies, taking an insightful step towards trustworthy and reliable AI. Code to build and utilize the Chimeric U-Net is made available under: https://github.com/kenrickschulze/Chimeric-UNet---Half-invertible-UNet-in-Pytorch

We consider autocovariance operators of a stationary stochastic process on a Polish space that is embedded into a reproducing kernel Hilbert space. We investigate how empirical estimates of these operators converge along realizations of the process under various conditions. In particular, we examine ergodic and strongly mixing processes and obtain several asymptotic results as well as finite sample error bounds. We provide applications of our theory in terms of consistency results for kernel PCA with dependent data and the conditional mean embedding of transition probabilities. Finally, we use our approach to examine the nonparametric estimation of Markov transition operators and highlight how our theory can give a consistency analysis for a large family of spectral analysis methods including kernel-based dynamic mode decomposition.

Deep convolutional neural networks (DCNNs) are routinely used for image segmentation of biomedical data sets to obtain quantitative measurements of cellular structures like tissues. These cellular structures often contain gaps in their boundaries, leading to poor segmentation performance when using DCNNs like the U-Net. The gaps can usually be corrected by post-hoc computer vision (CV) steps, which are specific to the data set and require a disproportionate amount of work. As DCNNs are Universal Function Approximators, it is conceivable that the corrections should be obsolete by selecting the appropriate architecture for the DCNN. In this article, we present a novel theoretical framework for the gap-filling problem in DCNNs that allows the selection of architecture to circumvent the CV steps. Combining information-theoretic measures of the data set with a fundamental property of DCNNs, the size of their receptive field, allows us to formulate statements about the solvability of the gap-filling problem independent of the specifics of model training. In particular, we obtain mathematical proof showing that the maximum proficiency of filling a gap by a DCNN is achieved if its receptive field is larger than the gap length. We then demonstrate the consequence of this result using numerical experiments on a synthetic and real data set and compare the gap-filling ability of the ubiquitous U-Net architecture with variable depths. Our code is available at https://github.com/ai-biology/dcnn-gap-filling.

BACKGROUND: High-throughput proteomics techniques, such as mass spectrometry (MS)-based approaches, produce very high-dimensional data-sets. In a clinical setting one is often interested in how mass spectra differ between patients of different classes, for example spectra from healthy patients vs. spectra from patients having a particular disease. Machine learning algorithms are needed to (a) identify these discriminating features and (b) classify unknown spectra based on this feature set. Since the acquired data is usually noisy, the algorithms should be robust against noise and outliers, while the identified feature set should be as small as possible. RESULTS: We present a new algorithm, Sparse Proteomics Analysis (SPA), based on the theory of compressed sensing that allows us to identify a minimal discriminating set of features from mass spectrometry data-sets. We show (1) how our method performs on artificial and real-world data-sets, (2) that its performance is competitive with standard (and widely used) algorithms for analyzing proteomics data, and (3) that it is robust against random and systematic noise. We further demonstrate the applicability of our algorithm to two previously published clinical data-sets.

Nucleoside analogs (NAs) are used to treat numerous viral infections and cancer. They compete with endogenous nucleotides (dNTP/NTP) for incorporation into nascent DNA/RNA and inhibit replication by preventing subsequent primer extension. To date, an integrated mathematical model that could allow the analysis of their mechanism of action, of the various resistance mechanisms, and their effect on viral fitness is still lacking. We present the first mechanistic mathematical model of polymerase inhibition by NAs that takes into account the reversibility of polymerase inhibition. Analytical solutions for the model point out the cellular- and kinetic aspects of inhibition. Our model correctly predicts for HIV-1 that resistance against nucleoside analog reverse transcriptase inhibitors (NRTIs) can be conferred by decreasing their incorporation rate, increasing their excision rate, or decreasing their affinity for the polymerase enzyme. For all analyzed NRTIs and their combinations, model-predicted macroscopic parameters (efficacy, fitness and toxicity) were consistent with observations. NRTI efficacy was found to greatly vary between distinct target cells. Surprisingly, target cells with low dNTP/NTP levels may not confer hyper-susceptibility to inhibition, whereas cells with high dNTP/NTP contents are likely to confer natural resistance. Our model also allows quantification of the selective advantage of mutations by integrating their effects on viral fitness and drug susceptibility. For zidovudine triphosphate (AZT-TP), we predict that this selective advantage, as well as the minimal concentration required to select thymidine-associated mutations (TAMs) are highly cell-dependent. The developed model allows studying various resistance mechanisms, inherent fitness effects, selection forces and epistasis based on microscopic kinetic data. It can readily be embedded in extended models of the complete HIV-1 reverse transcription process, or analogous processes in other viruses and help to guide drug development and improve our understanding of the mechanisms of resistance development during treatment.

Non-Gaussian component analysis (NGCA) introduced in offered a method for high-dimensional data analysis allowing for identifying a low-dimensional non-Gaussian component of the whole distribution in an iterative and structure adaptive way. An important step of the NGCA procedure is identification of the non-Gaussian subspace using principle component analysis (PCA) method. This article proposes a new approach to NGCA called sparse NGCA which replaces the PCA-based procedure with a new the algorithm we refer to as convex projection.