In supervised learning, understanding an input’s proximity to the training data can help a model decide whether it has sufficient evidence for reaching a reliable prediction. While powerful probabilistic models such as Gaussian Processes naturally have this property, deep neural networks often lack it. In this paper, we introduce Distance Aware Bottleneck (DAB), i.e., a new method for enriching deep neural networks with this property. Building on prior information bottleneck approaches, our method learns a codebook that stores a compressed representation of all inputs seen during training. The distance of a new example from this codebook can serve as an uncertainty estimate for the example. The resulting model is simple to train and provides deterministic uncertainty estimates by a single forward pass. Finally, our method achieves better out-of-distribution (OOD) detection and misclassification prediction than prior methods, including expensive ensemble methods, deep kernel Gaussian Processes, and approaches based on the standard information bottleneck.

Many real-world data represent sequences of interdependent events unfolding over time. They can be modeled naturally as realizations of a point process. Despite many potential applications, existing point process models are limited in their ability to capture complex patterns of interaction. Hawkes processes admit many efficient inference algorithms, but are limited to mutually excitatory effects. Non- linear Hawkes processes allow for more complex influence patterns, but for their estimation it is typically necessary to resort to discrete-time approximations that may yield poor generative models. In this paper, we introduce the first general class of Bayesian point process models extended with a nonlinear component that allows both excitatory and inhibitory relationships in continuous time. We derive a fully Bayesian inference algorithm for these processes using Polya-Gamma augmentation and Poisson thinning. We evaluate the proposed model on single and multi-neuronal spike train recordings. Results demonstrate that the proposed model, unlike existing point process models, can generate biologically-plausible spike trains, while still achieving competitive predictive likelihoods.

Probabilistic Boolean networks (PBNs) have previously been proposed so as to gain insights into complex dynamical systems. However, identification of large networks and their underlying discrete Markov chain which describes their temporal evolution still remains a challenge. In this paper, we introduce an equivalent representation for PBNs, the stochastic conjunctive normal form network (SCNFN), which enables a scalable learning algorithm and helps predict long-run dynamic behavior of large-scale systems. State-of-the-art methods turn out to be 400 times slower for middle-sized networks (i.e., containing 100 nodes) and incapable of terminating for large networks (i.e., containing 1000 nodes) compared to the SCNFN-based learning, when attempting to achieve comparable accuracy. In addition, in contrast to the currently used methods which introduce strict restrictions on the structure of the learned PBNs, the hypothesis space of our training paradigm is the set of all possible PBNs. Moreover, SCNFNs enable efficient sampling so as to statistically infer multistep transition probabilities which can provide information on the activity levels of individual nodes in the long run. Extensive experimental results showcase the scalability of the proposed approach both in terms of sample and runtime complexity. In addition, we provide examples to study large and complex cell signaling networks to show the potential of our model. Finally, we suggest several directions for future research on model variations, theoretical analysis, and potential applications of SCNFNs.

The inverse of the inductance matrix (reluctance matrix) is amenable to sparsification to a much greater extent than the inductance matrix itself. However, the inversion and subsequent truncation of a large dense inductance matrix to obtain the sparse inverse is very time-consuming, and previously proposed window-based techniques cannot provide adequate accuracy. In this paper we propose a method for selective inversion of the inductance matrix to a prescribed sparsity ratio, which is also amenable to parallelization on modern architectures. Experimental results demonstrate its potential to provide efficient and accurate approximation of the reluctance matrix for simulation of large-scale RLC circuits.