The challenge to automatically select the best among models of varying dimensions remains open, especially in the context of complex models, sparse data, and noisy data. Bayesian neural networks employ Markov chain Monte Carlo (MCMC) and variational inference methods for training (sampling) model parameters. However, the progress of MCMC methods in deep learning has been slow due to high computational requirements and uninformative priors of model parameters. Reversible jump MCMC allows sampling of model parameters of variable lengths; hence, it has the potential to train Bayesian neural networks effectively. In this paper, we implement reversible jump MCMC for training dynamic Bayesian neural networks that feature cascaded neural networks with dynamic hidden and input neurons. We apply the methodology to a wide range of regression and classification problems from the literature. The results show that our proposed framework provides an effective approach for the dynamic exploration of models while featuring uncertainty quantification that not only caters to model parameters but also extends to model topology. This opens up the road for uncertainty quantification in dynamic neural networks where hidden and input neurons can change over time.

Variational Bayes (VB) has become a widely-used tool for Bayesian inference in statistics and machine learning. VB algorithms are generally restricted to the case where the variational parameter space is Euclidean. Recently, the VB framework has been extended to the case where the variational parameter space is a Riemannian manifold. Such a manifold-based algorithm, by utilizing the natural gradient, can exploit the geometric structures to derive an efficient VB procedure. However, natural gradient in high dimension can be prohibitively computationally expensive to compute. This work proposes using a quantum natural gradient within the manifold VB framework. We demonstrate that the quantum natural gradient does not suffer from the well-known quantum-classical readout computational bottleneck, and hence delivers a highly computational efficiency. The resulting quantum manifold VB algorithm is provably convergent.

Recently, container orchestration technology has been widely adopted by telco companies in network function virtualization (NFV) architecture. The communication between containerized virtual network functions (VNFs) is implemented by the networking models which contain switches and router functions in the host system. Different kinds of networking models which support packet acceleration features (OvS-DPDK, VPP, SR-IOV, eBPF, or combination), and different container deployment configurations (CPU isolation, NUMA alignment, service function chaining, inter-node networking technologies) have significant impacts on the performance of container network. This paper aims to accurately benchmark container network performance based on these aspects. Especially, we categorize and cover some missing models and configuration scenarios that were not presented in previous related works.

Bayesian inference using Markov Chain Monte Carlo (MCMC) on large datasets has developed rapidly in recent years. However, the underlying methods are generally limited to relatively simple settings where the data have specific forms of independence. We propose a novel technique for speeding up MCMC for time series data by efficient data subsampling in the frequency domain. For several challenging time series models, we demonstrate a speedup of up to two orders of magnitude while incurring negligible bias compared to MCMC on the full dataset. We also propose alternative control variates for variance reduction based on data grouping and coreset constructions.

Activation in deep neural networks is fundamental to achieving non-linear mappings. Traditional studies mainly focus on finding fixed activations for a particular set of learning tasks or model architectures. The research on flexible activation is quite limited in both designing philosophy and application scenarios. In this study, three principles of choosing flexible activation components are proposed and a general combined form of flexible activation functions is implemented. Based on this, a novel family of flexible activation functions that can replace sigmoid or tanh in LSTM cells are implemented, as well as a new family by combining ReLU and ELUs. Also, two new regularisation terms based on assumptions as prior knowledge are introduced. It has been shown that LSTM models with proposed flexible activations P-Sig- Ramp provide significant improvements in time series forecasting, while the proposed P-E2-ReLU achieves better and more stable performance on lossy image compression tasks with convolutional auto-encoders. In addition, the proposed regularization terms improve the convergence, performance and stability of the models with flexible activation functions. The code for this paper is available at https://github.com/9NXJRDDRQK/Flexible_Activation.

Hamiltonian Monte Carlo (HMC) samples efficiently from high-dimensional posterior distributions with proposed parameter draws obtained by iterating on a discretized version of the Hamiltonian dynamics. The iterations make HMC computationally costly, especially in problems with large data sets, since it is necessary to compute posterior densities and their derivatives with respect to the parameters. Naively computing the Hamiltonian dynamics on a subset of the data causes HMC to lose its key ability to generate distant parameter proposals with high acceptance probability. The key insight in our article is that efficient subsampling HMC for the parameters is possible if both the dynamics and the acceptance probability are computed from the same data subsample in each complete HMC iteration. We show that this is possible to do in a principled way in a HMC-within-Gibbs framework where the subsample is updated using a pseudo marginal MH step and the parameters are then updated using an HMC step, based on the current subsample. We show that our subsampling methods are fast and compare favorably to two popular sampling algorithms that use gradient estimates from data subsampling. We also explore the current limitations of subsampling HMC algorithms by varying the quality of the variance reducing control variates used in the estimators of the posterior density and its gradients.