Testing of deep learning models is challenging due to the excessive number and complexity of the computations involved. As a result, test data selection is performed manually and in an ad hoc way. This raises the question of how we can automatically select candidate data to test deep learning models. Recent research has focused on defining metrics to measure the thoroughness of a test suite and to rely on such metrics to guide the generation of new tests. However, the problem of selecting/prioritising test inputs (e.g., to be labelled manually by humans) remains open. In this article, we perform an in-depth empirical comparison of a set of test selection metrics based on the notion of model uncertainty (model confidence on specific inputs). Intuitively, the more uncertain we are about a candidate sample, the more likely it is that this sample triggers a misclassification. Similarly, we hypothesise that the samples for which we are the most uncertain are the most informative and should be used in priority to improve the model by retraining. We evaluate these metrics on five models and three widely used image classification problems involving real and artificial (adversarial) data produced by five generation algorithms. We show that uncertainty-based metrics have a strong ability to identify misclassified inputs, being three times stronger than surprise adequacy and outperforming coverage-related metrics. We also show that these metrics lead to faster improvement in classification accuracy during retraining: up to two times faster than random selection and other state-of-the-art metrics on all models we considered.

The efficiency of cache-placement algorithms in edge-caching networks depends on the accuracy of the content request’s statistical model and the estimation method based on the postulated model. This paper studies these two important issues. First, we introduce a new model for content requests in stationary environments. The common approach to model the requests is through the Poisson stochastic process. However, the Poisson stochastic process is not a very flexible model since it cannot capture the correlations between contents. To resolve this limitation, we instead introduce the Poisson Factor Analysis (PFA) model for this purpose. In PFA, the correlations are modeled through additional random variables embedded in a low dimensional latent space. The correlations provide rich information about the underlying statistical properties of content requests which can be used for advanced cache-placement algorithms. Secondly, to learn the model, we use Bayesian Learning, an efficient framework which does not overfit. This is crucial in edge-caching systems since only partial view of the entire request set is available at the local cache and the learning method should be able to estimate the content popularities without overfitting. In the simulation results, we compare the performance of our approach with the existing popularity estimation method.

Edge-caching is recognized as an efficient technique for future wireless cellular networks to improve network capacity and user-perceived quality of experience. Due to the random content requests and the limited cache memory, designing an efficient caching policy is a challenge. To enhance the performance of caching systems, an accurate content request prediction algorithm is essential. Here, we introduce a flexible model, a Poisson regressor based on a Gaussian process, for the content request distribution in stationary environments. Our proposed model can incorporate the content features as side information for prediction enhancement. In order to learn the model parameters, which yield the Poisson rates or alternatively content popularities, we invoke the Bayesian approach which is very robust against over-fitting. However, the posterior distribution in the Bayes formula is analytically intractable to compute. To tackle this issue, we apply a Monte Carlo Markov Chain (MCMC) method to approximate the posterior distribution. Two types of predictive distributions are formulated for the requests of existing contents and for the requests of a newly-added content. Finally, simulation results are provided to confirm the accuracy of the developed content popularity learning approach.

In this paper, an on-line interactive method is proposed for learning a linear classifier. This problem is studied within the Active Learning (AL) framework where the learning algorithm sequentially chooses unlabelled training samples and requests their class labels from an oracle in order to learn the classifier with the least queries to the oracle possible. Additionally' a constraint is introduced into this interactive learning process which limits the percentage of the samples from one “unwanted” class under a certain threshold. An optimal AL solution is derived and implemented with a sophisticated, accurate and fast Bayesian Learning method, the Expectation Propagation (EP) and its performance is demonstrated through numerical simulations.

In this paper, an intelligent probing method for interference constraint learning is proposed to allow a centralized cognitive radio network (CRN) to access the frequency band of a primary user (PU) in an underlay cognitive communication scenario. The main idea is that the CRN probes the PU and subsequently eavesdrops the reverse PU link to acquire the binary ACK/NACK packet. This feedback is implicit channel state information of the PU link, indicating whether the probing-induced interference is harmful or not. The intelligence of this sequential probing process lies in the selection of the power levels of the secondary users, which aims to minimize the number of probing attempts, a clearly active learning (AL) procedure, and expectantly the overall PU QoS degradation. The enhancement introduced in this paper is that we incorporate the probability of each feedback being correct into this intelligent probing mechanism by using a multivariate Bayesian AL method. This technique is inspired by the probabilistic bisection algorithm and the deterministic cutting plane methods (CPMs). The optimality of this multivariate Bayesian AL method is proven and its effectiveness is demonstrated through numerical simulations. Computationally cheap CPM adaptations are also presented, which outperform existing AL methods.

In this paper, an intelligent probing method for interference constraint learning is proposed to allow a centralized Cognitive Radio Network (CRN) to access the frequency band of a Primary User (PU) operating based on an Adaptive Coding and Modulation (ACM) protocol. The main idea is that the CRN probes the PU and subsequently applies a Modulation and Coding Classification (MCC) technique to acquire the Modulation and Coding scheme (MCS) of the PU. This feedback is an implicit channel state information (CSI) of the PU link, indicating how harmful the probing induced interference is. The intelligence of this sequential probing process lies on the selection of the power levels of the Secondary Users (SUs) which aims to minimize the number of probing attempts, a clearly Active Learning (AL) procedure, and consequently the overall PU QoS degradation. The enhancement introduced in this work is that we incorporate the probability of each feedback being correct into this intelligent probing mechanism by using a univariate Bayesian Nonparamet-ric AL method, the Probabilistic Bisection Algorithm (PBA). An adaptation of the PBA is implemented for higher dimensions and its effectiveness as an uncertainty driven AL method is demonstrated through numerical simulations.