We propose a systematic analysis of deep neural networks (DNNs) based on a signal processing technique for network parameter removal, in the form of synaptic filters that identifies the fragility, robustness and antifragility characteristics of DNN parameters. Our proposed analysis investigates if the DNN performance is impacted negatively, invariantly, or positively on both clean and adversarially perturbed test datasets when the DNN undergoes synaptic filtering. We define three filtering scores for quantifying the fragility, robustness and antifragility characteristics of DNN parameters based on the performances for (i) clean dataset, (ii) adversarial dataset, and (iii) the difference in performances of clean and adversarial datasets. We validate the proposed systematic analysis on ResNet-18, ResNet-50, SqueezeNet-v1.1 and ShuffleNet V2 x1.0 network architectures for MNIST, CIFAR10 and Tiny ImageNet datasets. The filtering scores, for a given network architecture, identify network parameters that are invariant in characteristics across different datasets over learning epochs. Vice-versa, for a given dataset, the filtering scores identify the parameters that are invariant in characteristics across different network architectures. We show that our synaptic filtering method improves the test accuracy of ResNet and ShuffleNet models on adversarial dataset when only the robust and antifragile parameters are selectively retrained at any given epoch, thus demonstrating applications of the proposed strategy in improving model robustness.

(DL)We present a novel approach to rank Deep Learning hyper-parameters through the application of Sensitivity Analysis (SA). DL hyper-parameter tuning is crucial to model accuracy however, choosing optimal values for each parameter is time and resource-intensive. SA provides a quantitative measure by which hyper-parameters can be ranked in terms of contribution to model accuracy. Learning rate decay was ranked highest, with model performance being sensitive to this parameter regardless of architecture or dataset. The influence of a model’s initial learning rate was proven to be low, contrary to the literature. Additionally, the importance of a parameter is closely linked to model architecture. Shallower models showed susceptibility to hyper-parameters affecting the stochasticity of the learning process whereas deeper models showed sensitivity to hyper-parameters affecting the convergence speed. Furthermore, the complexity of the dataset can affect the margin of separation between the sensitivity measures of the most and the least influential parameters, making the most influential hyper-parameter an ideal candidate for tuning compared to the other parameters.

In this work, we define cooperative games on simplicial complexes, generalizing the study of probabilistic values of Weber (1988) and quasi-probabilistic values of Bilbao et al. (2001). Applications to Multi-Touch Attribution and the interpretability of the Machine-Learning prediction models motivate these new developments (Lundberg and Lee, 2017; Ribeiro et al., 2016; Strumbelj and Kononenko, 2013; Shrikumar et al., 2017; Datta et al., 2016; Bach et al., 2015). We deal with the axiomatization provided by the λi-dummy and the monotonicity requirements together with a probabilistic form of the symmetric and the efficiency axioms. We also characterize combinatorially the set of probabilistic participation influences as the facet polytope of the simplicial complex.