The coronavirus pandemic has fostered an explosion of misinformation about the disease, including the risk and effectiveness of vaccination. AI tools for automatic Scientific Claim Verification (SCV) can be crucial to defeat misinformation campaigns spreading through social media channels. However, over the past years, many concerns have been raised about the robustness of AI to adversarial attacks, and the field of automatic SCV is not exempt. The risk is that such SCV tools may reinforce and legitimize the spread of fake scientific claims rather than refute them. This article investigates the problem of generating adversarial attacks for SCV tools and shows that it is far more difficult than the generic NLP adversarial attack problem. The current NLP adversarial attack generators, when applied to SCV, often generate modified claims with entirely different meaning from the original. Even when the meaning is preserved, the modification of the generated claim is too simplistic (only a single word is changed), leaving many weaknesses of the SCV tools undiscovered. We propose T5-ParEvo, an iterative evolutionary attack generator, that is able to generate more complex and creative attacks while better preserving the semantics of the original claim. Using detailed quantitative and qualitative analyses, we demonstrate the efficacy of T5-ParEvo in comparison with existing attack generators.

In an age of the growing use of Machine-learning, it has become an imperative task to be able to explain the processes behind the functions of many "black box" models. The explainability feature of artificial intelligence is key to building trust between humans and computers' algorithmic predictions. One of the main ways to generate this interpretability is through attribution methods, which produce importance values of each feature for a single instance in a dataset. There are many different ways of attribution for various Machine-learning models, including ones designed for specific models or "model agnostic" attribution methods—ones that do not require a specific model to achieve importance values. These attribution methods are valued because of their easily understood nature. While evaluation procedures exist such as generality and precision for rule-based explanation methods, these have not been used on attribution methods until recently. A recent experiment by Ratul et al. [1] proved that the two most popular local model-agnostic attribution methods, LIME and SHAP, have poor precision and generality. In this paper, we propose a new attribution method, the Generality and Precision Shapley Attributions (GAPS). To evaluate these models, we use the generality and precision equations used previously to evaluate the other models. We present our findings that GAPS produces higher generality and precision scores than the existing LIME and SHAP models.

Interpretability is a key feature to broaden a conscious adoption of machine learning models in domains involving safety, security, and fairness. To achieve the interpretability of complex machine learning models, one approach consists in explaining the outcome of machine learning models through input features attribution. Attribution consists in scoring the features of an input instance by establishing how important is each feature value in a fixed instance to obtain a specific classification outcome from the machine learning model. In literature, several attribution methods are defined for specific machine learning models (e.g., neural networks) or more general ones that are model agnostic (i.e., can interpret any machine learning models). Attribution is particularly appreciated for its easy understanding of the interpretation, which is the attribution. In domains involving safety, security, and fairness, properties of the explanation such as precision and generality are crucial to establish human trust in machine learning interpretability and then on the machine learning model itself. However, even if precision and generality are clearly defined in rule-based interpretation models, they are not defined or measure on attribution models. In this work, we propose a general methodology to estimate the degree of precision and generality in attribution methods. In addition, we propose a way to measured consistency in attribution between two attribution methods. Our experiments focus on the two most popular model agnostic attribution methods, SHAP and LIME, and we evaluate them to two real applications in the field of attack detection. Our proposed methodology shows in these experiments that both SHAP and LIME lack precision, generality, and consistency and that still more investigation in the attribution research field is required.

As online image sharing has become commonplace, researchers have acknowledged the need to assist users in detecting sensitive (or private) images. However, image privacy classification tasks have shown to be nontrivial, as the designation of an image sensitivity requires considerations of the visual concepts in the image. In this paper, we propose an innovative framework that combines the power of knowledge transfer for efficient, personalized learning of individuals’ privacy preferences toward images.Our approach defines a meta-model, which, given the query image and a small set of labeled images (used for the user-privacy customization), identifies if the query image is private for a target user. A generic user can efficiently customize this model by providing a small labeled training set. Moreover, our proposed framework includes transfer learning techniques to import basic patterns for image processing learned from other domains. Transfer learning enables fast and accurate processing of images, and allows few shot learning to focus on customization. This helps speed up the training process and avoid risk of overfitting. Our proposed framework significantly outperforms several baselines, including advanced object-oriented approaches and other CNN-based methods.