Due to the model aging problem, Deep Neural Networks (DNNs) need updates to adjust them to new data distributions. The common practice leverages incremental learning (IL), e.g., Class-based Incremental Learning (CIL) that updates output labels, to update the model with new data and a limited number of old data. This avoids heavyweight training (from scratch) using conventional methods and saves storage space by reducing the number of old data to store. But it also leads to poor performance in fairness. In this paper, we show that CIL suffers both dataset and algorithm bias problems, and existing solutions can only partially solve the problem. We propose a novel framework, CILIATE, that fixes both dataset and algorithm bias in CIL. It features a novel differential analysis guided dataset and training refinement process that identifies unique and important samples overlooked by existing CIL and enforces the model to learn from them. Through this process, CILIATE improves the fairness of CIL by 17.03%, 22.46%, and 31.79% compared to state-of-the-art methods, iCaRL, BiC, and WA, respectively, based on our evaluation on three popular datasets and widely used ResNet models. Our code is available at https://github.com/Antimony5292/CILIATE.

Deep Neural Networks are well known to be vulnerable to adversarial attacks and backdoor attacks, where minor modifications on the input are able to mislead the models to give wrong results. Although defenses against adversarial attacks have been widely studied, investigation on mitigating backdoor attacks is still at an early stage. It is unknown whether there are any connections and common characteristics between the defenses against these two attacks. We conduct comprehensive studies on the connections between adversarial examples and backdoor examples of Deep Neural Networks to seek to answer the question: can we detect backdoor using adversarial detection methods. Our insights are based on the observation that both adversarial examples and backdoor examples have anomalies during the inference process, highly distinguishable from benign samples. As a result, we revise four existing adversarial defense methods for detecting backdoor examples. Extensive evaluations indicate that these approaches provide reliable protection against backdoor attacks, with a higher accuracy than detecting adversarial examples. These solutions also reveal the relations of adversarial examples, backdoor examples and normal samples in model sensitivity, activation space and feature space. This is able to enhance our understanding about the inherent features of these two attacks and the defense opportunities.

With Deep Neural Network (DNN) being integrated into a growing number of critical systems with far-reaching impacts on society, there are increasing concerns on their ethical performance, such as fairness. Unfortunately, model fairness and accuracy in many cases are contradictory goals to optimize during model training. To solve this issue, there has been a number of works trying to improve model fairness by formalizing an adversarial game in the model level. This approach introduces an adversary that evaluates the fairness of a model besides its prediction accuracy on the main task, and performs joint-optimization to achieve a balanced result. In this paper, we noticed that when performing backward propagation based training, such contradictory phenomenon are also observable on individual neuron level. Based on this observation, we propose FairNeuron, a DNN model automatic repairing tool, to mitigate fairness concerns and balance the accuracy-fairness trade-off without introducing another model. It works on detecting neurons with contradictory optimization directions from accuracy and fairness training goals, and achieving a trade-off by selective dropout. Comparing with state-of-the-art methods, our approach is lightweight, scaling to large models and more efficient. Our evaluation on three datasets shows that FairNeuron can effectively improve all models' fairness while maintaining a stable utility.

As in-the-wild data are increasingly involved in the training stage, machine learning applications become more susceptible to data poisoning attacks. Such attacks typically lead to test-time accuracy degradation or controlled misprediction. In this paper, we investigate the third type of exploitation of data poisoning - increasing the risks of privacy leakage of benign training samples. To this end, we demonstrate a set of data poisoning attacks to amplify the membership exposure of the targeted class. We first propose a generic dirty-label attack for supervised classification algorithms. We then propose an optimization-based clean-label attack in the transfer learning scenario, whereby the poisoning samples are correctly labeled and look "natural" to evade human moderation. We extensively evaluate our attacks on computer vision benchmarks. Our results show that the proposed attacks can substantially increase the membership inference precision with minimum overall test-time model performance degradation. To mitigate the potential negative impacts of our attacks, we also investigate feasible countermeasures.

Recently, deep neural networks have achieved state-of-the-art performance in multiple computer vision tasks, and become core parts of computer vision applications. In most of their implementations, a standard input preprocessing component called image scaling is embedded, in order to resize the original data to match the input size of pre-trained neural networks. This article demonstrates content disguising attacks by exploiting the image scaling procedure, which cause machine's extracted content to be dramatically dissimilar with that before scaled. Different from previous adversarial attacks, our attacks happen in the data preprocessing stage, and hence they are not subject to specific machine learning models. To achieve a better deceiving and disguising effect, we propose and implement three feasible attack approaches with L0- and L∞-norm distance metrics. We have conducted a comprehensive evaluation on various image classification applications, including three local demos and two remote proprietary services. We also investigate the attack effects on a YOLO-v3 object detection demo. Our experimental results demonstrate successful content disguising against all of them, which validate our approaches are practical.

Analyzing mouse-interaction behaviors for implicitly identifying computer users has received growing interest from security and biometric researchers. This study presents a simple but efficient active user authentication system by modeling mouse-interaction behavior, which is accurate and competent for future deployments. A pattern-growth-based mining method is proposed to extract frequent behavior segments, in obtaining a stable and discriminative representation of mouse-interaction behavior. Then procedural features are extracted to provide an accurate and fine-grained characterization of the behavior segments. A SVM-based decision procedure using one-class learning techniques is applied to the feature space for performing authentication. Analyses are conducted using data from around 1,526,400 mouse operations of 159 participants, and the authentication performance is evaluated across various application scenarios and tasks. Our experimental results show that characteristics from frequent behavior segments are more stable and discriminative than those from holistic behavior, and the system achieves a practically useful level of performance with FAR of 0.09 percent and FRR of 1 percent. Additional experiments on usability to sample length, reliability to application task, scalability to user size, robustness to mimic attack, and response to behavior change are provided to further explore the applicability. We also compare the proposed approach with the state-of-the-art approaches for the collected data.

The increasing usage of wearable devices for ambulatory monitoring and pervasive computing systems has given rise to the need of convenient and efficient activity recognition techniques. Hand-dominated activity recognition has great potential in understanding users' gesture and providing context-aware computing services. This paper investigates the feasibility and applicability on the usage of wristband-interaction behavior for recognizing hand-dominated activities, with the advantage of great compliance and long wearing time. For each action, sensor data from wristband are analyzed to obtain kinematic sequences. The sequences are then depicted by statistics-, frequency-, and wavelet-domain features for providing accurate and fine-grained characterization of hand-dominated actions, and the correlation between the wristband-sensor features and the actions is analyzed. Classification techniques (Naive Bayes, nearest neighbor, neural network, support vector machine, and Random Forest) are applied to the feature space for performing hand-dominated activity recognition. Analyses are conducted using the data from 51 participants with a diversity in gender, age, weight, and height. Extensive experiments demonstrate the efficacy of the proposed approach, achieving a recognition rate of 97.29% and an F-score above 0.94. Additional experiments on the effect of feature selection and wristband sampling rate are provided to further examine the effectiveness of our approach. Our data are publicly available.

The increasing use of smartphones as personal computing platforms to access personal information has stressed the demand for secure and usable authentication techniques, and for constantly protecting privacy. Smartphone sensors can measure users' unique behavioral characteristics when they interact with smartphones, based on different habits, gestures, and angle preferences of touch actions. This paper investigates the reliability and applicability of using motion-sensor behavior for active and continuous smartphone authentication across various operational scenarios, and presents a systematic evaluation of the distinctiveness and permanence properties of the behavior. For each sample of sensor behavior, kinematic information sequences are extracted and analyzed, which are characterized by statistic-, frequency-, and wavelet-domain features, to provide accurate and fine-grained characterization of users' touch actions. A Markov-based decision procedure, using one-class learning techniques, is developed and applied to the feature space for performing authentication. Analyses are conducted using the sensor data of 520 200 touch actions from 102 subjects across various operational scenarios. Extensive experiments show that motion-sensor behavior exhibits sufficient discriminability and stability for active and continuous authentication, and can achieve a false-rejection rate of 5.03% and a false-acceptance rate of 3.98%. Additional experiments on usability to operation length, sensitivity to application scenario, scalability to user size, contribution to different sensors, and response to behavior change are provided to further explore the effectiveness and applicability. We also implement an authentication system into the Android system that can react to the presence of the legitimate user.

The pervasiveness of wearable devices furnished with state-of-the-art sensors has shown the powerful capability in context-aware applications. However, embedded sensors also become targets for adversaries to launch potential side-channel attacks. In this paper, we present a self-adaptive and pretraining-independent pattern attack that infers a graphical password by recovering the victim's hand movement trajectory via motion sensors of a wrist-worn smart device. With the adaptive pattern inference algorithm, the discovered attack can be launched remotely without requiring previous training data from victims or the prior knowledge about the keyboard input settings. Toward the proposed attack, we create a method to detect the sliding behavior that draws a graphical password on the screen. We also propose an inference algorithm to generate password candidates from hand movement trajectories for different keypad input settings. We implement the discovered attack on a smartwatch and conduct experiments to evaluate the impact of this attack. The evaluation results show that for complex graphical patterns, with a single try, the attack can infer the passwords at a success rate as high as 80%, and the success rate can be further boosted to over 90% within five attempts, which reveals the overlooked privacy information threat caused by sensor data leakage.