Explainable AI (XAI) techniques have been widely adopted to enhance the interpretability and reliability of deep learning applications. To extend that success to adversarial example detection, we propose a new framework to extract decision logic from explanations, leverage that information to summarize common critical neurons, and utilize their status to distinguish normal and adversarial examples. Our first approach uses decision logic for detection, demonstrating that differences in critical neuron distributions can be leveraged to distinguish normal and adversarial examples. We then propose a best-layer selection strategy to enhance previous layer-wise SHAP value detection. Selecting the layer with the most common critical neurons improves performance in terms of both accuracy and computational efficiency. These two approaches achieve high detection accuracy but require runtime computation of SHAP values. To avoid such runtime overhead, we further propose a new activation status detection approach where we show that using the activation status of common critical neurons offers lightweight yet effective detection. This efficacy extends to untrained attack detection. We conduct a comprehensive study on the CIFAR-10, MNIST, SVHN, CIFAR-100, Tiny ImageNet, and ImageNet datasets to evaluate the prediction accuracy, resource consumption, and transferability of the proposed approaches against several state-of-the-art adversarial attacks. The activation status approach achieves 81.89% accuracy with the optimized parameter set, demonstrating its effectiveness and efficiency in detecting adversarial examples in high-resolution data.

Cell states' complexity and heterogeneity pose significant challenges in uncovering biological patterns in high-dimensional single-cell data. To address this, we developed scGHSOM, an enhanced framework based on the Growing Hierarchical Self-Organizing Map (GHSOM), for hierarchical clustering and visualization of high-dimensional datasets such as Mass Cytometry by Time-Of-Flight (CyTOF) and single-cell RNA sequencing. scGHSOM organizes data hierarchically, expanding clusters to satisfy within- and between-cluster variation thresholds. We propose a novel Significant Attributes Identification algorithm within the scGHSOM framework to identify features that minimize intra-cluster variation while maximizing inter-cluster variation, enabling targeted data analysis. To enhance interpretability, scGHSOM introduces two visualization tools: the Cluster Feature Map, which highlights feature distributions across hierarchical clusters, and the Cluster Distribution Map, which visualizes leaf clusters as circles sized by data volume and filled with different colors to represent features such as cell types or other attributes. Performance evaluation on three CyTOF datasets demonstrates that scGHSOM is compatible with state-of-the-art methods. Specifically, it achieves the best CH index in two of the three datasets. Furthermore, the proposed visualization tools significantly improve clarity and efficiency in interpreting scGHSOM results, effectively revealing clustering patterns and features.

The deployment of deepfake face-swapping technology has matured, becoming widespread on the Internet. The misuse of this technology raises significant concerns for application security and privacy. To counter deepfake threats, we propose a sugar-coated poison defense targeting the latent vectors of generative models. This strategy aims to impact visual effects without substantially increasing reconstruction loss. We establish metrics for visual effects and reconstruction loss to assess perturbation effects on latent vectors, emphasizing those with the most significant impact on visual effects while minimizing reconstruction loss. Our approach begins by utilizing a facial feature extraction model to convert faces into latent representations. We then introduce two latent selection methods: 1) shap-based latent selection using a linear regression model for approximation, and 2) grid search latent selection employing heuristics of adversarial attacks. These methods pinpoint vectors that, when perturbed, can increase face landmark distances while maintaining low mean square errors, commonly used as the optimization metric in deepfake reconstruction models. We apply inconsistent perturbations to selected latent vectors in video frames, acting as sugar-coated poison for deepfake face-swapping applications. Preliminary results demonstrate that these perturbations can be applied to individual videos, resulting in low reconstruction loss. Importantly, they induce measurable consistency reduction in deepfake videos, making them more discernible and accessible to identify.

The predictive capabilities of machine learning models have improved significantly in recent years, leading to their widespread use in various fields. However, these models remain vulnerable to adversarial attacks, where carefully crafted inputs can mislead predictions and compromise the security of critical systems. Therefore, it is crucial to develop effective methods for detecting and preventing such attacks. Given that many neural network models are implemented using Python, this study addresses the issue of detecting adversarial examples from a new perspective by investigating information leakage in their Python model executions. To realize this objective, we propose a novel Python interpreter that utilizes Python bytecode instrumentation to profile layer-wise instruction-level program executions. We then search for information leakage on both legal and adversarial inputs, identifying their side-channel differences in call executions (i.e., call count, return values, and execution time) and synthesize the detection rule accordingly. Our approach is evaluated against TorchAttacks, AdvDoor, and RNN-Test attacks, targeting various models and applications. Our findings indicate that while there is call-return-value leakage on TorchAttacks images, there is no leakage to detect AdvDoor and RNN-Test attacks based on execution time or return values of string, integer, float, and Boolean type functions.

Social media allows any individual to disseminate information without third-party restrictions, making it difficult to verify the authenticity of a source. The proliferation of fake news has severely affected people’s intentions and behaviors in trusting online sources. Applying AI approaches for fake news detection on social media is the focus of recent research, most of which, however, focuses on enhancing AI performance. This study proposes XFlag, an innovative explainable AI (XAI) framework which uses long short-term memory (LSTM) model to identify fake news articles, layer-wise relevance propagation (LRP) algorithm to explain the fake news detection model based on LSTM, and situation awareness-based agent transparency (SAT) model to increase transparency in human-AI interaction. The developed XFlag framework has been empirically validated. The findings suggest the use of XFlag supports users in understanding system goals (perception), justifying system decisions (comprehension), and predicting system uncertainty (projection), with little cost of perceived cognitive workload.

Strings with length constraints are prominent in software security analysis. Recent endeavors have made significant progress in developing constraint solvers for strings and integers. Most prior methods are based on deduction with inference rules or analysis using automata. The former may be inefficient when the constraints involve complex string manipulations such as language replacement; the latter may not be easily extended to handle length constraints and may be inadequate for counterexample generation due to approximation. Inspired by recent work on string analysis with logic circuit representation, we propose a new method for solving string with length constraints by an implicit representation of automata with length encoding. The length-encoded automata are of infinite states and can represent languages beyond regular expressions. By converting string and length constraints into a dependency graph of manipulations over length-encoded automata, a symbolic model checker for infinite state systems can be leveraged as an engine for the analysis of string and length constraints. Experiments show that our method has its unique capability of handling complex string and length constraints not solvable by existing methods.

Recently, symbolic program analysis techniques have been extended to quantitative analyses using model counting constraint solvers. Given a constraint and a bound, a model counting constraint solver computes the number of solutions for the constraint within the bound. We present a parameterized model counting constraint solver for string and numeric constraints. We first construct a multi-track deterministic finite state automaton that accepts all solutions to the given constraint. We limit the numeric constraints to linear integer arithmetic, and for non-regular string constraints we over-approximate the solution set. Counting the number of accepting paths in the generated automaton solves the model counting problem. Our approach is parameterized in the sense that, we do not assume a finite domain size during automata construction, resulting in a potentially infinite set of solutions, and our model counting approach works for arbitrarily large bounds. We experimentally demonstrate the effectiveness of our approach on a large set of string and numeric constraints extracted from software applications. We experimentally compare our tool to five existing model counting constraint solvers for string and numeric constraints and demonstrate that our tool is as efficient and as or more precise than other solvers. Moreover, our tool can handle mixed constraints with string and integer variables that no other tool can.

We present a code- and input-sensitive sanitization synthesis approach for repairing string vulnerabilities that are common in web applications. The synthesized sanitization patch modifies the user input in an optimal way while guaranteeing that the repaired web application is not vulnerable. Given a web application, an input pattern and an attack pattern, we use automata-based static string analysis techniques to compute a sanitization signature that characterizes safe input values that obey the given input pattern and are safe with respect to the given attack pattern. Using the sanitization signature, we synthesize an optimal sanitization patch that converts malicious user inputs to benign ones with minimal editing. When the generated patch is added to the web application, it is guaranteed that the repaired web application is no longer vulnerable. We present refinements to previous sanitization synthesis algorithms that reduce the runtime sanitization cost significantly. We evaluate our approach on open source web applications using common input and attack patterns, demonstrating the effectiveness of our approach.

Detecting anomaly behavior in large network traffic data has presented a great challenge in designing effective intrusion detection systems. We propose an adaptive model to learn majority patterns under a dynamic changing environment. We first propose unsupervised learning on data abstraction to extract essential features of samples. We then adopt incremental majority learning with iterative evolutions on fitting envelopes to characterize the majority of samples within moving windows. A network traffic sample is considered an anomaly if its abstract feature falls on the outside of the fitting envelope. We justify the effectiveness of the presented approach against 150000+ traffic samples from the NSL-KDD dataset in training and testing, demonstrating positive promise in detecting network attacks by identifying samples that have abnormal features.

Anomalous patterns are observations that lie far away from the fitting function deduced from the bulk of the given observations. This work addresses the research issue to effectively identify anomalous patterns in both contexts of resistant learning, where there is no assumption about the fitting function form, and of changing environments. The resistant learning means that the learning procedure is not impacted significantly by the outlying observations. In literature, there is the resistant learning with searching a near-perfect fitting function for identifying the bulk of the majority of observations. However, the learning algorithm with searching a near-perfect fitting function suffers from time inefficiency. To effectively identify anomalous patterns in both contexts of resistant learning and changing environments, this study proposes a new resistant learning algorithm with envelope module that learns to evolve a nonlinear fitting function wrapped with a constant-width envelope for containing the majority of observations and thus identifying anomalous patterns. An illustrative experiment is set up to justify the effectiveness of the envelope module and the experimental result shows the positive promise.

We pioneer the study on applying both SOMs and GHSOMs to cluster mobile apps based on their behaviors, showing that the SOM family works well for clustering samples with more than ten thousands of attributes. The behaviors of apps are characterized by system method calls that are embedded in their executable, but may not be perceived by users. In the data preprocessing stage, we propose a novel static binary analysis to resolve and count implicit system method calls of iOS executable. Since an app can make thousands of system method calls, it is needed a large dimension of attributes to model their behaviors faithfully. On collecting 115 apps directly downloaded from Apple app store, the analysis result shows that each app sample is represented with 18000+ kinds of methods as their attributes. Theoretically, such a sample representation with more than ten thousand attributes raises a challenge to traditional clustering mechanisms. However, our experimental result shows that apps that have similar behaviors (due to having been developed from the same company or providing similar services) can be clustered together via both SOMs and GHSOMs.

We present a symbolic approach for checking consistency of OpenMP parallel programs. A parallel program is consistent if it yields the same result as its sequential version despite the execution order among threads. We find race conditions of an OpenMP parallel program, construct the formal model of its raced segments under relaxed memory models, and perform guided symbolic simulation to search consistency violations. The simulation terminates when (1) a witness has been found (the program is inconsistent), or (2) all reachable states have been explored (the program is consistent). We have developed the tool Pathg by incorporating Omega library to solve race constraints and Red symbolic simulator to perform guided search. We show that Pathg can prove consistency of programs, identify races that modern OpenMP checkers failed to report, and find inconsistency witnesses effectively against benchmarks from the OpenMP Source Code Repository and the NAS Parallel benchmark suite.

We present automata-based static string analysis techniques that automatically generate sanitization statements for patching vulnerable web applications. Our approach consists of three phases: Given an attack pattern we first conduct a vulnerability analysis to identify if strings that match the attack pattern can reach the security-sensitive functions. Next, we compute vulnerability signatures that characterize all input strings that can exploit the discovered vulnerability. Given the vulnerability signatures, we then construct sanitization statements that 1) check if a given input matches the vulnerability signature and 2) modify the input in a minimal way so that the modified input does not match the vulnerability signature. Our approach is capable of generating relational vulnerability signatures (and corresponding sanitization statements) for vulnerabilities that are due to more than one input.

We present a modular approach for verification of synchronization behavior in concurrent programs that use reentrant locks. Our approach decouples the verification of the lock implementation from the verification of the threads that use the lock. This decoupling is achieved using lock interfaces that characterize the allowable execution order for the lock operations. We use a thread modular verification approach to check that each thread obeys the lock interface. We verify the lock implementation assuming that the threads behave according to the lock interface. We demonstrate that this approach can be used to verify synchronization behavior in Java programs that use reentrant lock implementations for synchronization.

Given a program and an attack pattern (specified as a regular expression), we automatically generate string-based vulnerability signatures, i.e., a characterization that includes all malicious inputs that can be used to generate attacks. We use an automata-based string analysis framework. Using forward reachability analysis we compute an over-approximation of all possible values that string variables can take at each program point. Intersecting these with the attack pattern yields the potential attack strings if the program is vulnerable. Using backward analysis we compute an over-approximation of all possible inputs that can generate those attack strings. In addition to identifying existing vulnerabilities and their causes, these vulnerability signatures can be used to filter out malicious inputs. Our approach extends the prior work on automata-based string analysis by providing a backward symbolic analysis that includes a symbolic pre-image computation for deterministic finite automata on common string manipulating functions such as concatenation and replacement.

We propose a novel method for modular verification of web service compositions. We first use symbolic fixpoint computations to derive conditions on the incoming messages and relations among the incoming and outgoing messages of individual BPEL web services. These pre- and post-conditions are accumulated and serve as a repository of summarizations of individual web services. We then compose the summaries of the invoked BPEL services to model external invocations, resulting in a scalable verification approach for web service compositions. Our technical contributions include (1) an efficient symbolic encoding for modeling the concurrency semantics of systems having both multi-threading and message passing, and (2) a scalable method for summarizing concurrent processes that interact with each other using synchronous message passing, along with a modular framework that utilizes these summaries for scalable verification.

An essential tool in object oriented modeling is the specification of cardinalities of associations between classes. In Object Constraint Language (OCL) such constraints are expressed as conditions on the sizes of the collections that correspond to associations. In this paper we present tools and techniques for automated verification of size properties of collection types in OCL. We automatically verify invariants related to the sizes of the collections of a class with respect to the pre and post-conditions of the methods of that class. Our approach is based on a size abstraction that abstracts away the contents of the collections, but preserves the constraints on their sizes. We implemented a tool which automates this abstraction by converting OCL expressions on collections to arithmetic expressions on their sizes. Following this translation, we employ an infinite state model checker, called Action Language Verifier (ALV), for size analysis. Size abstraction reduces the state space of the system and, hence, the cost of automated verification, and by focusing on size properties, enables us to use efficient, domain specific model checking techniques for automated verification. To demonstrate the effectiveness of our approach we conducted a case study on the OCL specification of the Java Card API. The OCL specification of the Java Card API consists of 31 classes and 150 methods. Using our tool, we translated the OCL specification of each class to Action Language and verified the size properties using ALV. Verification with ALV took only a few seconds per class and we revealed errors in 26 out of the 150 method specifications.

Inevitability properties in branching temporal logics are of the syntax foralldiamphi, where phi is an arbitrary (timed) CTL (computation tree logic) formula. Such inevitability properties in dense-time logics can be analyzed with the greatest fixpoint calculation. We present algorithms to model-check inevitability properties. We discuss a technique for early decision on greatest fixpoint calculation which has shown promising performance against several benchmarks. We have experimented with various issues which may affect the performance of TCTL inevitability analysis. Specifically, our algorithms come with a parameter for the measurement of time-progress. We report the performance of our implementation with regard to various parameter values and with or without the non-Zeno computation requirement in the evaluation of greatest fixpoints. We have also experimented with safe abstraction techniques for model-checking TCTL inevitability properties. The experiment results help us in deducing rules for setting the parameter for verification performance. Finally, we summarize suggestions for configurations of efficient TCTL inevitability evaluation procedure

The authors describe the use of bounded model checking (BMC) for verifying Web application code. Vulnerable sections of code are patched automatically with runtime guards, allowing both verification and assurance to occur without user intervention. Model checking techniques are relatively complex compared to the typestate-based polynomial-time algorithm (TS) we adopted in an earlier paper, but they offer three benefits - they provide counterexamples, more precise models, and sound and complete verification. Compared to conventional model checking techniques, BMC offers a more practical approach to verifying programs containing large numbers of variables, but requires fixed program diameters to be complete. Formalizing Web application vulnerabilities as a secure information flow problem with fixed diameter allows for BMC application without drawback. Using BMC-produced counterexamples, errors that result from propagations of the same initial error can be reported as a single group rather than individually. This offers two distinct benefits. First, together with the counterexamples themselves, they allow for more descriptive and precise error reports. Second, it allows for automated patching at locations where errors are initially introduced rather than at locations where the propagated errors cause problems. Results from a TS-BMC comparison test using 230 open-source Web applications showed a 41.0% decrease in runtime instrumentations when BMC was used. In the 38 vulnerable projects identified by TS, BMC classified the TS-reported 980 individual errors into 578 groups, with each group requiring a minimal set of patches for repair.

Security remains a major roadblock to universal acceptance of the Web for many kinds of transactions, especially since the recent sharp increase in remotely exploitable vulnerabilities have been attributed to Web application bugs. Many verification tools are discovering previously unknown vulnerabilities in legacy C programs, raising hopes that the same success can be achieved with Web applications. In this paper, we describe a sound and holistic approach to ensuring Web application security. Viewing Web application vulnerabilities as a secure information flow problem, we created a lattice-based static analysis algorithm derived from type systems and typestate, and addressed its soundness. During the analysis, sections of code considered vulnerable are instrumented with runtime guards, thus securing Web applications in the absence of user intervention. With sufficient annotations, runtime overhead can be reduced to zero. We also created a tool named.WebSSARI (Web application Security by Static Analysis and Runtime Inspection) to test our algorithm, and used it to verify 230 open-source Web application projects on SourceForge.net, which were selected to represent projects of different maturity, popularity, and scale. 69 contained vulnerabilities. After notifying the developers, 38 acknowledged our findings and stated their plans to provide patches. Our statistics also show that static analysis reduced potential runtime overhead by 98.4%.

Dynamic data-structures with pointer links, which are heavily used in real-world software, cause extremely difficult verification problems. Currently, there is no practical framework for the efficient verification of such software systems. We investigated symmetry reduction techniques for the verification of software systems with C-like indirect reference chains like x/spl rarr/y/spl rarr/z/spl rarr/w. We formally defined the model of software with pointer data structures and developed symbolic algorithms to manipulate conditions and assignments with indirect reference chains using BDD technology. We relied on two techniques, inactive variable elimination and process-symmetry reduction in the data-structure configuration, to reduce time and memory complexity. We used binary permutation for efficiency, but we also identified the possibility of an anomaly of false image reachability. We implemented the techniques in tool Red 5.0 and compared performance with Mur/spl phi/ and SMC against several benchmarks.