The increasing complexity of software systems has made them more prone to bugs, prompting the development of automated bug localization techniques to ensure software reliability. Despite these techniques having demonstrated notable success at the file level, their application and optimization at the function level often encounter serious performance cliffs. This limitation underscores the urgent need for a dedicated framework for function-level bug localization, which we address through FunLoc, a novel framework that takes coarse-grained source files as input units and identifies fine-grained buggy functions as output. To address the critical challenges of handling domain-specific bug reports and managing vast function-level sample space, we introduce two key innovations that are seamlessly integrated into FunLoc. First, we design a contrastive learning-based domain-adaptive language model to enhance the framework's ability to process and interpret specialized bug reports effectively. Second, we propose an active learning-based dynamic negative sampling strategy to address the scalability issues arising from the extensive function-level sample space. To evaluate the effectiveness of our approach, we extend and release a function-level bug localization dataset derived from large-scale real-world projects. Extensive experiments demonstrate that our approach outperforms state-of-the-art techniques.

BACKGROUND: Named entity recognition is a fundamental task in natural language processing. Recognizing entities in biomedical text, known as the BioNER, is particularly crucial for cutting-edge applications. However, BioNER poses greater challenges compared to traditional NER due to (1) nested structures and (2) category correlations inherent in biomedical entities. Recently, various BioNER models have been developed based on region classification or large language models. Despite being successful, these models still struggle to balance handling nested structures and capturing category knowledge. RESULTS: We present a novel parallel BioNER model, BEAN, designed to address the unique properties of biomedical entities while achieving a reasonable balance between handling nested structures and incorporating category correlations. Extensive experiments on five public NER datasets, including four biomedical datasets, demonstrate that BEAN achieves state-of-the-art performance. CONCLUSIONS: The proposed BEAN is elaborately designed to achieve two key objectives of the BioNER task: clearly detecting entity boundaries and correctly classifying entity categories. It is the first BioNER model to handle nested structures and category correlations in parallel. We exploit head, tail, and contextualized features to efficiently detect entity boundaries via a triaffine model. To the best of our knowledge, we are the first to introduce a multi-label classification model for the BioNER task to extract entity category information without boundary guidance.

Abstract Contrastive learning has emerged as an essential approach in self-supervised visual representation learning. Its main goal is to maximize the similarities between augmented versions of the same image (positive pairs), while minimizing the similarities between different images (negative pairs). Recent studies have demonstrated that harder negative samples, i.e., those that are more challenging to differentiate from the anchor sample perform a more crucial function in contrastive learning. However, many existing contrastive learning methods ignore the role of hard negative samples. In order to provide harder negative samples for the network model more efficiently. This paper proposes a novel feature-level sample sampling method, namely sampling synthetic hard negative samples for contrastive learning (SSCL). Specifically, we generate more and harder negative samples by mixing them through linear combination and ensure their reliability by debiasing. Finally, we execute weighted sampling of these negative samples. Compared to state-of-the-art methods, our method can provide more high-quality negative samples. Experiments show that SSCL improves the classification performance on different image datasets and can be readily integrated into existing methods.

Machine Learning (ML) techniques can facilitate the automation ofmalicious software(malware for short) detection, but suffer from evasion attacks. Many studies counter such attacks in heuristic manners, lacking theoretical guarantees and defense effectiveness. In this article, we propose a new adversarial training framework, termedPrincipledAdversarial MalwareDetection (PAD), which offers convergence guarantees for robust optimization methods. PAD lays on a learnable convex measurement that quantifies distribution-wise discrete perturbations to protect malware detectors from adversaries, whereby for smooth detectors, adversarial training can be performed with theoretical treatments. To promote defense effectiveness, we propose a new mixture of attacks to instantiate PAD to enhance deep neural network-based measurements and malware detectors. Experimental results on two Android malware datasets demonstrate: (i) the proposed method significantly outperforms the state-of-the-art defenses; (ii) it can harden ML-based malware detection against 27 evasion attacks with detection accuracies greater than 83.45%, at the price of suffering an accuracy decrease smaller than 2.16% in the absence of attacks; (iii) it matches or outperforms many anti-malware scanners in VirusTotal against realistic adversarial malware.

Deep neural networks have achieved remarkable results across various tasks. However, they are susceptible to adversarial examples, which are generated by adding adversarial perturbations to original data. Adversarial training (AT) is the most effective defense mechanism against adversarial examples and has received significant attention. Recent studies highlight the importance of example exploitation, where the model’s learning intensity is altered for specific examples to extend classic AT approaches. However, the analysis methodologies employed by these studies are varied and contradictory, which may lead to confusion in future research. To address this issue, we provide a comprehensive summary of representative strategies focusing on exploiting examples within a unified framework. Furthermore, we investigate the role of examples in AT and find that examples which contribute primarily to accuracy or robustness are distinct. Based on this finding, we propose a novel example-exploitation idea that can further improve the performance of advanced AT methods. This new idea suggests that critical challenges in AT, such as the accuracy-robustness trade-off, robust overfitting, and catastrophic overfitting, can be alleviated simultaneously from an example-exploitation perspective. The code can be found in https://github.com/geyao1995/advancing-example-exploitation-in-adversarial-training.

Virtualization technologies provide solutions for cloud computing. Virtual resource scheduling is a crucial task in data centers, and the power consumption of virtual resources is a critical foundation of virtualization scheduling. Containers are the smallest unit of virtual resource scheduling and migration. Although many practical models for estimating the power consumption of virtual machines (VMs) have been proposed, few power estimation models of containers have been put forth. In this paper, we propose a fast-training piecewise regression model based on a decision tree for VM power metering and estimate the power of containers configured on the VM by treating the container as a group of processes on the VM. We select appropriate features from the collected metrics of VMs/containers to help our model fit the nonlinear relationship between power and features well. Besides, we optimize the leaf nodes of the regression tree, realizing the effective power metering of virtualization environments. We evaluate the proposed model on 13 tasks in PARSEC and compare it with several commonly used models in data centers. The experimental results prove the effectiveness of the proposed model, and the estimated power of containers is in line with expectations.

Power forecasting has a guiding effect on power-aware scheduling strategies to reduce unnecessary power consumption in data centers. Many metrics related to power consumption can be collected in physical servers, such as the status of CPU, memory, and other components. However, most existing methods empirically exploit a small number of metrics to forecast power consumption. To this end, this article uses feature selection based on causality to explore the metrics that strongly influence the power consumption of different tasks. Moreover, we propose a tensor-based dual-stage attention LSTM to forecast the non-linear and non-periodic power consumption. In the proposed model, a multi-way delay embedding transform is utilized to convert the time series into tensors along the temporal direction. The LSTM combines with the tensor technique and the attention mechanism to capture the temporal pattern effectively. In addition, we adopt the cost-sensitive loss function to optimize the specific power forecasting problem in data centers. The experimental results demonstrate that our method can achieve up to 1.4% to 4.3% forecasting accuracy improvement compared with the state-of-the-art models.

Various automated bug localization technologies have recently emerged that require adequate bug-fix records available to train a predictive model. However, many projects in practice might not provide these necessities, especially for new projects in the first release, due to the expensive human effort for constructing a large amount of bug-fix records. Aiming to capture the potential relevance distribution between the bug report and code file from a limited number of available bug-fix records, we present the first semi-supervised bug localization model named BL-GAN in this paper. For this purpose, the promising Generative Adversarial Network is introduced in BL-GAN, in which synthetic bug-fix records close to the real ones are constructed by searching the project directory tree to generate file paths instead of traversing the contents of all code files. For processing bug reports, the proposed BL-GAN adopts an attention-based Transformer architecture to capture semantic and sequence information. In order to capture the proprietary structural information in code files, BL-GAN incorporates a novel multilayer Graph Convolutional Network to process the source code in a graphical view. Extensive experiments on large-scale real-world datasets reveal that our model BL-GAN significantly outperforms the state-of-the-art on all evaluation measures.

As an important stage in machine learning pipeline, feature selection techniques are mainly used to improve the generalization performance and training efficiency of machine learning model, but few works have focused on the robustness of machine learning models from the perspective of feature selection when dealing with adversarial attacks. In this paper, we propose the adversarial training (AT) based feature selection framework, i.e. AT based feature selection, to improve the robustness of machine learning model built on the feature selection result, which is inspired by using adversarial training to improve the robustness of deep learning model. AT based feature selection framework is the combination of adversarial training with some traditional feature selection algorithm, which can be divided into AT in-processing and AT post-processing feature selection. On the other hand, stability is also a very important property for feature selection. Then we experimentally analyze the relationship between robustness and stability of AT based feature selection, especially theoretically analyze the stability of $\ell_{2}$ regularized AT in-processing feature selection algorithm in two different adversarial training forms. Our experimental results on benchmark data sets show that AT based feature selection algorithm is effective to improve the robustness of machine learning model, however, obtain lower stability than corresponding feature selection model without AT.

Summary Due to the rapid increase in the number and scale of data centers, the information and communication technology (ICT) equipment in data centers consumes an enormous amount of power. A power prediction model is therefore essential for decision‐making optimization and power management of ICT equipment. However, it is difficult to predict the power consumption of data centers accurately due to the complex power patterns and nonlinear interdependencies among components. Existing methods either rely on standard formulas, or simply treat it as time series, both leading to poor power prediction accuracy. To overcome those limitations, in this article, we present a systematic power prediction framework called characteristic aware attention‐augmented deep learning‐based prediction method. In particular, we first analyze the different power consumption series to illustrate their different temporal characteristics. Second, we perform different data processing for the corresponding characteristics of power series samples. Third, we propose an accurate and efficient neural network model to predict future power consumption with the pretreated data. The experimental results show that the proposed model is able to achieve superior prediction accuracy.

With the rapid development of text mining, many studies observe that text generally contains a variety of implicit information, and it is important to develop techniques for extracting such information. Named Entity Recognition (NER), the first step of information extraction, mainly identifies names of persons, locations, and organizations in text. Although existing neural-based NER approaches achieve great success in many language domains, most of them normally ignore the nested nature of named entities. Recently, diverse studies focus on the nested NER problem and yield state-of-the-art performance. This survey attempts to provide a comprehensive review on existing approaches for nested NER from the perspectives of the model architecture and the model property, which may help readers have a better understanding of the current research status and ideas. In this survey, we first introduce the background of nested NER, especially the differences between nested NER and traditional (i.e., flat) NER. We then review the existing nested NER approaches from 2002 to 2020 and mainly classify them into five categories according to the model architecture, including early rule-based, layered-based, region-based, hypergraph-based, and transition-based approaches. We also explore in greater depth the impact of key properties unique to nested NER approaches from the model property perspective, namely entity dependency, stage framework, error propagation, and tag scheme. Finally, we summarize the open challenges and point out a few possible future directions in this area. This survey would be useful for three kinds of readers: (i) Newcomers in the field who want to learn about NER, especially for nested NER. (ii) Researchers who want to clarify the relationship and advantages between flat NER and nested NER. (iii) Practitioners who just need to determine which NER technique (i.e., nested or not) works best in their applications.

Vector of Locally Aggregated Descriptors (VLAD) method, which aggregates descriptors and produces a compact image representation, has achieved great success in the field of image classification and retrieval. However, the original VLAD method is a hard assignment strategy that only assigns each descriptor to the nearest neighbor visual word in dictionary, which leads to large quantization error. In this paper, improved VLAD based on adaptive bases and saliency weights is proposed to solve the above problem. The new method considers the local density distribution when assigning local descriptors, adaptively selects several nearest neighbor visual words, and takes the coding coefficients obtained by utilizing saliency as the weights of the selected visual words. Experimental results on Corel 10, 15 Scenes and UIUC Sports Event datasets show that the new coding method proposed in this paper achieves better classification performance compared with the existing five VLAD based methods and two commonly used representation methods.

Software vulnerability detection in source code is a fundamental problem in cyber-security. Aiming at discovering the vulnerability automatically, this paper proposes an open source software vulnerability detection method based on attention-based bidirectional recurrent neural networks. Based on the high-level and generalizable function representations that obtained from the abstract syntax tree(AST), an attention-based bidirectional recurrent neural networks is devised to capture the sequential and important code elements in vulnerability detection from the large number of features that the deep learning model has learned. Experimental results confirm that the huge potential of the proposed new vulnerability detection method which is not only more effective than Convolutional Neural Networks(CNN) but also better than traditional Bidirectional Recurrent Neural Networks(BRNN) in reducing the false negative rate at the price of increasing the false positive rate.

Virtualization technologies provide solutions of cloud computing. Virtual resource scheduling is a crucial task in data centers, and the power consumption of virtual resources is a critical foundation of virtualization scheduling. Containers are the smallest unit of virtual resource scheduling and migration. Although many effective models for estimating power consumption of virtual machines (VM) have been proposed, few power estimation models of containers have been put forth. In this paper, we offer a fast-training piecewise regression model based on decision tree to build a VM power estimation model and estimate the containers' power by treating the container as a group of processes on the VM. In our model, we characterize the nonlinear relationship between power and features and realize the effective estimation of the containers on the VM. We evaluate the proposed model on 13 workloads in PARSEC and compare it with several models. The experimental results prove the effectiveness of our proposed model on most workloads. Moreover, the estimated power of the containers is in line with expectations.

Named Entity Recognition (NER) is a fundamental task in natural language processing.In order to identify entities with nested structure, many sophisticated methods have been recently developed based on either the traditional sequence labeling approaches or directed hypergraph structures.Despite being successful, these methods often fall short in striking a good balance between the expression power for nested structure and the model complexity.To address this issue, we present a novel nested NER model named HIT.Our proposed HIT model leverages two key properties pertaining to the (nested) named entity, including (1) explicit boundary tokens and (2) tight internal connection between tokens within the boundary.Specifically, we design (1) Head-Tail Detector based on the multi-head selfattention mechanism and bi-affine classifier to detect boundary tokens, and (2) Token Interaction Tagger based on traditional sequence labeling approaches to characterize the internal token connection within the boundary.Experiments on three public NER datasets demonstrate that the proposed HIT achieves state-ofthe-art performance.

Bug localization plays an important role in software quality control. Many supervised machine learning models have been developed based on historical bug-fix information. Despite being successful, these methods often require sufficient historical data (i.e., labels), which is not always available especially for newly developed software projects. In response, cross-project bug localization techniques have recently emerged whose key idea is to transferring knowledge from label-rich source project to locate bugs in the target project. However, a major limitation of these existing techniques lies in that they fail to capture the specificity of each individual project, and are thus prone to negative transfer. To address this issue, we propose an adversarial transfer learning bug localization approach, focusing on only transferring the common characteristics (i.e., public information) across projects. Specifically, our approach (CooBa) learns the indicative public information from cross-project bug reports through a shared encoder, and extracts the private information from code files by an individual feature extractor for each project. CooBa further incorporates adversarial learning mechanism to ensure that public information shared between multiple projects could be effectively extracted. Extensive experiments on four large-scale real-world data sets demonstrate that the proposed CooBa significantly outperforms the state of the art techniques.

Named Entity Recognition (NER) is dedicated to recognizing different types of named entity. Previous works have shown that part-of-speech, as an important feature, provides complementary syntactical information to NER systems. However, these studies suffer from two limitations: (i) the previous models do not consider the noise from part-of-speech; (ii) the previous models need to re-extract features from token representations. In this paper, we propose a novel approach that can alleviate the above issues as well as make full use of part-of-speech features via attention mechanism and adversarial training. We evaluate our model on three NER datasets, and the experimental results demonstrate that our model achieves a state-of-the-art F1-score of Twitter dataset while matching a state-of-the-art performance on the CoNLL-2003 and Weibo datasets.

With the requirements of natural language applications, multi-task sequence labeling methods have some immediate benefits over the single-task sequence labeling methods. Recently, many state-of-the-art multi-task sequence labeling methods were proposed, while still many issues to be resolved including (C1) exploring a more general relationship between tasks, (C2) extracting the task-shared knowledge purely and (C3) merging the task-shared knowledge for each task appropriately. To address the above challenges, we propose MTAA, a symmetric multi-task sequence labeling model, which performs an arbitrary number of tasks simultaneously. Furthermore, MTAA extracts the shared knowledge among tasks by adversarial learning and integrates the proposed multi-representation fusion attention mechanism for merging feature representations. We evaluate MTAA on two widely used data sets: CoNLL2003 and OntoNotes5.0. Experimental results show that our proposed model outperforms the latest methods on the named entity recognition and the syntactic chunking task by a large margin, and achieves state-of-the-art results on the part-of-speech tagging task.

With the development of location-based social networks (LBSNs), Point-of-Interest (POI) recommendation has attracted lots of attention. Most of the existing studies focus on recommending POIs to users based on their recent check-ins. However, the recent check-ins may contain some daily check-ins that users are not really interested in. If a model treats the recent check-ins equally, it is non-trivial to capture the actual preference of users. To address the issue of mining the actual preferences of users in the POI recommendation, we propose an attention-based deep learning POI recommendation framework (ADPR), which consists of a latent representation method and an attention-based deep convolutional neural network. To learn the embedding of users and POIs, we propose a latent representation method, which incorporates the geographical influence and the categories of POIs to capture the relationships between POIs better. Further, we propose an attention-based deep convolutional neural network, which employs the attention mechanism to filter the important information in the recent check-ins, to recommend POIs to users based on the latent representations of users and the recent check-ins. We conduct experiments on a real-world LBSN dataset to evaluate our framework, and the experimental results show the effectiveness of our framework.

Feature selection is usually a necessary step for data mining and machine learning. Currently, secure machine learning, especially in privacy preservation, has attracted much attention. However, feature selection with privacy preservation is still a new issue, especially for ensemble feature selection. In this paper, a differentially private ensemble feature selection algorithms is presented. The basic idea behind the proposed algorithm is the output perturbation where the density of perturbation noise depends on the privacy degree and sensitivity of original feature selection algorithm. Besides the theoretical proof, the experimental results also demonstrated their high performance under certain privacy preservation degree.

Concept Factorization (CF) divides a matrix into the product of three matrices. It is considered as one variant of Non-negative Matrix Factorization (NMF). The biggest difference between the two methods is that CF can be executed in a kernel space. Because of this characteristic, many schemes based on CF have been proposed in computer vision and pattern recognition fields. Recent studies have shown that high dimensional data is often located in a low dimensional manifold space, in order to improve the performance and reduce the storage space, how to find the mapping function is particularly important. In addition, the development of supervised learning methods show that label information is critical to enhance the model's ability. In this paper, a supervised graph regularized discriminative concept factorization (SGDCF) method is presented for image clustering. In the SGDCF, we make use of local manifold geometry structure and label information. The corresponding multiplicative update solutions and convergence verification are given. Clustering results on four image data sets reveal that the SGDCF outperforms the state-of-the-art algorithms in terms of accuracy and normalized mutual information.

This paper studies the problem of relationship prediction in heterogeneous information networks. Our goal is not only to predict links/relationships more accurately but also to provide more viable paths to facilitate the formation of new links/relationships. A relationship prediction method based on multi-label learning named ML3P is proposed. In ML3P, each meta-path between nodes is regarded as a type of relationship and is given a label. Under the framework of multi-label learning, any potential relationship including the target relationship can be predicted. The results of comparative experiments in DBLP and Twitter datasets show that ML3P better uses heterogeneous information in supervised learning process and thus achieves better performance. Moreover, our method can output the correlation between relationships.

Vocal impairment is a common symptom for the vast majority of Parkinson's disease (PD) subjects. And it needs long term rehabilitation through personalized one-to-one periodic rehabilitation meetings with clinical speech experts. The significant challenge is that there are not enough experts to deliver the in-person treatments that is needed and for many people with PD, it is difficult to visit the experts for monitoring and treatments. Then there is the need for reliable clinical tools to assist the rehabilitation. This study aims to investigate the potential of using sustained vowel phonations towards objectively and automatically replicating the speech experts' assessments of PD subjects' voices as "acceptable" (a clinician would allow persisting during in-person rehabilitation treatment) or "unacceptable" (a clinician would not allow persisting during in-person rehabilitation treatment). The phonation is usually characterized by many dysphonia measures, which are extracted by clinical speech signal processing algorithms. For this aim, we need to select a stable dys-phonia measures subset, and adopt it to automatically distinguish the PD subjects' voices (acceptable versus unacceptable). In this paper, a diversity regularized ensemble feature weighting algorithm DREFW is presented to choose the stable dysphonia measures subset. The experimental results on real speech rehabilitation data set have shown the proposed algorithm can obtain high stability and classification performance for speech assessment. The findings of this paper is a first step towards improving the effectiveness of an automated rehabilitative speech assessment tool.

Internet traffic classification is one of the key foundations for research works and traffic engineering in Internet. With the rapid increase of Internet applications and the number of Internet flow, the technique challenges are coupled with development of traffic classification all the time. Currently, the machine learning-based technique has attracted much attention, since it can address the issues that the usage of the dynamic port numbers and the encryption technique at the transport layer in traffic. As we have known, feature selection is one of the key problems in machine learning. In this paper, in order to improve the efficiency of feature selection in dealing with large scale traffic data problem, especially to imbalance classification problem that occur in traffic classification, a Min-Max Ensemble Feature Selection (M2-EFS) is proposed to deal with traffic data, which based on balanced data partition and min-max ensemble strategy. The experimental results demonstrate that the M2-EFS can obtain higher performance in most cases, and it could efficiently deal with imbalanced problems.

Two factors characterize a good feature selection algorithm: its accuracy and stability. This paper aims at introducing a new approach to stable feature selection algorithms. The innovation of this paper centers on a class of stable feature selection algorithms called feature weighting as regularized energy-based learning (FREL). Stability properties of FREL using L1 or L2 regularization are investigated. In addition, as a commonly adopted implementation strategy for enhanced stability, an ensemble FREL is proposed. A stability bound for the ensemble FREL is also presented. Our experiments using open source real microarray data, which are challenging high dimensionality small sample size problems demonstrate that our proposed ensemble FREL is not only stable but also achieves better or comparable accuracy than some other popular stable feature weighting methods.

The privacy-preserving data analysis has been gained significant interest across several research communities. The current researches mainly focus on privacy-preserving classification and regression. However, feature selection is also an essential component for data analysis, which can be used to reduce the data dimensionality and can be utilized to discover knowledge, such as inherent variables in data. In this paper, in order to efficiently mine sensitive data, a privacy preserving feature selection algorithm is proposed and analyzed in theory based on local learning and differential privacy. We also conduct some experiments on benchmark data sets. The Experimental results show that our algorithm can preserve the data privacy to some extent.

Recently, besides the performance, the stability (robustness, i.e., the variation in feature selection results due to small changes in the data set) of feature selection is received more attention. Ensemble feature selection where multiple feature selection outputs are combined to yield more robust results without sacrificing the performance is an effective method for stable feature selection. In order to make further improvements of the performance (classification accuracy), the diversity regularized ensemble feature weighting framework is presented, in which the base feature selector is based on local learning with logistic loss for its robustness to huge irrelevant features and small samples. At the same time, the sample complexity of the proposed ensemble feature weighting algorithm is analyzed based on the VC-theory. The experiments on different kinds of data sets show that the proposed ensemble method can achieve higher accuracy than other ensemble ones and other stable feature selection strategy (such as sample weighting) without sacrificing stability

Min-Max Modular Support Vector Machine (M3-SVM) is a powerful ensemble learning method for large scale data processing, which consists of the data decomposition and min-max combination rule. However, when the data contains many redundant or irrelevant features, the ensemble learning performance of M3-SVM will degrade. To address this issue, reduce the computation complexity and enhance the diversity among base classifiers, we propose a method that the feature selection is integrated to the M3-SVM using two integration models. In order to understand the effect of feature selection for ensemble learning, the diversity among base classifiers caused by feature selection is also explored. Experimental results on two large scale data sets including one imbalance data set show that the proposed M3-SVM with feature selection can gain a better performance and higher diversity than original one.

The Parkinson's disease (PD) detection based on dysphonia has been drawn significant attention. However, all dysphonia measurements differ in the uncontrolled acoustic environments. In order to gain as much reliability as possible, measurements should be assessed and the robust ones are chosen. In this study, motivated by statistical learning theory, the problem of PD detection is addressed to classify the participant as healthy or PD using support vector machine (SVM) with the dysphonia measurements as the input feature vector. Therefore an energy-based feature-ranking algorithm is adopted to assess the dysphonia measurements. Moreover, in order to improve the stability of the proposed algorithm, an ensemble version is also presented where multiple feature-ranking results are aggregated. The experimental results on PD data sets have shown the proposed algorithm outperforms other classic ones, and the ensemble version obtain the higher stability than single one.