We investigate imbalanced regression with tabular data that have an imbalance ratio larger than 1,000 ("highly imbalanced"). Accurately estimating the target values of rare instances is important in applications such as forecasting the intensity of rare harmful Solar Energetic Particle (SEP) events. For regression, the MSE loss does not consider the correlation between predicted and actual values. Typical inverse importance functions allow only convex functions. Uniform sampling might yield mini-batches that do not have rare instances. We propose CISIR that incorporates correlation, Monotonically Decreasing Involution (MDI) importance, and stratified sampling. Based on five datasets, our experimental results indicate that CISIR can achieve lower error and higher correlation than some recent methods. Also, adding our correlation component to other recent methods can improve their performance. Lastly, MDI importance can outperform other importance functions. Our code can be found in https://github.com/Machine-Earning/CISIR.

Assuming unknown classes could be present during classification, the open set recognition (OSR) task aims to classify an instance into a known class or reject it as unknown. In this paper, we use a two-stage training strategy for OSR problems. In the first stage, we introduce a self-supervised feature decoupling method that finds the content features of the input samples from the known classes. Specifically, our feature decoupling approach learns a representation that can be split into content features and transformation features. In the second stage, we fine-tune the content features with the class labels. The fine-tuned content features are then used for the OSR problems. To measure representation quality, we introduce intra-inter ratio (IIR). Our experimental results indicate that our proposed self-supervised approach outperforms others in image and malware OSR problems. Also, our analyses indicate that IIR is correlated with and can explain OSR performance.

Open set recognition (OSR) problem has been a challenge in many machine learning (ML) applications, such as security. As new/unknown malware families occur regularly, it is difficult to exhaust samples that cover all the classes for the training process in ML systems. An advanced malware classification system should classify the known classes correctly while sensitive to the unknown class. In this paper, we introduce a self-supervised pre-training approach for the OSR problem in malware classification. We propose two transformations for the function call graph (FCG) based malware representations to facilitate the pretext task. Also, we present a statistical thresholding approach to find the optimal threshold for the unknown class. Moreover, the experiment results indicate that our proposed pre-training process can improve different performances of different downstream loss functions for the OSR problem.

We propose an unsupervised open set recognition approach that uses an Adversarial Autoencoder (AAE) for clustering and combines it with ii-loss to learn a representation that facilitates open set recognition. We evaluate our approach on two malware datasets. To show the applicability of the proposed method outside the malware domain we also evaluate it with an image dataset. Our proposed approach gives a statistically significant improvement over other evaluated approaches.

In this paper, we present a neural network based representation for the Open Set Recognition problem. In this representation instances from the same class are close to each other while instances from different classes are further apart. When used for Open Set Recognition tasks, evaluated on three datasets from two different domains, the proposed approach results in a statistically significant improvement compared to other approaches.

The task of identifying pros and cons from product reviews has applications in decision support for consumers. It becomes even more useful when the pros and cons are identified for product aspects so consumers can quickly see strengths and weaknesses of each aspect of the product without reading all reviews. Given a collection of product reviews, we automatically extract relevant product aspects, find the most significant sentences that represent pros and cons for each aspect, and provide a summary for each aspect. We introduce SS2 to select sentences that are likely to represent pros/cons and are semantically related to the aspect to which they are associated. Our results on three data sets indicate that compared to an existing algorithm, our algorithm can generate more meaningful summarized aspects, along with a list of pros and cons more closely related to each aspect.

The task of mining pros and cons from actions has applications in decision support. Given an action query and social media data, we mine related pros and cons of the action via extracting significant events as potential outcomes of the action. We propose using actions and characteristics to select relevant messages, and adjective vectors to establish similarity among adjectives. We introduce SS to select event headlines, and to rank them in the final pros-and-cons table. Our results on two data sets indicate our algorithm can generate more meaningful pros and cons than an existing algorithm.

In an attempt to preserve the structural information in malware binaries during feature extraction, function call graph-based features have been used in various research works in malware classification. However, the approach usually employed when performing classification on these graphs, is based on computing graph similarity using computationally intensive techniques. Due to this, much of the previous work in this area incurred large performance overhead and does not scale well. In this paper, we propose a linear time function call graph (FCG) vector representation based on function clustering that has significant performance gains in addition to improved classification accuracy. We also show how this representation can enable using graph features together with other non-graph features.

Based on the Flow Authority model, we propose the VSM (Vector-based Spread Maximization) algorithm that estimates the SSS (Steady-state Spread) objective function for multiple seeds based on SSS values of individual seeds to reduce computation. Based on three real-world large sparse networks, our empirical results indicate that VSM is more effective than two existing algorithms, and two orders of magnitude more efficient than one of them.

Training Artificial Neural Networks (ANN) is relatively slow compared to many other machine learning algorithms. In this study, we focus on instance selection to improve training speed. We first evaluate the effectiveness of instance selection algorithms for k-nearest neighbor algorithms with ANN. We then analyze factors in accuracy -- distance from decision boundary, dense regions, and class distributions, and propose new instance selection algorithms. We discuss the trade off between accuracy and training speed, and introduce a measure for the trade off. Our empirical results on real data sets indicate that our proposed RDI is more effective with ANN.

One sub-field of Genetic Programming (GP) which has gained recent interest is semantic GP, in which programs are evolved by manipulating program semantics instead of program syntax. This paper introduces a new semantic GP algorithm, called SGP+, which is an extension of an existing algorithm called SGP. New crossover and mutation operators are introduced which address two of the major limitations of SGP: large program trees and reduced accuracy on high-arity problems. Experimental results on "deceptive" Boolean problems show that programs created by the SGP+ are 3.8 times smaller while still maintaining accuracy as good as, or better than, SGP. Additionally, a statistically significant improvement in program accuracy is observed for several high-arity Boolean problems.

Boarding or holding in the Emergency Department (ED) reduces capacity of the ED and delays patients from receiving specialized care. Estimating accurately the number of admissions from the ED can help determine appropriate level of staffing to reduce holding. We propose a randomized non-linear regression algorithm, RT-KGERS, to estimate the number of admissions a week in advance. We also devise features based on cyclical patterns found with a Fast Fourier Transform analysis on the hospital admission data. We evaluate the accuracy and efficiency of RT-KGERS and three existing algorithms in a dataset provided by a local hospital. We then compare our features with related features. Initial experimental results from RT-KGERS encouraged the hospital and us to conduct a live trial study which yielded similar levels of accuracy using RT-KGERS and the six features we devised.

Popularity of mobile devices is accompanied by widespread security problems, such as MAC address spoofing in wireless networks. We propose a probabilistic approach to temporal anomaly detection using smoothing technique for sparse data. Our technique builds up on the Markov chain, and clustering is presented for reduced storage requirements. Wireless networks suffer from oscillations between locations, which result in weaker statistical models. Our technique identifies such oscillations, resulting in higher accuracy. Experimental results on publicly available wireless network data sets indicate that our technique is more effective than Markov chain to detect anomalies for location, time, or both.

For intrusion detection, the LERAD algorithm learns a succinct set of comprehensible rules for detecting anomalies, which could be novel attacks. LERAD validates the learned rules on a separate held-out validation set and removes rules that cause false alarms. However, removing rules with possible high coverage can lead to missed detections. We propose to retain these rules and associate weights to them. We present three weighting schemes and our empirical results indicate that, for LERAD, rule weighting can detect more attacks than pruning with minimal computational overhead.

The prevalent use of computers and internet has enhanced the quality of life for many people, but it has also attracted undesired attempts to undermine these systems. This special topic contains several research studies on how machine learning algorithms can help improve the security of computer systems.

Our goal is to generate comprehensible and accurate models from multiple time series for anomaly detection. The models need to produce anomaly scores in an online manner for real-life monitoring tasks. We introduce three algorithms that work in a constructed feature space and evaluate them with a real data set from the NASA shuttle program. Our offline and online evaluations indicate that our algorithms can be more accurate than two existing algorithms.

Finding meaningful phrases in a document has been studied in various information retrieval systems in order to improve the performance. Many previous statistical phrase-finding methods had a different aim such as document classification. Some are hybridized with statistical and syntactic grammatical methods; others use correlation heuristics between words. We propose a new phrase-finding algorithm that adds correlated words one by one to the phrases found in the previous stage, maintaining high correlation within a phrase. Our results indicate that our algorithm finds more meaningful phrases than an existing algorithm. Furthermore, the previous algorithm could be improved by applying different correlation junctions.

Many clustering and segmentation algorithms both suffer from the limitation that the number of clusters/segments is specified by a human user. It is often impractical to expect a human with sufficient domain knowledge to be available to select the number of clusters/segments to return. We investigate techniques to determine the number of clusters or segments to return from hierarchical clustering and segmentation algorithms. We propose an efficient algorithm, the L method that finds the "knee" in a '# of clusters vs. clustering evaluation metric' graph. Using the knee is well-known, but is not a particularly well-understood method to determine the number of clusters. We explore the feasibility of this method, and attempt to determine in which situations it will and will not work. We also compare the L method to existing methods based on the accuracy of the number of clusters that are determined and efficiency. Our results show favorable performance for these criteria compared to the existing methods that were evaluated.

Most of the prevalent anomaly detection systems use some training data to build models. These models are then utilized to capture any deviations resulting from possible intrusions. The efficacy of such systems is highly dependent upon a training data set free of attacks. "Clean" or labeled training data is hard to obtain. This paper addresses the very practical issue of refinement of unlabeled data to obtain a clean data set which can then train an online anomaly detection system.

We introduce an algorithm called LERAD that learns rules for finding rare events in nominal time-series data with long range dependencies. We use LERAD to find anomalies in network packets and TCP sessions to detect novel intrusions. We evaluated LERAD on the 1999 DARPA/Lincoln Laboratory intrusion detection evaluation data set and on traffic collected in a university departmental server environment.

To provide a more robust context for personalization, we desire to extract a continuum of general (long-term) to specific (short-term) interests of a user. Our proposed approach is to learn a user interest hierarchy (UIH) from a set of web pages visited by a user. We devise a divisive hierarchical clustering (DHC) algorithm to group words (topics) into a hierarchy where more general interests are represented by a larger set of words. Each web page can then be assigned to nodes in the hierarchy for further processing in learning and predicting interests. This approach is analogous to building a subject taxonomy for a library catalog system and assigning books to the taxonomy. Our approach does not need user involvement and learns the UIH "implicitly." Furthermore, it allows the original objects, web pages, to be assigned to multiple topics (nodes in the hierarchy). In this paper, we focus on learning the UIH from a set of visited pages. We propose a few similarity functions and dynamic threshold-finding methods, and evaluate the resulting hierarchies according to their meaningfulness and shape

Traditional intrusion detection systems (IDS) detect attacks by comparing current behavior to signatures of known attacks. One main drawback is the inability of detecting new attacks which do not have known signatures. In this paper we propose a learning algorithm that constructs models of normal behavior from attack-free network traffic. Behavior that deviates from the learned normal model signals possible novel attacks. Our IDS is unique in two respects. First, it is nonstationary, modeling probabilities based on the time since the last event rather than on average rate. This prevents alarm floods. Second, the IDS learns protocol vocabularies (at the data link through application layers) in order to detect unknown attacks that attempt to exploit implementation errors in poorly tested features of the target software. On the 1999 DARPA IDS evaluation data set [9], we detect 70 of 180 attacks (with 100 false alarms), about evenly divided between user behavioral anomalies (IP addresses and ports, as modeled by most other systems) and protocol anomalies. Because our methods are unconventional there is a significant non-overlap of our IDS with the original DARPA participants, which implies that they could be combined to increase coverage.

Intrusion detection systems (IDSs) must be capable of detecting new and unknown attacks, or anomalies. We study the problem of building detection models for both pure anomaly detection and combined misuse and anomaly detection (i.e., detection of both known and unknown intrusions). We propose an algorithm to generate artificial anomalies to coerce the inductive learner into discovering an accurate boundary between known classes (normal connections and known intrusions) and anomalies. Empirical studies show that our pure anomaly detection model trained using normal and artificial anomalies is capable of detecting more than 77% of all unknown intrusion classes with more than. 50% accuracy per intrusion class. The combined misuse and anomaly detection models are as accurate as a pure misuse detection model in detecting known intrusions and are capable of detecting at least 50% of unknown intrusion classes with accuracy measurements between 75% and 100% per class.

In this paper, we propose meta-leaming as a general technique to combine the results of multiple learning algorithms, Mets-learner Met: ID3 96.6+

Knowledge-discovery systems face challenging problems from real-world databases, which tend to be dynamic, incomplete, redundant, noisy, sparse, and very large. These problems are addressed and some techniques for handling them are described. A model of an idealized knowledge-discovery system is presented as a reference for studying and designing new systems. This model is used in the comparison of three systems: CoverStory, EXPLORA, and the Knowledge Discovery Workbench. The deficiencies of existing systems relative to the model reveal several open problems for future research.>

Although the problem of increasing the speed of rule-based programs has been studied for a long while, so far the level of parallelism achieved under various parallel processing schemes fails to meet expectations of high performance gains. Most of the work has focused on manipulating existing rule programs to accommodate parallel architectures. However, without changing the inherently sequential algorithms encoded in rule form and without using intrinsic parallel rule languages to exploit parallelism, speedup is severely limited. In this paper we demonstrate that to maximize parallelism, manipulations must take place at the algorithmic level in addition to the program level. However, traditional rule languages are not equipped to easily express parallel algorithms. Thus, an inherently parallel rule language must be devised to enable maximum parallelism. We present our initial specification of such a language, named PARULEL. Preliminary performance results are detailed for one test case program. PARULEL is one part of a larger research effort that considers four key approaches to parallel rule processing: inherently parallel execution semantics; the use of redaction meta-rules to allow programmer control of the set of parallel instantiations; an incremental evaluation algorithm that incorporates, in an efficient manner, updates to the initial set of assumptions; and an algorithm that maps a program to a parallel architecture.