In anomaly detection, the normal behavior of a process is characterized by a model, and deviations from the model are called anomalies. In behavior-based approaches to anomaly detection, the model of normal behavior is constructed from an observed sample of normally occurring patterns. Models of normal behavior can represent either the set of allowed patterns (positive detection) or the set of anomalous patterns (negative detection). A formal framework is given for analyzing the tradeoffs between positive and negative detection schemes in terms of the number of detectors needed to maximize coverage. For realistically sized problems, the universe of possible patterns is too large to represent exactly (in either the positive or negative scheme). Partial matching rules generalize the set of allowable (or unallowable) patterns, and the choice of matching rule affects the tradeoff between positive and negative detection. A new match rule is introduced, called r-chunks, and the generalizations induced by different partial matching rules are characterized in terms of the crossover closure. Permutations of the representation can be used to achieve more precise discrimination between normal and anomalous patterns. Quantitative results are given for the recognition ability of contiguous-bits matching together with permutations.

This paper examines the problem of prioritizing actions under uncertainty. Our motivating applications come from the domain of data mining. Data mining problems present the user with a huge collection of individual items (e.g., abstracts, medical histories, and computer users' command histories) and require that these items be prioritized according to which should be pursued thoroughly. More precisely, each data item is assumed to be generated by one of two processes: A large majority of the data comes from a common, mundane process and a very small fraction comes from a rare, phenomenon process. The problem is to rank the information so as to optimally direct the user in his or her pursuit of the data items that were generated by the phenomenon process. Our previous work has developed the theoretical foundations of the information prioritization problem. The current paper summarizes these foundations, derives new theoretical results, and details initial experimental results of a prioritization system based on the theory. We focus here on feature selection techniques and the method of model surrogates, each tailored to the classes of prioritization applications of greatest current interest. Our results demonstrate the effectiveness of the techniques and motivate further research to improve the existing system.

We present new results on a distributable change-detection method inspired by the natural immune system. A weakness in the original algorithm was the exponential cost of generating detectors. Two detector-generating algorithms are introduced which run in linear time. The algorithms are analyzed, heuristics are given for setting parameters based on the analysis, and the presence of holes in detector space is examined. The analysis provider a basis for assessing the practicality of the algorithms in specific settings, and some of the implications are discussed.

The authors present exact characterizations of structures on which the greedy algorithm produces optimal solutions. Our characterization, which are called matroid embeddings, complete the partial characterizations of Rado [A note on independent functions, Proc. London Math. Soc., 7 (1957), pp. 300–320], Gale [Optimal assignments in an ordered set, J. Combin. Theory, 4 (1968), pp. 176–180], and Edmonds [Matroids and the greedy algorithm, Math. Programming, 1 (1971), pp. 127–136], (matroids), and of Korte and Lovasz [Greedoids and linear object functions, SIAM J. Alg. Discrete Meth., 5 (1984), pp. 239–248] and [Mathematical structures underlying greedy algorithms, in Fundamentals of Computational Theory, LNCS 177, Springer-Verlag, 1981, pp. 205–209] (greedoids). It is shown that the greedy algorithm optimizes all linear objective functions if and only if the problem structure (phrased in terms of either accessible set systems or hereditary languages) is a matroid embedding. An exact characterization of the objective functions optimized by the greedy algorithm on matroid embeddings is also presented. Finally, the authors present an exact characterization of the structures on which the greedy algorithm optimizes all bottleneck functions, structures that are less constrained than matroid embeddings.

We model computer transactions as generated by two stationary stochastic processes, the legitimate (normal) process N and the misuse process M. We define misuse (anomaly) detection to be the identification of transactions most likely to have been generated by M. We formally demonstrate that the accuracy of misuse detectors is bounded by a function of the difference of the densities of the processes N and M over the space of transactions. In practice, detection accuracy can be far below this bound, and generally improves with increasing sample size of historical (training) data. Careful selection of transaction attributes also can improve detection accuracy; we suggest several criteria for attribute selection, including adequate sampling rate and separation between models. We demonstrate that exactly optimizing even the simplest of these criteria is NP-hard, thus motivating a heuristic approach. We further differentiate between modeling (density estimation) and nonmodeling approaches.>

Computer use is modeled as a mixture of two stochastic processes, normal and misuse. Intrusion detection is formally defined as identifying those transactions generated by the misuse process. Bounds for detection performance are derived in terms of the ratios of the densities of the processes at the individual transactions. It is shown that any optimal intrusion detection system must rank transaction suspicion consistently with these ratios. Sparsity of data requires that transactions be grouped into equivalence classes that preserve the order of the true ratio ranking and reduce the number of singleton and unobserved transactions. Results are described that demonstrate that in general this 'singleton reduction' problem is NP-hard.>

The logic of many query optimizers corresponds to searching a graph that contains all alternative intermediate results considered by the optimizer. We consider the problem, “Given a query Q, determine the cost of a best strategy that uses intermediate result v to compute Q, assuming that v is available at no cost.” Our main result is an algorithm that mass-produces such cost information for all intermediates v considered by the query optimizer, in time linear in the size of the graph. To illustrate potential applications for the algorithm, we describe rules for identifying and eliminating suboptimal alternatives in multiple query optimization and demonstrate how the algorithm rapidly computes the necessary cost bounds. We also describe applications of the algorithm to speeding physical database design. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.

A new model for dynamic programming and branch and bound algorithms is presented. The model views these algorithms as utilizing computationally feasible dominance relations to infer the orderings of application objects, thereby implicitly enumerating a finite solution space. The formalism is broad enough to apply the computational strategies of dynamic programming and branch and bound to problems with nonassociative objects, and can model both oblivious and nonoblivious algorithms, as well as parallel algorithms. The model is used to classify computations based, in part, on the types of computationally feasible dominances that they employ. It is demonstrated that the model is computationally precise enough to support the derivation of lower bounds on the number of operations required to solve various types of problems.