Managing inputs that are novel, unknown, or out-of-distribution is critical as an agent moves from the lab to the open world. Novelty-related problems include being tolerant to novel perturbations of the normal input, detecting when the input includes novel items, and adapting to novel inputs. While significant research has been undertaken in these areas, a noticeable gap exists in the lack of a formalized definition of novelty that transcends problem domains. As a team of researchers spanning multiple research groups and different domains, we have seen, first hand, the difficulties that arise from ill-specified novelty problems, as well as inconsistent definitions and terminology. Therefore, we present the first unified framework for formal theories of novelty and use the framework to formally define a family of novelty types. Our framework can be applied across a wide range of domains, from symbolic AI to reinforcement learning, and beyond to open world image recognition. Thus, it can be used to help kick-start new research efforts and accelerate ongoing work on these important novelty-related problems.

Detection of near duplicate documents is an important problem in many data mining and information filtering applications. When faced with massive quantities of data, traditional duplicate detection techniques relying on direct inter-document similarity computation (e.g., using the cosine measure) are often not feasible given the time and memory performance constraints. On the other hand, fingerprint-based methods, such as I-Match, are very attractive computationally but may be brittle with respect to small changes to document content. We focus on approaches to near-replica detection that are based upon large-collection statistics and present a general technique of increasing their robustness via multiple lexicon randomization. In experiments with large web-page and spam-email datasets the proposed method is shown to consistently outperform traditional I-Match, with the relative improvement in duplicate-document recall reaching as high as 40-60%. The large gains in detection accuracy are offset by only small increases in computational requirements.

We have continued our study of a parallel perturbative learning method [Alspector et al., 1993] and implications for its implemen(cid:173) tation in analog VLSI. Our new results indicate that, in most cases, a single parallel perturbation (per pattern presentation) of the func(cid:173) tion parameters (weights in a neural network) is theoretically the best course. This is not true, however, for certain problems and may not generally be true when faced with issues of implemen(cid:173) tation such as limited precision. In these cases, multiple parallel perturbations may be best as indicated in our previous results.

In feedback neural networks, especially for static pattern learning, a reliable method of settling is required. Simulated annealing has been used but it is often difficult to determine how to set the annealing schedule. Often the specific heat is used as a measure of when to slow down the annealing process, but this is difficult to measure. We propose another measure, volatility, which is easy to measure and related to the Edwards-Anderson model in spin-glass physics. This paper presents the concept of volatility, an argument for its similarity to specific heat, simulations of dynamics in Boltzmann and mean-field networks, and a method of using it to speed up learning.

Boltzmann-based models with asymmetric connections are investigated. Although they are initially unstable, these networks spontaneously self-stabilize as a result of learning. Moreover, pairs of weights symmetrize during learning; however, the symmetry is not enough to account for the observed stability. To characterize the system it is useful to consider how its entropy is affected by learning and the entropy of the information stream. The stability of an asymmetric network is confirmed with an electronic model.