Endotherms such as rats and mice huddle together to keep warm. The huddle is considered to be an example of a self-organising system, because complex properties of the collective group behaviour are thought to emerge spontaneously through simple interactions between individuals. Groups of rodent pups display two such emergent properties. First, huddling undergoes a 'phase transition', such that pups start to aggregate rapidly as the temperature of the environment falls below a critical temperature. Second, the huddle maintains a constant 'pup flow', where cooler pups at the periphery continually displace warmer pups at the centre. We set out to test whether these complex group behaviours can emerge spontaneously from local interactions between individuals. We designed a model using a minimal set of assumptions about how individual pups interact, by simply turning towards heat sources, and show in computer simulations that the model reproduces the first emergent property--the phase transition. However, this minimal model tends to produce an unnatural behaviour where several smaller aggregates emerge rather than one large huddle. We found that an extension of the minimal model to include heat exchange between pups allows the group to maintain one large huddle but eradicates the phase transition, whereas inclusion of an additional homeostatic term recovers the phase transition for large huddles. As an unanticipated consequence, the extended model also naturally gave rise to the second observed emergent property--a continuous pup flow. The model therefore serves as a minimal description of huddling as a self-organising system, and as an existence proof that group-level huddling dynamics emerge spontaneously through simple interactions between individuals. We derive a specific testable prediction: Increasing the capacity of the individual to generate or conserve heat will increase the range of ambient temperatures over which adaptive thermoregulatory huddling will emerge.

Long after a new language has been learned and forgotten, relearning a few words seems to trigger the recall of other words. This "free-lunch learning" (FLL) effect has been demonstrated both in humans and in neural network models. Specifically, previous work proved that linear networks that learn a set of associations, then partially forget them all, and finally relearn some of the associations, show improved performance on the remaining (i.e., nonrelearned) associations. Here, we prove that relearning forgotten associations decreases performance on nonrelearned associations; an effect we call negative free-lunch learning. The difference between free-lunch learning and the negative free-lunch learning presented here is due to the particular method used to induce forgetting. Specifically, if forgetting is induced by isotropic drifting of weight vectors (i.e., by adding isotropic noise), then free-lunch learning is observed. However, as proved here, if forgetting is induced by weight values that simply decay or fall towards zero, then negative free-lunch learning is observed. From a biological perspective, and assuming that nervous systems are analogous to the networks used here, this suggests that evolution may have selected physiological mechanisms that involve forgetting using a form of synaptic drift rather than synaptic decay, because synaptic drift, but not synaptic decay, yields free-lunch learning.

After a language has been learned and then forgotten, relearning some words appears to facilitate spontaneous recovery of other words. More generally, relearning partially forgotten associations induces recovery of other associations in humans, an effect we call free-lunch learning (FLL). Using neural network models, we prove that FLL is a necessary consequence of storing associations as distributed representations. Specifically, we prove that (1) FLL becomes increasingly likely as the number of synapses (connection weights) increases, suggesting that FLL contributes to memory in neurophysiological systems, and (2) the magnitude of FLL is greatest if inactive synapses are removed, suggesting a computational role for synaptic pruning in physiological systems. We also demonstrate that FLL is different from generalization effects conventionally associated with neural network models. As FLL is a generic property of distributed representations, it may constitute an important factor in human memory.

Animals with rudimentary innate abilities require substantial learning to transform those abilities into useful skills, where a skill can be considered as a set of sensory-motor associations. Using linear neural network models, it is proved that if skills are stored as distributed representations, then within-lifetime learning of part of a skill can induce automatic learning of the remaining parts of that skill. More importantly, it is shown that this "free-lunch" learning (FLL) is responsible for accelerated evolution of skills, when compared with networks which either 1) cannot benefit from FLL or 2) cannot learn. Specifically, it is shown that FLL accelerates the appearance of adaptive behaviour, both in its innate form and as FLL-induced behaviour, and that FLL can accelerate the rate at which learned behaviours become innate.

A measure of temporal predictability is defined and used to separate linear mixtures of signals. Given any set of statistically independent source signals, it is conjectured here that a linear mixture of those signals has the following property: the temporal predictability of any signal mixture is less than (or equal to) that of any of its component source signals. It is shown that this property can be used to recover source signals from a set of linear mixtures of those signals by finding an un-mixing matrix that maximizes a measure of temporal predictability for each recovered signal. This matrix is obtained as the solution to a generalized eigenvalue problem; such problems have scaling characteristics of O(N3), where N is the number of signal mixtures. In contrast to independent component analysis, the temporal predictability method requires minimal assumptions regarding the probability density functions of source signals. It is demonstrated that the method can separate signal mixtures in which each mixture is a linear combination of source signals with supergaussian, subgaussian, and gaussian probability density functions and on mixtures of voices and music.

We demonstrate that imperfect recall of a set of associations can usually be improved by training on a new, unrelated set of associations. This spontaneous recovery of associations is a consequence of the high dimensionality of weight spaces, and is therefore not peculiar to any single type of neural net. Accordingly, this work may have implications for spontaneous recovery of memory in the central nervous system.

A model is presented for unsupervised learning of low level vision tasks, such as the extraction of surface depth. A key assumption is that perceptually salient visual parameters (e.g., surface depth) vary smoothly over time. This assumption is used to derive a learning rule that maximizes the long-term variance of each unit's outputs, whilst simultaneously minimizing its short-term variance. The length of the half-life associated with each of these variances is not critical to the success of the algorithm. The learning rule involves a linear combination of anti-Hebbian and Hebbian weight changes, over short and long time scales, respectively. This maximizes the information throughput with respect to low-frequency parameters implicit in the input sequence. The model is used to learn stereo disparity from temporal sequences of random-dot and gray-level stereograms containing synthetically generated subpixel disparities. The presence of temporal discontinuities in disparity does not prevent learning or generalization to previously unseen image sequences. The implications of this class of unsupervised methods for learning in perceptual systems are discussed.

Introduces adaptive scale filtering, a general method for deriving shape from texture under perspective projection without recourse to prior segmentation of the image into geometric texture elements (texels), and without thresholding of filtered images. If texels on a given surface can be identified in an image then the orientation of that surface can be obtained. However, there is no general characterization of texels for arbitrary textures. Furthermore, even if the size and shape of texels on the surface is invariant with regard to position, perspective projection ensures that the size and shape of the corresponding image texels vary by orders of magnitude. Commencing with an initial set F/sub O/ of identical image filters, adaptive scale filtering iteratively derives a set F/sub N/ which contains a unique filter for each image position. Each element of F/sub N/ is tuned to the three-dimensional structure of the surface; that is, all image filters in F/sub N/ back-project to an identical shape and size on the surface. Thus image texels of various sizes, but associated with a single spatial scale on the surface, can be identified in different parts of the image. When combined with conventional shape from texture methods, edges derived using F/sub N/ provide accurate estimates of surface orientation. Results for planar surfaces are presented.>

We present a neural network model for the unsupervised learning of high order visual invariances. The model is demonstrated on the problem of estimating sub-pixel stereo disparity from a temporal sequence of unprocessed image pairs. After learning on a given image sequence, the model's ability to detect sub-pixel disparity generalises, without additional learning, to image pairs from other sequences. 1

If texels on a given surface can be identified in an image then the orientation of that surface can be obtained [1]. Unfortunately there is no known procedure for identifying texels for arbitrary textures. Even if the size and shape of texels on the surface is invariant with regard to position, perspective projection ensures that the size and shape of the corresponding image texels will vary by orders of magnitude.