Copyright holders have been investigating technological solutions to prevent distribution of copyrighted materials in peer-to-peer file sharing networks. A particularly popular technique consists in "poisoning" a specific item (movie, song, or software title) by injecting a massive number of decoys into the peer-to-peer network, to reduce the availability of the targeted item. In addition to poisoning, pollution, that is, the accidental injection of unusable copies of files in the network, also decreases content availability. In this paper, we attempt to provide a first step toward understanding the differences between pollution and poisoning, and their respective impact on content availability in peer-to-peer file sharing networks. To that effect, we conduct a measurement study of content availability in the four most popular peer-to-peer file sharing networks, in the absence of poisoning, and then simulate different poisoning strategies on the measured data to evaluate their potential impact. We exhibit a strong correlation between content availability and topological properties of the underlying peer-to-peer network, and show that the injection of a small number of decoys can seriously impact the users' perception of content availability.

No abstract available.

We propose, implement and evaluate an approach to predicting conditional distribution tail percentiles, which corresponds to value-at-risk (VaR) when applied to financial asset return series. Our approach differs from current methods for measuring VaR in two basic ways. Firstly, while the standard variance-covariance framework assumes that asset return distributions are normal, we make no assumptions about the shape of the entire distribution; instead we focus on estimating only the relevant percentile. Secondly, while the standard approach utilizes only the historical returns and covariances of the assets comprising the portfolio being measured, our method depends on other additional relevant exogenous variables. We use the mean weighted absolute error function (MWAE), a generalization of the well-known mean absolute error (MAE) cost function. It is a familiar statistical property that the MAE is minimized at the median, or 50th percentile of the data; the generalization allows for estimation of arbitrary p-percentiles. There are two methods to estimating values from data: a lazy method keeps all of the data to compute the desired value at any given point; in contrast, an eager method uses the data to estimate a model which is used to generate the desired output. We investigate two approaches for applying the MWAE to estimate tail percentiles, one using each method. The first, kernel percentile regression, is a lazy method, while the second, neural network percentile regression, is an eager method.

Addressing the problem of non-normal portfolio returns, we introduce a novel approach for estimating the distribution of portfolio returns considering higher order mutual information. It allows us to extend the standard variance-covariance framework and efficiently re-compute measures of market risk such as the standard Value-at-Risk or any other probability density based measure. The approach combines two clean and transparent methodologies-independent component analysis and finite Gaussian mixture distributions-and is formulated algorithmically in three steps.

Since the stock market crash in October of 1987, prices of index options deviate significantly from Black-Scholes theory. This fact is prominently documented in the literature as the volatility smile (M. Rubinstein 1994). The pricing error is a sign that the assumptions of the model do not capture all relevant information embedded in option prices. As response to this problem, previous research has relaxed some of the underlying assumptions in order to arrive at more realistic prices. Examples are changes in the data generating process of the underlying security, e.g., jump diffusion or constant elasticity of variance models. We reverse this direction of thought: we take recorded option prices as given and estimate the implied pricing kernel that is consistent with current market valuations. To be flexible in the shape of the risk-neutral density and to allow probabilities to be non-Gaussian, we use the concept of mixture distributions. We apply our methodology to a recent dataset of options on the S&P 500 index future traded on the Chicago Mercantile Exchange. The data spans the year 1998 and contains settlement prices for 22497 American-style futures options.

Introduces a tool for exploratory data analysis and data mining called scale-sensitive gated experts (SSGE) which can partition a complex nonlinear regression surface into a set of simpler surfaces (which we call features). The set of simpler surfaces has the property that each element of the set can be efficiently modeled by a single feedforward neural network. The degree to which the regression surface is partitioned is controlled by an external scale parameter. The SSGE consists of a nonlinear gating network and several competing nonlinear experts. Although SSGE is similar to the mixture of experts model of Jacobs et al. (1991) the mixture of experts model gives only one partitioning of the input-output space, and thus a single set of features, whereas the SSGE gives the user the capability to discover families of features. One obtains a new member of the family of features for each setting of the scale parameter. We derive the scale-sensitive gated experts and demonstrate its performance on a time series segmentation problem. The main results are: (1) the scale parameter controls the granularity of the features of the regression surface, (2) similar features are modeled by the same expert and different kinds of features are modeled by different experts, and (3) for the time series problem, the SSGE finds different regimes of behavior, each with a specific and interesting interpretation.

The paper discusses the application of a signal processing technique known as independent component analysis (ICA), also called blind source separation, to multivariate financial time series. The key idea of ICA is to linearly map observed multivariate time series (such as a portfolio of stocks) into a new space of components that are statistically independent. The authors apply ICA to daily returns of the 28 largest Japanese stocks and compare the ICA results to principal component analysis. Their results indicate that the estimated ICs fall into two categories, (i) infrequent but large shocks (responsible for the major changes in the stock prices), and (ii) frequent but rather small fluctuations (contributing little to the overall level of the stocks). They show that the overall stock price can be reconstructed surprisingly well by thresholding the weighted ICs and using, on average, only one such shock per quarter. In contrast, when using shocks derived from principal components instead of independent components, the reconstructed price does not resemble the original one. The technique of ICA is shown to be a potentially powerful method to analyze and understand driving mechanisms in financial time series.

Exposes problems of the commonly used technique of splitting the available data into training, validation, and test sets that are held fixed, warns about drawing too strong conclusions from such static splits, and shows potential pitfalls of ignoring variability across splits. Using a bootstrap or resampling method, we compare the uncertainty in the solution stemming from the data splitting with neural-network specific uncertainties (parameter initialization, choice of number of hidden units, etc.). We present two results on data from the New York Stock Exchange. First, the variation due to different resamplings is significantly larger than the variation due to different network conditions. This result implies that it is important to not over-interpret a model (or an ensemble of models) estimated on one specific split of the data. Second, on each split, the neural-network solution with early stopping is very close to a linear model; no significant nonlinearities are extracted.

The standard method of embedding time series data is to use a moving window of past values. By the inverse relationship between time and frequency localisation, all information contained in the lower frequencies are lost using this scheme. Increasing the window size comes at the price of adding more degrees of freedom, and thereby worsening the curse of dimensionality. Wavelets provide a solution to this problem. Using multiresolution analysis the authors separate the different time-scales in a given time series. By separating the time series into its component time-scales using the translation-invariant wavelet transform, they determine at which time-scale the series is most predictable.

Most traditional time series models are global models based on local time information: they assume that the state can be fully and locally (in time) characterized with a finite embedding space. Prediction then amounts to simple regression. Unfortunately, there are many situations in which simple regression is not sufficient to model the temporal structure in a time series. The authors introduce an architecture that they call hidden Markov experts. It is based on hidden Markov models used in speech recognition research. By introducing the concept of hidden states, hidden Markov experts model time dependency of time series explicitly as a first-order Markov model with transitions between these hidden states. Within each state, local models are applied to estimate the probability density, which can be linear or nonlinear depending on the situation. The paper first discusses the statistical framework and the learning algorithm of hidden Markov experts, then applies them to daily S&P500 data and to high frequency currency exchange rate data. The hidden Markov experts have better profit than the linear and nonlinear global models. The volatilities of the time series can be characterized by the hidden states.

This paper explores the application of a signal processing technique known as independent component analysis (ICA) or blind source separation to multivariate financial time series such as a portfolio of stocks. The key idea of ICA is to linearly map the observed multivariate time series into a new space of statistically independent components (ICs). We apply ICA to three years of daily returns of the 28 largest Japanese stocks and compare the results with those obtained using principal component analysis. The results indicate that the estimated ICs fall into two categories, (i) infrequent large shocks (responsible for the major changes in the stock prices), and (ii) frequent smaller fluctuations (contributing little to the overall level of the stocks). We show that the overall stock price can be reconstructed surprisingly well by using a small number of thresholded weighted ICs. In contrast, when using shocks derived from principal components instead of independent components, the reconstructed price is less similar to the original one. ICA is shown to be a potentially powerful method of analyzing and understanding driving mechanisms in financial time series. The application to portfolio optimization is described in Chin and Weigend (1998).

While many trading strategies are based on price prediction, traders in financial markets are typically interested in optimizing risk-adjusted performance such as the Sharpe Ratio, rather than the price predictions themselves. This paper introduces an approach which generates a nonlinear strategy that explicitly maximizes the Sharpe Ratio. It is expressed as a neural network model whose output is the position size between a risky and a risk-free asset. The iterative parameter update rules are derived and compared to alternative approaches. The resulting trading strategy is evaluated and analyzed on both computer-generated data and real world data (DAX, the daily German equity index). Trading based on Sharpe Ratio maximization compares favorably to both profit optimization and probability matching (through cross-entropy optimization). The results show that the goal of optimizing out-of-sample risk-adjusted profit can indeed be achieved with this nonlinear approach.

In time series problems, noise can be divided into two categories: dynamic noise which drives the process, and observational noise which is added in the measurement process, but does not influence future values of the system. In this framework, we show that empirical volatilities (the squared relative returns of prices) exhibit a significant amount of observational noise. To model and predict their time evolution adequately, we estimate state space models that explicitly include observational noise. We obtain relaxation times for shocks in the logarithm of volatility ranging from three weeks (for foreign exchange) to three to five months (for stock indices). In most cases, a two-dimensional hidden state is required to yield residuals that are consistent with white noise. We compare these results with ordinary autoregressive models (without a hidden state) and find that autoregressive models underestimate the relaxation times by about two orders of magnitude since they do not distinguish between observational and dynamic noise. This new interpretation of the dynamics of volatility in terms of relaxators in a state space model carries over to stochastic volatility models and to GARCH models, and is useful for several problems in finance, including risk management and the pricing of derivative securities. Data sets used: Olsen & Associates high frequency DEM/USD foreign exchange rates (8 years). Nikkei 225 index (40 years). Dow Jones Industrial Average (25 years).

In the analysis and prediction of real-world systems, two of the key problems are nonstationarity (often in the form of switching between regimes) and overfitting (particularly serious for noisy processes). This article addresses these problems using gated experts, consisting of a (nonlinear) gating network, and several (also nonlinear) competing experts. Each expert learns to predict the conditional mean, and each expert adapts its width to match the noise level in its regime. The gating network learns to predict the probability of each expert, given the input. This article focuses on the case where the gating network bases its decision on information from the inputs. This can be contrasted to hidden Markov models where the decision is based on the previous state(s) (i.e. on the output of the gating network at the previous time step), as well as to averaging over several predictors. In contrast, gated experts soft-partition the input space, only learning to model their region. This article discusses the underlying statistical assumptions, derives the weight update rules, and compares the performance of gated experts to standard methods on three time series: (1) a computer-generated series, obtained by randomly switching between two nonlinear processes; (2) a time series from the Santa Fe Time Series Competition (the light intensity of a laser in chaotic state); and (3) the daily electricity demand of France, a real-world multivariate problem with structure on several time scales. The main results are: (1) the gating network correctly discovers the different regimes of the process; (2) the widths associated with each expert are important for the segmentation task (and they can be used to characterize the sub-processes); and (3) there is less overfitting compared to single networks (homogeneous multilayer perceptrons), since the experts learn to match their variances to the (local) noise levels. This can be viewed as matching the local complexity of the model to the local complexity of the data.

Most traditional prediction techniques deliver a single point, usually the mean of a probability distribution. For multimodal processes, instead of predicting the mean, it is important to predict the full distribution. This article presents a new connectionist method to predict the conditional probability distribution in response to an input. The main idea is to transform the problem from a regression problem to a classification problem. The conditional probability distribution network can perform both direct predictions and iterated predictions, the latter task being specific for time series problems. We compare this new method to fuzzy logic and discuss important differences, and also demonstrate the architecture on two time series. The first is the benchmark laser series used in the Santa Fe competition, a deterministic chaotic system. The second is a time series from a Markov process which exhibits structure on two time scales. The network produces multimodal predictions for this series. We compare the predictions of the network with a nearest-neighbor predictor and find that the conditional probability network is more than twice as likely a model.

We present a new method for obtaining local error bars for nonlinear regression, i.e., estimates of the confidence in predicted values that de(cid:173) pend on the input. We approach this problem by applying a maximum(cid:173) likelihood framework to an assumed distribution of errors. We demon(cid:173) strate our method first on computer-generated data with locally varying, normally distributed target noise. We then apply it to laser data from the Santa Fe Time Series Competition where the underlying system noise is known quantization error and the error bars give local estimates of model misspecification. In both cases, the method also provides a weighted(cid:173) regression effect that improves generalization performance. 1 Learning Local Error Bars Using a Maximum Likelihood Framework: Motivation, Concept, and Mechanics Feed-forward artificial neural networks used for nonlinear regression can be interpreted as predicting the mean of the target distribution as a function of (conditioned on) the input pattern (e.g., Buntine & Weigend, 1991; Bishop, 1994), typically using one linear output unit per output variable. If parameterized, this conditional target distribution (CID) may also be ·http://www.cs.colorado.edu/~andreas/Home.html. This paper is available with figures in colors as ftp://ftp.cs.colorado.edu/pub/ Time-Series/MyPapers/nix.weigenCLnips7.ps.Z . 490 David A. Nix, Andreas S. Weigend viewed as an error model (Rumelhart et al., 1995). Here, we present a simple method that provides higher-order information about the cm than simply the mean. Such additional information could come from attempting to estimate the entire cm with connectionist methods (e.g., "Mixture Density Networks," Bishop, 1994; "fractional binning, "Srivastava & Weigend, 1994) or with non-connectionist methods such as a Monte Carlo on a hidden Markov model (Fraser & Dimitriadis, 1994). While non-parametric estimates of the shape of a C1D require large quantities of data, our less data-hungry method (Weigend & Nix, 1994) assumes a specific parameterized form of the C1D (e.g., Gaussian) and gives us the value of the error bar (e.g., the width of the Gaussian) by finding those parameters which maximize the likelihood that the target data was generated by a particular network model. In this paper we derive the specific update rules for the Gaussian case. We would like to emphasize, however, that any parameterized unimodal distribution can be used for the em in the method presented here. j------------, I I------T-------, , I A I A2,. ) o y(x) 0 cr IX /\ i '.

The calculation of second derivatives is required by recent training and analysis techniques of connectionist networks, such as the elimination of superfluous weights, and the estimation of confidence intervals both for weights and network outputs. We review and develop exact and approximate algorithms for calculating second derivatives. For networks with |w| weights, simply writing the full matrix of second derivatives requires O(|w|(2)) operations. For networks of radial basis units or sigmoid units, exact calculation of the necessary intermediate terms requires of the order of 2h+2 backward/forward-propagation passes where h is the number of hidden units in the network. We also review and compare three approximations (ignoring some components of the second derivative, numerical differentiation, and scoring). The algorithms apply to arbitrary activation functions, networks, and error functions.

We investigate the effectiveness of connectionist architectures for predicting the future behavior of nonlinear dynamical systems. We focus on real-world time series of limited record length. Two examples are analyzed: the benchmark sunspot series and chaotic data from a computational ecosystem. The problem of overfitting, particularly serious for short records of noisy data, is addressed both by using the statistical method of validation and by adding a complexity term to the cost function ("back-propagation with weight-elimination"). The dimension of the dynamics underlying the time series, its Liapunov coefficient, and its nonlinearity can be determined via the network. We also show why sigmoid units are superior in performance to radial basis functions for high-dimensional input spaces. Furthermore, since the ultimate goal is accuracy in the prediction, we find that sigmoid networks trained with the weight-elimination algorithm outperform traditional nonlinear statistical approaches.