We study the connections between causal relations and conditional independence within the settable systems extension of the Pearl causal model (PCM). Our analysis clearly distinguishes between causal notions and probabilistic notions, and it does not formally rely on graphical representations. As a foundation, we provide definitions in terms of suitable functional dependence for direct causality and for indirect and total causality via and exclusive of a set of variables. Based on these foundations, we provide causal and stochastic conditions formally characterizing conditional dependence among random vectors of interest in structural systems by stating and proving the conditional Reichenbach principle of common cause, obtaining the classical Reichenbach principle as a corollary. We apply the conditional Reichenbach principle to show that the useful tools of d-separation and D-separation can be employed to establish conditional independence within suitably restricted settable systems analogous to Markovian PCMs.

We illustrate the need to use higher-order (specifically sixth-order) expansions in order to properly determine the asymptotic distribution of a standard artificial neural network test for neglected nonlinearity. The test statistic is a quasi-likelihood ratio (QLR) statistic designed to test whether the mean square prediction error improves by including an additional hidden unit with an activation function violating the no-zero condition in Cho, Ishida, and White (2011). This statistic is also shown to be asymptotically equivalent under the null to the Lagrange multiplier (LM) statistic of Luukkonen, Saikkonen, and Teräsvirta (1988) and Teräsvirta (1994). In addition, we compare the power properties of our QLR test to one satisfying the no-zero condition and find that the latter is not consistent for detecting a DGP with neglected nonlinearity violating an analogous no-zero condition, whereas our QLR test is consistent.

Tests for regression neglected nonlinearity based on artificial neural networks (ANNs) have so far been studied by separately analyzing the two ways in which the null of regression linearity can hold. This implies that the asymptotic behavior of general ANN-based tests for neglected nonlinearity is still an open question. Here we analyze a convenient ANN-based quasi-likelihood ratio statistic for testing neglected nonlinearity, paying careful attention to both components of the null. We derive the asymptotic null distribution under each component separately and analyze their interaction. Somewhat remarkably, it turns out that the previously known asymptotic null distribution for the type 1 case still applies, but under somewhat stronger conditions than previously recognized. We present Monte Carlo experiments corroborating our theoretical results and showing that standard methods can yield misleading inference when our new, stronger regularity conditions are violated.

There are several phases that an emerging field goes through before it reaches maturity, and computational finance is no exception. There is usually a trigger for the birth of the field. In our case, new techniques such as neural networks, significant progress in computing technology, and the need for results that rely on more realistic assumptions inspired new researchers to revisit the traditional problems of finance, problems that have often been tackled by introducing simplifying assumptions in the past. The result has been a wealth of new approaches to these time-honored problems, with significant improvements in many cases.

We propose tests for individual and joint irrelevance of network inputs. Such tests can be used to determine whether an input or group of inputs "belong" in a particular model, thus permitting valid statistical inference based on estimated feedforward neural-network models. The approaches employ well-known statistical resampling techniques. We conduct a small Monte Carlo experiment showing that our tests have reasonable level and power behavior, and we apply our methods to examine whether there are predictable regularities in foreign exchange rates. We find that exchange rates do appear to contain information that is exploitable for enhanced point prediction, but the nature of the predictive relations evolves through time.

We obtain an improved approximation rate (in Sobolev norm) of r/sup -1/2-/spl alpha//(d+1)/ for a large class of single hidden layer feedforward artificial neural networks (ANN) with r hidden units and possibly nonsigmoid activation functions when the target function satisfies certain smoothness conditions. Here, d is the dimension of the domain of the target function, and /spl alpha//spl isin/(0, 1) is related to the smoothness of the activation function. When applying this class of ANNs to nonparametrically estimate (train) a general target function using the method of sieves, we obtain new root-mean-square convergence rates of Op([n/log(n)]/sup -/(1+2/spl alpha//(d+1))/[4(1+/spl alpha//(d+1))])=op(n/sup -1/4/) by letting the number of hidden units /spl tau//sub n/, increase appropriately with the sample size (number of training examples) n. These rates are valid for i.i.d. data as well as for uniform mixing and absolutely regular (/spl beta/-mixing) stationary time series data. In addition, the rates are fast enough to deliver root-n asymptotic normality for plug-in estimates of smooth functionals using general ANN sieve estimators. As interesting applications to nonlinear time series, we establish rates for ANN sieve estimators of four different multivariate target functions: a conditional mean, a conditional quantile, a joint density, and a conditional density. We also obtain root-n asymptotic normality results for semiparametric model coefficient and average derivative estimators.

May 01 1995 Bootstrapping Confidence Intervals for Clinical Input Variable Effects in a Network Trained to Identify the Presence of Acute Myocardial Infarction In Special Collection: CogNet William G. Baxt, William G. Baxt Department of Emergency Medicine and Medicine, University of California, San Diego Medical Center, San Diego, CA USA 92093 *Present address: Department of Emergency Medicine, Hospital of the University of Pennsylvania, 3400 Spruce Street, Philadelphia, PA 19104 USA. Search for other works by this author on: This Site Google Scholar Halbert White Halbert White Department of Economics and Institute for Neural Compilation, University of California, San Diego, San Diego, CA USA 92093 Search for other works by this author on: This Site Google Scholar Author and Article Information William G. Baxt Department of Emergency Medicine and Medicine, University of California, San Diego Medical Center, San Diego, CA USA 92093 Halbert White Department of Economics and Institute for Neural Compilation, University of California, San Diego, San Diego, CA USA 92093 *Present address: Department of Emergency Medicine, Hospital of the University of Pennsylvania, 3400 Spruce Street, Philadelphia, PA 19104 USA. Received: April 21 1994 Accepted: July 22 1994 Online ISSN: 1530-888X Print ISSN: 0899-7667 © 1995 Massachusetts Institute of Technology1995 Neural Computation (1995) 7 (3): 624–638. https://doi.org/10.1162/neco.1995.7.3.624 Article history Received: April 21 1994 Accepted: July 22 1994 Cite Icon Cite Permissions Share Icon Share Facebook Twitter LinkedIn Email Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Peer Review Search Site Citation William G. Baxt, Halbert White; Bootstrapping Confidence Intervals for Clinical Input Variable Effects in a Network Trained to Identify the Presence of Acute Myocardial Infarction. Neural Comput 1995; 7 (3): 624–638. doi: https://doi.org/10.1162/neco.1995.7.3.624 Download citation file: Ris (Zotero) Reference Manager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex toolbar search Search Dropdown Menu toolbar search search input Search input auto suggest filter your search All ContentAll JournalsNeural Computation Search Advanced Search This content is only available as a PDF. © 1995 Massachusetts Institute of Technology1995 Article PDF first page preview Close Modal You do not currently have access to this content.

In a recent paper, Poggio and Girosi (1990) proposed a class of neural networks obtained from the theory of regularization. Regularized networks are capable of approximating arbitrarily well any continuous function on a compactum. In this paper we consider in detail the learning problem for the one-dimensional case. We show that in the case of output data observed with noise, regularized networks are capable of learning and approximating (on compacta) elements of certain classes of Sobolev spaces, known as reproducing kernel Hilbert spaces (RKHS), at a nonparametric rate that optimally exploits the smoothness properties of the unknown mapping. In particular we show that the total squared error, given by the sum of the squared bias and the variance, will approach zero at a rate of n(-2m)/(2m+1), where m denotes the order of differentiability of the true unknown function. On the other hand, if the unknown mapping is a continuous function but does not belong to an RKHS, then there still exists a unique regularized solution, but this is no longer guaranteed to converge in mean square to a well-defined limit. Further, even if such a solution converges, the total squared error is bounded away from zero for all n sufficiently large.

We consider the problem of approximating a smooth target function and its derivatives by networks involving superpositions and translations of a fixed activation function. The approximation is with respect to the sup-norm and the rate is shown to be of order O(n/sup -1/2/); that is, the rate is independent of the dimension d. The results apply to neural and wavelet networks and extend the work of Barren(see Proc. 7th Yale Workshop on Adaptive and Learning Systems, May, 1992, and ibid., vol.39, p.930, 1993). The approach involves probabilistic methods based on central limit theorems for empirical processes indexed by classes of functions.>

Recently Barron (1993) has given rates for hidden layer feedforward networks with sigmoid activation functions approximating a class of functions satisfying a certain smoothness condition. These rates do not depend on the dimension of the input space. We extend Barron's results to feedforward networks with possibly nonsigmoid activation functions approximating mappings and their derivatives simultaneously. Our conditions are similar but not identical to Barron's, but we obtain the same rates of approximation, showing that the approximation error decreases at rates as fast as n −1/2 , where n is the number of hidden units. The dimension of the input space appears only in the constants of our bounds.

We give a rigorous analysis of the convergence properties of a backpropagation algorithm for recurrent networks containing either output or hidden layer recurrence. The conditions permit data generated by stochastic processes with considerable dependence. Restrictions are offered that may help assure convergence of the network parameters to a local optimum, as some simulations illustrate.

The authors derive a method for selecting exemplars for training a multilayer feedforward network architecture to estimate an unknown (deterministic) mapping from clean data, i.e., data measured either without error or with negligible error. The objective is to minimize the data requirement of learning. The authors choose a criterion for selecting training examples that works well in conjunction with the criterion used for learning, here, least squares. They proceed sequentially, selecting an example that, when added to the previous set of training examples and learned, maximizes the decrement of network squared error over the input space. When dealing with clean data and deterministic relationships, concise training sets that minimize the integrated squared bias (ISB) are desired. The ISB is used to derive a selection criterion for evaluating individual training examples, the DISB, that is maximized to select new exemplars. They conclude with graphical illustrations of the method, and demonstrate its use during network training. Experimental results indicate that training upon exemplars selected in this fashion can save computation in general purpose use as well.

It is shown that feedforward networks having bounded weights are not undesirable restricted, but are in fact universal approximators, provided that the hidden-layer activation function belongs to one of several suitable broad classes of functions: polygonal functions, certain piecewise polynomial functions, or a class of functions analytic on some open interval. These results are obtained by trading bounds on network weights for possible increments to network complexity, as indexed by the number of hidden nodes. The hidden-layer activation functions used include functions not admitted by previous universal approximation results, so the present results also extend the already broad class of activation functions for which universal approximation results are available. A theorem which establishes the approximate ability of these arbitrary mappings to learn when examples are generated by a stationary ergodic process is given

The premise of this article is that learning procedures used to train artificial neural networks are inherently statistical techniques. It follows that statistical theory can provide considerable insight into the properties, advantages, and disadvantages of different network learning methods. We review concepts and analytical results from the literatures of mathematical statistics, econometrics, systems identification, and optimization theory relevant to the analysis of learning in artificial neural networks. Because of the considerable variety of available learning procedures and necessary limitations of space, we cannot provide a comprehensive treatment. Our focus is primarily on learning procedures for feedforward networks. However, many of the concepts and issues arising in this framework are also quite broadly relevant to other network learning paradigms. In addition to providing useful insights, the material reviewed here suggests some potentially useful new training methods for artificial neural networks.