Electromagnetic source imaging (ESI) and independent component analysis (ICA) are two popular and apparently dissimilar frameworks for M/EEG analysis. This letter shows that the two frameworks can be linked by choosing biologically inspired source sparsity priors. We demonstrate that ESI carried out by the sparse Bayesian learning (SBL) algorithm yields source configurations composed of a few active regions that are also maximally independent from one another. In addition, we extend the standard SBL approach to source imaging in two important directions. First, we augment the generative model of M/EEG to include artifactual sources. Second, we modify SBL to allow for efficient model inversion with sequential data. We refer to this new algorithm as recursive SBL (RSBL), a source estimation filter with potential for online and offline imaging applications. We use simulated data to verify that RSBL can accurately estimate and demix cortical and artifactual sources under different noise conditions. Finally, we show that on real error-related EEG data, RSBL can yield single-trial source estimates in agreement with the experimental literature. Overall, by demonstrating that ESI can produce maximally independent sources while simultaneously localizing them in cortical space, we bridge the gap between the ESI and ICA frameworks for M/EEG analysis.

Phase-Amplitude Coupling (PAC) in electrophysiological signals refers to the transient interplay of activities in different frequency ranges, wherein phase in a low-frequency band and amplitude in a high-frequency band are in some way dependent. PAC phenomena have received increasing interest in neuroscience given the growing evidence of their apparent role in both normal and pathological brain processes. This interest has resulted in publication of a wave of methods for PAC estimation, each with its own advantages and drawbacks relative to others. Motivated by the widespread study of this phenomenon, most academic open source software environments for analyzing electrophysiological signals (e.g., Fieldtrip, Brainstorm, MNE) have implemented at least one method of PAC estimation. Here we describe an EEGLAB plug-in release, PACTools for MATLAB (The Mathworks, Inc.), that computes PAC in either continuous or event-related data using any of five methods for PAC estimation: Mean Vector Length Modulation Index, Kullback- Leibler Modulation Index, Phase-Locking Value, General Linear Model Modulation Index and Mutual Information PAC (MIPAC). PACTools uses parallelized code for efficient performance and offers built-in direct access to online high-performance computing resources made freely available for nonprofit research by the Neurosciences Gateway (nsgportal.org). PACTools features intuitive graphic user interfaces and equivalent command line calls that make its use straightforward and seamless in the EEGLAB environment. We discuss toolbox implementation, architecture, dependencies, and the implemented methods of PAC computation and visualization.

The power budget for embedded hardware implementations of Deep Learning algorithms can be extremely tight. To address implementation challenges in such domains, new design paradigms, like Approximate Computing, have drawn significant attention. Approximate Computing exploits the innate error-resilience of Deep Learning algorithms, a property that makes them amenable for deployment on low-power computing platforms. This paper describes an Approximate Computing design methodology, AX-DBN, for an architecture belonging to the class of stochastic Deep Learning algorithms known as Deep Belief Networks (DBNs). Specifically, we consider procedures for efficiently implementing the Discriminative Deep Belief Network (DDBN), a stochastic neural network which is used for classification tasks, extending Approximation Computing from the analysis of deterministic to stochastic neural networks. For the purpose of optimizing the DDBN for hardware implementations, we explore the use of: (a) Limited precision of neurons and functional approximations of activation functions; (b) Criticality analysis to identify the nodes in the network which can operate at reduced precision while allowing the network to maintain target accuracy levels; and (c) A greedy search methodology with incremental retraining to determine the optimal reduction in precision for all neurons to maximize power savings. Using the AX-DBN methodology proposed in this paper, we present experimental results across several network architectures that show significant power savings under a user-specified accuracy loss constraint with respect to ideal full precision implementations.

Restricted Boltzmann Machines and Deep Belief Networks have been successfully used in probabilistic generative model applications such as image occlusion removal, pattern completion and motion synthesis. Generative inference in such algorithms can be performed very efficiently on hardware using a Markov Chain Monte Carlo procedure called Gibbs sampling, where stochastic samples are drawn from noisy integrate and fire neuron s implemented on neuromorphic substrates. Currently, no satisfactory metrics exist for evaluating the generative performance of such algorithms implemented on high-dimensional data for neuromorphic platforms. This paper demonstrates the application of nonparametric goodness-of-fit testing to both quantify the generative performance as well as provide decision-directed criteria for choosing the parameters of the neuromorphic Gibbs sampler and optimizing usage of hardware resources used during sampling.

Restricted Boltzmann Machines and Deep Belief Networks have been successfully used in a wide variety of applications including image classification and speech recognition. Inference and learning in these algorithms uses a Markov Chain Monte Carlo procedure called Gibbs sampling. A sigmoidal function forms the kernel of this sampler which can be realized from the firing statistics of noisy integrate-and-fire neurons on a neuromorphic VLSI substrate. This paper demonstrates such an implementation on an array of digital spiking neurons with stochastic leak and threshold properties for inference tasks and presents some key performance metrics for such a hardware-based sampler in both the generative and discriminative contexts.

We analyze the jointly-sparse signal recovery problem in the regime where the number of sources k is larger than the number of measurements M. We show that the support set of sources can still be recovered with sparse Bayesian learning (M-SBL) even if k ≥ M. We provide sufficient conditions on the dictionary and sources which theoretically guarantee support set recovery in the noiseless case of M-SBL. We validate our sufficient conditions with experiments and also demonstrate that M-SBL outperforms M-CoSaMP, the algorithm recently used to localize more sources than sensors. Finally, we experimentally show robustness of the approach in the presence of noise.

We propose a method to solve the multiple measurement vector (MMV) sparse signal recovery problem in a robust manner when data contains outlier points which do not fit the shared sparsity structure otherwise contained in the data. This scenario occurs frequently in the applications of MMV models due to only partially known source dynamics. The algorithm we propose is a modification of MMV-based sparse bayesian learning (M-SBL) by incorporating the idea of least trimmed squares (LTS), which has previously been developed for robust linear regression. Experiments show a significant performance improvement over the conventional M-SBL under different outlier ratios and amplitudes.

Restricted Boltzmann Machines (RBMs) and Deep Belief Networks (DBNs) have been demonstrated to perform efficiently on a variety of applications, such as dimensionality reduction and classification. Implementation of RBMs on neuromorphic platforms, which emulate large-scale networks of spiking neurons, has significant advantages from concurrency and low-power perspectives. This work outlines a neuromorphic adaptation of the RBM, which uses a recently proposed neural sampling algorithm (Buesing et al. 2011), and examines its algorithmic efficiency. Results show the feasibility of such alterations, which will serve as a guide for future implementation of such algorithms in neuromorphic very large scale integration (VLSI) platforms.

Because of the increasing portability and wearability of noninvasive electrophysiological systems that record and process electrical signals from the human brain, automated systems for assessing changes in user cognitive state, intent, and response to events are of increasing interest. Brain-computer interface (BCI) systems can make use of such knowledge to deliver relevant feedback to the user or to an observer, or within a human-machine system to increase safety and enhance overall performance. Building robust and useful BCI models from accumulated biological knowledge and available data is a major challenge, as are technical problems associated with incorporating multimodal physiological, behavioral, and contextual data that may in the future be increasingly ubiquitous. While performance of current BCI modeling methods is slowly increasing, current performance levels do not yet support widespread uses. Here we discuss the current neuroscientific questions and data processing challenges facing BCI designers and outline some promising current and future directions to address them.

We propose a model for the density of cross-spectral coefficients using normal variance mean mixtures. We show that this model density generalizes the corresponding marginal density of the complex Wishart distribution for the cross-spectral density. The maximum likelihood estimate of parameters in the distribution is derived, and examples are given from alpha brain wave sources in separated EEG data.

We derive an asymptotic Newton algorithm for quasi-maximum likelihood estimation of the ICA mixture model, using the ordinary gradient and Hessian. The probabilistic mixture framework yields an algorithm that can accommodate non-stationary environments and arbitrary source densities. We prove asymptotic stability when the source models match the true sources. An example application to EEC segmentation is given.

Determining adversarial intent is important on a battlefield. In this paper, we propose a method for intent analysis using evolutionary computation. The proposed approach defines the model as an optimization problem then gives an algorithm for determining the parameters of the model in a Valuated State Spacereg domain. Example experiments using simulation data are discussed

A variety of Bayesian methods have recently been introduced for finding sparse representations from overcomplete dictionaries of candidate features. These methods often capitalize on latent structure inherent in sparse distributions to perform standard MAP estimation, variational Bayes, approximation using convex duality, or evidence maximization. Despite their reliance on sparsity-inducing priors however, these approaches may or may not actually lead to sparse representations in practice, and so it is a challenging task to determine which algorithm and sparse prior is appropriate. Rather than justifying prior selections and modelling assumptions based on the credibility of the full Bayesian model as is commonly done, this paper bases evaluations on the actual cost functions that emerge from each method. Two minimal conditions are postulated that ideally any sparse learning objective should satisfy. Out of all possible cost functions that can be obtained from the methods described above using (virtually) any sparse prior, a unique function is derived that satisfies these conditions. Both sparse Bayesian learning (SBL) and basis pursuit (BP) are special cases. Later, all methods are shown to be performing MAP estimation using potentially non-factorable implicit priors, which suggests new sparse learning cost functions.

We present a hierarchical architecture and learning algorithm for visual recognition and other visual inference tasks such as imagination, reconstruction of occluded images, and expectation-driven segmentation. Using properties of biological vision for guidance, we posit a stochastic generative world model and from it develop a simplified world model (SWM) based on a tractable variational approximation that is designed to enforce sparse coding. Recent developments in computational methods for learning overcomplete representations (Lewicki & Sejnowski, 2000; Teh, Welling, Osindero, & Hinton, 2003) suggest that overcompleteness can be useful for visual tasks, and we use an overcomplete dictionary learning algorithm (Kreutz-Delgado, et al., 2003) as a preprocessing stage to produce accurate, sparse codings of images. Inference is performed by constructing a dynamic multilayer network with feedforward, feedback, and lateral connections, which is trained to approximate the SWM. Learning is done with a variant of the back-propagation-through-time algorithm, which encourages convergence to desired states within a fixed number of iterations. Vision tasks require large networks, and to make learning efficient, we take advantage of the sparsity of each layer to update only a small subset of elements in a large weight matrix at each iteration. Experiments on a set of rotated objects demonstrate various types of visual inference and show that increasing the degree of overcompleteness improves recognition performance in difficult scenes with occluded objects in clutter.

We consider criteria for variational representations of non-Gaussian latent variables, and derive variational EM algorithms in general form. We establish a general equivalence among convex bounding methods, evidence based methods, and ensemble learning/Variational Bayes methods, which has previously been demonstrated only for particular cases.

We compare machine learning methods applied to a difficult real-world problem: predicting computer hard-drive failure using attributes monitored internally by individual drives. The problem is one of detecting rare events in a time series of noisy and nonparametrically-distributed data. We develop a new algorithm based on the multiple-instance learning framework and the naive Bayesian classifier (mi-NB) which is specifically designed for the low false-alarm case, and is shown to have promising performance. Other methods compared are support vector machines (SVMs), unsupervised clustering, and non-parametric statistical tests (rank-sum and reverse arrangements). The failure-prediction performance of the SVM, rank-sum and mi-NB algorithm is considerably better than the threshold method currently implemented in drives, while maintaining low false alarm rates. Our results suggest that nonparametric statistical tests should be considered for learning problems involving detecting rare events in time series data. An appendix details the calculation of rank-sum significance probabilities in the case of discrete, tied observations, and we give new recommendations about when the exact calculation should be used instead of the commonly-used normal approximation. These normal approximations may be particularly inaccurate for rare event problems like hard drive failures.

Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial "25 words or less"), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error).

A sparse representation of a signal, i.e., a representation using a small number of vectors chosen from a dictionary of vectors, is highly desirable in many applications. Here, we extend the backward elimination sparse representation algorithm presented by Reeves (see ibid., vol.6, p.266-68, Oct. 1999) to allow for an overcomplete dictionary and develop recursions for its implementation. In the overcomplete case, the representation error cannot be used as a general criterion for elimination of a dictionary vector and other criteria must be considered. Simulation on a test-case dictionary shows that the performance of the proposed algorithm can improve upon that of forward selection methods.

Improved methods are proposed for disk-drive failure prediction. The SMART (self monitoring and reporting technology) failure prediction system is currently implemented in disk-drives. Its purpose is to predict the near-term failure of an individual hard disk-drive, and issue a backup warning to prevent data loss. Two experimental tests of SMART show only moderate accuracy at low false-alarm rates. (A rate of 0.2% of total drives per year implies that 20% of drive returns would be good drives, relative to /spl ap/1% annual failure rate of drives). This requirement for very low false-alarm rates is well known in medical diagnostic tests for rare diseases, and methodology used there suggests ways to improve SMART. Two improved SMART algorithms are proposed. They use the SMART internal drive attribute measurements in present drives. The present warning-algorithm based on maximum error thresholds is replaced by distribution-free statistical hypothesis tests. These improved algorithms are computationally simple enough to be implemented in drive microprocessor firmware code. They require only integer sort operations to put several hundred attribute values in rank order. Some tens of these ranks are added up and the SMART warning is issued if the sum exceeds a prestored limit. These new algorithms were tested on 3744 drives of 2 models. They gave 3-4 times higher correct prediction accuracy than error thresholds on will-fail drives, at 0.2% false-alarm rate. The highest accuracies achievable are modest (40%-60%). Care was taken to test will-fail drive prediction accuracy on data independent of the algorithm design data. Additional work is needed to verify and apply these algorithms in actual drive design. They can also be useful in drive failure analysis engineering. It might be possible to screen drives in manufacturing using SMART attributes. Marginal drives might be detected before substantial final test time is invested in them, thereby decreasing manufacturing cost, and possibly decreasing overall field failure rates.

Sparse solutions to the linear inverse problem Ax=Y and the determination of an environmentally adapted overcomplete dictionary (the columns of A) depend upon the choice of a "regularizing function" d(x) in several previously proposed procedures. We discuss the interpretation of d(x) within a Bayesian framework, and the desirable properties that "good" (i.e., sparsity ensuring) regularizing functions, d(x) might have. These properties are: Schur-concavity (d(x) is consistent with majorization); concavity (d(x) has sparse minima); parameterizability (d(x) is drawn from a large, parameterizable class); and factorizability of the gradient of d(x) in a certain manner. The last property (which naturally leads one to consider separable regularizing functions) allows d(x) to be efficiently minimized subject to Ax=Y using an affine scaling transformation (AST)-like algorithm "adapted" to the choice of d(x). A Bayesian framework allows the algorithm to be interpreted as an independent component analysis (ICA) procedure.

A novel approach for estimating 3D lip posture from monocular video se-quences is presented. The lips are modeled as a four body closed kinematic chain with each body possessing translational, rotational and prismatic (to ac-count for deformations) degrees of freedom. Geometric constraints relating these bodies to each other, and to the face as a whole, are used to constrain the space of possible lip postures recovered from each image. These constraints are used with the recently proposed Expectation Constrained Maximization algorithm to estimate the lip posture from video frames that have been pro-cessed (using a color segmentation algorithm described here) to identify lip regions. 1

A general framework based on majorization, Schur-concavity, and concavity is given that facilitates the analysis of algorithm performance and clarifies the relationships between existing proposed diversity measures useful for best basis selection. Admissible sparsity measures are given by the Schur-concave functions, which are the class of functions consistent with the partial ordering on vectors known as majorization. Concave functions form an important subclass of the Schur-concave functions which attain their minima at sparse solutions to the basis selection problem. Based on a particular functional factorization of the gradient, we give a general affine scaling optimization algorithm that converges to a sparse solution for measures chosen from within this subclass.

The problem of sensorimotor control is underdetermined due to excess (or "redundant") degrees of freedom when there are more joint variables than the minimum needed for positioning an end-effector. A method is presented for solving the nonlinear inverse kinematics problem for a redundant manipulator by learning a natural parameterization of the inverse solution manifolds with self-organizing maps. The parameterization approximates the topological structure of the joint space, which is that of a fiber bundle. The fibers represent the "self-motion manifolds" along which the manipulator can change configuration while keeping the end-effector at a fixed location. The method is demonstrated for the case of the redundant planar manipulator. Data samples along the self-motion manifolds are selected from a large set of measured input-output data. This is done by taking points in the joint space corresponding to end-effector locations near "query points", which define small neighborhoods in the end-effector work space. Self-organizing maps are used to construct an approximate parameterization of each manifold which is consistent for all of the query points. The resulting parameterization is used to augment the overall kinematics map so that it is locally invertible. Joint-angle and end-effector position data, along with the learned parameterizations, are used to train neural networks to approximate direct inverse functions.

Dynamic analysis and control of robotic systems with more than a few degrees of freedom, taking into account the full dynamics of the system, is a difficult task. Thus, quite often assumptions are made to simplify the dynamics of the system. In this paper we investigate the validity of certain assumptions that result in approximate dynamic decoupling of parts of a closed chain system. Specifically, we consider robots with a star topology structure and investigate the dynamic interaction of the branches of such a robot when one or more branches are in contact or collision with the environment (thereby forming closed chains), comparing the exact and approximate dynamic models. Examples of star topology systems, where approximate dynamic decoupling can be applied, include multilegged (or bipedal) walking robots and multifingered hands.

An operator form of a scale-invariant singularity-robust generalized inverse of the Jacobian, J /spl epsiv/ R/sup mxn/, for a serial-chain n-link manipulator is developed. This allows the direct computation of a solution to the resolved-rate problem /spl beta/(0)=J/spl theta//spl dot/ via an O(n) recursive algorithm. There is no restriction on the number of links or the dimension, m/spl les/6, of the end-effector velocity vector. The generalized inverse requires knowledge of the mass and inertia properties of the manipulator links. However its use is central to the problem of solving closed chain forward dynamics and computing velocity changes due to inelastic collision between a manipulator end-effector and the environment.

Many systems of interest in robotics have a star topology structure (multiple serial chains radiating from a common mobile, non-articulated base). In this paper the author utilize the special structure of star topology-based robots to develop algorithms useful for simulating such systems. The algorithm can handle a variety of walking, grasping, and contact situations. As a specific example, a multipedal walking system is analyzed.>

The workspace of a redundant manipulator can be partitioned into a finite number of invertible subsets such that the inverse image of each subset has trivial fiber bundle topology, with the manifolds of manipulator self-motion as the fibers. As a consequence we show that a canonical representation of the self-motion manifolds is possible over these well-defined subsets. The canonical representation parameterizes a family of inverse functions over the work space subsets. This parameterization can be used to approximate a continuous inverse kinematic function over each workspace subset. Setting appropriate valves of the parameters yields particular inverse functions which provide access to all solution branches and to the full range of the redundant degrees of freedom in order to satisfy any side constraints which may be imposed during operation.>

This paper advances two linear operator factorizations of the manipulator mass matrix. Embedded in the factorizations are many of the techniques that are regarded as very efficient computational solutions to inverse and forward dynamics problems. The operator factorizations provide a high-level architectural understanding of the mass matrix and its inverse, which is not visible in the detailed algorithms. They also lead to an approach to the development of computer programs or the organisation of complexity in robot dynamics.>

Operator forms of the inverse and Moore-Penrose pseudoinverse of the Jacobian for a nonsingular n-link serial spatial manipulator are developed. It is shown that the Jacobian pseudoinverse operation that maps an end-effector velocity to a corresponding joint-space velocity can be performed using an O(n) recursive algorithm involving one 6*6 matrix inversion. It is also shown that an alternative recursive algorithm that produces a kinematic singularity measure as a byproduct can be used to perform the Jacobian inverse operation without the need for a matrix inversion.>

A spatial operator algebra approach to modeling and analysis of multibody robotic systems is discussed. It is shown that this modeling and analysis method can be used to develop O(n) recursive algorithms that compute the operational space mass matrix and the operational space coriolis/centrifugal and gravity terms of an n-link serial chain manipulator. It is also shown that, taken together, these algorithms enable an O(n) recursive implementation of operational space control.>

A kinematic analysis of anthropomorphic 7 DOF serial link spatial manipulators with revolute joints is presented. To determine joint angles uniquely for a given end-effector position and orientation, the redundancy is parameterized by a scalar variable which corresponds to the angle between the arm plane and a reference plane. The forward kinematic mappings from joint-space to end-effector coordinates and arm angle and the augmented Jacobian matrix which gives end-effector and arm angle rates as functions of joint rates are given. Conditions under which the augmented Jacobian becomes singular are given and are shown to correspond to the arm being either at a kinematically singular configuration or at a nonsingular configuration for which the arm angle ceases to parameterize the redundancy.>

The spatial operator algebra developed by the authors for modeling control and trajectory design of manipulation is discussed, with emphasis on the analytical formulation of the operator algebra and its simple implementation in the Ada programming language. The elements of this algebra are linear operators whose domain and range spaces consist of forces, moments, velocities, and accelerations. The effect of these operators is equivalent to a spatial recursion along the span of the manipulator. Inversion of operators can be efficiently obtained using techniques of recursive filtering and smoothing. The operator algebra provides a high-level framework for describing the dynamic and kinematic behavior of a manipulator and control and trajectory design algorithms. The interpretation of expressions within the algebraic framework leads to enhanced conceptual and physical understanding of manipulator dynamics and kinematics. Furthermore, implementable recursive algorithms can be immediately derived from the abstract operator expressions by inspection. Thus, the transition from an abstract problem formulation and solution to the detailed mechanization of specific algorithm is greatly simplified.>

The authors address the problem of motion and force control of multiple robot arms manipulating an object. A general control paradigm that decouples the motion and force control problems is introduced. For motion control, different control strategies are constructed on the basis of control input variables. There are three natural choices: joint torques, arm tip force vectors, and the acceleration of a generalized coordinate. The first choice allows relatively model-independent control by exploiting the Hamiltonian structure of the open-loop system. The latter two require the full model information but produce simpler control design problems. The motion control determines the joint torque only to within a manifold, owing to the multiple-arm kinematic constraint. To resolve the nonuniqueness of the joint torques, two methods are introduced. If the arm and object models are available, the allocation of the desired end-effector control force to the joint actuators can be optimized. The other possibility is to control the internal force about some set point. Effective force regulation can be achieved with little model information.>