The statistical dependencies which independent component analysis (ICA) cannot remove often provide rich information beyond the ICA components. It would be very useful to estimate the dependency structure from data. However, most models have concentrated on higher-order correlations such as energy correlations, neglecting linear correlations. Linear correlations might be a strong and informative form of a dependency for some real data sets, but they are usually completely removed by ICA and related methods, and not analyzed at all. In this paper, we propose a probabilistic model of non-Gaussian components which are allowed to have both linear and energy correlations. The dependency structure of the components is explicitly parametrized by a parameter matrix, which defines an undirected graphical model over the latent components. Furthermore, the estimation of the parameter matrix is shown to be particularly simple because using score matching, the objective function is a quadratic form. Using artificial data, we demonstrate that the proposed method is able to estimate non-Gaussian components and their dependency structures, as it is designed to do. When applied to natural images and outputs of simulated complex cells in the primary visual cortex, novel dependencies between the estimated features are discovered.

Data representation methods related to ICA and sparse coding have successfully been used to model neural representation. However, they are highly abstract methods, and the neural encoding does not correspond to a detailed neuron model. This limits their power to provide deeper insight into the sensory systems on a cellular level. We propose here data representation where the encoding happens with a spiking neuron. The data representation problem is formulated as an optimization problem: Encode the input so that it can be decoded from the spike train, and optionally, so that energy consumption is minimized. The optimization leads to a learning rule for the encoder and decoder which features synergistic interaction: The decoder provides feedback affecting the plasticity of the encoder while the encoder provides optimal learning data for the decoder.