This work proposes Multi-task Meta Learning (MTML), integrating two learning paradigms Multi-Task Learning (MTL) and meta learning, to bring together the best of both worlds. In particular, it focuses simultaneous learning of multiple tasks, an element of MTL and promptly adapting to new tasks, a quality of meta learning. It is important to highlight that we focus on heterogeneous tasks, which are of distinct kind, in contrast to typically considered homogeneous tasks (e.g., if all tasks are classification or if all tasks are regression tasks). The fundamental idea is to train a multi-task model, such that when an unseen task is introduced, it can learn in fewer steps whilst offering a performance at least as good as conventional single task learning on the new task or inclusion within the MTL. By conducting various experiments, we demonstrate this paradigm on two datasets and four tasks: NYU-v2 and the taskonomy dataset for which we perform semantic segmentation, depth estimation, surface normal estimation, and edge detection. MTML achieves state-of-the-art results for three out of four tasks for the NYU-v2 dataset and two out of four for the taskonomy dataset. In the taskonomy dataset, it was discovered that many pseudo-labeled segmentation masks lacked classes that were expected to be present in the ground truth; however, our MTML approach was found to be effective in detecting these missing classes, delivering good qualitative results. While, quantitatively its performance was affected due to the presence of incorrect ground truth labels. The the source code for reproducibility can be found at https://github.com/ricupa/MTML-learn-how-to-adapt-to-unseen-tasks.

This article presents a new transfer learning method named group learning, that jointly aligns multiple domains (many-to-many) and an extension named fast alignment that aligns any further domain to previously aligned group of domains (many-to-one). The proposed group alignment algorithm (GALIA) is evaluated on brain-computer interface (BCI) data and optimal hyper-parameter values of the algorithm are studied for classification performance and computational cost. Six publicly available P300 databases comprising 333 sessions from 177 subjects are used. As compared to the conventional subject-specific train/test pipeline, both group learning and fast alignment significantly improve the classification accuracy except for the database with clinical subjects (average improvement: 2.12±1.88%). GALIA utilizes cyclic approximate joint diagonalization (AJD) to find a set of linear transformations, one for each domain, jointly aligning the feature vectors of all domains. Group learning achieves a many-to-many transfer learning without compromising the classification performance on non-clinical BCI data. Fast alignment further extends the group learning for any unseen domains, allowing a many-to-one transfer learning with the same properties. The former method creates a single machine learning model using data from previous subjects and/or sessions, whereas the latter exploits the trained model for an unseen domain requiring no further training of the classifier.

We consider the problem of removing gradient artifact from electroencephalogram (EEG) signal, registered during a functional magnetic resonance imaging (fMRI) acquisition, by calculating and utilizing the statistical properties of the artifacts. We propose a new approach to EEG data organization for extracting artifactual components using independent vector analysis. This new approach estimates the gradient artifact signal as a single component thus alleviating the need of using advanced order selection algorithm before back reconstruction of EEG data. Experimental results are compared with average artifact subtraction method on real EEG data collected concurrently with fMRI data.

Estimating likelihood or entropy rate is one of the key issues in many signal processing problems. Mutual information rate, which leads to the minimization of entropy rate, provides a natural cost for achieving blind source separation (BSS). In many complex-valued BSS applications, the latent sources are non-Gaussian, noncircular, and possess sample dependence. Consequently, an effective estimator of entropy rate that jointly considers all three properities of the sources is required. In this paper, we propose such an entropy rate estimator that assumes the sources are generated by invertible filters. With this new entropy rate estimator, we propose a complex entropy rate bound minimization algorithm. Simulation results show that the new method exploits all three properties effectively.

The recent extensions of independent component analysis (ICA) to exploit source dependence across multiple datasets, termed independent vector analysis (IVA), have thus far only considered two multivariate source distribution models: the Gaussian and a second-order uncorrelated Laplacian distribution. In this paper, we introduce the use of the Kotz distribution family as a more flexible source distribution model which exploits both second and higher-order statistics. The Cramér-Rao lower bound (CRLB) for IVA performance prediction is shown to be analogous to the bound for blind source separation (BSS). Lastly, we provide an analytic expression for the CRLB when the sources follow the multivariate power exponential (MPE) subclass of distributions within the Kotz family.

Recent advances in Joint Blind Source Separation (JBSS) extend the BSS framework to the simultaneous source separation of multiple datasets. In this paper we provide a comparative study of four such JBSS algorithms on human dual-electroencephalographic (dual-EEG) data. Appropriateness of second order JBSS is demonstrated for concurrent estimation of correlated sources in a multi-subject synchronous steady-state visually evoked potentials experiment. This approach gives a new starting point for the exploration of brain activities in a hyperscanning framework.

Co-training is a practical and powerful semi-supervised learning method. It yields high classification accuracy with a training data set containing only a small set of labeled data. Successful performance in co-training requires two important conditions on the features: diversity and sufficiency. In this paper, we propose a novel mutual information (MI) based approach inspired by the idea of dependent component analysis (DCA) to achieve feature splits that are maximally independent between-subsets (diversity) or within-subsets (sufficiency). We evaluate the relationship between the classification performance and the relative importance of the two conditions. Experimental results on actual tire data indicate that compared to diversity, sufficiency has a more significant impact on their classification accuracy. Further results show that co-training with feature splits obtained by the MI-based approach yields higher accuracy than supervised classification and significantly higher when using a small set of labeled training data.

Since in many blind source separation applications, latent sources are both non-Gaussian and have sample dependence, it is desirable to exploit both non-Gaussianity and sample dependency. In this paper, we use the Markov model to construct a general framework for the analysis and derivation of algorithms that take both properties into account. We also present two algorithms using two effective source priors. The first one is a multivariate generalized Gaussian distribution and the second is an autoregressive model driven by a generalized Gaussian distributed process. We derive the Cramér-Rao lower bound and demonstrate that the performance of the algorithms approach the lower bound especially when the underlying model matches the parametric model. We also demonstrate that a flexible semi-parametric approach exhibits very desirable performance.

When applied to functional magnetic resonance imaging (fMRI) data, independent vector analysis (IVA) provides superior performance in capturing subject variability within one group, as compared to the widely used group independent component analysis (ICA) approach. However, the effectiveness of IVA algorithms in preserving variability between different groups of subjects has not been studied yet, although it is of great interest in most fMRI studies, especially for identifying biomarkers for diagnosis of mental disorders. In this paper, we introduce a methodology that uses graph-theoretical analysis and statistical analysis for assessing the ability of IVA algorithms to capture group variability. We generate multi-subject fMRI-like datasets with increasing spatial variability for a selected component between two groups and compare a robust IVA algorithm to group ICA approach. Our experimental results show that IVA can successfully preserve group variability, indicating its potential in extracting biomarkers across groups of subjects in fMRI analysis.

In many machine learning applications, like Brain - Computer Interfaces (BCI), only high-dimensional noisy data are available rendering the discrimination task non-trivial. In this work, we focus on feature selection, more precisely on optimal electrode selection and weighting, as an efficient tool to improve the BCI classification procedure. The proposed framework closely integrates spatial feature selection and weighting within the classification task itself. Spatial weights are considered as hyper-parameters to be learned by a Support Vector Machine (SVM). The resulting spatially weighted SVM (sw-SVM) is then designed to maximize the margin between classes whilst minimizing the generalization error. Experimental studies on eight Error Related Potential (ErrP) data sets, illustrate the efficiency of the sw-SVM from a physiological and a machine learning point of view.

This brief deals with the problem of blind source separation (BSS) via independent component analysis (ICA). We prove that a linear combination of the separator output fourth-order marginal cumulants (kurtoses) is a valid contrast function for ICA under prewhitening if the weights have the same sign as the source kurtoses. If, in addition, the source kurtoses are different and so are the linear combination weights, the contrast eliminates the permutation ambiguity typical to ICA, as the estimated sources are sorted at the separator output according to their kurtosis values in the same order as the weights. If the weights equal the source kurtoses, the contrast is a cumulant matching criterion based on the maximum-likelihood principle. The contrast can be maximized by means of a cost-efficient Jacobi-type pairwise iteration. In the real-valued two-signal case, the asymptotic variance of the resulting Givens angle estimator is determined in closed form, leading to the contrast weights with optimal finite-sample performance. A fully blind solution can be implemented by computing the optimum weights from the initial source estimates obtained by a classical ICA stage. An experimental study validates the features of the proposed technique and shows its superior performance compared to related previous methods.

In this work we present an efficient coding scheme suitable for lossy image compression using a lattice vector quantizer (LVQ) based on statistically independent data projections. The independence of these components guarantees the optimality of the quantizer. However, this introduces an overload in coding since the projection matrix rendering the components independent needs to be transmitted to the decoder. This issue is tackled by modeling the data such that the projection matrix can be recovered at the decoder side based solely on the model parameters. The original data can thus be recovered based on a reduced descriptive data model and the statistically independent components. Results show that the coding of independent components with a lattice vector quantizer is highly efficient compared with scalar or simple LVQ. Furthermore, the independent data obtained by a model-based projection shows better efficiency without the penalizing coding load of the projection matrix.

A contrast function for independent component analysis (ICA) is presented incorporating the prior knowledge on the sub-Gaussian or super-Gaussian character of the sources as described by their kurtosis signs. The contrast is related to the maximum likelihood principle, reduces the permutation indeterminacy typical of ICA, and proves particularly useful in the direct extraction of a source signal with distinct kurtosis sign. In addition, its numerical maximization can be performed cost-effectively by a Jacobi-like pairwise iteration. Extensions to standardized cumulants of orders other than four are also given.