The canonical polyadic decomposition (CPD) can be used as a low-rank representation of tensors appearing in various applications. However, obtaining a CPD can lead to a difficult optimization problem when, e.g., the rank is higher than some of the tensor dimensions. During the optimization, a degenerate subtensor can then be formed, i.e., an ill-conditioned tensor that tries to approximate a tensor of higher rank. This can lead to regions in the optimization landscape of very slow convergence, informally known as ‘swamps’. An efficient optimization framework is proposed to detect the degenerate terms and let them become complex using a parameterization of the factor vectors in their real and imaginary parts. This gives more freedom to the decomposition during convergence. The eventual solution remains real-valued because of the uniqueness of the CPD. We illustrate the approach for difficult tensors that appear in underdetermined independent component analysis, showing significantly improved accuracy and computational efficiency.

Convolutional Neural Networks (CNNs) with U-shaped architectures have dominated medical image segmentation, which is crucial for various clinical purposes. However, the inherent locality of convolution makes CNNs fail to fully exploit global context, essential for better recognition of some structures, e.g., brain lesions. Transformers have recently proven promising performance on vision tasks, including semantic segmentation, mainly due to their capability of modeling long-range dependencies. Nevertheless, the quadratic complexity of attention makes existing Transformer-based models use self-attention layers only after somehow reducing the image resolution, which limits the ability to capture global contexts present at higher resolutions. Therefore, this work introduces a family of models, dubbed Factorizer, which leverages the power of low-rank matrix factorization for constructing an end-to-end segmentation model. Specifically, we propose a linearly scalable approach to context modeling, formulating Nonnegative Matrix Factorization (NMF) as a differentiable layer integrated into a U-shaped architecture. The shifted window technique is also utilized in combination with NMF to effectively aggregate local information. Factorizers compete favorably with CNNs and Transformers in terms of accuracy, scalability, and interpretability, achieving state-of-the-art results on the BraTS dataset for brain tumor segmentation and ISLES'22 dataset for stroke lesion segmentation. Highly meaningful NMF components give an additional interpretability advantage to Factorizers over CNNs and Transformers. Moreover, our ablation studies reveal a distinctive feature of Factorizers that enables a significant speed-up in inference for a trained Factorizer without any extra steps and without sacrificing much accuracy. The code and models are publicly available at https://github.com/pashtari/factorizer.

The Canonical Polyadic Decomposition (CPD) is a fundamental tensor decomposition which has widespread use in signal processing due to its ability to extract component information. A popular for algorithm CPD is the generalized eigenvalue decomposition (GEVD) which is based on the generalized eigenvectors of a subpencil of a tensor. GEVD plays an important role in applications as it provides strong algebraic initializations for optimization routines for CPD computation. In fact, using GEVD initializations can improve final accuracy and reduce computation time. However, despite GEVD’s success, the algorithm underperforms in some settings and exhibits pencil-based instability.We present a recursive generalized eigenspace decomposition (GESD) for CPD computation. Rather using one sub-pencil, GESD combines generalized eigenspace information from many subpencils to compute a CPD. GESD is more accurate than GEVD and thereby improves the reliability of the components extracted by CPD. We also give a Cramér-Rao based analysis for the accuracy of GESD.

The canonical decomposition (CPD) is a fundamental tensor decomposition which expresses a tensor as a sum of rank one tensors. In stark contrast to the matrix case, with light assumptions, the CPD of a low rank tensor is (essentially) unique. The essential uniqueness of CPD makes this decomposition a powerful tool in many applications as it allows for extraction of component information from a signal of interest. One popular algorithm for algebraic computation of a CPD is the generalized eigenvalue decomposition (GEVD) which selects a matrix subpencil of a tensor, then computes the generalized eigenvectors of the pencil. In this article, we present a simplification of GEVD which improves the accuracy of the algorithm. Surprisingly, the generalized eigenvector computation in GEVD is in fact unnecessary and can be replaced by a QZ decomposition which factors a pair of matrices as a product of unitary and upper triangular matrices. Computing a QZ decomposition is a standard first step when computing generalized eigenvectors, so our algorithm can been seen as a direct simplification of GEVD.

The Canonical Polyadic Decomposition (CPD) is one of the most basic tensor models used in signal processing and machine learning. Despite its wide applicability, identifiability conditions and algorithms for CPD in cases where the tensor is incomplete are lagging behind its practical use. We first present a tensor-based framework for bilinear factorizations subject to monomial constraints, called monomial factorizations. Next, we explain that the CPD of a tensor that has missing fibers can be interpreted as a monomial factorization problem. Finally, using the monomial factorization interpretation, we show that CPD recovery conditions can be obtained that only rely on the observed fibers of the tensor.

Tensor updating methods enable tensor decompositions to adapt quickly when new data is added to the tensor. At present, updating methods for the canonical polyadic decomposition (CPD) give every tensor entry the same weight. In practice, however, data quality or relative importance might differ between tensor entries, which warrants the use of more general weighting schemes. In this paper, an NLS updating method is developed for the CPD that uses a weighted least squares (WLS) approach with a low-rank weight tensor. This weight tensor itself can also be updated to allow dynamic weighting schemes. By exploiting the CPD structure of both the data and weight tensors, the algorithm obtains better accuracy than the unweighted updating methods, while being more time- and memory efficient than batch WLS methods.

Higher order tensors and their decompositions are well-known tools in signal processing. More specifically, tensors admitting decompositions with structured factor matrices arise in various applications, such as telecommunications or convolutive independent component analysis. These applications motivate the development of efficient algorithms for structured tensor decompositions. In this letter, we develop a method for canonical polyadic decompositions (CPDs) with block-circulant factor matrices by extending a method for circulant factors. Block-circulant factor matrices arise in the identification of homogeneous Wiener-Hammerstein models. By transforming the data tensor to the frequency domain, the available structure is exploited using simple element-wise divisions. This approach reformulates the structured CPD as an unstructured CPD of lower rank. The resulting CPD of a possibly incomplete tensor can then be solved algebraically or using optimization-based methods.

Independent component analysis (ICA) research has been driven by various applications in biomedical signal separation, telecommunications, speech analysis, and more. One particular class of algorithms for instantaneous ICA uses tensors, which have useful properties. In an attempt to port these properties to convolutive methods, we zoom in on an existing method that uses second-order statistics. By pointing out links in the literature, we show that this method is in fact a typical tensor-based method, even though this was not recognized by the authors at the time. The existing method mentioned above can be interpreted as a tensorization step followed by a deconvolution step. However, as sometimes done in literature, one may consider using the opposite approach; starting with a deconvolution step and then tensorizing the remaining instantaneous mixture. Because subspace-based deconvolution can be slow, we propose a fast variant which uses only partial information. We then use this variant to compare the approach starting with tensorization and the one starting with deconvolution.

Nonnegative matrix factorization is a key tool in many data analysis applications such as feature extraction, compression, and noise filtering. Many existing algorithms impose additional constraints to take into account prior knowledge and to improve the physical interpretation. This letter proposes a novel algorithm for nonnegative matrix factorization, in which the factors are modeled by nonnegative polynomials. Using a parametric representation of finite-interval nonnegative polynomials, we obtain an optimization problem without external nonnegativity constraints, which can be solved using conventional quasi-Newton or nonlinear least-squares methods. The polynomial model guarantees smooth solutions and may realize a noise reduction. A dedicated orthogonal compression enables a significant reduction of the matrix dimensions, without sacrificing accuracy. The overall approach scales well to large matrices. The approach is illustrated with applications in hyperspectral imaging and chemical shift brain imaging.

Magnetic resonance spectroscopic imaging (MRSI) reveals chemical information that characterizes different tissue types in brain tumors. Blind source separation techniques are used to extract the tissue-specific profiles and their corresponding distribution from the MRSI data. We focus on automatic detection of the tumor, necrotic and normal brain tissue types by constructing a 3D MRSI tensor from in vivo 2D-MRSI data of individual glioma patients. Nonnegative canonical polyadic decomposition (NCPD) is applied to the MRSI tensor to differentiate various tissue types. An in vivo study shows that NCPD has better performance in identifying tumor and necrotic tissue type in glioma patients compared to previous matrix-based decompositions, such as nonnegative matrix factorization and hierarchical nonnegative matrix factorization.

Coupled decompositions of multiple tensors are fundamental tools for multi-set data fusion. In this paper, we introduce a coupled version of the rank-(Lm, Ln, •) block term decomposition (BTD), applicable to joint independent subspace analysis. We propose two algorithms for its computation based on a coupled block simultaneous generalized Schur decomposition scheme. Numerical results are given to show the performance of the proposed algorithms.

A novel deterministic blind signal separation technique for separating signals into rational functions is proposed, applicable in various situations. This new technique is based on a tensorization of the observed data matrix into a set of Löwner matrices. The obtained tensor can then be decomposed with a block tensor decomposition, resulting in a unique separation into rational functions under mild conditions. This approach provides a viable alternative to independent component analysis (ICA) in cases where the independence assumption is not valid or where the sources can be modeled well by rational functions, such as frequency spectra. In contrast to ICA, this technique is deterministic and not based on statistics, and therefore works well even with a small number of samples.

Blind deterministic receivers for DS-CDMA systems based on the PARAFAC model have been proposed in several papers since their conception in. In many cases, the transmitted signals can be considered uncorrelated. Hence, we develop PARAFAC receivers for uncorrelated signals. We introduce several numerical algorithms for orthogonality constrained PARAFAC on which receivers for uncorrelated signals can be based. Simulation results show an increase in performance when the PARAFAC receiver takes the uncorrelatedness of the transmitted signals into account.

Matrix decompositions have always been at the heart of signal, circuit and system theory. In particular, the singular value decomposition (SVD) has been an important tool. There is currently a shift of paradigm in the algebraic foundations of these fields. Quite recently, nonnegative matrix factorization (NMF) has been shown to outperform SVD at a number of tasks. Increasing research efforts are spent on the study and application of decompositions of higher-order tensors or multi-way arrays. This paper is a partial survey on tensor generalizations of the SVD and their applications. We also touch on nonnegative tensor factorizations.

Newton's method for solving the matrix equation F(X) identical to AX-XX(T) AX = 0 runs up against the fact that its zeros are not isolated. This is due to a symmetry of F by the action of the orthogonal group. We show how differential-geometric techniques can be exploited to remove this symmetry and obtain a "geometric" Newton algorithm that finds the zeros of F. The geometric Newton method does not suffer from the degeneracy issue that stands in the way of the original Newton method.

We propose a new algorithm to impose independence constraints based on second order statistics in one mode of the Parallel Factor Analysis / Canonical Decomposition, also known as the CP model, and show with simulations that it outperforms in some cases the ordinary CP model.

This paper presents a new subspace-based technique for automatic detection of the number of exponentially damped sinusoids. It consists in studying the shift-invariance of the dominant subspace of the Hankel data matrix. No threshold setting and no penalization terms are necessary. This model-based method, easy to implement, can be plugged into most subspace-based harmonic retrieval algorithms.

In this paper, we present a technique for the blind separation of DS-CDMA signals received on an antenna array, for a multi-path propagation scenario with inter symbol interference. Our method relies on a new third-order tensor decomposition, which is a generalization of the parallel factor model. We start with the observation that the temporal, spatial and spectral diversities give a third-order tensor structure to the received data. This tensor is then decomposed in a sum of contributions by means of an alternating least squares algorithm, where each contribution fully characterizes one user

We present a subspace-based scheme for the estimation of the poles (angular-frequencies and damping-factors) of a sum of damped and delayed sinusoids. In our model each component is supported over a different time frame, depending on the delay parameter. Classical subspace based methods are not suited to handle signals with varying time-support. In this contribution, we propose a solution based on the best rank-(R/sub 1/, R/sub 2/, R/sub 3/) approximation of a partially structured Hankel tensor on which the data are mapped. We show, by means of an example, that our approach outperforms the current tensor and matrix-based approaches in terms of the accuracy of the damping parameter estimates.

We address the problem of blind identification of a convolutive Multiple-Input Multiple-Output (MIMO) system with more inputs than outputs, and in particular, the 3-input 2-output case. We assume that the inputs are temporally white, non-Gaussian distributed, spatially independent and that the system impulse response can be complex. In this paper, we look at the problem in the frequency domain, where, for each frequency we construct two tensors based on cross-polyspectra of the output. These tensors lead to the system frequency response within frequency dependent scaling and permutation ambiguities. We propose ways to resolve these ambiguities, and show that it is possible to obtain the system response within a scalar and a linear phase.

This paper deals with the extraction of the antepartum foetal electrocardiogram (ECG) from multilead cutaneous potential recordings, and the modelling of the transfer to the electrodes. We give an overview of a class of algebraic approaches, based on variants of the singular value decomposition (SVD): the concept, pros and cons of techniques relying on the ordinary SVD, quotient SVD and multilinear SVD are discussed.

This paper deals with the blind identification of multiple-input multiple-output finite impulse response filters. We limit ourselves to the case of 2 outputs and 2 inputs. After a classical prewhitening, the remaining problem is the blind identification of a paraunitary filter. For this task, we derive a multilinear algebraic algorithm. This procedure is a generalization of the algorithm for independent component analysis described in Comon (1994). The performance is illustrated by means of some numerical experiments.