Deep equilibrium (DEQ) models are widely recognized as a memory efficient alternative to standard neural networks, achieving state-of-the-art performance in language modeling and computer vision tasks. These models solve a fixed point equation instead of explicitly computing the output, which sets them apart from standard neural networks. However, existing DEQ models often lack formal guarantees of the existence and uniqueness of the fixed point, and the convergence of the numerical scheme used for computing the fixed point is not formally established. As a result, DEQ models are potentially unstable in practice. To address these drawbacks, we introduce a novel class of DEQ models called positive concave deep equilibrium (pcDEQ) models. Our approach, which is based on nonlinear Perron-Frobenius theory, enforces nonnegative weights and activation functions that are concave on the positive orthant. By imposing these constraints, we can easily ensure the existence and uniqueness of the fixed point without relying on additional complex assumptions commonly found in the DEQ literature, such as those based on monotone operator theory in convex analysis. Furthermore, the fixed point can be computed with the standard fixed point algorithm, and we provide theoretical guarantees of its geometric convergence, which, in particular, simplifies the training process. Experiments demonstrate the competitiveness of our pcDEQ models against other implicit models.

We use fixed point theory to analyze nonnegative neural networks, which we define as neural networks that map nonnegative vectors to nonnegative vectors. We first show that nonnegative neural networks with nonnegative weights and biases can be recognized as monotonic and (weakly) scalable mappings within the framework of nonlinear Perron-Frobenius theory. This fact enables us to provide conditions for the existence of fixed points of nonnegative neural networks having inputs and outputs of the same dimension, and these conditions are weaker than those recently obtained using arguments in convex analysis. Furthermore, we prove that the shape of the fixed point set of nonnegative neural networks with nonnegative weights and biases is an interval, which under mild conditions degenerates to a point. These results are then used to obtain the existence of fixed points of more general nonnegative neural networks. From a practical perspective, our results contribute to the understanding of the behavior of autoencoders, and we also offer valuable mathematical machinery for future developments in deep equilibrium models.

We introduce the concept of inverse feasibility for linear forward models as a tool to enhance Over-the-Air (OTA) federated learning (FL) algorithms. Inverse feasibility is defined as an upper bound on the condition number of the forward operator as a function of its parameters. We analyze an existing OTA FL model using this definition, identify areas for improvement, and propose a new OTA FL model. Numerical experiments illustrate the main implications of the theoretical results. The proposed framework, which is based on inverse problem theory, can potentially complement existing notions of security and privacy by providing additional desirable characteristics to networks.

Spectral fingerprints (SFs) are unique power spectra signatures of human brain regions of interest (ROIs, Keitel & Gross, 2016). SFs allow for accurate ROI identification and can serve as biomarkers of differences exhibited by non-neurotypical groups. At present, there are no open-source, versatile tools to calculate spectral fingerprints. We have filled this gap by creating a modular, highly-configurable MATLAB Toolbox for Frequency-based Fingerprinting (ToFFi). It can transform magnetoencephalographic and electroencephalographic signals into unique spectral representations using ROIs provided by anatomical (AAL, Desikan-Killiany), functional (Schaefer), or other custom volumetric brain parcellations. Toolbox design supports reproducibility and parallel computations.

We present a convexly-constrained beamformer design for brain activity reconstruction from non-invasive electroencephalography (EEG) signals. An intrinsic gap between the output variance and the mean squared errors is highlighted that occurs due to the presence of interfering activities correlated with the desired activity. The key idea of the proposed beamformer is reducing this gap without amplifying the noise by imposing a quadratic constraint that bounds the total power of interference leakage together with the distortionless constraint. The proposed beamformer can be implemented efficiently by the multi-domain adaptive filtering algorithm. Numerical examples show the clear advantages of the proposed beamformer over the minimum-variance distortionless response (MVDR) and nulling beamformers.

We consider the problem of electroencephalography (EEG) and magnetoencephalography (MEG) source localization using beamforming techniques. Specifically, we propose a reduced-rank extension of the recently derived multi-source activity index (MAI), which itself is an extension of the classical neural activity index to the multi-source case. We show that, for uncorrelated dipole sources and any nonzero rank constraint, the proposed reduced-rank multi-source activity index (RR-MAI) achieves the global maximum when evaluated at the true source positions. Therefore, the RR-MAI can be used to localize multiple sources simultaneously. Furthermore, we propose another version of the RR-MAI which can be seen as a natural generalization of the proposed index to arbitrarily correlated sources. We present a series of numerical simulations showing that the RR-MAI can achieve a more precise source localization than the full-rank MAI in the case when the EEG/MEG forward model becomes ill-conditioned, which in our settings corresponds to the case of closely positioned sources and low signal-to-noise ratio.

We consider the problem of dipole source signals estimation in electroencephalography (EEG) using beamforming techniques in ill-conditioned settings. We take advantage of the link between the linearly constrained minimum-variance (LCMV) beamformer in sensor array processing and the best linear unbiased estimator (BLUE) in linear regression modeling. We show that the recently introduced reduced-rank extension of BLUE, named minimum-variance pseudo-unbiased reduced-rank estimator (MV-PURE), achieves much lower estimation error not only than LCMV beamformer, but also than the previously derived reduced-rank principal components (PC) and cross-spectral metrics (CSM) beamformers in ill-conditioned settings. The practical scenarios where the considered estimation model becomes ill-conditioned are discussed, then we show the applicability of MV-PURE dipole source estimator under those conditions through realistic simulations.

The stochastic MV-PURE estimator is a linear estimator for stochastic linear model that is highly robust to mismatches in model knowledge and which is specially designed for efficient estimation in noisy and ill-conditioned cases. To date, its properties were analyzed in the theoretical settings of perfect model knowledge and thus could not explain clearly the reason behind its superior performance compared to the Wiener filter observed in simulations in practical cases of imperfect model knowledge. In this paper we derive closed form expressions of the mean-square-error (MSE) of both Wiener filter and the stochastic MV-PURE estimator for the case of perturbed singular values of a model matrix in the linear model considered. These expressions provide in particular conditions under which the stochastic MV-PURE estimator achieves smaller MSE not only than Wiener filter, but also than its full-rank version, the minimum-variance distortionless (MVDR) estimator in such settings. We provide numerical simulations confirming the main theoretical results presented.

The stochastic MV-PURE estimator has recently emerged as the robust solution for frequently occuring in practice problem of linear estimation in ill-conditioned and imperfectly known linear stochastic model. In this paper we provide theoretical results showing that the stochastic MV-PURE estimator can be used to the greatest effect in highly noisy settings. In such settings, we discuss the relation between the stochastic MV-PURE estimator and the well-known reduced rankWiener filter. We verify the theoretical results presented by a means of numerical simulations.