A common assumption in matrix completion (MC) and tensor completion (TC) is that the missing locations are sampled randomly. However, in real-world scenarios, the unobserved elements are often not arbitrarily located, and may concentrate within entire rows or columns. We refer to this missing mechanism as structural missingness, and traditional MC and TC schemes suffer from drastic degradation under these circumstances. This work addresses the challenge of restoring structural missingness by introducing a novel framework for simultaneously reconstructing multiple matrices, called multi-matrix completion (MMC). In MMC, tri-factorization across matrices captures the correlation between matrices, and Tikhonov regularization on each matrix exploits its correlation. This design enables MMC to efficiently handle both random and structural missingness. In addition, MMC is not affected by the smoothness along matrices which makes it suitable for a wider variety of data compared to Fourier transform based TC methods. The alternating direction method of multipliers is utilized to solve the resultant optimization problem. The global convergence of the algorithm is supported by comprehensive theoretical analyses. We demonstrate the versatility of MMC through extensive experiments in image and video restoration, and showcase its superior performance in comparison to traditional MC and TC methods.

This paper proposes a robust, energy-efficient multi-user OFDM chaotic vector cyclic shift keying (MU-OFDM-CVCSK) system for channels impaired by multi-user interference (MUI), Gaussian, and impulsive noise. The system’s co-designed transceiver uses cyclic shifts of a single reference signal for energy efficiency, while a novel receiver combines low-rank matrix factorization with a truncated-quadratic loss function to suppress both MUI and noise. In particular, the transmitted signal’s cyclical structure enables iterative reference refinement, further enhancing performance. We provide a comprehensive analysis of the algorithm’s convergence, complexity, energy efficiency, power spectral density, peak-to-average power ratio, and derive the bit error rate (BER) expressions. Simulations demonstrate superior BER performance compared to benchmark schemes in challenging interference and noise conditions.

Byzantine-robust federated learning aims to maintain resilient performance in the presence of malicious attacks that can impede the convergence of learning algorithms. Although numerous robust aggregators have been developed to merge the collected gradient information in the server, they either require data homogeneity and are suboptimal for heterogeneous data, or their breakdown points—the smallest proportion of outliers that can make the aggregators fail—are not theoretically analyzed or less than 0.5. In contrast to existing aggregators, this paper formulates the aggregation process as a low-rank plus sparse decomposition model, where the low-rank component, with a rank of one, facilitates accurate gradient computation, while the sparse component, penalized by the ℓ2,0-norm, mitigates the impact of outliers. We prove that the devised rule achieves the maximum breakdown point of 0.5. Besides, we apply our aggregation rule to Byzantine-robust federated learning and employ the Polyak’s momentum to reduce gradient variance among honest workers. It is analyzed that our aggregator achieves order-optimal Byzantine-resilient federated learning for heterogeneous data. Experimental results using MNIST, Fashion-MNIST and CIFAR-10 demonstrate that the developed approach yields higher classification accuracy than the competing aggregators under different attack types and heterogeneity levels.

Byzantine-robust distributed or federated learning (FL) refers to providing reliable performance under Byzantine attacks, which violate the prescribed protocols and transmit arbitrary information to the server to hamper the convergence of machine learning (ML) algorithms, via designing resilient aggregation rules to combat attacks. Although numerous robust rules have been suggested, their performance degrades for heterogeneous data. A few techniques have been exploited to handle this problem, but they either require preaggregation operations, hence increasing the computational load, or lack breakdown point analysis of their rules. This article proposes a new aggregation rule, which clips the gradients received from all workers according to the distance between the gradient and the aggregation center. That is, when the distance is larger than the radius $\gamma $ , the gradient will be clipped, and the longer the distance, the closer the clipped gradient is to the center. We theoretically analyze that the breakdown point of the developed rule is 0.5, the maximum value for robust aggregators. Moreover, our rule achieves order-optimal Byzantine-robust training error under data heterogeneity, while the median-based schemes, such as coordinate-wise median (CM) and geometric median (GM), are suboptimal. Experimental results demonstrate that the devised aggregation mechanism can handle different attacks well and outperforms the existing rules.

As large language models continue to grow in size, parameter-efficient fine-tuning (PEFT) has become increasingly crucial.While lowrank adaptation (LoRA) offers a solution through low-rank updates, its static rank allocation may yield suboptimal results.Adaptive low-rank adaptation (AdaLoRA) improves this with dynamic allocation but remains sensitive to initial and target rank configurations.We introduce AROMA, a framework that automatically constructs layer-specific updates by iteratively building up rank-one components with very few trainable parameters that gradually diminish to zero.Unlike existing methods that employ rank reduction mechanisms, AROMA introduces a dual-loop architecture for rank growth.The inner loop extracts information from each rank-one subspace, while the outer loop determines the number of rankone subspaces, i.e., the optimal rank.We reset optimizer states to maintain subspace independence.AROMA significantly reduces parameters compared to LoRA and AdaLoRA while achieving superior performance on natural language understanding and generation, commonsense reasoning, offering new insights into adaptive PEFT.

In robust matrix completion (MC), the Welsch function, also referred to as the maximum correntropy criterion with Gaussian kernel, has been widely employed. However, it suffers from the drawback of down-weighing normal data. This work is the first to uncover the explicit regularizer (ER) for the Welsch function based on the multiplicative form of half-quadratic (HQ) minimization. Leveraging this discovery, we develop a new function called t-Welsch, also with ER, which provides unity weight to normal data and exhibits stronger robustness against large-magnitude outliers compared to Huber's weight. We apply the t-Welsch to rank-one matching pursuit, enabling accurate and robust low-rank matrix recovery without the need of rank information and singular value decomposition (SVD). The resultant MC algorithm is realized via block coordinate descent (BCD), whose analyses of convergence and computational complexity are produced. Experiments are conducted using synthetic random data, as well as real-world images with salt-and-pepper noise and multiple-input multiple-output (MIMO) radar signals in the presence of Gaussian mixture disturbances. In all three scenarios, the proposed algorithm outperforms the state-of-the-art robust MC methods in terms of recovery accuracy. The code is available at https://github.com/ShuDun23/t-Welsch-and-RAR1MC.

BACKGROUND: Whole genome variants offer sufficient information for genetic prediction of human disease risk, and prediction of animal and plant breeding values. Many sophisticated statistical methods have been developed for enhancing the predictive ability. However, each method has its own advantages and disadvantages, so far, no one method can beat others. RESULTS: We herein propose an Ensemble Learning method for Prediction of Genetic Values (ELPGV), which assembles predictions from several basic methods such as GBLUP, BayesA, BayesB and BayesCπ, to produce more accurate predictions. We validated ELPGV with a variety of well-known datasets and a serious of simulated datasets. All revealed that ELPGV was able to significantly enhance the predictive ability than any basic methods, for instance, the comparison p-value of ELPGV over basic methods were varied from 4.853E-118 to 9.640E-20 for WTCCC dataset. CONCLUSIONS: ELPGV is able to integrate the merit of each method together to produce significantly higher predictive ability than any basic methods and it is simple to implement, fast to run, without using genotype data. is promising for wide application in genetic predictions.

Tensor completion (TC) refers to restoring the missing entries in a given tensor by making use of the low-rank structure. Most existing algorithms have excellent performance in Gaussian noise or impulsive noise scenarios. Generally speaking, the Frobenius-norm-based methods achieve excellent performance in additive Gaussian noise, while their recovery severely degrades in impulsive noise. Although the algorithms using the$\ell_{p}$-norm ($0<p<2$) or its variants can attain high restoration accuracy in the presence of gross errors, they are inferior to the Frobenius-norm-based methods when the noise is Gaussian-distributed. Therefore, an approach that is able to perform well in both Gaussian noise and impulsive noise is desired. In this work, we use a capped Frobenius norm to restrain outliers, which corresponds to a form of the truncated least-squares loss function. The upper bound of our capped Frobenius norm is automatically updated using normalized median absolute deviation during iterations. Therefore, it achieves better performance than the$\ell_{p}$-norm with outlier-contaminated observations and attains comparable accuracy to the Frobenius norm without tuning parameter in Gaussian noise. We then adopt the half-quadratic theory to convert the nonconvex problem into a tractable multivariable problem, that is, convex optimization with respect to (w.r.t.) each individual variable. To address the resultant task, we exploit the proximal block coordinate descent (PBCD) method and then establish the convergence of the suggested algorithm. Specifically, the objective function value is guaranteed to be convergent while the variable sequence has a subsequence converging to a critical point. Experimental results based on real-world images and videos exhibit the superiority of the devised approach over several state-of-the-art algorithms in terms of recovery performance. MATLAB code is available at https://github.com/Li-X-P/Code-of-Robust-Tensor-Completion.

In this paper, we devise a sparse-coded orthogonal frequency division multiplexing (OFDM) differential chaos shift keying (DCSK) communication system based on low-rank matrix recovery which can handle Gaussian background noise and outlier-contaminated symbols simultaneously. As the noise-free OFDM-DCSK symbol matrix has rank 1, we exploit the vector outer product for its modeling, while sparse coding is also applied to reduce the transmission energy. To demodulate information bits from the sparse-coded signal, we formulate an objective function which consists of a sum of Frobenius norm for rank-1 matrix recovery and$\ell _{0}$-norm for identifying the possibly outlier-contaminated symbols, with a self-adaptive weight parameter. The resultant optimization problem is solved iteratively via block coordinate descent, and the Laplacian kernel with the Silverman’s rule is adopted for outlier detection. Theoretical analysis including convergence of the objective function, bit error rate (BER), energy efficiency and computational complexity, are provided. Simulation results show that the proposed system has comparable mean square error and BER performance with the$\ell _{p}$-norm minimization based matrix recovery approach at$p=2$in additive white Gaussian noise, and is superior to that of$p=1$in Middleton class A noise, even when sparse coding is applied. Moreover, compared with other binary DCSK systems, our system achieves higher energy efficiency thanks to the sparse coding.

Robust matrix completion refers to recovering a low-rank matrix given a subset of the entries corrupted by gross errors, and has various applications since many real-world signals can be modeled as low-rank matrices. Most of the existing methods only perform well for noise-free data or those with zero-mean white Gaussian noise, and their performance will be degraded in the presence of outliers. In this paper, based on the factorization framework, we propose a novel robust matrix completion scheme via using the truncated-quadratic loss function, which is non-convex and non-smooth, and half-quadratic theory is adopted for its optimization. By introducing an auxiliary variable, half-quadratic optimization (HO) can transform the loss function into two tractable forms, that is, additive and multiplicative formulations. Block coordinate descent method is then exploited as their solver. Compared with the additive form, the multiplicative variant has lower computational cost since we attempt to take the observations contaminated by outliers as missing entries. Numerical simulations and experimental results based on image inpainting and hyperspectral image recovery demonstrate that our algorithms are superior to the state-of-the-art methods in terms of restoration accuracy and runtime. MATLAB code is available athttps://github.com/bestzywang.

Matrix completion refers to recovering a matrix from a small subset of its entries. It is an important topic because numerous real-world data can be modeled as low-rank matrices. One popular approach for matrix completion is based on low-rank matrix factorization, but it requires knowing the matrix rank, which is difficult to accurately determine in many practical scenarios. We propose a novel algorithm based on rank-one approximation that a matrix can be decomposed as a sum of outer products. The key idea is to find the basis vectors of the underlying matrix according to the observed entries, and gradually increase the vector number until an appropriate rank estimate is reached. In contrast to the conventional rank-one schemes that employ unchanging rank-one basis matrices, our algorithm performs completion from the vector viewpoint and is able to generate continuously updated rank-one basis matrices. Besides, we theoretically show that the developed method has a linear convergence rate and a smaller recovery error than existing rank-one based algorithms. Experimental results using both synthetic data and real-world images demonstrate that our solution has the best recovery performance among the competing algorithms when the observations are contaminated by Gaussian noise.