We investigate a difference-of-convex (DC) formulation where the second term is allowed to be weakly convex. We examine the precise behavior of a single iteration of the difference-of-convex algorithm (DCA), providing a tight characterization of the objective function decrease, distinguishing between six distinct parameter regimes. Our proofs, inspired by the performance estimation framework, are notably simplified compared to related prior research. We subsequently derive sublinear convergence rates for the DCA towards critical points, assuming at least one of the functions is smooth. Additionally, we explore the underexamined equivalence between proximal gradient descent (PGD) and DCA iterations, demonstrating how DCA, a parameter-free algorithm, without the need for a stepsize, serves as a tool for studying the exact convergence rates of PGD. Finally, we propose a method to optimize the DC decomposition to achieve optimal convergence rates, potentially transforming the subtracted function to become weakly convex.

We consider the classical gradient method with constant stepsizes where some error is introduced in the computation of each gradient.More specifically, we assume relative inexactness, in the sense that the norm of the difference between the true gradient and its approximate value is bounded by a certain fraction of the gradient norm.We establish a sublinear convergence rate for this inexact method when applied to smooth convex functions, and illustrate on a logistic regression example.1 Stochasticity may be viewed as another source of inexactness, but we limit the scope of this work to deterministic methods 125

Bayesian optimization (BO) is a type of black-box method used to optimize a costly objective function for which we have no access to derivatives.In practice, it is frequent that a series of similar problems has to be solved, with the problem data changing moderately between instances.We investigate a transfer learning approach based on BO that reuses information from a previous configuration in order to speed up subsequent optimizations.Our approach involves learning the noise variance to apply to the function values of the previous configuration and adapting the exploration-exploitation trade-off of the acquisition function from the previous configuration.We apply those ideas to the calibration of a beam line in proton therapy where the goal is to find magnet currents to obtain a desired shape for the beam of protons, and for which the calibration has to be repeated for several configurations.We show that reusing information from a previous configuration allows a reduction in the number of iterations by more than 80%, and that using BO is superior to the conventional Nelder-Mead algorithm for black box optimization and transfer learning.

Iterative power difference balancing (IPDB) has recently been proposed as a first real-time dynamic spectrum management (RT-DSM) algorithm. It consists of a primal coordinate ascent approach where each coordinate step is performed using an exhaustive discrete grid line search. In this paper we present an iterative convex approximation based approach to perform the coordinate ascent search so as to reduce its computational complexity. By exploiting the problem structure, a closed-form solution is derived for the convex approximations. The resulting RT-DSM algorithm is referred to as fast IPDB (F-IPDB). Compared to IPDB, F-IPDB exhibits similar data rate performance with significantly reduced computational complexity, while also providing smoother final transmit spectra.

Dynamic spectrum management is recognized as a key technique to tackle interference in multi-user multi-carrier communication systems and networks. However existing dynamic spectrum management algorithms may not be suitable when the available computation time and compute power are limited, i.e., when a very fast responsiveness is required. In this paper, we present a new paradigm, theory and algorithm for real-time dynamic spectrum management (RT-DSM). Specifically, a RT-DSM algorithm is real-time in the sense that it can be stopped at any point in time while guaranteeing a feasible and improved solution. This is enabled by the introduction of a novel difference-of-variables (DoV) transformation and problem reformulation, for which a primal coordinate ascent approach is proposed with exact line search via a logarithmically-scaled grid search. The proposed algorithm is referred to as iterative power difference balancing (IPDB). Simulations for different realistic wireline and wireless interference-limited systems demonstrate its good performance, low complexity and wide applicability under different configurations.

Spectrum optimization is a promising means to tackle the crosstalk problem in DSL systems, and corresponds to a challenging nonconvex optimization problem. Iterative convex approximation (ICA) methods have been proposed in the literature to deal with this optimization problem. These methods consist in solving a series of improving convex approximations and are typically implemented in a per-user iterative approach. In this paper we develop a novel class of iterative methods that focus explicitly on per-user iterative implementations, and which consist of improved per-user approximations that are tighter and simpler to solve (in closed-form) than state-of-the-art ICA methods. As a result, the proposed methods improve the convergence speed as fewer approximations are required to converge, and display a significantly lower computational cost. Furthermore, three of the proposed methods can tackle the issue of getting stuck in bad locally optimal solutions, and hence improve solution quality with respect to existing ICA methods.

Nonnegative matrix factorization (NMF) is a data analysis technique used in a great variety of applications such as text mining, image processing, hyperspectral data analysis, computational biology, and clustering. In this letter, we consider two well-known algorithms designed to solve NMF problems: the multiplicative updates of Lee and Seung and the hierarchical alternating least squares of Cichocki et al. We propose a simple way to significantly accelerate these schemes, based on a careful analysis of the computational cost needed at each iteration, while preserving their convergence properties. This acceleration technique can also be applied to other algorithms, which we illustrate on the projected gradient method of Lin. The efficiency of the accelerated algorithms is empirically demonstrated on image and text data sets and compares favorably with a state-of-the-art alternating nonnegative least squares algorithm.

In this study, we use nonnegative matrix factorization (NMF) and nonnegative matrix underapproximation (NMU) approaches to generate feature vectors that can be used to cluster aviation safety reporting system (ASRS) documents obtained from the distributed national ASAP archive (DNAA). By preserving nonnegativity, both the NMF and NMU facilitate a sum-of-parts representation of the underlying term usage patterns in the ASRS document collection. Both the training and test sets of ASRS documents are parsed and then factored by both algorithms to produce a reduced-rank representations of the entire document space. The resulting feature and coefficient matrix factors are used to cluster ASRS documents so that the (known) associated anomalies of training documents are directly mapped to the feature vectors. Dominant features of test documents are then used to generate anomaly relevance scores for those documents.We demonstrate that the approximate solution obtained by NMU using Lagrangrian duality can lead to a better sum-of-parts representation and document classification accuracy.