In recent years, various subspace algorithms have been developed to handle large-scale optimization problems. Although existing subspace Newton methods require fewer iterations to converge in practice, the matrix operations and full gradient computation are bottlenecks when dealing with large-scale problems. We propose a subspace quasi-Newton method that is restricted to a deterministic subspace together with a subspace gradient based on random matrix theory. Our method does not require full gradients, let alone Hessian matrices. Yet, it achieves the same order of worst-case iteration complexity in expectation for both convex and nonconvex cases as existing subspace methods. In numerical experiments, we confirm the superiority of our algorithm in terms of computation time.

In recent years, random subspace methods have been actively studied for large-dimensional nonconvex problems. Recent subspace methods have improved theoretical guarantees such as iteration complexity and local convergence rate while reducing computational costs by deriving descent directions in randomly selected low-dimensional subspaces. This paper proposes the Random Subspace Homogenized Trust Region (RSHTR) method with the best theoretical guarantees among random subspace algorithms for nonconvex optimization. RSHTR achieves an $\varepsilon$-approximate first-order stationary point in $O(\varepsilon^{-3/2})$ iterations, converging locally at a linear rate. Furthermore, under rank-deficient conditions, RSHTR satisfies $\varepsilon$-approximate second-order necessary conditions in $O(\varepsilon^{-3/2})$ iterations and exhibits a local quadratic convergence. Experiments on real-world datasets verify the benefits of RSHTR.

Optimization with orthogonality constraints frequently arises in various fields such as machine learning. Riemannian optimization offers a powerful framework for solving these problems by equipping the constraint set with a Riemannian manifold structure and performing optimization intrinsically on the manifold. This approach typically involves computing a search direction in the tangent space and updating variables via a retraction operation. However, as the size of the variables increases, the computational cost of the retraction can become prohibitively high, limiting the applicability of Riemannian optimization to large-scale problems. To address this challenge and enhance scalability, we propose a novel approach that restricts each update on a random submanifold, thereby significantly reducing the per-iteration complexity. We introduce two sampling strategies for selecting the random submanifolds and theoretically analyze the convergence of the proposed methods. We provide convergence results for general nonconvex functions and functions that satisfy Riemannian Polyak–Łojasiewicz condition as well as for stochastic optimization settings. Additionally, we demonstrate how our approach can be generalized to quotient manifolds derived from the orthogonal manifold. Extensive experiments verify the benefits of the proposed method, across a wide variety of problems.

Abstract Random projection, a dimensionality reduction technique, has been found useful in recent years for reducing the size of optimization problems. In this paper, we explore the use of sparse sub-gaussian random projections to approximate semidefinite programming (SDP) problems by reducing the size of matrix variables, thereby solving the original problem with much less computational effort. We provide some theoretical bounds on the quality of the projection in terms of feasibility and optimality that explicitly depend on the sparsity parameter of the projector. We investigate the performance of the approach for semidefinite relaxations appearing in polynomial optimization, with a focus on combinatorial optimization problems. In particular, we apply our method to the semidefinite relaxations of Maxcut and Max-2-sat . We show that for large unweighted graphs, we can obtain a good bound by solving a projection of the semidefinite relaxation of Maxcut . We also explore how to apply our method to find the stability number of four classes of imperfect graphs by solving a projection of the second level of the Lasserre Hierarchy. Overall, our computational experiments show that semidefinite programming problems appearing as relaxations of combinatorial optimization problems can be approximately solved using random projections as long as the number of constraints is not too large.

Abstract We are interested in the design of robust (or resilient) capacitated rooted Steiner networks in the case of terminals with uniform demands. Formally, we are given a graph, capacity, and cost functions on the edges, a root, a subset of vertices calledterminals, and a bound on the number of possible edge failures. We first study the problem where and the network that we want to design must be a tree covering the root and the terminals: we give complexity results and propose models to optimize both the cost of the tree and the number of terminals disconnected from the root in the worst case of an edge failure, while respecting the capacity constraints on the edges. Secondly, we consider the problem of computing a minimum‐cost survivable network, that is, a network that covers the root and terminals even after the removal of any edges, while still respecting the capacity constraints on the edges. We also consider the possibility of protecting a given number of edges. We propose three different formulations: a bilevel formulation (with an attacker and a defender), a cutset‐based formulation and a flow‐based one. We compare the formulations from a theoretical point of view, and we propose algorithms to solve them and compare their efficiency in practice.

The use of random projections in mathematical programming allows standard solution algorithms to solve instances of much larger sizes, at least approximately. Approximation results have been derived in the relevant literature for many specific problems, as well as for several mathematical programming subclasses. Despite the theoretical developments, it is not always clear that random projections are actually useful in solving mathematical programs in practice. In this paper we provide a computational assessment of the application of random projections to linear programming.

The automated real time control of an electrical network is achieved through the estimation of its state using phasor measurement units. Given an undirected graph representing the network, we study the problem of finding the minimum number of phasor measurement units to place on the edges such that the graph is fully observed. This problem is also known as the Power Edge Set problem, a variant of the Power Dominating Set problem. It is naturally modeled using an iteration‐indexed binary linear program, whose size turns out to be too large for practical purposes. We use a fixed‐point argument to remove the iteration indices and obtain a more compact bilevel formulation. We then reformulate the latter to a single‐level mixed‐integer linear program, which performs better than the natural formulation. Lastly, we provide an algorithm that solves the bilevel program directly and much faster than a commercial solver can solve the previous models. We also discuss robust variants and extensions of the problem. © 2016 Wiley Periodicals, Inc. NETWORKS, Vol. 68(2), 104–120 2016