Abstract Euclidean projection onto the intersection of the sparsity constraint set and an underlying problem-dependent constraint set is an important technique in sparsity or cardinality constrained optimization. When the underlying constraint set is symmetric, it is known that sparse projection can be efficiently computed via a sorting function. Motivated by the observation that some class of underlying constraint sets is partially symmetric, i.e., symmetric with respect to a certain index subset, this paper introduces the notion of semi-symmetry and develops conditions of sparse projection onto a semi-symmetric set. These conditions are characterized by a sorting function and certain combinatorial conditions. When the sparsity level or the number of multi-semi-symmetric index subsets is small or moderate, these conditions can be efficiently verified. Stationary point conditions are obtained for semi-symmetric sets, and the convergence of the projected gradient descent scheme with an improved step length is established in a general setting and is applied to sparse optimization. The proposed semi-symmetric sparse projection based schemes are tested on the joint sparsity and group sparsity problem from statistics and two finance problems subject to semi-symmetric constraints; numerical results demonstrate improved performance in terms of numerical accuracy and computation time.

The core of a good model is in its ability to focus only on important information that reflects the basic patterns and consistencies, thus pulling out a clear, noise-free signal from the dataset. This necessitates using a simplified model defined by fewer parameters. The importance of theoretical foundations becomes clear in this context, as this paper relies on established results from the domain of advanced sparse optimization, particularly those addressing nonlinear differentiable functions. The need for such theoretical foundations is further highlighted by the trend that as computational power for training NNs increases, so does the complexity of the models in terms of a higher number of parameters. In practical scenarios, these large models are often simplified to more manageable versions with fewer parameters.Understanding why these simplified models with less number of parameters remain effective raises a crucial question. Understanding why these simplified models with fewer parameters remain effective raises an important question. This leads to the broader question of whether there is a theoretical framework that can clearly explain these empirical observations. Recent developments, such as establishing necessary conditions for the convergence of iterative hard thresholding (IHT) to a sparse local minimum—a sparse method analogous to gradient descent, are promising. The IHT algorithm’s remarkable capacity to accurately identify and learn the locations of nonzero parameters underscores its practical effectiveness and utility.This paper aims to investigate whether the theoretical prerequisites for such convergence are applicable in the realm of neural network (NN) training by providing justification for all the necessary conditions for convergence. Then, these conditions are validated by experiments on a single-layer NN, using the IRIS dataset as a testbed. Our empirical results demonstrate that convergence conditions can be reliably ensured during the training of a neural network (NN), and under these conditions, the IHT algorithm consistently converges to a sparse local minimizer.

Recent approaches to vision-language tasks are built on the remarkable capabilities of large vision-language models (VLMs). These models excel in zero-shot and few-shot learning, enabling them to learn new tasks without parameter updates. However, their primary challenge lies in their design, which primarily accommodates 2D input, thus limiting their effectiveness for medical images, particularly radiological images like MRI and CT, which are typically 3D. To bridge the gap between state-of-the-art 2D VLMs and 3D medical image data, we developed an innovative, one-pass, unsupervised representative slice selection method called Vote-MI, which selects representative 2D slices from 3D medical imaging. To evaluate the effectiveness of vote-MI when implemented with VLMs, we introduce BrainMD, a robust, multimodal dataset comprising 2,453 annotated 3D MRI brain scans with corresponding textual radiology reports and electronic health records. Based on BrainMD, we further develop two benchmarks, BrainMD-select (including the most representative 2D slice of 3D image) and BrainBench (including various vision-language downstream tasks). Extensive experiments on the BrainMD dataset and its two corresponding benchmarks demonstrate that our representative selection method significantly improves performance in zero-shot and few-shot learning tasks. On average, Vote-MI achieves a 14.6\% and 16.6\% absolute gain for zero-shot and few-shot learning, respectively, compared to randomly selecting examples. Our studies represent a significant step toward integrating AI in medical imaging to enhance patient care and facilitate medical research. We hope this work will serve as a foundation for data selection as vision-language models are increasingly applied to new tasks.

A Deep Neural Network (DNN) is a composite function of vector-valued functions, and in order to train a DNN, it is necessary to calculate the gradient of the loss function with respect to all parameters. This calculation can be a non-trivial task because the loss function of a DNN is a composition of several nonlinear functions, each with numerous parameters. The Backpropagation (BP) algorithm leverages the composite structure of the DNN to efficiently compute the gradient. As a result, the number of layers in the network does not significantly impact the complexity of the calculation. The objective of this paper is to express the gradient of the loss function in terms of a matrix multiplication using the Jacobian operator. This can be achieved by considering the total derivative of each layer with respect to its parameters and expressing it as a Jacobian matrix. The gradient can then be represented as the matrix product of these Jacobian matrices. This approach is valid because the chain rule can be applied to a composition of vector-valued functions, and the use of Jacobian matrices allows for the incorporation of multiple inputs and outputs. By providing concise mathematical justifications, the results can be made understandable and useful to a broad audience from various disciplines.

CAV platooning technology has received considerable attention, driven by the next generation smart transportation systems. This paper considers nonlinear vehicle dynamics and develops fully distributed optimization based CAV platooning control schemes via the platoon centered MPC approach for a possibly heterogeneous CAV platoon. The nonlinear vehicle dynamics leads to major difficulties in distributed algorithm development and control analysis. Specifically, the underlying MPC optimization problem is nonconvex and densely coupled. Further, the closed loop dynamics becomes a time-varying nonlinear system with non-vanishing external perturbations, making stability analysis rather complicated. To overcome these difficulties, we formulate the underlying MPC optimization problem as a locally coupled, albeit nonconvex, optimization problem and develop a sequential convex programming based fully distributed scheme for a general MPC horizon. Such a scheme can be effectively implemented for real-time computing using operator splitting methods. To analyze the closed loop stability, we apply various tools from global implicit function theorems, stability of linear time-varying systems, and Lyapunov theory for input-to-state stability to show that the closed loop system is locally input-to-state stable uniformly in all small coefficients pertaining to the nonlinear dynamic effects. Numerical tests on a heterogeneous CAV platoon in a real traffic condition illustrate the effectiveness of the proposed method.

This paper proposes an efficient way to compute the ℒ2-gains of discrete-time switched linear systems. Using the notion of generating functions, generalized versions of ℒ2-gains under arbitrary switching are studied. An efficient numerical algorithm is formulated by which these generalized ℒ2-gains can be estimated. The proposed method mitigates the problem of conservative bounds. Numerical examples are provided to illustrate the algorithm.

Exponential stability of switched linear systems under both arbitrary and proper switching is studied through two suitably defined families of functions called the strong and the weak generating functions. Various properties of the generating functions are established. It is found that the radii of convergence of the generating functions characterize the exponential growth rate of the trajectories of the switched linear systems. In particular, necessary and sufficient conditions for the exponential stability of the systems are derived based on these radii of convergence. Numerical algorithms for computing estimates of the generating functions are proposed and examples are presented for illustration purpose.