Structure-based drug discovery, encompassing the tasks of protein-ligand docking and pocket-aware 3D drug design, represents a core challenge in drug discovery. However, no existing work can deal with both tasks to effectively leverage the duality between them, and current methods for each task are hindered by challenges in modeling 3D information and the limitations of available data. To address these issues, we propose 3DMolFormer, a unified dual-channel transformer-based framework applicable to both docking and 3D drug design tasks, which exploits their duality by utilizing docking functionalities within the drug design process. Specifically, we represent 3D pocket-ligand complexes using parallel sequences of discrete tokens and continuous numbers, and we design a corresponding dual-channel transformer model to handle this format, thereby overcoming the challenges of 3D information modeling. Additionally, we alleviate data limitations through large-scale pre-training on a mixed dataset, followed by supervised and reinforcement learning fine-tuning techniques respectively tailored for the two tasks. Experimental results demonstrate that 3DMolFormer outperforms previous approaches in both protein-ligand docking and pocket-aware 3D drug design, highlighting its promising application in structure-based drug discovery. The code is available at: https://github.com/HXYfighter/3DMolFormer.

Gradient regularization (GR), which aims to penalize the gradient norm atop the loss function, has shown promising results in training modern over-parameterized deep neural networks. However, can we trust this powerful technique? This paper reveals that GR can cause performance degeneration in adaptive optimization scenarios, particularly with learning rate warmup. Our empirical and theoretical analyses suggest this is due to GR inducing instability and divergence in gradient statistics of adaptive optimizers at the initial training stage. Inspired by the warmup heuristic, we propose three GR warmup strategies, each relaxing the regularization effect to a certain extent during the warmup course to ensure the accurate and stable accumulation of gradients. With experiments on Vision Transformer family, we confirm the three GR warmup strategies can effectively circumvent these issues, thereby largely improving the model performance. Meanwhile, we note that scalable models tend to rely more on the GR warmup, where the performance can be improved by up to 3% on Cifar10 compared to baseline GR. Code is available at https://github.com/zhaoyang-0204/gnp.

*De novo* drug design is a pivotal issue in pharmacology and a new area of focus in AI for science research. A central challenge in this field is to generate molecules with specific properties while also producing a wide range of diverse candidates. Although advanced technologies such as transformer models and reinforcement learning have been applied in drug design, their potential has not been fully realized. Therefore, we propose MolRL-MGPT, a reinforcement learning algorithm with multiple GPT agents for drug molecular generation. To promote molecular diversity, we encourage the agents to collaborate in searching for desirable molecules in diverse directions. Our algorithm has shown promising results on the GuacaMol benchmark and exhibits efficacy in designing inhibitors against SARS-CoV-2 protein targets. The codes are available at: https://github.com/HXYfighter/MolRL-MGPT.

How to train deep neural networks (DNNs) to generalize well is a central concern in deep learning, especially for severely overparameterized networks nowadays. In this paper, we propose an effective method to improve the model generalization by additionally penalizing the gradient norm of loss function during optimization. We demonstrate that confining the gradient norm of loss function could help lead the optimizers towards finding flat minima. We leverage the first-order approximation to efficiently implement the corresponding gradient to fit well in the gradient descent framework. In our experiments, we confirm that when using our methods, generalization performance of various models could be improved on different datasets. Also, we show that the recent sharpness-aware minimization method (Foretet al., 2021) is a special, but not the best, case of our method, where the best case of our method could give new state-of-art performance on these tasks. Code is available at https://github.com/zhaoyang-0204/gnp.

Detecting the presence of target subspace signals with unknown clutters is a well-known hard problem encountered in various signal processing applications. Traditional methods fails to solve this problem because prior knowledge of clutter subspace is required, which can not be obtained when target and clutter are intimately mixed. In this paper, we propose a novel subspace detector that can detect target signal buried in clutter without knowledge of clutter subspace. This detector makes use of the geometrical relation between target and clutter subspaces and is derived based upon the calculation of volume of high dimensional geometrical objects. Moreover, the proposed detector can accomplish the detection simultaneously with the learning processes of clutter, a property called “detecting while learning”. The performance of detector was showed by theoretical analysis and numerical simulation.

Compressive sensing (CS) provides a new perspective for data reduction without compromising performance when the signal of interest is sparse or has intrinsically low-dimensional structure. The theoretical foundation for most of the existing studies on CS is based on the stable embedding (i.e., a distance-preserving property) of vectors that are sparse or in a union of subspaces via random measurement matrices. To the best of our knowledge, few existing literatures of CS have clearly discussed the stable embedding of linear subspaces via compressive measurement systems. In this paper, we explore a volume-based stable embedding of multidimensional signals based on Grassmann manifold, via Gaussian random measurement matrices. The Grassmann manifold is a topological space, in which each point is a linear vector subspace, and is widely regarded as an ideal model for multidimensional signals generated from linear subspaces. In this paper, we formulate the linear subspace spanned by multidimensional signal vectors as points on the Grassmann manifold, and use the volume and the product of sines of principal angles (also known as the product of principal sines) as the generalized norm and distance measure for the space of Grassmann manifold. We prove a volume-preserving embedding property for points on the Grassmann manifold via Gaussian random measurement matrices, i.e., the volumes of all parallelotopes from a finite set in Grassmann manifold are preserved upon compression. This volume-preserving embedding property is a multidimensional generalization of the conventional stable embedding properties, which only concern the approximate preservation of lengths of vectors in certain unions of subspaces. In addition, we use the volume-preserving embedding property to explore the stable embedding effect on a generalized distance measure of Grassmann manifold induced from volume. It is proved that the generalized distance measure, i.e., the product of principal sines between different points on the Grassmann manifold, is well preserved in the compressed domain via Gaussian random measurement matrices. Numerical simulations are also provided for validation.

Compressive Sensing (CS) provides a new perspective for dimensionnality reduction without compromising performance. The theoretical foundation for most of existing studies of CS is a stable embedding (i.e., a distance-preserving property) of certain low-dimensional signal models such as sparse signals or signals in a union of linear subspaces. However, few existing literatures clearly discussed the embedding effect of points on the Grassmann manifold in under-sampled linear measurement systems. In this paper, we explore the stable embedding property of multi-dimensional signals based on Grassmann manifold, which is a topological space with each point being a linear subspace of ℝN(or ℂN), via the Gaussian random matrices. It should be noted that the stability mentioned here is about the volume-preserving instead of distance-preserving, because volume is the key characteristic for linear subspace spanned by multiple vectors. The theorem of the volume-preserving stable embedding property is proposed, and sketched proofs as well as discussions about our theorem is also given.

In this paper, we propose a new algorithm, TLS-FOCUSS, for sparse recovery for large underdetermined linear systems, based on total least square (TLS) method and FOCUSS(FOCal Underdetermined System Solver). The problem of sparse recovery when perturbations appear in both the measurements and the dictionary (sensing matrix) is considered. FOCUSS algorithm is extended with main idea of TLS to reduce the impact of the perturbation of dictionary on the performance of sparse recovery. The simulation results illustrate the advantage of TLS-FOCUSS on accuracy and stability compared with ordinary FOCUSS algorithm.

Using multiple measurement vectors (MMV), we propose an algorithm based on weighted ℓ1minimization for direction- of-arrival (DOA) estimation, in which the weights are obtained by exploiting the orthogonality between the noise subspace and the array manifold matrix. The proposed algorithm penalizes the nonzero entries whose indices correspond to the row support of the jointly sparse signals by smaller weights and the other entries whose indices are more likely to be outside of the row support of the jointly sparse signals by larger weights, and therefore it can encourage sparsity at the true source locations. Numerical examples prove that the proposed algorithm has better performance than existing algorithms based on regular ℓ1minimization.

An algorithm based on matching pursuit (MP) technique is proposed for inverse synthetic aperture radar (ISAR) imaging of maneuvering targets. The received ISAR echo is decomposed into many basis sub-signals that are generated by discretizing the target spatial domain and synthesizing the ISAR data for every discretized spatial position, and then the ISAR imaging problem is converted into the sub-signal selection problem. The basis sub-signals that indeed contribute to the ISAR echo are selected by using the MP technique, and the projection coefficients of the ISAR echo on the selected basis sub-signals represent the ISAR image. In the case of unknown rotation rate of the target, the true rotation rate is obtained by combining the MP technique with the maximum contrast search. Numerical examples show that the proposed algorithm can produce high resolution and remove sidelobe artifacts.

A novel STAP algorithm based on sparse recovery technique, called CS-STAP, were presented. Instead of using conventional maximum likelihood estimation of covariance matrix, our method utilizes the echo statistics on spatial-temporal plane, which is extracted from sample data of only ONE training range cell with Compressed Sensing techniques, to construct a new estimator of covariance matrix, and build the optimal detector based on it. Full description of CS-STAP is given. Numerical result on real data has provided the evidence for great potential of CS-STAP as a effective approach when clutter is non-stationary because it need much less training data compared with common STAP methods.

We presented a novel Bayesian estimator for the correlation parameters of the multivariate Nakagami-m distributions. The proposed estimator has a two-stage structure. First, a generalized Gibbs sampler - the group-based sampler is constructed to analyze the posterior distributions of the parameters, which leads to a lower time complexity compared to the traditional Gibbs sampler. Second, a multi-proposal Metropolis-Hastings algorithm is implemented within the group-based sampler. In the Metropolis-Hastings stage, estimation values are obtained by computing the weighted mean of both accepted and rejected states. It makes the estimation variance lower compared to the traditional methods using only the accepted states. Numerical simulation results demonstrate that our new estimator is practically unbiased except some points and it offers a relatively small estimation variance.

A method for evaluating both the robustness and detection performance of constant false alarm rate (CFAR) detectors is presented in this paper, which is based on the powerful methodology of influence function (IF) developed in the literature on robust statistics. The robustness of different kinds of CFAR detector can be evaluated and compared by calculating the first derivative of the false alarm probability (FAP) at an underlying distribution, which is named IF-FAP. From the analysis of IF-FAP, several conclusions are given here: the robustness of the detector can be asymptotically represented by the IF of the clutter power estimator, while the detection performance can be approximately evaluated by the asymptotic variance of the clutter power estimator which is an explicit functional of the corresponding IF.

In this paper, a new approach to speckle filtering of synthetic aperture radar (SAR) data is presented. We define a parameter space consisting of two orthogonal subspaces-the signal subspace and the noise subspace. Then, the full polarimetric information from the signal subspace is obtained after speckle filtering. In this way, edges of different kinds of targets are preserved. The effectiveness of this method is demonstrated using the National Aeronautics and Space Administration Jet Propulsion Laboratory airborne L-band polarimetric SAR data.