Heterogeneous Large Language Model (LLM) fusion integrates the strengths of multiple source LLMs with different architectures into a target LLM with low computational overhead. While promising, existing methods suffer from two major limitations: 1) **reliance on real data from limited domain** for knowledge fusion, preventing the target LLM from fully acquiring knowledge across diverse domains, and 2) **fixed data allocation proportions** across domains, failing to dynamically adjust according to the target LLM's varying capabilities across domains, leading to a capability imbalance. To overcome these limitations, we propose Bohdi, a synthetic-data-only heterogeneous LLM fusion framework. Through the organization of knowledge domains into a hierarchical tree structure, Bohdi enables automatic domain exploration and multi-domain data generation through multi-model collaboration, thereby comprehensively extracting knowledge from source LLMs. By formalizing domain expansion and data sampling proportion allocation on the knowledge tree as a Hierarchical Multi-Armed Bandit problem, Bohdi leverages the designed DynaBranches mechanism to adaptively adjust sampling proportions based on the target LLM's performance feedback across domains. Integrated with our proposed Introspection-Rebirth (IR) mechanism, DynaBranches dynamically tracks capability shifts during target LLM's updates via Sliding Window Binomial Likelihood Ratio Testing (SWBLRT), further enhancing its online adaptation capability. Comparative experimental results on a comprehensive suite of benchmarks demonstrate that Bohdi significantly outperforms existing baselines on multiple target LLMs, exhibits higher data efficiency, and virtually eliminates the imbalance in the target LLM's capabilities.

Abstract. Deep learning-based image denoising models demonstrate remarkable performance, but their lack of robustness analysis remains a significant concern. A major issue is that these models are susceptible to adversarial attacks, where small, carefully crafted perturbations to input data can cause them to fail. Surprisingly, perturbations specifically crafted for one model can easily transfer across various models, including convolutional neural networks, transformers, unfolding models, and plug-and-play models, leading to failures in those models as well. Such high adversarial transferability is not observed in classification models. We analyze the possible underlying reasons behind the high adversarial transferability through a series of hypotheses and validation experiments. By characterizing the manifolds of Gaussian noise and adversarial perturbations using the concept of a typical set and the asymptotic equipartition property, we prove that adversarial samples deviate slightly from the typical set of the original input distribution, causing the models to fail. Based on these insights, we propose a novel adversarial defense method: the out-of-distribution typical set sampling (TSS) training strategy. TSS training strategy not only significantly enhances the model’s robustness but also marginally improves denoising performance compared to the original model.

Pansharpening aims to recover the spectral information and spatial details of high-resolution multispectral (HRMS) images with high accuracy. In this context, we design a triple-scale multi-level network guided by Hessian spatial loss for pansharpening (HTMP). Firstly, we propose a novel Hessian spatial loss designed to establish deep mapping relationships in the spatial domain. Hessian spatial loss guides the network in enhancing its ability to characterize the edges of blurred regions while maintaining the consistency of spatial details. Secondly, we employ a triple-scale multi-level feature extraction network (TMFENet) to obtain comprehensive spectral and spatial features, thereby enhancing the interaction of multi-scale contextual information from coarse-grained to fine-grained scales. To efficiently integrate and represent cross-modal long-distance information, the spectral and spatial features are treated as a whole and input into the triple-scale adaptive sparse transformer (TASTrans) to extract global features. Finally, the multi-level image reconstruction network (MIRNet) combines multi-modal features at different resolutions to progressively generate HRMS images rich in semantics. Experiments performed on datasets demonstrate that our method produces fused images with superior visual quality compared to state-of-the-art methods. Furthermore, it is obvious from the ablation experiments that incorporating the Hessian spatial loss significantly enhances the fusion performance of deep learning models.

The rapid development of deep learning has significantly advanced Synthetic Aperture Radar (SAR) despeckling techniques. However, as the uncertainty and vulnerability of the network structure is ignited by the fuse of adversarial attacks, its authenticity and widespread applicability are subsequently drawn into question. This study examines the robustness and performance of deep learning-based image despeckling models under a denoising-PGD adversarial attack. Furthermore, we investigate the impact of varying feature extraction approaches on model performance, with the goal of providing reliable guidelines for model training.

Abstract Image denoising is one of the fundamental problems in image processing. Convolutional neural network (CNN) based denoising approaches have achieved better performance than traditional methods, such as STROLLR and BM3D. However, CNNs can easily bring unexplainable artifacts to denoised images. In this article, a Deep Learning‐Informed Diffusion Equation (DLI‐DE) framework utilizing the image prior or the image gradient prior for image denoising is proposed. The image priors and gradient priors are learned from CNN models and used as coefficients in diffusion equations. The solution of DLI‐DE is infinitely smooth from the uniqueness of existence theorem, which guarantees that the denoised image is free of artifacts. Good properties of DLI‐DE also ensure high‐quality of denoising. The experimental analysis confirms that the denoising performance of DLI‐DE is comparable to that of contemporary CNN‐based denoising methods such as TNRD and DnCNN, while effectively preventing artifacts.

Pansharpening, as a widely used technique, plays a crucial role in the field of remote sensing image processing. In this letter, we propose a novel structure tensor-driven block-based variational pansharpening model with adaptive coefficients. First, a structure descriptor derived from the structure tensor of the panchromatic (PAN) image is integrated into the regularization term. It can not only effectively capture the edge information of the PAN image, but also contribute to better preserving the spatial details of the PAN image in the fused product. Furthermore, we partition the degraded PAN image and the upsampled multispectral (MS) image into several equal-sized blocks and utilize a regression-based approach to calculate the adaptive coefficients within each block. As a result, a more accurate constraint relationship between the PAN image and the high-resolution MS image can be established. Then, by incorporating the regularization term and the fidelity term, the proposed variational model is formulated. An explicit finite difference scheme is employed to efficiently solve the gradient descent flow of the proposed model. Experiments conducted on different datasets demonstrate that the proposed pansharpening method outperforms the state-of-the-art techniques in both qualitative and quantitative evaluations.

In the hyperspectral image classification (HSIC) task, the most commonly used model validation paradigm is partitioning the training-test dataset through pixelwise random sampling. By training on a small amount of data, the deep learning model can achieve almost perfect accuracy. However, in our experiments, we found that the high accuracy was reached because the training and test datasets share a lot of information. On nonoverlapping dataset partitions, well-performing models suffer significant performance degradation. To this end, we propose a spectral-spatial axial aggregation transformer model, namely, SaaFormer, which preserves generalization across dataset partitions. SaaFormer applies a multilevel spectral extraction structure to segment the spectrum into multiple spectrum clips such that the wavelength continuity of the spectrum across the channel is preserved. For each spectrum clip, the axial aggregation attention mechanism, which integrates spatial features along multiple spectral axes, is applied to mine the spectral characteristic. The multilevel spectral extraction and the axial aggregation attention emphasize spectral characteristics to improve the model generalization. The experimental results on five publicly available datasets demonstrate that our model exhibits comparable performance on the random partition while significantly outperforming other methods on nonoverlapping partitions. Moreover, SaaFormer shows excellent performance on background classification.

The transferability of adversarial perturbations provides an effective shortcut for black-box attacks. Targeted perturbations have greater practicality but are more difficult to transfer between models. In this paper, we experimentally and theoretically demonstrated that neural networks trained on the same dataset have more consistent performance in High-Sample-Density-Regions (HSDR) of each class instead of low sample density regions. Therefore, in the target setting, adding perturbations towards HSDR of the target class is more effective in improving transferability. However, density estimation is challenging in high-dimensional scenarios. Further theoretical and experimental verification demonstrates that easy samples with low loss are more likely to be located in HSDR. Perturbations towards such easy samples in the target class can avoid density estimation for HSDR location. Based on the above facts, we verified that adding perturbations to easy samples in the target class improves targeted adversarial transferability of existing attack methods. A generative targeted attack strategy named Easy Sample Matching Attack (ESMA) is proposed, which has a higher success rate for targeted attacks and outperforms the SOTA generative method. Moreover, ESMA requires only $5\%$ of the storage space and much less computation time comparing to the current SOTA, as ESMA attacks all classes with only one model instead of seperate models for each class. Our code is available at https://github.com/gjq100/ESMA

The curvature regularization method is well-known for its good geometric interpretability and strong priors in the continuity of edges, which has been applied to various image processing tasks. However, due to the non-convex, non-smooth, and highly non-linear intrinsic limitations, most existing algorithms lack a convergence guarantee. This paper proposes an efficient yet accurate scalar auxiliary variable (SAV) scheme for solving both mean curvature and Gaussian curvature minimization problems. The SAV-based algorithms are shown unconditionally energy diminishing, fast convergent, and very easy to be implemented for different image applications. Numerical experiments on noise removal, image deblurring, and single image super-resolution are presented on both gray and color image datasets to demonstrate the robustness and efficiency of our method. Source codes are made publicly available athttps://github.com/Duanlab123/SAV-curvature.

In recent years, deep learning algorithms for speckle noise removal have attracted much attention. However, speckle noise is strongly heavy-tailed and signal dependent, which makes it difficult to remove. In this paper, we propose a two-stream convolutional neural network with hybrid truncation loss to eliminate multiplicative noise (HTNet). HTNet combines the major task of multiplicative noise removal and the auxiliary task of noise estimation to improve the despeckling effect while preserving texture details. The main branch of HTNet is composed of a feature extraction block and an improved U-Net that can extract multi-scale information, which is mainly used for speckle noise removal. The noise estimation auxiliary branch is designed to fit the speckle noise. A hybrid truncation loss function is to applied for robust estimation of heavy-tailed distribution instead of mean squared error. Extensive experimental results show that HTNet can effectively remove speckle noise and outperforms the state-of-the-art methods on both simulated and real SAR images. In addition, HTNet has advantages on textured images.

Multiplicative noise removal problems have attracted much attention in recent years. Unlike additive noise removal problems, multiplicative noise destroys almost all information of the original image, especially for texture images. In this paper, a fractional-order nonlinear diffusion model is proposed to denoise the texture images corrupted by multiplicative noise. In the model, a gray level indicator is introduced to remove multiplicative noise and preserve structure details for texture images. By virtue of the discrete Fourier transform, the model is solved by an iterative scheme in the frequency domain. Then an algorithm in the spatial domain is developed based on the definition of the Grünwald--Letnikov fractional-order derivative. Inspired by the discrepancy principle used for additive noise, we develop a new stopping criterion based on the mean and variance of the noise. Numerical examples are presented to demonstrate the effectiveness and efficiency of the proposed method. Experimental results show that the proposed model can handle multiplicative noise removal and texture preservation quite well.

This paper considers colorization-based image compression in RGB color space. In compression, we store only the compressed luminance component of the original color image and a few representative pixels extracted from the original color image. In decompression, by explicitly introducing the relation between the luminance component and the original color image into diffusion equations, a linear reaction-diffusion system with Perona--Malik type diffusion coefficient is proposed to reconstruct R, G, and B channels simultaneously. The Perona--Malik type diffusion coefficient is a function of the luminance component and leads to interior degenerations, in general. It yields anisotropic smoothing in the restored color image and constrains the geometry of the restored image to follow the geometry of the luminance component. The existence and uniqueness of solutions for the proposed system with a specific class of diffusion coefficients are proved in a weighted Sobolev space. The selection of representative pixels has a big impact on reconstruction results. We also propose a local-optimal strategy that splits the original color image into a series of different size subimages and searches the optimal representative pixel in each subimage. Comparisons with recent colorization-based image compression methods, as well as transform-based JPEG and JPEG2000 standards, are performed to show the potential for successful compression applications of the proposed method.

Multiplicative noise removal is a challenging task in image processing. Inspired by the impressive performance of nonlinear diffusion models in additive noise removal, we address this problem in the view of nonlinear diffusion equation theories rather than the traditional variation methods. We develop a nonlinear diffusion filter denoising framework, which considers not only the information of the gradient of the image, but also the information of gray levels of the image. Furthermore, under this framework, we propose a doubly degenerate diffusion model for multiplicative noise removal, which is analyzed with respect to some of its properties and behavior in denoising process. In numerical aspects, we present an efficient scheme which uses a stabilization by fast explicit diffusion for the implementation of the multiplicative noise removal model. Finally, the experimental results illustrate effectiveness and efficiency of the proposed model.

This paper introduces a class of adaptive Perona-Malik (PM) diffusion, which combines the PM equation with the heat equation. The PM equation provides a potential algorithm for image segmentation, noise removal, edge detection, and image enhancement. However, the defect of traditional PM model is tending to cause the staircase effect and create new features in the processed image. Utilizing the edge indicator as a variable exponent, we can adaptively control the diffusion mode, which alternates between PM diffusion and Gaussian smoothing in accordance with the image feature. Computer experiments indicate that the present algorithm is very efficient for edge detection and noise removal.