Model merging enables powerful capabilities in neural networks without requiring additional training.In this paper, we introduce a novel perspective on model merging by leveraging the fundamental mechanisms of neural network representation.Our approach is motivated by the linear representation hypothesis, which states that neural networks encode information through linear combinations of feature vectors.We propose a method that superposes task-specific features from individual models into a merged model.Our approach specifically targets linear transformation matrices, which are crucial for feature activation and extraction in deep networks.By formulating the merging process as a linear system, we can preserve task-specific features from individual models and create merged models that effectively maintain multi-task capabilities compared to existing methods.Extensive experiments across diverse benchmarks and models demonstrate that our method outperforms existing techniques.Code is available at https://github.com/LARS-research/STF. and task vectors (Ilharco et al., 2022;Du et al., 2024).However, these methods primarily focus on parameter-level operations and do not explicitly incorporate the fundamental working mechanisms of neural networks in their design.We argue that a principled approach to model merging should be conditioned on how deep neural networks represent and process information.Therefore, to design our merging method, we draw upon e n c .0 e n c .1 e n c .2 e n c .3 e n c .4 e n c .5 e n c .6 e n c .7 e n c .8 e n c .9 e n c .1 0 e n c .1 1 d e c .0 d e c .1 d e c .2 d e c .3 d e c .4 d e c .5 d e c .6 d e c .7 d e c .8 d e c .9 d e c .1 0 d e c .1 1 Layer 0.0 0.2 0.4 0.6 0.8 1.0 " * 𝐮 !" 𝐯 !" $ % !"&' ( !&' Merged Matrix 𝐌 (c) Identify task-specific features (e) Merging by solving linear system Task 1 Task T

Deep learning's success is accompanied by challenges in interpretability and efficiency. This research explores disentanglement mechanisms in deep learning across three core dimensions: model expressivity, optimization paradigms, and interpretability. We first analyze how Graph Neural Networks learn logical rules through representation disentanglement, establishing theoretical foundations for their expressivity. We then develop efficient parameter merging strategies by leveraging feature decomposition in network parameters. Finally, we design neural architectures aligned with symbolic formulas for modeling complex network dynamics, enhancing transparency through architecture disentanglement. Our research provides both theoretical insights and practical methodologies for building more interpretable and efficient AI systems. Future work will further advance these directions through precision-focused expressivity analysis, semantic-aware parameter optimization, and crossdomain applications of interpretable modeling.

Rule learning is critical to improving knowledge graph (KG) reasoning due to their ability to provide logical and interpretable explanations. Recently, Graph Neural Networks (GNNs) with tail entity scoring achieve the state-of-the-art performance on KG reasoning. However, the theoretical understandings for these GNNs are either lacking or focusing on single-relational graphs, leaving what the kind of rules these GNNs can learn an open problem. We propose to fill the above gap in this paper. Specifically, GNNs with tail entity scoring are unified into a common framework. Then, we analyze their expressivity by formally describing the rule structures they can learn and theoretically demonstrating their superiority. These results further inspire us to propose a novel labeling strategy to learn more rules in KG reasoning. Experimental results are consistent with our theoretical findings and verify the effectiveness of our proposed method. The code is publicly available at https://github.com/LARS-research/Rule-learning-expressivity.

In this paper, we study nonconvex tensor robust principal component analysis (RPCA) based on the $t$-SVD. We first propose an alternating projection method, i.e., APT, which converges linearly to the ground-truth under the incoherence conditions of tensors. However, as the projection to the low-rank tensor space in APT can be slow, we further propose to speedup such a process by utilizing the property of the tangent space of low-rank. The resulting algorithm, i.e., EAPT, is not only more efficient than APT but also keeps the linear convergence. Compared with existing tensor RPCA works, the proposed method, especially EAPT, is not only more effective due to the recovery guarantee and adaption in the transformed (frequency) domain but also more efficient due to faster convergence rate and lower iteration complexity. These benefits are also empirically verified both on synthetic data, and real applications, e.g., hyperspectral image denoising and video background subtraction.

Low-rank tensor recovery (LRTR) is a natural extension of low-rank matrix recovery (LRMR) to high-dimensional arrays, which aims to reconstruct an underlying tensor from incomplete linear measurements M(X). However, LRTR ignores the error caused by quantization, limiting its application when the quantization is low-level. In this work, we take into account the impact of extreme quantization and suppose the quantizer degrades into a comparator that only acquires the signs of M(X). We still hope to recover X from these binary measurements. Under the tensor Singular Value Decomposition (t-SVD) framework, two recovery methods are proposedthe first is a tensor hard singular tube thresholding method; the second is a constrained tensor nuclear norm minimization method. These methods can recover a real n1 n2 n3 tensor X with tubal rank r from m random Gaussian binary measurements with errors decaying at a polynomial speed of the oversampling factor := m/((n1+ n2)n3r). To improve the convergence rate, we develop a new quantization scheme under which the convergence rate can be accelerated to an exponential function of . Numerical experiments verify our results, and the applications to real-world data demonstrate the promising performance of the proposed methods.

Snapshot compressive imaging (SCI) is a new type of compressive imaging system that compresses multiple frames of images into a single snapshot measurement, which enjoys low cost, low bandwidth, and high-speed sensing rate. By applying the existing SCI methods to deal with hyperspectral images, however, could not fully exploit the underlying structures, and thereby demonstrate unsatisfactory reconstruction performance. To remedy such issue, this paper aims to propose a new effective method by taking advantage of two intrinsic priors of the hyperspectral images, namely deep image denoising and total variation (TV) priors. Specifically, we propose an optimization objective to utilize these two priors. By solving this optimization objective, our method is equivalent to incorporate a weighted FFDNet and a 2DTV or 3DTV denoiser into the plug-andplay framework. Extensive numerical experiments demonstrate the outperformance of the proposed method over several state-of-the-art alternatives. Additionally, we provide a detailed convergence analysis of the resulting plug-andplay algorithm under relatively weak conditions such as without using diminishing step sizes. The code is available at https://github.com/ucker/SCI-TVFFDNet.

In this paper, the generative model is used to introduce the structural properties of the signal to replace the common sparse hypothesis, and a non-convex compressed sensing sparse deviation model based on the generative model (ℓq-Gen) is proposed. By establishing ℓqvariant of the restricted isometry property (q-RIP) and Set-Restricted Eigenvalue Condition (q-S-REC), the error upper bound of the optimal decoder is derived when the recovered signal is within the sparse deviation range of the generator. Furthermore, it is proved that the Gaussian matrix satisfying a certain number of measurements is sufficient to ensure a good recovery for the generating function with high probability. Finally, a series of experiments are carried out to verify the effectiveness and superiority of the ℓq-Gen model.