A prevalent approach in Parameter-Efficient Fine-Tuning (PEFT) of pre-trained Vision Transformers (ViT) involves freezing the majority of the backbone parameters and solely learning low-rank adaptation weight matrices to accommodate downstream tasks. These low-rank matrices are commonly derived through the multiplication structure of down-projection and up-projection matrices, exemplified by methods such as LoRA and Adapter. In this work, we observe an approximate orthogonality among any two row or column vectors within any weight matrix of the backbone parameters; however, this property is absent in the vectors of the down/up-projection matrices. Approximate orthogonality implies a reduction in the upper bound of the model's generalization error, signifying that the model possesses enhanced generalization capability. If the fine-tuned down/up-projection matrices were to exhibit this same property as the pre-trained backbone matrices, could the generalization capability of fine-tuned ViTs be further augmented? To address this question, we propose an Approximately Orthogonal Fine-Tuning (AOFT) strategy for representing the low-rank weight matrices. This strategy employs a single learnable vector to generate a set of approximately orthogonal vectors, which form the down/up-projection matrices, thereby aligning the properties of these matrices with those of the backbone. Extensive experimental results demonstrate that our method achieves competitive performance across a range of downstream image classification tasks, confirming the efficacy of the enhanced generalization capability embedded in the down/up-projection matrices.

Multi-source sensors, such as LiDAR, cameras, Inertial Measurement Units (IMU), and suspension displacement sensors, can describe road surface characteristics from different dimensions. Sensor fusion, which incorporates vehicle dynamic characteristics, is a key to improving the accuracy of road surface identification. Therefore, we propose a road surface identification method that combines segmented image features, statistically analyzed and extracted LiDAR features, and vehicle state features, with suspension displacement serving as a supervisory signal. The visual features from cameras and LiDAR inputs are extracted using the Transfuser backbone. Meanwhile, vehicle state features are encoded separately using Fourier Feature Mapping and a Multilayer Perceptron (MLP), and are subsequently fused with the visual features. The accuracy of road surface identification is further improved through the use of a state feature dropout module and a suspension displacement supervision module during the training process. Experimental results show that our method effectively combines multi-source sensor information and achieves higher accuracy in road surface identification compared to single-sensor-based methods and other multi-sensor fusion approaches. Furthermore, comparative road surface identification tests under constant and variable vehicle speeds, conducted under the same road conditions, demonstrate that our method is not affected by the type of vehicle motion, due to the supervision module based on the decoupled suspension displacement signal.

A common strategy for Parameter-Efficient Fine-Tuning (PEFT) of pre-trained Vision Transformers (ViTs) involves adapting the model to downstream tasks by learning a low-rank adaptation matrix. This matrix is decomposed into a product of down-projection and up-projection matrices, with the bottleneck dimensionality being crucial for reducing the number of learnable parameters, as exemplified by prevalent methods like LoRA and Adapter. However, these low-rank strategies typically employ a fixed bottleneck dimensionality, which limits their flexibility in handling layer-wise variations. To address this limitation, we propose a novel PEFT approach inspired by Singular Value Decomposition (SVD) for representing the adaptation matrix. SVD decomposes a matrix into the product of a left unitary matrix, a diagonal matrix of scaling values, and a right unitary matrix. We utilize Householder transformations to construct orthogonal matrices that efficiently mimic the unitary matrices, requiring only a vector. The diagonal values are learned in a layer-wise manner, allowing them to flexibly capture the unique properties of each layer. This approach enables the generation of adaptation matrices with varying ranks across different layers, providing greater flexibility in adapting pre-trained models. Experiments on standard downstream vision tasks demonstrate that our method achieves promising fine-tuning performance.

In this paper, we consider the problem of joint beam selection and link activation across a set of communication pairs to effectively control the interference between communication pairs via inactivating part communication pairs in ultra-dense device-to-device (D2D) mmWave communication networks. The resulting optimization problem is formulated as an integer programming problem that is nonconvex and NP-hard. Consequently, the global optimal solution, even the local optimal solution, cannot be generally obtained. To overcome this challenge, this paper resorts to design a deep learning architecture based on graph neural network to finish the joint beam selection and link activation, with taking the network topology information into account. Meanwhile, we present an unsupervised Lagrangian dual learning framework to train the parameters of the GBLinks model. Numerical results show that the proposed GBLinks model can converge to a stable point with the number of iterations increases, in terms of the weighted sum rate. Furthermore, the GBLinks model can reach near-optimal solutions through comparing with the exhaustive scheme in small-scale ultra-dense D2D mmWave communication networks and outperforms GreedyNoSched and the SCA-based method. It also shows that the GBLinks model can generalize to varying network densities and network coverage regions of ultra-dense D2D mmWave communication networks.

Let d be a positive integer and x a real number. Let $$A_{d, x}$$ be a $$d\times 2d$$ matrix with its entries $$\begin{aligned} a_{i,j}=\left\{ \begin{array}{ll} x\ \ &{} \quad \text{ for } \ 1\leqslant j\leqslant d+1-i,\\ 1\ \ &{} \quad \text{ for } \ d+2-i\leqslant j\leqslant d+i,\\ 0\ \ &{} \quad \text{ for } \ d+1+i\leqslant j\leqslant 2d. \end{array} \right. \end{aligned}$$ Further, let $$R_d$$ be a set of sequences of integers as follows: $$\begin{aligned} R_d=\left\{ (\rho _1, \rho _2,\ldots , \rho _d)|1\leqslant \rho _i\leqslant d+i, 1\leqslant i \leqslant d,\ \text{ and }\ \rho _r\ne \rho _s\ \quad \text{ for }\ r\ne s\right\} . \end{aligned}$$ and define $$\begin{aligned} \Omega _d(x)=\sum _{\rho \in R_d}a_{1,\rho _1}a_{2, \rho _2}\ldots a_{d,\rho _d}. \end{aligned}$$ In order to give a better bound on the size of spheres of permutation codes under the Chebychev distance, Kløve introduced the above function and conjectured that $$\begin{aligned} \Omega _d(x)=\sum _{m=0}^d{d\atopwithdelims ()m}(m+1)^d(x-1)^{d-m}. \end{aligned}$$ In this paper, we settle down this conjecture positively.

Various kinds of secure codes and their related hash families are broadly studied combinatorial structures for protecting copyrighted materials. The codewords in such a structure can be regarded as a subset of$Q^{N}$, the set of all$q$-ary vectors of given length$N$, satisfying some constraints. We use a hypergraph model to characterize the combinatorial structure. By applying a result of Dukeet al.on the lower bound of the independence number of hypergraphs, we provide a new approach to evaluate the lower bounds for several kinds of secure codes and related hash families. In particular, the general method is illustrated via the examples of existence results on some perfect hash families, frameproof codes, and separable codes.

Permutation codes have been shown to be useful in power line communications, block ciphers, and multilevel flash memory models. Construction of such codes is extremely difficult. In fact, the only general lower bound known is the Gilbert-Varshamov type bound. In this paper, we establish a connection between permutation codes and independent sets in certain graphs. Using the connection, we improve the Gilbert-Varshamov bound asymptotically by a factor log(n), when the code lengthngoes to infinity.

The routing number $rt(G)$ of a connected graph G is the minimum integer r so that every permutation of vertices can be routed in r steps by swapping the ends of disjoint edges. In this paper, we study the routing numbers of cycles, complete bipartite graphs, and hypercubes. We prove that $rt(C_n)=n-1$ (for $n\geq3$) and for $s\geq t$, $rt(K_{s,t})=\lfloor\frac{3s}{2t}\rfloor+O(1)$. We also prove $n+1\leq rt(Q_n)\leq2n-2$ for $n\geq3$. The lower bound $rt(Q_n)\geq n+1$ was previously conjectured by Alon, Chung, and Graham [SIAM J. Discrete Math., 7 (1994), pp. 513–530]. A variation, called fractional routing number, is also considered in this paper.

Sorting permutations by transpositions is an important and difficult problem in genome rearrangements. The transposition diameter $TD(n)$ is the maximum transposition distance among all pairs of permutations in $S_n$. It was previously conjectured [H. Eriksson et al., Discrete Math., 241 (2001), pp. 289–300] that $TD(n)\leq\lceil\frac{n+1}{2}\rceil$. This conjecture was disproved by Elias and Hartman [IEEE/ACM Trans. Comput. Biol. Bioinform., 3 (2006), pp. 369–379] by showing $TD(n)\geq\lfloor\frac{n+1}{2}\rfloor+1$. In this paper we improved the lower bound to $TD(n)\geq\frac{17}{33}n+\frac{1}{33}$ via computation.