The "Fine-Tuning-as-a-Service" paradigm exposes large language models to catastrophic safety degradation from less harmful samples.Alignment-stage defenses address this by proactively injecting adversarial perturbations to bolster the model's inherent robustness against harmful drift.However, existing methods rely on perturbation directions that often conflict with harmful gradients, inadvertently facilitating the acquisition of malicious features rather than suppressing them.To address this issue, we propose Orthogonal and Adaptive Safety Alignment Strategy (OASIS) to mathematically decouple safety enforcement from harmful feature acquisition.By projecting perturbations orthogonal to harmful gradients and concentrating optimization on adaptively selected safetycritical layers, OASIS effectively resolves directional conflicts while maximizing parameter efficiency.Extensive experiments on four LLMs across three datasets (SST2, GSM8K, and AGNews) demonstrate that OASIS reduces the Harmful Score by approximately 60% compared to competitive baselines, while maintaining stable downstream task utility.Our code is publicly available at https://github. com/xiaoroyi/OASIS.

Large language models (LLMs) have gained a lot of attention and achievements recently because of their significant comprehension and generative abilities. However, the large-scale parameters of LLMs require considerable computational resources in the training and inference process, which restricts their wide application. To overcome this challenge, we propose an efficient mixed precision weight quantization (EMWQ) method for LLMs in this article. Specifically, we introduce a new outlier detection method by analyzing the weight distribution instead of the conventional weight magnitude. Then, we propose a dual-quantization strategy that quantizes both the outlier critical columns and the residual matrices with different precision. Besides, we introduce two effective EMWQ-based application frameworks, the EMWQ-R and EMWQ-O in our study. Comprehensive experiments are conducted on the Penn Treebank (PTB), C4, ARC-Easy datasets, and MMLU benchmark across various tasks. The comparison results demonstrate that the proposed EMWQ achieves state-of-the-art performance in mixed precision quantization and further reduces computational memory cost. Besides, it has higher generalizability compared with conventional methods.