Large Language Models (LLM) incur substantial resource costs during inference, driving widespread interest in Parameter-Efficient Fine-Tuning (PEFT) techniques. Among these, LoRA dramatically reduces overhead by updating only a few low-rank adapters. However, Multi-tenant LoRA LLM inference faces challenges from heterogeneous requests and latency-throughput trade-offs. Moreover, inefficient adapter reuse, poor cache management, and non-adaptive batching strategies severely restrict inference efficiency, service quality, resource utilization, and fairness. To address these challenges, we propose CoLoRA—a collaborative scheduling framework for multi-tenant LoRA LLM inference, comprising four core modules: (1) Adaptive Priority Scheduling (APS), which dynamically integrates queue waiting time, adapter residency status, and SLA urgency to compute task priorities; (2) Adapter-Aware Scheduling (AAS), which enhances cache management by prioritizing SLA-critical, frequently used, and fairly shared adapters, thus reducing cold-start latency and fragmentation; (3) Load-Aware Batch Scheduling (LBS), which combines real-time GPU utilization and queue depth to adaptively form batches and coalesce tasks targeting the same adapter, thereby improving parallelism while controlling latency; and (4) Unified Scheduler (US), which periodically gathers system metadata to orchestrate the submodules collaboratively and employs a feedback loop to optimize global strategies online. Evaluation on realistic multi-tenant workloads and popular open-source LLM shows that CoLoRA, compared to conventional baselines, increases overall system throughput by 56.5%, reduces P95 latency of online requests by 34%, and significantly enhances GPU utilization and tenant-level fairness, demonstrating its promise for large-scale inference services.

Robust variants represent a promising method to enhance model robustness against adversarial attacks through exploring diverse neural network architectures. However, the significant computational demand of training multiple variants often restricts adversarial defense techniques to a narrow range of model architectures, thus failing to fully exploit the robustness benefits of architectural variations. In this paper, we first reveal that function-preserving knowledge transfer can significantly speed up adversarial training of different architecture variants. Then, we propose ROBIN, a framework for accelerating adversarially robust multi-variant training. By utilizing the architectural similarities among variants, ROBIN facilitates efficient weight transformation across models via two tensor-level atomic operations, hastening the convergence of multiple variants. Our experiments indicate that ROBIN can accelerate the adversarial training process of various architecture variants by 2.56 × to 4.27 ×, enabling efficient exploration of robust network architectures.

Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning approach that adaptes pre-trained Large Language Models (LLMs) to multi-tenant tasks by generating a variety of LoRA adapters. However, this approach faces significant security challenges and is particularly susceptible to malicious servers stealing model parameters and sensitive data. Existing research on addressing security risks in multi-tenant environments remains constrained and insufficient. This paper explores the security challenges and proposes the LoRATEE framework, which embeds LoRA adapters within a server-side Trusted Execution Environment (TEE) and employs a lightweight One-Time Pad (OTP) encryption mechanism to ensure secure data transmission. Additionally, we design a dynamic LoRA adapter prefetching mechanism to reduce I/O latency. Moreover, a LoRA adapter module equivalence-sharing strategy based on Parameter-Efficient Fine-Tuning (PEFT) and minimalist design principles was introduced to optimize adapters loading. Experimental results show that LoRATEE maintains inference efficiency while securing multi-tenant LoRA LLMs systems.

To enhance the robustness and accuracy of the small model, existing approaches combine adversarial training with knowledge distillation, introducing a comprehensive single-teacher model to improve the performance of the student model (small model). However, due to the limited knowledge of a teacher model, it appears "knowledge gain saturation" phenomenon. Therefore, we propose a PMP-Guided Adaptive Multi-Teacher Distillation with Meta-Learning. Pontryagin’s Maximum Principle is employed to solve the issue of inconsistent teaching objectives among teachers causing distinct optimization directions. Meanwhile, Meta-learning-network is designed to tackle the problem of a student struggling to balance the learned knowledge. A series of experiments conducted on public datasets demonstrate that our approach outperforms the state-of-the-art methods against various adversarial attacks.

In the face of huge threats from adversarial attacks, developing an efficient defense mechanism is crucial for deep learning systems. Adversarial ensemble defense method is one of the most effective techniques for defending against adversarial attacks, which constructs ensembles of multiple DNNs to improve model's robustness. However, deploying ensemble defense methods on existing DNN inference systems is inefficient and impractical due to their dynamics and randomness. To this end, we propose an inference system for adversarial ensemble defense called Garrison, which can deliver robust and low-latency predictions using Multi-Instance GPUs. Garrison employs a multi-granularity GPU partitioning strategy, optimizing hardware utilization by capitalizing on the intrinsic heterogeneity of GPUs. It also integrates a reinforcement learning-based scheduling mechanism, enabling random ensemble of diverse defense models to enhance robustness while maintaining bounded latency. Our evaluations show that Garrison can improve adversarial robustness by up to 24.5%, while accelerating ensemble inference by up to 6.6X compared to the state-of-the-art inference framework.