With the rapid proliferation of large language models (LLMs), model pools have become increasingly heterogeneous in both capability and efficiency.Larger LLMs can improve quality but incur higher latency and cost, while smaller LLMs are the opposite, making perquery model selection crucial in practice.This has spawned LLM routers that dispatch each query to an appropriate model.Existing routers lack fine-grained resource awareness across deployment settings, which degrades efficiency metrics in real-world serving.To this end, we propose FLARE, a length-centric, resourceaware multi-LLM routing framework that uses length-based models to estimate per-query latency and cost.FLARE formulates routing as a discrete multi-objective optimization problem to achieve an efficient trade-off.Experiments show that FLARE reduces latency and cost by up to 68% and 75% while achieving sufficient accuracy, and can be easily applied to new datasets and LLMs.

Kolmogorov–Arnold Networks (KANs) promise higher expressive capability and stronger interpretability than Multilayer Perceptron, particularly in the domain of AI for Science. However, practical adoption has been hindered by low GPU utilization of existing parallel implementations. To address this challenge, we present a GPU-accelerated operator library, named PolyKAN, which is the first general open-source implementation of KAN and its variants. PolyKAN fuses the forward and backward passes of polynomial KAN layers into a concise set of optimized CUDA kernels. Four orthogonal techniques underpin the design: (i) lookup-table with linear interpolation that replaces runtime expensive math-library functions; (ii) 2D tiling to expose thread-level parallelism with preserving memory locality; (iii) a two-stage reduction scheme converting scattered atomic updates into a single controllable merge step; and (iv) coefficient-layout reordering yielding unit-stride reads under the tiled schedule. PolyKAN can deliver 1.2–10 × faster inference and 1.4–12 × faster training than a Triton + cuBLAS baseline, with identical accuracy on speech, audio-enhancement, and tabular-regression workloads on both highend GPU and consumer-grade GPU.

Deep Neural Network (DNN) inference has become a cornerstone of latency-sensitive applications such as autonomous driving and augmented reality. While GPUs offer high throughput for DNN inference, they often suffer from underutilization due to coarse-grained scheduling and limited concurrency. Existing GPU-sharing methods either lack awareness of fine-grained kernel interference or fail to meet service-level objectives (SLOs) under multi-tenant, multi-priority workloads. In this paper, we propose IasRT, a runtime framework that enables interferenceaware and SLO-driven GPU sharing for real-time DNN inference. IasRT profiles kernel-level resource usage and interference sensitivity, and dynamically partitions GPU streaming multiprocessors (SMs) to collocate jobs with minimal performance degradation. Furthermore, it introduces a dynamic SLO controller to maintain latency targets for multiple latency-sensitive (LS) jobs simultaneously. Evaluations on real-world DNN workloads show that IasRT reduces the 99th percentile latency of LS jobs by up to 38% compared to the state-of-the-art GPU sharing methods, while maintaining similar overall throughput from multiple collocated workloads, demonstrating its effectiveness in high-concurrency environments.