Large Language Models (LLMs) have attracted significant attention for on-device applications, delivering strong performance across a variety of real-world tasks. However, hardware constraints on edge devices limit model capacity, often resulting in suboptimal performance. A promising remedy is LLM cascading, where a lightweight local model defers selected hard queries to a more capable server model for response generation. While prior work has primarily optimized the performance--cost trade-off, real-world deployments must also address privacy concerns i.e., user information leakage, a requirement that remains largely overlooked. In this work, we go beyond existing confidence- and logit-based cascade methods and propose P3Defer, a novel Chain-of-Thought (CoT)-enhanced policy learning framework coupled with a private memory for privacy-preserved deferral decision-making. By jointly optimizing for performance, cost, and privacy, our approach improves performance-cost cascade efficiency while mitigating privacy risks. Extensive experiments on three benchmark datasets demonstrate that P3Defer consistently outperforms existing methods in both accuracy and privacy preservation.

Learning multiple domains/tasks with a single model is important for improving data efficiency and lowering inference cost for numerous vision tasks, especially on resource-constrained mobile devices. However, hand-crafting a multi-domain/task model can be both tedious and challenging. This paper proposes a novel approach to automatically learn a multi-path network for multi-domain visual classification on mobile devices. The proposed multi-path network is learned from neural architecture search by applying one reinforcement learning controller for each domain to select the best path in the super-network created from a MobileNetV3-like search space. An adaptive balanced domain prioritization algorithm is proposed to balance optimizing the joint model on multiple domains simultaneously. The determined multi-path model selectively shares parameters across domains in shared nodes while keeping domain-specific parameters within non-shared nodes in individual domain paths. This approach effectively reduces the total number of parameters and FLOPS, encouraging positive knowledge transfer while mitigating negative interference across domains. Extensive evaluations on the Visual Decathlon dataset demonstrate that the proposed multi-path model achieves state-of-the-art performance in terms of accuracy, model size, and FLOPS against other approaches using MobileNetV3-like architectures. Furthermore, the proposed method improves average accuracy over learning single-domain models individually, and reduces the total number of parameters and FLOPS by 78% and 32% respectively, compared to the approach that simply bundles single-domain models for multi-domain learning.