The rapid advancement of Large Language Models (LLMs) has significantly enhanced performance across various natural language processing (NLP) tasks, yet the high computational costs and latency associated with deploying such models continue to pose critical bottlenecks, limiting their broader applicability.To mitigate these challenges, we propose a dynamic hybrid inference framework, Firewall Routing, which efficiently selects between a strong and a weak LLMs based on the complexity of the query.A lightweight routing model is trained to optimize resource allocation by learning from response quality and preventing longtail queries, which are often too hard to solve by LLMs, from being routed to the stronger model.Moreover, our method incorporates multiple sampling to enhance query evaluation reliability while leveraging Hard Blocking and Soft Blocking to handle long-tail queries along with refining labels for model selection.Extensive experiments show our method outperforms existing routing strategies by up to 5.29% in APGR, demonstrating state-of-the-art performance across multiple benchmarks.

Post-training processes are essential phases in grounding pre-trained language models to real-world tasks, with learning from demonstrations or preference signals playing a crucial role in this adaptation. We present a unified theoretical framework bridging Supervised Fine-Tuning (SFT) and preference learning in Large Language Model (LLM) post-training. Through rigorous mathematical derivation, we demonstrate that both SFT and preference learning methods like Direct Preference Optimization (DPO) operate within the same optimal policy-reward subspace, with SFT representing a special case of implicit reward learning. Our analysis reveals a critical limitation in conventional SFT: the KL divergence term in distribution matching becomes constant with respect to the policy during optimization, failing to constrain model updates. To address this, we propose a simple yet effective learning rate reduction approach that yields significant performance improvements (up to \textbf{25\%} relative gain and \textbf{6\%} absolute win rate increase in instruction following tasks. Additionally, we derive alternative SFT objectives from various f-divergence functions that preserve the KL term during optimization, further enhancing post-DPO model performance. Finally, we extend the theoretical relationship between LLM logits and Q-functions from preference learning to the SFT context, providing mathematical derivations and experimental validation.

Human parsing and pose estimation, regarded as two fundamental tasks to analyze human in the wild, are the basis of upper-level tasks, such as human action recognition and person re-identification. The lack of a comprehensive multi-person dataset, which contains the annotations of both human part labels and skeleton keypoint labels, makes that most of the joint learning work for pose estimation and human parsing can only focus on the single-person scene. To fill this gap, we proposed a comprehensive multi-person dataset Look into Multi-Person (LIMP) with 10,082 multi-person images. And we adopt data augmentation strategies to enrich the dataset's diversity which contains more texture, more occlusion, and more in-the-wild images than CIHP. To the best of our knowledge, this is the first comprehensive multi-person dataset for pose estimation and human parsing.