Large Language Models are widely utilized in natural language processing tasks but often necessitate expensive fine-tuning to accommodate specific domains. Parameter-efficient fine-tuning (PEFT) techniques help mitigate these costs by reducing the magnitude of trainable parameters. Meanwhile, some recent studies have employed Multi-Task Learning (MTL) to address issues of data scarcity and imbalance in PEFT, demonstrating its effectiveness in overcoming these challenges. The Mixture of Experts (MoE) approach further enhances knowledge sharing in MTL by dynamically assigning experts to different tasks. However, existing MoE approaches face challenges in accurately routing experts due to suboptimal routing methods and random routing. To address these limitations, we propose the Dual Routing Strategy for Mixture of Experts (DRMOE). DRMOE proposes a Dual Routing Strategy to refine expert allocation and constructs a Task-Specified Loss to alleviate randomness in routing decisions. Extensive experiments show that DRMOE effectively enhances performance across tasks, demonstrating its robustness in multi-task scenarios. The code is available at https://github.com/lhyscau/DRMOE.

J1C2 Presentation Abstract at IEEE SERVICES 2022 for IEEE Transactions on Services Computing, doi: 10.1109/TSC.2021.3086023

Traditional data sharing systems are facing new challenges when implementing access control with more and more complex data sharing requirements. Flexibility of user revocation in completely decentralized environments needs to be taken into account. In this article, we propose a Key-Policy Attribute-Based Encryption scheme with Multiple and Flexible Revocation (MAFR-KP-ABE) to achieve the features of decentralized authorization and flexible revocation. We prove the security of our MAFR-KP-ABE scheme in the standard model and provide the comparison with relevant schemes to demonstrate its efficiency. Then we propose a fine-grained access control system based on MAFR-KP-ABE scheme and blockchain that matches the need of paid data sharing services with several security properties enhanced. Security analysis and system implementation are given subsequently to demonstrate our system efficient and secure.