Several studies have demonstrated that large language models (LLMs) exhibit positional bias when answering multiple-choice questions (MCQs). Previous methods have identified such bias to be detrimental, leading to the development of techniques to mitigate it. However, we observe that certain permutations of options can actually improve the performance. Therefore, instead of eliminating such bias, we propose an EMbracing the Bias EquivaRiantly (EMBER) network. Specifically, the EMBER network, which outputs a permutation of options in MCQs, is optimized towards the beneficial permutations to which the LLM is biased. Additionally, to solve the positional bias among different permutations of options, the EMBER network is designed to grant the equivariance to the permutation to the LLMs. Theoretically and empirically, we show that the proposed EMBER network can effectively utilize the positional bias and demonstrate state-of-the-art performance over various baselines.

Federated Semi-Supervised Learning alleviates the necessity for fully labeled data in Federated Learning. However, it does not sufficiently prioritize model privacy or the personalized requirements of clients. To address these concerns, our key idea is to communicate no longer individual model parameters but their black-box models, implying to provide other clients with only an input-output interface of individual models. The communication mechanism is model-agnostic, thereby facilitating adaption to heterogeneous models for each client through customization. Consequently, we propose a framework called Personalized Federated Semi-Supervised Learning with Black-Box Models (B2PFSSL) to enhance the privacy of communication. To prevent negative knowledge transfer due to data heterogeneity, we design a two-stage strategy that filters at both the model and data levels, enabling clients to obtain large training datasets by including more high-quality pseudo-labeled data under conditions of scarce labeled data. The experimental results indicate that B2PFSSL achieves competitive performance while reducing the amount of information exposed during communication. Furthermore, it can foster productive collaboration among diverse model architectures in model heterogeneous Federated Learning.