Recently, Direct Alignment Algorithms (DAAs) such as Direct Preference Optimization (DPO) have emerged as alternatives to the standard Reinforcement Learning from Human Feedback (RLHF) for aligning large language models (LLMs) with human values. Surprisingly, while DAAs do not use a separate proxy reward model as in RLHF, their performance can still deteriorate over the course of training -- an over-optimization phenomenon found in RLHF where the learning policy exploits the overfitting to inaccuracies of the reward model to achieve high rewards. One attributed source of over-optimization in DAAs is the under-constrained nature of their offline optimization, which can gradually shift probability mass toward non-preferred responses not presented in the preference dataset. This paper proposes a novel importance-sampling approach to mitigate the distribution shift problem of offline DAAs. This approach, called (IS-DAAs), multiplies the DAA objective with an importance ratio that accounts for the reference policy distribution. IS-DAAs additionally avoid the high variance issue associated with importance sampling by clipping the importance ratio to a maximum value. Our extensive experiments demonstrate that IS-DAAs can effectively mitigate over-optimization, especially under low regularization strength, and achieve better performance than other methods designed to address this problem.

High-Utility Itemset Mining (HUIM) is designed to discover sets of itemsets that can bring high profits from the database. However, HUIM encounters several challenges in picking a suitable minimum utility threshold for each database. A class of algorithms that select the top-k itemsets based on their utility has been proposed to address this issue. Although traditional top-k HUI mining algorithms do not require a specific threshold, they tend to be very time-consuming and memory-intensive when dealing with large datasets. To tackle the combinational complexity involved in HUIM algorithms, nature-inspired methods have been suggested and adopted. Nonetheless, these algorithms have traditionally focused on handling conventional, often overlooking critical data structures like product hierarchies. Consequently, they fail to extract crucial insight from this novel database format. Thus, our research introduces a heuristic-based algorithm designed to leverage top-k itemsets from databases enriched with item taxonomy data. We propose a technique involving the early pruning of unpromising items to enhance mining efficiency. Experimental evaluations are conducted on several datasets to assess the method’s performance, both with and without adopting this strategy, demonstrating its effectiveness.