In this paper, we introduce a novel approach for addressing the multi-objective optimization problem in large language model merging via black-box multi-objective optimization algorithms. The goal of model merging is to combine multiple models, each excelling in different tasks, into a single model that outperforms any of the individual source models. However, the effectiveness of conventional model merging methods is constrained by human intuition or domain knowledge. While existing optimization-based model merging methods can automatically search for model merging parameter configurations, they often struggle to find a satisfactory configuration within a limited evaluation budget. To address this challenge, we propose a novel and sample-efficient automated model merging method, named MM-MO. This method leverages multi-objective Bayesian optimization algorithms to autonomously search for great merging configurations across various tasks. In MMMO, we proposed an enhanced acquisition strategy and an auxiliary optimization objective to improve the search process. Our enhanced acquisition strategy integrates a weak-to-strong method to refine the acquisition function, enabling previously evaluated superior configurations to guide the search for new ones. Meanwhile, Fisher information is utilized to further filter these configurations, increasing the possibility of finding high-quality merging configurations. Additionally, we design a sparsity metric as an auxiliary optimization objective, further enhance the models generalization performance across different tasks. We conducted comprehensive experiments with other mainstream model merging methods, demonstrating that the proposed MMMO algorithm is competitive and effective in achieving high-quality model merging.

A series of surrogate-assisted evolutionary algorithms (SAEAs) have been proposed for expensive multi-objective optimization problems (EMOPs), building cheap surrogate models to replace the expensive real function evaluations. However, the search efficiency of these SAEAs is not yet satisfactory. More efforts are needed to further exploit useful information from the real function evaluations in order to better guide the search process. Facing this challenge, this paper proposes a Hyperbolic Neural Network (HNN) based preselection operator to accelerate the optimization process based on limited evaluated solutions. First, the preselection task is modeled as a multi-label classification problem where solutions are classified into different layers (ordinal categories) through -relaxed objective aggregation. Second, in order to resemble the hierarchical structure of candidate solutions, a hyperbolic neural network is applied to tackle the multi-label classification problem. The reason for using HNN is that hyperbolic spaces more closely resemble hierarchical structures than Euclidean spaces. Moreover, to alleviate the data deficiency issue, a data augmentation strategy is employed for training the HNN. In order to evaluate its performance, the proposed HNN-based preselection operator is embedded into two surrogate-assisted evolutionary algorithms. Experimental results on two benchmark test suites and three real-world problems with up to 11 objectives and 150 decision variables involving seven state-of-the-art algorithms demonstrate the effectiveness of the proposed method.

Large-scale multiobjective optimization problems (LSMOPs), characterized by a substantial number of decision variables, pose significant challenges for many existing evolutionary algorithms. However, the search efficiency of these algorithms is not yet satisfactory. This is mainly because that the search efficiency of these algorithms may deteriorate dramatically since the search space increases exponentially with the number of decision variables. Having this in mind, we proposed a large-Scale multiobjective optimization framework named causal inference-based competitive swarm optimizer (CI-CSO). Specifically, a causal-information-(CI)-based operator is designed for competitive swarm optimizers. First, a causal inference technique named information geometric causal inference (IGCI) is introduced to adequately explore the CI between decision variables and fitness values. To further distinguish the positive or negative impacts of these critical variables on solution quality, a CI processing module is designed, facilitating targeted optimization. To enhance search efficiency, CI-based offspring generator are employed, leveraging the variance of causal effects to dynamically adjust the search step size and sampling range. To evaluate its performance, the proposed CI-based operator is embedded into two multiobjective evolutionary algorithms (MOEAs) (LSTPA and LMOCSO). To demonstrate the effectiveness of the proposed framework, experimental results are presented using the LSMOP test suite and five real-world problems, each involving up to 10 000 decision variables. In addition, six classic algorithms are included for comparison.

Personalized recommendation serves as a ubiquitous channel for users to discover information tailored to their interests. However, traditional recommendation models primarily rely on unique IDs and categorical features for user-item matching, potentially overlooking the nuanced essence of raw item contents across multiple modalities such as text, image, audio, and video. This underutilization of multimodal data poses a limitation to recommender systems, especially in multimedia services like news, music, and short-video platforms. The recent advancements in large multimodal models offer new opportunities and challenges in developing content-aware recommender systems. This survey seeks to provide a comprehensive exploration of the latest advancements and future trajectories in multimodal pretraining, adaptation, and generation techniques, as well as their applications in enhancing recommender systems. Furthermore, we discuss current open challenges and opportunities for future research in this dynamic domain. We believe that this survey, alongside the curated resources, will provide valuable insights to inspire further advancements in this evolving landscape.

Large Language Models (LLM) based agents have shown promise in autonomously completing tasks across various domains, e.g., robotics, games, and web navigation. However, these agents typically require elaborate design and expert prompts to solve tasks in specific domains, which limits their adaptability. We introduce AutoManual, a framework enabling LLM agents to autonomously build their understanding through interaction and adapt to new environments. AutoManual categorizes environmental knowledge into diverse rules and optimizes them in an online fashion by two agents: 1) The Planner codes actionable plans based on current rules for interacting with the environment. 2) The Builder updates the rules through a well-structured rule system that facilitates online rule management and essential detail retention. To mitigate hallucinations in managing rules, we introduce a *case-conditioned prompting* strategy for the Builder. Finally, the Formulator agent compiles these rules into a comprehensive manual. The self-generated manual can not only improve the adaptability but also guide the planning of smaller LLMs while being human-readable. Given only one simple demonstration, AutoManual significantly improves task success rates, achieving 97.4\% with GPT-4-turbo and 86.2\% with GPT-3.5-turbo on ALFWorld benchmark tasks. The code is available at https://github.com/minghchen/automanual.

In the field of music information retrieval (MIR), cover song identification (CSI) is a challenging task that aims to identify cover versions of a query song from a massive collection. Existing works still suffer from high intra-song variances and inter-song correlations, due to the entangled nature of version-specific and version-invariant factors in their modeling. In this work, we set the goal of disentangling version-specific and version-invariant factors, which could make it easier for the model to learn invariant music representations for unseen query songs. We analyze the CSI task in a disentanglement view with the causal graph technique, and identify the intra-version and inter-version effects biasing the invariant learning. To block these effects, we propose the disentangled music representation learning framework (DisCover) for CSI. DisCover consists of two critical components: (1) Knowledge-guided Disentanglement Module (KDM) and (2) Gradient-based Adversarial Disentanglement Module (GADM), which block intra-version and inter-version biased effects, respectively. KDM minimizes the mutual information between the learned representations and version-variant factors that are identified with prior domain knowledge. GADM identifies version-variant factors by simulating the representation transitions between intra-song versions, and exploits adversarial distillation for effect blocking. Extensive comparisons with best-performing methods and in-depth analysis demonstrate the effectiveness of DisCover and the and necessity of disentanglement for CSI.

This paper considers the state smoothing problem for Markovian jump linear systems with non-Gaussian noises which obey Gaussian mixture distributions. On the basis of decomposing the total probability at the point of two adjacent Markov jumping parameters at the current and the next epochs, the posterior probability density of the state for smoothing is derived recursively. Then, through transforming the quotient of two Gaussian mixtures into the corresponding multiplication under the possible two adjacent Markov modes, a recursive Gaussian mixture smoother is designed with the conditional posterior probability density under each hypothesis being approximated by the Gaussian mixture. A maneuvering target tracking example with non-Gaussian noises validates the proposed method.

This paper considers the state estimation of Markovian jump linear systems with random parameters and estimate feedback. The state estimate at the previous epoch is introduced into the dynamical model to depict some phenomena that the system evolvement may depend on the most recent estimate. Then, the linear minimum mean square error estimator is derived for the considered system. A filtering framework for state estimation and data association for multiple maneuvering targets tracking is presented via the considered system, by using the state estimate at previous epoch to model the false echo which is dropped into the (overlapped) validation regions and the random parameters to describe the uncertainty between targets and possible echoes. A simulation about tracking two closely maneuvering targets in clutter shows the effectiveness of the proposed method.