Multi-modal large language models (MLLMs) are expanded from large language models (LLMs) with additional capabilities to infer multi-modal data. Current MLLM workflows, when dealing with complex tasks, typically begin by using an LLM to decompose the task into multiple subtasks, then heuristically select a specific pre-trained model to complete a subtask to get a result, and finally integrate all the results to obtain the final response. However, heuristically binding one model to one subtask may generate a less satisfying subtask result, thereby affecting the overall performance. Therefore, we propose CompeMLLM, which introduces an innovative method of dynamic orchestration of the workflows. It allows different models to compete on the same subtask instead of statically binding them. By dynamically integrating the results from diverse models, the optimal subtask result is determined, thereby improving the overall performance of MLLM. Specifically, given a certain complex task, CompeMLLM first decomposes it into subtasks, then employs multiple pre-trained models to execute the same subtask in parallel to compete, and then the optimal subtask result is chosen by dynamically evaluating these results using ensemble learning idea, and finally integrates these optimal results into a complete workflow, thus obtaining the best overall performance. We conducted extensive experiments using six advanced MLLMs as baselines across seven diverse datasets. The experimental results robustly demonstrate that CompeMLLM achieves significantly improved performance on all datasets, demonstrating its effectiveness.

Multi-agent reinforcement learning (MARL) effectively improves the learning speed of agents in sparse reward tasks with the guide of subgoals. However, existing works sever the consistency of the learning objectives of the subgoal generation and subgoal reached stages, thereby significantly inhibiting the effectiveness of subgoal learning. To address this problem, we propose a novel Potential field Subgoal-based Multi-Agent reinforcement learning (PSMA) method, which introduces the potential field (PF) to unify the two-stage learning objectives. Specifically, we design a state-to-PF representation model that describes agents' states as potential fields, allowing easy measurement of the interaction effect for both allied and enemy agents. With the PF representation, a subgoal selector is designed to automatically generate multiple subgoals for each agent, drawn from the experience replay buffer that contains both individual and total PF values. Based on the determined subgoals, we define an intrinsic reward function to guide the agent to reach their respective subgoals while maximizing the joint action-value. Experimental results show that our method outperforms the state-of-the-art MARL method on both StarCraft II micro-management (SMAC) and Google Research Football (GRF) tasks with sparse reward settings.

In some complex multi-agent environments, the types of relationships between agents are diverse and their intensity changes during the policy learning process. Theoretically, some of these relationships can facilitate cooperative policy learning. However, acquiring these relationships is an intractable problem. To tackle the problem, we propose a diverse effective relationship exploration based multi-agent reinforcement learning (DERE) method. Specifically, a potential fields model is firstly designed to represent relationships between agents. Then to encourage the exploration of effective relationships, we define an information-theoretic objective function. Finally, an intrinsic reward function is designed to optimize the information-theoretic objective, meanwhile, guide agents to learn more effective collaborative policies. Experimental results show that our method outperforms state-of-the-art methods on both super hard StarCraft II micromanagement tasks (SMAC) and Google Research Football (GRF).

The role-oriented learning approach could improve the performance of multi-agent reinforcement learning by decomposing complex multi-agent tasks into different roles. However, due to the dynamic environment and interactions among agents, the role undertaken by an agent changes rapidly with time going on. Therefore, the roles of agents should be adapted to the varying situation during the learning process. In this paper, we propose a role-oriented graph convolution based multi-agent reinforcement learning framework (ROGC). Firstly, we design a role assigner based on samples generated from the environment to learn roles for classifying agents into different groups. To further enhance cooperation among agents in the same group for higher performance, we design a graph convolutional module to achieve intra-role communications based on discovered roles. With roles and extracted role features, we design a role-oriented policy learning module that embeds the role information into the algorithm and generates effective policies for individuals. Further, we introduce an auto-encoder to learn the intra-role cooperation knowledge in the graph convolutional module, which ensures our framework executes in a decentralized way. Extensive experiments show that our framework can learn dynamic roles and make full use of learned roles, which makes it outperform popular MARL methods.

Modeling user preferences from users' historical sequences is one of the core problems of sequential recommendation. Previous studies only considered transition patterns between user-items, ignoring transition patterns between item features and item-item interactions. Recently, there has been interest in integrating knowledge graphs as auxiliary information into sequential recommendation. Most of the existing methods deal with the heterogeneous information in the knowledge graph in a coarse-grained manner. We believe that fine-grained processing of information in knowledge graphs can help recommendation systems understand changes in user preferences, i.e. dividing hetero-geneous information into item-to-item relationships and item-to-attribute relationships. In this paper, we propose a dual-level self-attention network for sequential recommendation. Specifically, we divide the knowledge graph heterogeneous information about items into relation level and attribute level, representing item-item relationship and item-attribute relationship, respectively. Afterwards, the self-attention network is used to learn user preferences at dual-level, respectively. Then, the outputs of the above dual levels are integrated for next item recommendation. Based on extensive experiments on three real-world data sets, our model achieves significant improvements compared to state-of-the-art baseline methods.

Heterogeneous behaviours are widespread in many multi-agent tasks, which have not been paid much attention in the community of multi-agent reinforcement learning. It would be a key factor for improving the learning performance to efficiently characterize and automatically find heterogeneous behaviours. In this paper, we introduce the concept of the skill to explore the ability of heterogeneous behaviours. We propose a novel skill-based multi-agent reinforcement learning framework to enable agents to master diverse skills. Specifically, our framework consists of the skill representation mechanism, the skill selector and the skill-based policy learning mechanism. We design an auto-encoder model to generate the latent variable as the skill representation by incorporating the environment information, which ensures the distinguishable of agents for skill selection and the discriminability for the skill learning. With the representation, a skill selection mechanism is invented to realize the assignment from agents to skills. Meanwhile, diverse skill-based policies are generated through a novel skill-based policy learning method. To promote efficient skill discovery, a mutual information based intrinsic reward function is constructed. Empirical results show that our framework obtains the best performance on three challenging benchmarks, i.e., StarCraft II micromanagement tasks, Google Research Football and GoBigger, over state-of-the-art MARL methods.

Knowledge graph completion can complete knowledge by predicting missing facts, which is a increasingly hot research topic in knowledge graph construction.Prevalent approaches propose to embed knowledge graphs in a lowdimensional vector space and use these embedding to predict, but they neglect either semantic information or graph structures.We propose a new approach to knowledge graph completion named as ATTWALK, which learns embedding by exploiting both structural and semantic features of a knowledge graph.This is achieved by leveraging a key insight that an entities' embedding is influenced by its multi-hop neighbors', which can be further distinguished by their semantic importance to the entity.ATTWALK orchestrates a two-step workflow by first evaluating neighbors' semantic weights using graph attention networks for each entity, then exploring the entities' local structural features by performing a semantic weight guided walk.We evaluate ATTWALK by conducting extensive experiments, which show that ATTWALK outperforms 12 representative approaches on average across 3 publicly available datasets.

Abstract Scene graph generation refers to the task of identifying the objects and specifically the relationships between the objects from an image. Existing scene graph generation methods generally use the bounding boxes region features of objects to identify the relationships between objects. However, we feel that the overlap region features of two objects may play an important role in fine‐grained relationship identification. In fact, some fine‐grained relationships can only be obtained from the overlap region features of two objects. Therefore, we propose the Multi‐Branch Feature Combination (MFC) module and Overlap Region Transformer (ORT) module to comprehensively obtain the visual features contained in the overlap regions of two objects. Concretely, the MFC module uses deconvolution and multi‐branch dilation convolution to obtain high‐pixels and multi‐receptive field features in the overlap regions. The ORT module uses the vision transformer to obtain the self‐attention of the overlap regions. The joint use of these two modules achieves the mutual complementation of local connectivity properties of convolution and the global connectivity properties of attention. We also design a Geometrical Center Augmented (GCA) module to obtain the relative position information of the geometric centers between two objects, to prevent the problem that only relying on the scale of the overlap region cannot accurately capture the relationship between two objects. Experiments show that our model ORGC (Overlap Region and Geometrical Center), the combination of the MFC module, the ORT module, and the GCA module, can enhance the performance of fine‐grained relation identification. On the Visual Genome dataset, our model outperforms the current state‐of‐the‐art model by 4.4% on the R@50 evaluation metric, reaching a state‐of‐the‐art result of 33.88.