Reinforcement learning has been shown to have great potential applications in autonomous driving. For collaborative driving scenarios, multi-agent reinforcement learning can be used to explore efficient and collaborative driving strategies. However, it still faces the challenge of non-stationary. Traditional methods focus on evaluating the similarities between the real state of the teammate and the modeled state. There is also the issue of partial observability. It can be addressed by establishing communication to share information with other surrounding agents. However, prior approaches overlook the efficient communication problem caused by unprocessed and redundant information. To tackle these two challenges, we propose an approach named Multi-Agent Collaboration via Cross-Attention and Communication (MACAC). MACAC leverages the agent’s local observations to analyze and capture environment and interaction information, while also incorporating teammate modeling through the exchange of concise state information via communication. In addition, to improve the learning process, we integrate the noisy advantage technique into MACAC to enhance the agent’s exploration capabilities. As a result, vehicles can effectively adapt to dynamic environments and exhibit efficient collaborative driving skills. In all, experiments conducted on an autonomous driving simulator demonstrate that our approach surpasses the performance of the baseline algorithms.

A modular reconfigurable robot can change its configuration adaptively according to the different environments and tasks. By adding or subtracting modules or recombining existing modules, the current configuration can be quickly changed to another one adapted to a new environment and tasks. Stable autonomous docking control directly affects the mobility, docking accuracy, and docking success rate of reconfigurable robots, and has become the focus of research in this field. In this paper, a multi-constrained trajectory planning algorithm is designed, and a multi-stage robust reconfiguration strategy is designed on the basis of this optimization algorithm. The strategy is also applied to a chain modular reconfigurable robot independently researched in the laboratory for testing, and the experimental results show that the method in this paper is effective and feasible.

Feed recommendation systems, which recommend a sequence of items for users to browse and interact with, have gained significant popularity in practical applications. In feed products, users tend to browse a large number of items in succession, so the previously viewed items have a significant impact on users' behavior towards the following items. Therefore, traditional methods that mainly focus on improving the accuracy of recommended items are suboptimal for feed recommendations because they may recommend highly similar items. For feed recommendation, it is crucial to consider both the accuracy and diversity of the recommended item sequences in order to satisfy users' evolving interest when consecutively viewing items. To this end, this work proposes a general re-ranking framework named Multi-factor Sequential Re-ranking with Perception-Aware Diversification~(MPAD) to jointly optimize accuracy and diversity for feed recommendation in a sequential manner. Specifically, MPAD first extracts users' different scales of interests from their behavior sequences through graph clustering-based aggregations. Then, MPAD proposes two sub-models to respectively evaluate the accuracy and diversity of a given item by capturing users' evolving interest due to the ever-changing context and users' personal perception of diversity from an item sequence perspective. This is consistent with the browsing nature of the feed scenario. Finally, MPAD generates the return list by sequentially selecting optimal items from the candidate set to maximize the joint benefits of accuracy and diversity of the entire list. MPAD has been implemented in Taobao's homepage feed to serve the main traffic and provide services to recommend billions of items to hundreds of millions of users every day.

Integrated recommendation, which aims at jointly recommending heterogeneous items from different channels in a main feed, has been widely applied to various online platforms. Though attractive, integrated recommendation requires the ranking methods to migrate from conventional user-item models to the new user-channel-item paradigm in order to better capture users' preferences on both item and channel levels. Moreover, practical feed recommendation systems usually impose exposure constraints on different channels to ensure user experience. This leads to greater difficulty in the joint ranking of heterogeneous items. In this paper, we investigate the integrated recommendation task with exposure constraints in practical recommender systems. Our contribution is forth-fold. First, we formulate this task as a binary online linear programming problem and propose a two-layer framework named Multi-channel Integrated Recommendation with Exposure Constraints~(MIREC) to obtain the optimal solution. Second, we propose an efficient online allocation algorithm to determine the optimal exposure assignment of different channels from a global view of all user requests over the entire time horizon. We prove that this algorithm reaches the optimal point under a regret bound of O (√T) with linear complexity. Third, we propose a series of collaborative models to determine the optimal layout of heterogeneous items at each user request. The joint modeling of user interests, cross-channel correlation, and page context in our models aligns more with the browsing nature of feed products than existing models. Finally, we conduct extensive experiments on both offline datasets and online A/B tests to verify the effectiveness of MIREC. The proposed framework has now been implemented on the homepage of Taobao to serve the main traffic.

Sequential user behavior modeling plays a crucial role in online user-oriented services, such as product purchasing, news feed consumption, and online advertising. The performance of sequential modeling heavily depends on the scale and quality of historical behaviors. However, the number of user behaviors inherently follows a long-tailed distribution, which has been seldom explored. In this work, we argue that focusing on tail users could bring more benefits and address the long tails issue by learning transferrable parameters from both optimization and feature perspectives. Specifically, we propose a gradient alignment optimizer and adopt an adversarial training scheme to facilitate knowledge transfer from the head to the tail. Such methods can also deal with the cold-start problem of new users. Moreover, it could be directly adaptive to various well-established sequential models. Extensive experiments on four real-world datasets verify the superiority of our framework compared with the state-of-the-art baselines.

Factorization Machine (FM) is a general supervised learning framework for many AI applications due to its powerful capability of feature engineering. Despite being extensively studied, existing FM methods have several limitations in common. First of all, most existing FM methods often adopt the squared loss in the modeling process, which can be very sensitive when the data for learning contains noises and outliers. Second, some recent FM variants often explore the low-rank structure of the feature interactions matrix by relaxing the low-rank minimization problem as a trace norm minimization, which cannot always achieve a tight approximation to the original one. To address the aforementioned issues, this paper proposes a new scheme of Robust Factorization Machine (RFM) by exploring a doubly capped norms minimization approach, which employs both a capped squared trace norm in achieving a tighter approximation of the rank minimization and a capped ℓ1-norm loss to enhance the robustness of the empirical loss minimization from noisy data. We develop an efficient algorithm with a rigorous convergence proof of RFM. Experiments on public real-world datasets show that our method outperforms the state-of-the-art FM methods significantly.

The process of agent migration is the major difference between logical code mobility of software agents and physical mobility of mobile nodes in ad hoc networks. Without considering agent transfer, it would make little sense to mention the modeling of strong code mobility, which aims to make a migrated agent restarted exactly from the state when it was stopped before migration. From the perspective of system's architecture, this paper proposes a two-layer approach for the formal modeling of logical agent mobility (LAM) using predicate/transition (PrT) nets. We view a mobile agent system as a set of agent spaces and agents could migrate from one space to another. Each agent space is explicitly abstracted to be a component, consisting of an environmental part and an internal connector dynamically binding agents with their environment. We use a system net, agent nets, and a connector net to model the environment, agents, and the connector, respectively. In particular, agent nets are packed up as parts of tokens in system nets, so that agent transfer and location change are naturally captured by transition firing (token game) in Petri nets. Agent nets themselves are active only at specific places and disabled at all the other places in a system net. The semantics of such a two-layer LAM model is defined by transforming it into a PrT net. This facilitates the analysis of several properties about location, state, and connection. In addition, this paper also presents a case study of modeling and analyzing an information retrieval system with mobile agents.