Accurate teeth delineation on 3-D dental models is essential for individualized orthodontic treatment planning. Pioneering works like PointNet suggest a promising direction to conduct efficient and accurate 3-D dental model analyses in end-to-end learnable fashions. Recent studies further imply that multistream architectures to concurrently learn geometric representations from different inputs/views (e.g., coordinates and normals) are beneficial for segmenting teeth with varying conditions. However, such multistream networks typically adopt simple late-fusion strategies to combine features captured from raw inputs that encode complementary but fundamentally different geometric information, potentially hampering their accuracy in end-to-end semantic segmentation. This article presents a hierarchical cross-stream aggregation (HiCA) network to learn more discriminative point/cell-wise representations from multiview inputs for fine-grained 3-D semantic segmentation. Specifically, based upon our multistream backbone with input-tailored feature extractors, we first design a contextual cross-steam aggregation (CA) module conditioned on interstream consistency to boost each view's contextual representation learning jointly. Then, before the late fusion of different streams' outputs for segmentation, we further deploy a discriminative cross-stream aggregation (DA) module to concurrently update all views' discriminative representation learning by leveraging a specific graph attention strategy induced by multiview prototype learning. On both public and in-house datasets of real-patient dental models, our method significantly outperformed state-of-the-art (SOTA) deep learning methods for teeth semantic segmentation. In addition, extended experimental results suggest the applicability of HiCA to other general 3-D shape segmentation tasks. The code is available at https://github.com/ladderlab-xjtu/HiCA.

While decentralized training is attractive in multi-agent reinforcement learning (MARL) for its excellent scalability and robustness, its inherent coordination challenges in collaborative tasks result in numerous interactions for agents to learn good policies. To alleviate this problem, action advising methods make experienced agents share their knowledge about what to do, while less experienced agents strictly follow the received advice. However, this method of sharing and utilizing knowledge may hinder the team's exploration of better states, as agents can be unduly influenced by suboptimal or even adverse advice, especially in the early stages of learning. Inspired by the fact that humans can learn not only from the success but also from the failure of others, this paper proposes a novel knowledge sharing framework called Cautiously-Optimistic kNowledge Sharing (CONS). CONS enables each agent to share both positive and negative knowledge and cautiously assimilate knowledge from others, thereby enhancing the efficiency of early-stage exploration and the agents' robustness to adverse advice. Moreover, considering the continuous improvement of policies, agents value negative knowledge more in the early stages of learning and shift their focus to positive knowledge in the later stages. Our framework can be easily integrated into existing Q-learning based methods without introducing additional training costs. We evaluate CONS in several challenging multi-agent tasks and find it excels in environments where optimal behavioral patterns are difficult to discover, surpassing the baselines in terms of convergence rate and final performance.

The training of multi-agent reinforcement learning (MARL) tasks with the public goods dilemma (PGD) is difficult because the selfish actions of individual agents for high personal rewards may reduce the collective utility of the whole group. Existing solutions to this problem, e.g., reward gifting or intrinsic rewards, although inducing cooperation among agents in small groups, cannot guarantee fairness among agents’ policies and fail to achieve optimal group utility in large-scale systems. In this paper, we propose F4PGD, an effective method to train large-scale MARL tasks with PGD in a decentralized manner, which is inspired by Adam’s equity theory that the match between a person’s payoff and his contribution is the key incentive for people to contribute to the common good. In F4PGD, a mechanism is designed to match an agent’s reward with its contribution, which suppresses agents from taking a free ride and meanwhile encourages well-learned agents to contribute to public goods. Experimental results show that F4PGD effectively learns optimal policies for the whole group and guarantees fairness among agents in several typical MARL tasks with PGD.

Learning effective strategies in sparse reward tasks is one of the fundamental challenges in reinforcement learning. This becomes extremely difficult in multi-agent environments, as the concurrent learning of multiple agents induces the non-stationarity problem and sharply increased joint state space. Existing works have attempted to promote multi-agent cooperation through experience sharing. However, learning from a large collection of shared experiences is inefficient as there are only a few high-value states in sparse reward tasks, which may instead lead to the curse of dimensionality in large-scale multi-agent systems. This paper focuses on sparse-reward multi-agent cooperative tasks and proposes an effective experience-sharing method, Multi-Agent Selective Learning (MASL), to boost sample-efficient training by reusing valuable experiences from other agents. MASL adopts a retrogression-based selection method to identify high-value traces of agents from the team rewards, based on which some recall traces are generated and shared among agents to motivate effective exploration. Moreover, MASL selectively considers information from other agents to cope with the non-stationarity issue while enabling efficient training for large-scale agents. Experimental results show that MASL significantly improves sample efficiency compared with state-of-the-art MARL algorithms in cooperative tasks with sparse rewards.

How to efficiently coordinate multiple readers to work together is critical for high throughput in RFID systems. Existing researchs focus on designing efficient reader scheduling strategies that arrange adjacent readers to work in different time to avoid signal collisions. However, the impact of unbalanced tag number of readers on tag read throughput is still challenging. In RFID systems, the distribution of tags is usually variable and uneven, making the number of tags covered by each reader (i.e., the load) imbalanced. This imbalance leads to different execution time for readers: the heavily loaded readers take longer time to collect all tags, while the other readers whose finish execution earlier have to wait in vain. To avoid this useless waiting and improve the system throughput, this paper focuses on the load balancing problem of multiple readers, which is an NP-hard problem. In this paper, we design heuristic algorithms to adjust readers interrogation regions and efficiently balance their loads. The amazing advantage of our algorithm is that it can be adopted by almost all existing protocols in multi-reader systems, including the reader scheduling protocol, to improve system throughput. Extensive experiments demonstrate that our algorithm can significantly improve the throughput in various scenarios.

Although multistage tasks involving multiple sequential goals are common in real-world applications, they are not fully studied in multi-agent reinforcement learning (MARL). To accomplish a multi-stage task, agents have to achieve cooperation on different subtasks. Exploring the collaborative patterns of different subtasks and the sequence of completing the subtasks leads to an explosion in the search space, which poses great challenges to policy learning. Existing works designed for single-stage tasks where agents learn to cooperate only once usually suffer from low sample efficiency in multi-stage tasks as agents explore aimlessly. Inspired by human’s improving cooperation through goal consistency, we propose Multi-Agent Goal Consistency (MAGIC) framework to improve sample efficiency for learning in multi-stage tasks. MAGIC adopts a goal-oriented actor-critic model to learn both local and global views of goal cognition, which helps agents understand the task at the goal level so that they can conduct targeted exploration accordingly. Moreover, to improve exploration efficiency, MAGIC employs two-level goal consistency training to drive agents to formulate a consistent goal cognition. Experimental results show that MAGIC significantly improves sample efficiency and facilitates cooperation among agents compared with state-of-art MARL algorithms in several challenging multistage tasks.

Weakly-supervised domain adaptation has been introduced to address the source domain with label noise or/and feature noise. However, the existing weakly-supervised domain adaptation methods only work under ideal assumptions, which assume either the annotated data of the target domain can be accessed or the noise rate of all the classes is identical and already known. This limits their practical application. To tackle this, we propose a universal weakly-supervised domain adaptation method called PDCAS which relaxes the ideal assumptions and makes it more general. Specially, PD- CAS includes two stages: progressive distillation and domain alignment. In progressive distillation, we iteratively distill out potentially corrected samples whose annotated labels are consistent with the prediction of model. By exploiting intrinsic similarity to extract initial corrected samples, this process does not need any supervision. In domain alignment, besides taking the global feature distributions into consideration, we also adopt Class-Aligned Sampling which balances the samples for both source and target domains to alleviate the shift of label distributions. Extensive experiments on Office-31 and Office-Home datasets demonstrate the effectiveness and robustness of our method compared to state-of-the-art methods.

Multi-agent reinforcement learning system is used to solve the problem that agents achieve specific goals in the interaction with the environment through learning policies. Almost all existing multi-agent reinforcement learning methods assume that the observation of the agents is accurate during the training process. It does not take into account that the observation may be wrong due to the complexity of the actual environment or the existence of dishonest agents, which will make the agent training difficult to succeed. In this paper, considering the limitations of the traditional multi-agent algorithm framework in noisy environments, we propose a multi-agent fault-tolerant reinforcement learning (MAFTRL) algorithm. Our main idea is to establish the agent's own error detection mechanism and design the information communication medium between agents. The error detection mechanism is based on the autoencoder, which calculates the credibility of each agent's observation and effectively reduces the environmental noise. The communication medium based on the attention mechanism can significantly improve the ability of agents to extract effective information. Experimental results show that our approach accurately detects the error observation of the agent, which has good performance and strong robustness in both the traditional reliable environment and the noisy environment. Moreover, MAFTRL significantly outperforms the traditional methods in the noisy environment.

Domain adaptation aims to transfer knowledge across different domains and bridge the gap between them. While traditional knowledge transfer considers identical domain, a more realistic scenario is to transfer from a larger and more diverse source domain to a smaller target domain, which is referred to as partial domain adaptation (PDA). However, matching the whole source domain to the target domain for PDA might produce negative transfer. Samples in the shared classes should be carefully selected to mitigate negative transfer in PDA. We observe that the correlations between different target domain samples and source domain samples are diverse: classes are not equally correlated and moreover, different samples have different correlation strengthes even when they are in the same class. In this study, we propose ECDT, a novel PDA method that Exploits the Correlation Diversity for knowledge Transfer between different domains. We propose a novel method to estimate target domain label space that utilizes the label distribution and feature distribution of target samples, based on which outlier source classes can be filtered out and their negative effects on transfer can be mitigated. Moreover, ECDT combines class-level correlation and instance-level correlation to quantity sample-level transferability in domain adversarial network. Experimental results on three commonly used cross-domain object data sets show that ECDT is superior to previous partial domain adaptation methods.