Multi-task Learning (MTL) involves training multiple tasks within a single model to improve overall performance by leveraging shared knowledge. However, this joint training can result in performance degradation due to task conflicts, typically manifesting as conflicts in task gradients. Existing solutions primarily focus on modeling task gradient relationships, which overlook the differences in how the same data sample influences different tasks. These differences are the source of intricate task gradient relationships and could further lead to varying degrees of impact from conflicts on tasks. To tackle these challenges, we propose DTJS, a novel adaptive Data and Task Joint Scheduling approach for MTL, which uniquely considers the influence of data within each task and the distinct task perception of gradient conflicts from an innovative scheduling perspective. Specifically, we design intra-task scheduling to quantify the difficulty level of the data based on its influence within each task, facilitating easy-to-hard data scheduling. Concurrently, inter-task scheduling is proposed to capture the diverse relationship among joint learning tasks via assessing the severity of conflicts between tasks and adaptively considering their effects on individual tasks through learnable task conflict perception. Furthermore, DTJS utilizes a bi-level optimization strategy that alternately updates model parameters and the learnable task conflict perception, taking into account their interdependence. Scheduled model gradients are used to optimize the MTL model, while implicit gradients refine the learnable task conflict perception. Extensive experimental results not only demonstrate that DTJS improves the performance of the MTL model over SOTA methods across various scenarios but also explain how DTJS schedules both data and tasks to bring performance improvements. The code is available at https://github.com/ZeyuLiu0706IDTJS.

Multi-task learning (MTL) has succeeded in various industrial applications by utilizing common knowledge among joint training tasks to enhance the generalization of MTL models, resulting in improved performance across all training tasks simultaneously. Unfortunately, training all tasks simultaneously often causes performance degradation compared to single-task models since different tasks might conflict with each other. Despite existing MTL methods that aim to mitigate task conflicts by manipulating task gradients at each iteration, they ignore the potential influence of noisy data from different batches on task gradients. Consequently, the current iteration's task gradient may not accurately reflect the task itself, leading to inadequate alleviation of the dilemma of task conflicts. Moreover, existing works seldom explore the potential source of task conflicts and merely pose an assumption. In this paper, we conduct an in-depth empirical investigation into the potential sources of performance degradation of MTL and find that task gradient conflict is one of the primary reasons for the performance degradation of tasks. Then, to address the task conflicts problem, we propose a novel gradient manipulation approach, namely MoCoGrad, which manipulates task gradients by leveraging the momentum information of the task to calibrate the gradients of conflicting tasks. In addition, we derive theoretical guarantees for the con-vergence of our proposed MoCoGrad and theoretically analyze the convergence rate of MoCoGrad. Finally, to evaluate the effectiveness of MoCoGrad, extensive experiments are conducted on six real-world datasets from different domains. Our approach yields the best performance across all tasks in all six MTL benchmarks, demonstrating the effectiveness and superiority of our method.

Graph contrastive learning (GCL) has gained increasing interest as a solution for graph representation learning. In GCL, graph augmentation is essential to generate contrastive samples used for contrastive learning. Recently, most existing methods employ learnable graph view generators to augment graphs based on the node probability distribution adaptively. However, these methods cannot ensure that semantic-related nodes are preserved during graph augmentation, leading to performance degradation. To tackle this issue, we propose a novel approach called Semantic-aware Graph Contrastive Learning (SGCL), which can generate high-quality contrastive samples by only augmenting semantic-unrelated nodes so as to facilitate the performance of GCL on downstream tasks. Specifically, we first design a Lipschitz constant generator to compute the Lipschitz constants that measure the semantic relevance of each node. Then, we propose the Lipschitz graph augmentation to augment graphs while only dropping these semantic-unrelated nodes with small Lipschitz constants. Furthermore, we propose semanticaware contrastive learning to obtain more refined representations by contrasting the graph-level representation of anchor graphs and high-quality generated samples. Experimental results on unsupervised learning and transfer learning demonstrate the effectiveness of SGCL compared to state-of-the-art methods.

Accurate forecasting of traffic flow plays a crucial role in building smart cities in the new era. Previous work has achieved success in learning inherent spatial and temporal patterns of traffic flow. However, existing works investigated the multiple periodicities (e.g., hourly, daily, and weekly) of traffic via entanglement learning, which has not yet dealt with distribution shift and interaction shift problems in traffic flow. In this paper, we propose a novel disentanglement learning network, called MUSE-Net, to tackle the limitations of entanglement learning by simultaneously factorizing the exclusiveness and interaction of multi-periodic patterns in traffic flow. Grounded in the theory of mutual information, we first learn and dis-entangle exclusive and interactive representations of traffics from multi-periodic patterns. Then, we utilize semantic-pushing and semantic-pulling regularizations to encourage the learned representations to be independent and informative. Moreover, we derive a lower bound estimator to tractably optimize the disentanglement problem with multiple variables and propose a joint training model for traffic forecasting. Extensive experimental results on several real-world traffic datasets demonstrate the effectiveness of the proposed framework. The code is available at: https://github.com/JianyangQin/MUSE-Net.

Fake news detection attracts many researchers’ attention due to the negative impacts on the society. Most existing fake news detection approaches mainly focus on semantic analysis of news’ contents. However, the detection performance will dramatically decrease when the content of news is short. In this paper, we propose a novelfake news detection multi-task learning (FDML)model based on the following observations: 1) some certain topics have higher percentages of fake news; and 2) some certain news authors have higher intentions to publish fake news. FDML model investigates the impact of topic labels for the fake news and introduce contextual information of news at the same time to boost the detection performance on the short fake news. Specifically, the FDML model consists of representation learning and multi-task learning parts to train the fake news detection task and the news topic classification task, simultaneously. As far as we know, this is the first fake news detection work that integrates the above two tasks. The experiment results show that the FDML model outperforms state-of-the-art methods on real-world fake news dataset.