Aspect-Based Multimodal Sentiment Analysis (ABMSA) aims to infer the users’ sentiment polarities over individual aspects using visual, textual, and acoustic signals. Although psychological studies have shown that personality has a direct impact on people's sentiment orientations, most existing methods disregard the potential personality character while executing ABMSA tasks. To tackle this issue, a novel psychological perspective, the people's personalities are introduced. To the best of our knowledge, this paper is the very first study in this field. Different from current pipelined multi-task sentiment analysis methods, an end-to-end ABMSA method called Personality-coupled mUlti-task leaRning framEwork (PURE) is proposed, which strongly couples personality mining and ABMSA tasks in a unified architecture to avoid error propagation and enhance the overall system robustness. Specifically, an adaptive personality feature extraction method is designed to accurately model the first impression of different people's personalities. Then, a multi-task ABMSA framework is designed to strongly couple the multimodal features of aspects extracted by the interactive attention fusion network with people's personalities. Subsequently, the proposed framework optimizes them parallel via extended Bayesian meta-learning. Finally, compared to the current optimal model, the classification accuracy and macro F1 score of the proposed model have both shown significant improvements on public datasets. In addition, PURE is transferable and can effectively couple personality modeling tasks with any other sentiment analysis methods.

Targeted sentiment analysis aims to identify the sentiment polarity of specific target mentions in a sentence. Existing methods employ neural networks to extract the relations between target mentions and their contexts. Recent approaches based on graph convolutional networks can model the syntactic relations extracted by an external parser into adjacency matrices. However, online reviews are informal and complex, the syntactic structures provided by the parser can be incorrect in these syntax-insensitive scenarios. To remedy this defect, we design a novel integrated syntactic and semantic tree (IS2tree) by labeling semantic relations between the target mention and contexts in a syntactic dependency tree. Furthermore, a dual-channel graph convolutional network (DCGCN) is proposed to encode the contextual information associated with the target mention by dynamic semantic pruning mechanisms and to also retain the syntactic relations. Experimental results demonstrate that the IS2tree has a favorable generalization capability comparing to the state-of-the-art baselines on four public datasets.

Targeted sentiment analysis aims to predict the sentiment polarity of the target in a sentence. Most traditional Graph Neural Network-based methods have focused only on the syntactic dependency information of sentences. However, they ignore the position information of words in the linear form of sentences, which leads the model to paying attention to the irrelevant syntactic dependency information to the target. To attenuate the irrelevant information, a novel model called Position-aware Dual Relational Graph Attention Network (PDRGAT) is proposed. Firstly, introducing the position-aware weight window of the syntactic dependency information to make the model pay more attention to the local syntactic information of words that neighbor the target. Secondly, a dual relational attention mechanism combining syntactic and position information is proposed. Experiments show that our model can effectively attenuate the irrelevant syntactic information and outperform state-of-the-art baselines on Accuracy and Macro-F1.

Deep learning has proved to be a powerful tool for medical image analysis in recent years. Data imbalance is a common problem in medical images. Dice Loss is widely used in medical image segmentation tasks to address the data imbalance problem. However, it only addresses the imbalance problem between foreground and background yet overlooks another imbalance between easy and hard examples that also severely affects the training process of a learning model. Empirically speaking, an easy example generally contributes less to the overall loss than a hard example. However, in practice, compared with hard examples, a large number of easy examples will be generated from a medical image and will dominate the training model, resulting in sub-optimal training or worse. To tackle this problem, we propose a novel Focal Dice Loss to alleviate the imbalance between hard examples and easy examples. Focal Dice Loss is able to reduce the contribution from easy examples and make the model focus on hard examples through our proposed novel balanced sampling strategy during the training process. Furthermore, to evaluate the effectiveness of our proposed loss functions, we conduct extensive experiments on two real-world medical image datasets with 2D and 3D convolutional neural networks. The experimental results show that our proposed Focal Dice Loss brings a significant improvement in segmentation performance compared to Dice Loss. Moreover, we find that our proposed Focal Dice Loss can effectively alleviate the over-fitting problem.

Due to their promising performance in clinical risk prediction with Electronic Health Records (EHRs), deep learning methods have attracted significant interest from healthcare researchers. However, there are 4 challenges: (i) Data insufficiency. Many methods require large amounts of training data to achieve satisfactory results. (ii) Interpretability. Results from many methods are hard to explain to clinicians (e.g., why the models make particular predictions and which events cause clinical outcomes). (iii) Domain knowledge integration. No existing method dynamically exploits complicated medical knowledge (e.g., relations such as cause and is-caused-by between clinical events). (iv) Time interval information. Most existing methods only consider the relative order of visits from EHRs, but ignore the irregular time intervals between neighboring visits. In the study, we propose a new model, Domain Knowledge Guided Recurrent Neural Networks (DG-RNN), by directly introducing domain knowledge from the medical knowledge graph into an RNN architecture, as well as taking the irregular time intervals into account. Experimental results on heart failure risk prediction tasks show that our model not only outperforms state-of-the-art deep-learning based risk prediction models, but also associates individual medical events with heart failure onset, thus paving the way for interpretable accurate clinical risk predictions.