Lightweight models are often deployed on edge devices (e.g., smartphones and wearable devices) to enhance their scalability and practicability. To enhance their generalization ability in dynamic changing environments, cloud-device collaborative learning (CDCL) is proposed to transfer the generalization ability from large models on cloud servers to lightweight models deployed on devices. However, current methods mainly focus on solving the data distribution shifts for a limited number of seen classes but ignore the continual incoming new classes. Though existing class-incremental learning (CIL) methods achieve impressive performance, two major challenges arise when adapting them into the CDCL setting: 1) poor generalization of lightweight models during CIL and 2) overfitting during data-incremental learning. In this article, we explore a new problem named class-incremental CI-CDCL, and propose a contrastive prototypical network based CI-CDCL framework, aiming to improve the effectiveness of cloud-device collaboration in both class-incremental and data-incremental learning. Extensive experiments are conducted on two public datasets and the experimental results can evaluate the effectiveness of our proposed model.

Much of commonsense knowledge in real world is the form of procudures or sequences of steps to achieve particular goals. In recent years, knowledge extraction on procedural documents has attracted considerable attention. However, they often focus on procedural text but ignore a common multimodal scenario in the real world. Images and text can complement each other semantically, alleviating the semantic ambiguity suffered in text-only modality. Motivated by these, in this paper, we explore a problem of grounded multimodal procedural entity recognition (GMPER), aiming to detect the entity and the corresponding bounding box groundings in image (i.e., visual entities). A new dataset (Wiki-GMPER) is bult and extensive experiments are conducted to evaluate the effectiveness of our proposed model.

Generating coherent and credible explanations remains a significant challenge in the field of AI. In recent years, researchers have delved into the utilization of entailment trees to depict explanations, which exhibit a reasoning process of how a hypothesis is deduced from the supporting facts. However, existing models often overlook the importance of generating intermediate conclusions with logical consistency from the given facts, leading to inaccurate conclusions and undermining the overall credibility of entailment trees. To address this limitation, we propose the logical pattern memory pre-trained model (LMPM). LMPM incorporates an external memory structure to learn and store the latent representations of logical patterns, which aids in generating logically consistent conclusions. Furthermore, to mitigate the influence of logically irrelevant domain knowledge in the Wikipedia-based data, we introduce an entity abstraction approach to construct the dataset for pre-training LMPM. The experimental results highlight the effectiveness of our approach in improving the quality of entailment tree generation. By leveraging logical entailment patterns, our model produces more coherent and reasonable conclusions that closely align with the underlying premises.

Cross-domain aspect-based sentiment analysis has recently attracted significant attention, which can effectively alleviate the problem of lacking large-scale labeled data for supervised learning based methods. Most of current methods mainly focus on extracting domain-shared syntactic features to conduct the domain adaptation. Due to the language and syntax are diverse between domains, these methods lack generalization and even lead to syntactic transfer errors. External knowledge graphs have rich domain commonsense and share the relational structures between source and target domains. The domain-shared relational structure can effectively bridge the gap across domains and solve the problem of syntactic transfer errors. Moreover, not all the introduced external knowledge is equally important for the cross-domain aspect-based sentiment analysis. Motivated by these, we propose a knowledge-enhanced and topic-guided cross domain aspect-based sentiment analysis model with the domain-shared commonsense relational structure learning module and the topic-guided knowledge attention module. Extensive experiments are conducted and the experimental results evaluate the effectiveness of our proposed model.

Relation Classification is one of the most important tasks in text mining. Previous methods either require large-scale manually-annotated data or rely on distant supervision approaches which suffer from the long-tail problem. To reduce the expensive manually-annotating cost and solve the long-tail problem, prototypical networks are widely used in few-shot RC tasks. Despite their remarkable performance, current prototypical networks ignore the different granularities of relations, which degrades the classification performance dramatically. Moreover, the optimization of current prototypical networks simply relies on the cross-entropy loss, which cannot consider the intra-relation compactness and the dispersion among relations in a semantic space. It is not robust enough for current prototypical network in real-world and complicated scenarios. In this paper, we propose an area prototypical network with a granularity-aware measurement, aiming to considering the different granularities of relations. Each relation is represented as an area whose width can reflect the granularity level of relation. Moreover, to improve the robustness, bimargin loss is designed to force area prototypical network to improve the intra-relation compactness and inter-relation dispersion for the feature representation in a semantic space. Extensive experiments on two public datasets are conducted and evaluate the effectiveness of our proposed model.

Millions of people post images and texts to express their feelings and point of views on social media everyday, especially on the short text social media such as Twitter or Weibo. As the images can provide important supplementary information for the text, many multimodal topic models have been developed to mine the topics from the multimodal social media content. We summarize three fundamental characteristics of the short text multimodal social media. The first is that the text of a short social media document generally belong to only one topic. The second is that the attached images can be relevant to multiple topics due to the rich information expressed in the images. The last is that although in most cases, text and images in social media posts are relevant, it should be noted that in a small number of cases, text and pictures are not relevant. However, most of the current multimodal topic models fail to model the these characteristics, and thus may produce low-quality topics. Based on these characteristics, we propose an unsupervised multimodal topic model SMMTM to model the short text multimodal social media documents. In the SMMTM model, only one topic is sampled for the the text while an image can belong to different topics. The correlation of the topics between the text and the images in a document are also formulated in an appropriate way. The experiments on three short text social media datasets with four evaluation metrics show the advantages of our model over the existing models.

Cross-domain sentiment analysis (SA) has recently attracted significant attention, which can effectively alleviate the problem of lacking large-scale labeled data for deep neural network based methods. However, exiting unsupervised cross-domain SA models ignore the relation between the aspect and opinion, which suffer from the sentiment transfer error problem. To solve this problem, we propose an aspect-opinion sentiment alignment SA model and extensive experiments are conducted to evaluate the effectiveness of our model.

Aspect Based Sentiment Analysis (ABSA) aims to extract aspect terms and identify the sentiment polarities towards each extracted aspect term. Currently, syntactic information is seen as the bridge for the domain adaptation and achieves remarkable performance. However, the transferable syntactic knowledge is complex and diverse, which causes the transfer error problem in domain adaptation. In our paper, we propose a domain-shared relational structure incorporated cross-domain ABSA model. The experimental results show the effectiveness of our model.

Sentiment analysis in social media is challenging since posts are short of context. As a popular way to express emotion on social media, stickers related to these posts can supplement missing sentiments and help identify sentiments precisely. However, research about stickers has not been investigated further. To this end, we present a Chinese sticker-based multimodal dataset for the sentiment analysis task (CSMSA). Compared with previous real-world photo-based multimodal datasets, the CSMSA dataset focuses on stickers, conveying more vivid and moving emotions. The sticker-based multimodal sentiment analysis task is challenging in three aspects: inherent multimodality of stickers, significant inter-series variations between stickers, and complex multimodal sentiment fusion. We propose SAMSAM to address the above three challenges. Our model introduces a flexible masked self-attention mechanism to allow the dynamic interaction between post texts and stickers. The experimental results indicate that our model performs best compared with other models. More researches need to be devoted to this field. The dataset is publicly available at https://github.com/Logos23333/CSMSA.

Cross-domain sentiment analysis has recently attracted significant attention, which can effectively alleviate the problem of lacking large-scale labeled data for deep neural network based methods. However, most of the existing cross-domain sentiment classification models neglect the domain-specific features, which limits their performance especially when the domain discrepancy becomes larger. Meanwhile, the relations between the aspect and opinion terms cannot be effectively modeled and thus the sentiment transfer error problem is suffered in the existing unsupervised domain-adaptation methods. To address these two issues, we propose an aspect-opinion correlation aware and knowledge-expansion few shot cross-domain sentiment classification model. Sentiment classification can be effectively conducted with only a few support instances of the target domain. Extensive experiments are conducted and the experimental results show the effectiveness of our proposed model.

Relation Classification (RC) is an important task in information extraction. In most real-world scenarios, the frequency of relations often follows a long-tailed and open-ended distribution. However, current efforts mainly focus on the partial frequency distribution of relations, which is limited in real-world applications. Meanwhile, prototypical network achieves remarkable performance among fields of deep supervised learning, few-shot learning and open set learning. Nevertheless, in the open world environment, it still suffers from the incompatible feature embedding problem as the novel and unknown relations come in. To address these problems, we propose an Open Generalized Prototypical Network with task-adaptive feature fusion for the open generalized few-shot relation classification. Extensive experiments are conducted on public large-scale datasets and our proposed model obtains the better performances.

The interaction between named entity recognition and relation classification is quite essential for the extraction of relational triplets. However, most of jointly extraction works only consider unidirectional interaction between the two sub-tasks. They even neglect the interactive information totally. In order to tackle these problems, we propose a novel unified joint extraction model which considers bidirection-interactive information between the two subtasks. Our model consists of two modules. The first module utilizes Bi-LSTM and GCN to capture the sequential and the structure-semantic features of a sentence, The second module utilizes two layers to capture bidirection-interactive information between the two subtasks and generates relational triplets respectively. The experimental results show that our proposed model outperforms the state-of-the-art models on two public datasets.

Relation Classification (RC) plays an important role in natural language processing (NLP).Current conventional supervised and distantly supervised RC models always make a closed-world assumption which ignores the emergence of novel relations in an open environment.To incrementally recognize the novel relations, current two solutions (i.e, re-training and lifelong learning) are designed but suffer from the lack of large-scale labeled data for novel relations.Meanwhile, prototypical network enjoys better performance on both fields of deep supervised learning and few-shot learning.However, it still suffers from the incompatible feature embedding problem when the novel relations come in.Motivated by them, we propose a two-phase prototypical network with prototype attention alignment and triplet loss to dynamically recognize the novel relations with a few support instances meanwhile without catastrophic forgetting.Extensive experiments are conducted to evaluate the effectiveness of our proposed model.