Multimodal aspect-based sentiment classification (MABSC) is a challenging task emerging in recent years, which aims to combine text and image to identify the sentiment polarity of each aspect. There exists a potential irrelevance between aspects and images, and mistakenly focusing on irrelevant image regions will introduce redundant and misalignment noise. Besides, existing methods implicitly mix visual and textual features, which may lead to the loss or blurring of modality-specific information. To address these challenges, we propose a Bottleneck-constrained Contrastive Decoupled Network (BCDN) for the MABSC task. We first design a bottleneck-constrained visual consistency module to reduce redundancy and misalignment noise in aspect-related visual features. Additionally, we employ modality decoupling to fully capture inter-modality knowledge. Specifically, we first decouple the aspect-related visual and textual representations into modality-invariant and modality-specific features. Afterwards, we propose novel contrastive regularizations to optimize the decoupled features. Extensive experiments on benchmark datasets demonstrate that our BCDN achieves superior performance and verify the effectiveness of our BCDN. The codes are released at https://github.com/ruiliu2020/BCDN.

Aspect sentiment triplet extraction (ASTE) aims to extract triplets composed of aspect terms, opinion terms, and sentiment polarities from given sentences. The table tagging method is a popular approach to addressing this task, which encodes a sentence into a 2-dimensional table, allowing for the tagging of relations between any two words. Previous efforts have focused on designing various downstream relation learning modules to better capture interactions between tokens in the table, revealing that a stronger capability in relation capture can lead to greater improvements in the model. Motivated by this, we attempt to directly utilize transformer layers as downstream relation learning modules. Due to the powerful semantic modeling capability of transformers, it is foreseeable that this will lead to excellent improvement. However, owing to the quadratic relation between the length of the table and the length of the input sentence sequence, using transformers directly faces two challenges: overly long table sequences and unfair local attention interaction. To address these challenges, we propose a novel Table-Transformer (T-T) for the tagging-based ASTE method. Specifically, we introduce a stripe attention mechanism with a loop-shift strategy to tackle these challenges. The former modifies the global attention mechanism to only attend to a 2-dimensional local attention window, while the latter facilitates interaction between different attention windows. Extensive and comprehensive experiments demonstrate that the T-T, as a downstream relation learning module, achieves state-of-the-art performance with lower computational costs.

Detecting harmful memes plays a crucial role in safeguarding social media users and maintaining a healthy network ecosystem. However, the ability of multimodal large language models(MLLMs) in harmful meme detection tasks remains to be fully developed. Fine-tuning MLLMs is prohibitively expensive due to the large number of parameters, making it essential to leverage these powerful models more efficiently rather than simply fine-tuning them. In this work, we propose GENHMD, a generated rationale-enhanced detection framework, which generates rationales by prompting MLLMs. Our key insight is that MLLMs, pre-trained on massive amounts of data containing rich knowledge, can enhance harmful meme detection by eliciting specialized knowledge. Specifically, inspired by the powerful capacity of MLLMs for text generation and reasoning, we first employ MLLMs to generate credible reasons through chain-of-thought reasoning. To further promote collaboration between harmfulness rationales and the multimodal information inherent in memes, we leverage a smaller multimodal model to learn the attention weights between different modal features using the attention mechanism. Comprehensive experiments on two public meme datasets demonstrate that GENHMD achieves outstanding performance in harmful meme detection. Further analysis shows that the generated rationales substantiate the explainability of the detection results.

Transformer-based architecture models like BERT have performed excellently for various Natural Language Processing (NLP) tasks. However, these models are usually computationally expensive with a large number of parameters. As a result, deploying them in edge devices has become a challenging task. The existing compression work on lower-precision quantization still has a severe accuracy decrease and rarely focuses on the information hidden in the different modules of the model. In this paper, we propose a distillation-aware quantization with mixed precision method combined with quantization and knowledge distillation. We achieve the ultra-low mixed precision quantization with the different sensitivity of different modules of BERT. Moreover, we leverage knowledge distillation to reduce the model accuracy degradation. We extensively test our method on four GLUE tasks. It shows that DQMix-BERT outperforms the other BERT compression methods and even achieves comparable performance to the original BERT model while achieving ~8x compression.

Event detection on social platforms can help people perceive essential events and make actionable decisions. Existing document-pivot streaming social event detection methods generally embed documents and perform text clustering. They face the challenges of constantly changing context and unknown event categories and struggle by designing compound text representation methods and various similarity measures. However, phased, well-designed methods are excessively fragile and unable to utilize the potential of text representations fully. Meanwhile, their complex threshold settings result in clustering-based event detection suffering the pain of ever-changing environments. We propose a text representation learning method namely Text Similarity Contrastive Learning Neural Network (Text-SimCLNN) to tackle these challenges. Text-SimCLNN uses smaller parts to learn the similarity probability of text pairs from semantic and structural perspectives, effectively bridging the gap between text representation learning and similarity measure in streaming event detection. Event discovery and merging in streams can be easily performed based on the learned representations, and we use various techniques to speed up such processes. Furthermore, we introduce an online update mechanism that uses heterogeneous graphs to generate high-quality samples to enable stable and reliable inductive learning. Extensive experiments on two real-world datasets demonstrate that our method far exceeds state-of-the-art (SOTA).

Anchor link prediction focuses on finding accounts related to the same natural person in different online platforms and can benefit several services like user modeling and cross-network recommendation systems. Recently, community structure information has attracted interest from researchers, while most of existing methods only treat community structure information as additional information and lack utilizing the interaction between communities. To address these limitations, we propose a cross-network community sensing model for anchor link prediction (CCALP). Our CCALP model regards communities as special nodes in each network and further models community level inter-network relationships by existing anchor links. We further utilize an attention mechanism to integrate cross-network community neighbors' vectors. This attention mechanism can help us balance the degree of utilization of community knowledge. Through extensive experiments on two real-world datasets, we demonstrate that CCALP outperforms the existing baseline approaches.

Recently, language models (LMs) or language representation models are widely used in natural language understanding (NLU) tasks.However, these LMs are usually trained on large unlabeled text corpora, while the finetuning process simply takes words or wordpieces as model input.Because of the differences between language model and NLU task objectives, the problem of lack of concern on some key words exists.Thus in this paper, we propose a method called reverse mapping bytepair encoding, which maps named-entity information and other word-level linguistic features back to subwords during the encoding procedure of bytepair encoding (BPE).We employ this method to the Generative Pre-trained Transformer (OpenAI GPT) (Radford et al., 2018) by adding a weighted linear layer after the embedding layer.We also propose a new model architecture named as the multi-channel separate transformer to evaluate the effectiveness of the newly introduced information by employing a training process without parameter-sharing.Experiments on Story Cloze, RTE, SciTail and SST-2 datasets demonstrate the effectiveness of our approach.Compared with the original results in GPT, our approach gains 1.58% absolute increase on Stories Cloze, 6.4% on RTE, 0.69% on SciTail and 0.8% on SST-2.