Multimodal knowledge graph completion (MKGC) focuses on predicting missing head or tail entities in multimodal knowledge graphs. While previous studies have emphasized the integration of images and descriptive sentences to enhance entity representation learning, our research highlights that not all multimodal information is beneficial for MKGC. In fact, irrelevant multimodal data can lead to incorrect predictions and diminished performance. This finding underscores the need for a balance between leveraging relevant multimodal information and minimizing the impact of irrelevant data in knowledge graph representation learning. To address this challenge, we propose a self-adaptive fusion model for MKGC (SAFKGC). SAFKGC utilizes a cross-modal transformer with an innovative input sequence to select features and assess the importance of different modalities. It also identifies irrelevant multimodal information based on prediction confidence, weakening its influence while maintaining reasoning consistency between multimodal and unimodal models through soft-label learning. Experimental results across four widely used MKGC datasets demonstrate that our model achieves competitive or superior performance compared to entity-aware the state-of-the-art approaches. Additional experiments reveal that SAFKGC can dynamically adjust unimodal KGC results to enhance multimodal KGC outcomes. The code is available at https://anonymous.4open.science/r/SAFKGC-BBE4/ .

Weakly supervised phrase grounding tasks aim to learn alignments between phrases and regions with coarse image-caption match information. One branch of previous methods established pseudo-label relationships between phrases and regions based on the Expectation-Maximization (EM) algorithm combined with contrastive learning. However, adopting a simplified batch-level local update (partial) of pseudo-labels in E-step is sub-optimal, while extending it to global update requires inefficiently numerous computations. In addition, their failure to consider potential false negative examples in contrastive loss negatively impacts the effectiveness of M-step optimization. To address these issues, we propose a Momentum Pseudo Labeling (MPL) method, which efficiently uses a momentum model to synchronize global pseudo-label updates on the fly with model parameter updating. Additionally, we explore potential relationships between phrases and regions from non-matching image-caption pairs and convert these false negative examples to positive ones in contrastive learning. Our approach achieved SOTA performance on 3 commonly used grounding datasets for weakly supervised phrase grounding tasks.

Recent Text-to-SQL methods leverage large language models (LLMs) by incorporating feedback from the database management system. While these methods effectively address execution errors in SQL queries, they struggle with database mismatches--errors that do not trigger execution exceptions. Database mismatches include issues such as condition mismatches and stricter constraint mismatches, both of which are more prevalent in real-world scenarios. To address these challenges, we propose a tool-assisted agent framework for SQL verification and refinement, equipping the LLM-based agent with two specialized tools: a retriever and a detector, designed to diagnose and correct SQL queries with database mismatches. These tools enhance the capability of LLMs to handle real-world questions more effectively. We also introduce SpiderMismatch, a new dataset specifically constructed to reflect the condition mismatch problems encountered in real-world scenarios. Empirical studies demonstrate the effectiveness of our proposed model on Spider and Spider-Realistic datasets in few-shot settings and confirm that our model outperforms baseline methods on SpiderMismatch.

Multi-Modal Entity Alignment aims to discover identical entities across heterogeneous knowledge graphs.While recent studies have delved into fusion paradigms to represent entities holistically, the elimination of features irrelevant to alignment and modal inconsistencies is overlooked, which are caused by inherent differences in multi-modal features.To address these challenges, we propose a novel strategy of progressive modality freezing, called PMF, that focuses on alignmentrelevant features and enhances multi-modal feature fusion.Notably, our approach introduces a pioneering cross-modal association loss to foster modal consistency.Empirical evaluations across nine datasets confirm PMF's superiority, demonstrating stateof-the-art performance and the rationale for freezing modalities.

Prompt-based Learning has shown significant success in few-shot classification. The mainstream approach is to concatenate a template for the input text to transform the classification task into a cloze-type task where label mapping plays an important role in finding the ground-truth labels. While current label mapping methods only use the contexts in one single input, it could be crucial if wrong information is contained in the text. Specifically, it is proved in recent work that even the large language models like BERT/RoBERTa make classification decisions heavily dependent on a specific keyword regardless of the task or the context. Such a word is referred to as a lexical cue and if a misleading lexical cue is included in the instance it will lead the model to make a wrong prediction. We propose a multi-mask prompt-based approach with Multi-Mask Label Mapping (MMLM) to reduce the impact of misleading lexical cues by allowing the model to exploit multiple lexical cues. To satisfy the conditions of few-shot learning, an instance augmentation approach for the cloze-type model is proposed and the misleading cues are gradually excluded through training. We demonstrate the effectiveness of MMLM by both theoretical analysis and empirical studies, and show that MMLM outperforms other existing label mapping approaches.

The semi-supervised text classification (SSTC) task aims at training text classification models with a few labeled data and massive unlabeled data.Recent works achieve this task by pseudo-labeling methods that assign pseudolabels to unlabeled data as additional supervision.However, these models may suffer from incorrect pseudo-labels caused by underfitting of decision boundaries and generating biased pseudo-labels on imbalanced data.We propose a prototype-guided semi-supervised model to address the above problems, which integrates a prototype-anchored contrasting strategy and a prototype-guided pseudo-labeling strategy.Particularly, the prototype-anchored constrasting constructs prototypes to cluster text representations with the same class, forcing them to be high-density distributed, thus alleviating the underfitting of decision boundaries.And the prototype-guided pseudo-labeling selects reliable pseudo-labeled data around prototypes based on data distribution, thus alleviating the bias from imbalanced data.Empirical results on 4 commonly-used datasets demonstrate that our model is effective and outperforms state-ofthe-art methods.

Document-level relation extraction (DocRE)involves identifying relations between entities distributed in multiple sentences within a document.Existing methods focus on building a heterogeneous document graph to model the internal structure of an entity and the external interaction between entities.However, there are two drawbacks in existing methods.On one hand, anaphor plays an important role in reasoning to identify relations between entities but is ignored by these methods.On the other hand, these methods achieve crosssentence entity interactions implicitly by utilizing a document or sentences as intermediate nodes.Such an approach has difficulties in learning fine-grained interactions between entities across different sentences, resulting in sub-optimal performance.To address these issues, we propose an Anaphor-Assisted (AA) framework for DocRE tasks.Experimental results on the widely-used datasets demonstrate that our model achieves a new state-of-the-art performance.1

Entity alignment is a vital task in knowledge fusion, which aims to align entities from different knowledge graphs and merge them into one single graph. Existing entity alignment models focus on local features and try to minimize the distance between pairs of pre-aligned entities. Despite their success, these models heavily rely on the number of existing pre-aligned entity pairs and the topology information from the rest large set of unaligned entities is still largely unexplored. To overcome the limitation of existing models, we propose a model, termed Global Alignment and Local Information Aggregation, or GALA. GALA constructs global features for the knowledge graphs to be aligned using entity embeddings. It aligns the entities in the graphs by forcing their global features to match with each other and progressively updating the entity embeddings by aggregating local information from the other network. Empirical studies on commonly-used KG alignment data sets confirm the effectiveness of the proposed model.

Each link prediction task requires different degrees of answer diversity. While a link prediction task may expect up to a couple of answers, another may expect nearly a hundred answers. Given this fact, the performance of a link prediction model can be estimated more accurately if a flexible number of obtained answers are estimated instead of a predefined number of answers. Inspired by this, in this article, we analyze two evaluation criteria for link prediction tasks, respectively ranking-based protocol and sampling-based protocol. Furthermore, we study two classes of models on link prediction task, direct model and latent-variable model respectively, to demonstrate that latent-variable model performs better under the sampling-based protocol. We then propose a latent-variable model where the framework of Conditional Variational AutoEncoder (CVAE) is applied. Experimental study suggests that the proposed model performs comparably to the current state-of-the-art even under the conventional rank-based protocol. Under the sampling-based protocol, the proposed model is shown to outperform various state-of-the-art models.

Text style transfer is an important task in natural language processing with broad applications.Existing models following the masking and filling scheme suffer two challenges: the word masking procedure may mistakenly remove unexpected words and the selected words in the word filling procedure may lack diversity and semantic consistency.To tackle both challenges, in this study, we propose a style transfer model, with an adversarial masking approach and a styled filling technique (AMSF).Specifically, AMSF first trains a mask predictor by adversarial training without manual configuration.Then two additional losses, i.e. an entropy maximization loss and a consistency regularization loss, are introduced in training the word filling module to guarantee the diversity and semantic consistency of the transferred texts.Experimental results and analysis on two benchmark text style transfer data sets demonstrate the effectiveness of the proposed approaches.

In this paper, we unify a diverse set of learning tasks in NLP, semantic retrieval and related areas, under a common umbrella, which we call unsupervised semantic association learning (USAL). Examples of this generic task include word sense disambiguation, answer selection and question retrieval. We then present a novel modeling framework to tackle such tasks. The framework introduces, under the deep learning paradigm, a latent label indexing the true target in the candidate target set. An EM algorithm is then developed for learning the deep model and inferring the latent variables, principled under variational techniques and noise contrastive estimation. We apply the model and algorithm to several semantic retrieval benchmark tasks and the superior performance of the proposed approach is demonstrated via empirical studies.