While Text-to-Image (T2I) diffusion models excel at generating visually appealing images of individual instances, they struggle to accurately position and control the features generation of multiple instances. The Layout-to-Image (L2I) task was introduced to address the positioning challenges by incorporating bounding boxes as spatial control signals, but it still falls short in generating precise instance features. In response, we propose the Instance Feature Generation (IFG) task, which aims to ensure both positional accuracy and feature fidelity in generated instances. To address the IFG task, we introduce the Instance Feature Adapter (IFAdapter). The IFAdapter enhances feature depiction by incorporating additional appearance tokens and utilizing an Instance Semantic Map to align instance-level features with spatial locations. The IFAdapter guides the diffusion process as a plug-and-play module, making it adaptable to various community models. For evaluation, we contribute an IFG benchmark and develop a verification pipeline to objectively compare models' abilities to generate instances with accurate positioning and features. Experimental results demonstrate that IFAdapter outperforms other models in both quantitative and qualitative evaluations.

Implicit discourse relation recognition aims to identify logical relations between two arguments without explicit connectives and is a challenging task in discourse analysis. Recent methods tend to leverage the label hierarchy to enhance discourse relation representations. However, they fail to fully utilize the connective information. Specifically, the methods overlook the guiding role of connectives in discourse relation classification by treating them as the last-level labels in the label hierarchy to leverage connective information, whereas it would be more appropriate to exploit connective information prior to relation classification. Moreover, these methods ignore the dependency degree of labels between different levels in the label hierarchy. In other words, they consider the label hierarchy as an unweighted undirected graph, and assume that the path weights between high-level labels and their corresponding low-level labels are the same, which leads to an insufficient construction of the label hierarchy. To overcome these issues, we propose a method for implicit discourse relation recognition (IDRR) utilizing Connective Prediction and Dependency-weighted Label Hierarchy (CP-DLH). Experimental results on PDTB 2.0 dataset show that our model achieves the state-of-the-art performance at all hierarchical levels.

Few-shot Named Entity Recognition aims to identify named entities from unstructured texts in various domains using a minimal amount of training samples and classify them into predefined categories. Many popular approaches decompose this process into two tasks: Span Detection and Entity Classification. However, they still have some issues: (1) Neglecting presentation optimization. Most of these methods emphasize classification, neglecting the optimization of span presentation during Span Detection. (2) Missing label semantics. They have not fully leveraged semantic information of entity type labels during Entity Classification. To address these issues, this paper proposes Multi-Granularity Dual-Aware Contrastive Learning (MGDAC) for few-shot NER. Specifically, we introduce multi-granularity contrastive learning for solve neglecting presentation optimization, focusing on both the overall vector and internal vector granularity to enhance features beneficial for span detection in token vector representations. Additionally, we design dual-aware contrastive learning for solve missing label semantics, effectively utilizing semantic information from both entity tokens and entity type labels during prototype construction to jointly optimize prototype representations. Finally, extensive experiments on the FewNERD dataset demonstrate that our proposed method exhibits improvements in both Span Detection and Entity Classification, outperforming other competitive baseline methods.

Frame-semantic Parsing (FSP) is a challenging and critical task in Natural Language Processing (NLP). Most of the existing studies decompose the FSP task into frame identification (FI) and frame semantic role labeling (FSRL) subtasks, and adopt a pipeline model architecture that clearly causes error propagation problem. However, recent jointly learning models aim to address the above problem and generally treat FSP as a span-level structured prediction task, which, unfortunately, leads to cascading error propagation problem between roles and less-efficient solutions due to huge search space of roles. To address these problems, we reformulate the FSRL task into a target-aware relation classification task and propose a novel and lightweight jointly learning framework that simultaneously processes three subtasks of FSP, including frame identification, argument identification, and role classification. The novel task formulation and jointly learning with interaction mechanisms among subtasks can help improve the overall system performance and reduce the search space and time complexity, compared with existing methods. Extensive experimental results demonstrate that our proposed model significantly outperforms 10 state-of-the-art models in terms of F1 score across two benchmark datasets.

Handwritten mathematical expression recognition aims to automatically generate LaTeX sequences from given images. Currently, attention-based encoder-decoder models are widely used in this task. They typically generate target sequences in a left-to-right (L2R) manner, leaving the right-to-left (R2L) contexts unexploited. In this paper, we propose an Attention aggregation based Bi-directional Mutual learning Network (ABM) which consists of one shared encoder and two parallel inverse decoders (L2R and R2L). The two decoders are enhanced via mutual distillation, which involves one-to-one knowledge transfer at each training step, making full use of the complementary information from two inverse directions. Moreover, in order to deal with mathematical symbols in diverse scales, an Attention Aggregation Module (AAM) is proposed to effectively integrate multi-scale coverage attentions. Notably, in the inference phase, given that the model already learns knowledge from two inverse directions, we only use the L2R branch for inference, keeping the original parameter size and inference speed. Extensive experiments demonstrate that our proposed approach achieves the recognition accuracy of 56.85 % on CROHME 2014, 52.92 % on CROHME 2016, and 53.96 % on CROHME 2019 without data augmentation and model ensembling, substantially outperforming the state-of-the-art methods. The source code is available in https://github.com/XH-B/ABM.

Xuefeng Su, Ru Li, Xiaoli Li, Jeff Z. Pan, Hu Zhang, Qinghua Chai, Xiaoqi Han. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.