Multimodal sarcasm detection is a critical and challenging task that requires understanding the implicit associations between images and text. The existing multimodal sarcasm detection methods frequently overlook implicit sarcastic cues and are inadequate in modeling complex inter-modal conflicts. To address these limitations, we propose a novel framework, termed Sarcasm Detection with Sarcasm Cues and Hypergraph (SCH-SD). Specifically, we design prompt templates to leverage a Multimodal Large Language Model (MLLM) for generating detailed image descriptions and extracting implicit sarcastic cues from the input samples. Next, we apply a shallow cross-modal fusion mechanism to the encoded features to facilitate preliminary cross-modal interactions. Finally, we design a hypergraph-based deep cross-modal information fusion module to model the complex conflicting information between different modalities. Experimental results on the public benchmarks MMSD1.0 and MMSD2.0 demonstrate that SCH-SD outperforms existing baselines, improving Acc by 1.68% and 0.83%, and F1 by 2.77% and 1.98%, respectively, achieving state-of-the-art performance.

Event Causality Extraction (ECE) aims to extract causal event pairs from text. Existing methods overlook the interplay between causal event pairs and their corresponding textual evidence (e.g., causal event mention pairs), and fail to effectively leverage global causal dependency information. To address these issues, we propose a Mention-Level Causal Evidence and Global Causal Graph Reasoning (MLCE-GCGR) framework to enhance ECE. First, we introduce an auxiliary Event Mention Causality Extraction (EMCE) task, which extracts causal event mention pairs, to provide evidence for the main ECE task, and design a Dual-Level Interaction Enhancement (DLIE) strategy to enhance the bidirectional interplay between event-level and mention-level causality. Second, we develop a Global Causal Graph Reasoning (GCGR) module that simulates human-like multi-turn reasoning, aiming to progressively refine the causal graph by capturing global dependencies among event mentions, types, and arguments. Experiments on four benchmark datasets show that our method outperforms state-of-the-art approaches. Moreover, by extracting causal event mention pairs as supporting evidence, our approach improves the interpretability of structured causality extraction.

ABSTRACT Purpose With the rapid development of artificial intelligence technology, highly intelligent and unmanned factories have become an important trend. In the complex environments of smart factories, the long‐term tracking and inspection of specified targets, such as operators and special products, as well as comprehensive visual recognition and decision‐making capabilities throughout the whole production process, are critical components of automated unmanned factories. However, challenges such as target occlusion and disappearance frequently occur, complicating long‐term tracking. Currently, there is limited research specifically focused on developing robust and comprehensive long‐term visual tracking frameworks for unmanned factories, particularly those designed to integrate with embedded platforms and overcome various challenges. Methods We first construct three new benchmark datasets in the complex workshop environment of a smart factory (referred to as SF‐Complex3 data), which include challenging conditions such as complete occlusion and partial occlusion of targets. A brain memory‐inspired approach is used to determine uncertainty estimation parameters, including confidence, peak‐to‐sidelobe ratio and average peak‐to‐correlation energy, to develop a continual learning‐based adaptive model update method. Additionally, we design a lightweight target detection model to automatically detect and locate targets in the initial frame and during re‐detection. Finally, we integrate the algorithm with ground mobile robots and unmanned aerial vehicles‐based imaging and processing equipment to build a new visual detection and tracking framework, smart factory complex recognition and tracking. Results We conducted extensive tests on the benchmark UAV20L and SF‐Complex3 datasets. The proposed algorithm demonstrates an average performance improvement of 6% when addressing key challenging attributes, compared to state‐of‐the‐art tracking methods. Additionally, the algorithm was capable of running efficiently on embedded platforms, including mobile robots and UAVs, at a real‐time speed of 36.4 frames per second. Conclusions The proposed SFC‐RT framework effectively addresses the challenges of target loss and occlusion in long‐term tracking within complex smart factory environments. The framework meets the requirements for real‐time performance, robustness and lightweight design, making it well suited for practical deployment.

Event Causality Extraction (ECE) aims to extract the cause-effect event pairs with their structured event information from plain texts. As far as we know, the existing ECE methods mainly focus on the correlation between arguments, without explicitly modeling the causal relationship between events, and usually design two independent frameworks to extract cause events and effect events, respectively, which cannot effectively capture the dependency between the subtasks. Therefore, we propose a joint multi-label extraction framework for ECE to alleviate the above limitations. In particular, 1) we design a heterogeneous-relation-aware graph module to learn the potential relationships between events and arguments, in which we construct the heterogeneous graph by taking the predefined event types and all the words in the sentence as nodes, and modeling three relationships of "event-event", "event-argument" and "argument-argument" as edges. 2) We also design a multi-channel label enhancing module to better learn the distributed representation of each label in the multi-label extraction framework, and further enhance the interaction between the subtasks by considering the preliminary results of cause-effect type identification and event argument extraction. The experimental results on the benchmark dataset ECE-CCKS show that our approach outperforms previous state-of-the-art methods, and that our model also performs well on the complex samples with multiple cause-effect event pairs.

Document-level Event Extraction (DEE) is a vital task in NLP as it seeks to automatically recognize and extract event information from a document. However, current approaches often overlook intricate relationships among events and subtle correlations among arguments within a document, which can significantly impact the effectiveness of event type recognition and the extraction of cross-sentence arguments in DEE task. This paper proposes a novel Correlation Association Interactive Network (CAINet), comprising two key components: event relationship graph and argument correlation graph. In particular, the event relationship graph models the relationship among various events through structural associations among event nodes and sentence nodes, to improve the accuracy of event recognition. On the other hand, the arguments correlation graph models the correlations among arguments by quantifying the strength of association among arguments, to effectively aggregate cross-sentence arguments, contributing to the overall success of DEE. Furthermore, we use the large language model to execute DEE task experiments. Experimental results show the proposed CAINet outperforms existing state-of-the-art models and large language models in terms of F1-score across two benchmark datasets.

Large pre-trained code generation models, such as OpenAI Codex, can generate syntax-and function-correct code, making the coding of programmers more productive. In this paper, we introduce CodeGeeX, a multilingual model with 13 billion parameters for code generation. CodeGeeX is pre-trained on 850 billion tokens of 23 programming languages as of June 2022. Our extensive experiments suggest that CodeGeeX outperforms multilingual code models of similar scale for both the tasks of code generation and translation on HumanEval-X. Building upon HumanEval (Python only), we develop the HumanEval-X benchmark for evaluating multilingual models by hand-writing the solutions in C++, Java, JavaScript, and Go. In addition, we build CodeGeeX-based extensions on Visual Studio Code, JetBrains, and Cloud Studio, generating 8 billion tokens for tens of thousands of active users per week. Our user study demonstrates that CodeGeeX can help to increase coding efficiency for 83.4% of its users. Finally, CodeGeeX is publicly accessible since Sep. 2022, we open-sourced its code, model weights, API, extensions, and HumanEval-X at https://github.com/THUDM/CodeGeeX.

Emotion inference in multi-turn conversations aims to predict the participant's emotion in the next upcoming turn without knowing the participant's response yet, and is a necessary step for applications such as dialogue planning.However, it is a severe challenge to perceive and reason about the future feelings of participants, due to the lack of utterance information from the future.Moreover, it is crucial for emotion inference to capture the characteristics of emotional propagation in conversations, such as persistence and contagiousness.In this study, we focus on investigating the task of emotion inference in multi-turn conversations by modeling the propagation of emotional states among participants in the conversation history, and propose an addresseeaware module to automatically learn whether the participant keeps the historical emotional state or is affected by others in the next upcoming turn.In addition, we propose an ensemble strategy to further enhance the model performance.Empirical studies on three different benchmark conversation datasets demonstrate the effectiveness of the proposed model over several strong baselines.

Detecting public sentiment drift is a challenging task due to sentiment change over time.Existing methods first build a classification model using historical data and subsequently detect drift if the model performs much worse on new data.In this paper, we focus on distribution learning by proposing a novel Hierarchical Variational Auto-Encoder (HVAE) model to learn better distribution representation, and design a new drift measure to directly evaluate distribution changes between historical data and new data.Our experimental results demonstrate that our proposed model achieves better results than three existing state-of-theart methods.