Conformal prediction for graph neural networks (GNNs) offers a promising framework for quantifying uncertainty, enhancing GNN reliability in high-stakes applications. However, existing methods predominantly focus on static graphs, neglecting the evolving nature of real-world graphs. Temporal dependencies in graph structure, node attributes, and ground truth labels violate the fundamental exchangeability assumption of standard conformal prediction methods, limiting their applicability. To address these challenges, in this paper, we introduce NCPNET, a novel end-to-end conformal prediction framework tailored for temporal graphs. Our approach extends conformal prediction to dynamic settings, mitigating statistical coverage violations induced by temporal dependencies. To achieve this, we propose a diffusion-based non-conformity score that captures both topological and temporal uncertainties within evolving networks. Additionally, we develop an efficiency-aware optimization algorithm that improves the conformal prediction process, enhancing computational efficiency and reducing coverage violations. Extensive experiments on diverse real-world temporal graphs, including WIKI, REDDIT, DBLP, and IBM Anti-Money Laundering dataset, demonstrate NCPNET's capability to ensure guaranteed coverage in temporal graphs, achieving up to a 31% reduction in prediction set size on the WIKI dataset, significantly improving efficiency compared to state-of-the-art methods. Our data and code are available at https://github.com/ODYSSEYWT/NCPNET.

In the fast-paced realm of open-world machine learning, Generalized Category Discovery (GCD) has emerged as a crucial task for identifying new classes within ever-evolving datasets. With the rise of multimodal data that includes text, images, audio, and video, traditional GCD methods, which often rely on parametric classifiers and single-modality inputs, face significant limitations. These approaches can lead to overfitting and hinder the ability to generalize to new categories effectively. This paper highlights the pressing need for innovative strategies that harness the richness of multimodal data to enhance contextual understanding and facilitate real-time category identification. We aim to establish a foundational framework for future GCD research, promoting a more agile and resilient approach to data classification in today’s complex information landscape. To achieve this, we propose a dynamic framework that integrates heterogeneous multi-agent systems, combining Large Language Models (LLMs) with diverse non-LLM methodologies. This approach not only enhances the adaptability and robustness of GCD solutions but also opens up transformative possibilities across critical fields such as autonomous driving, medical diagnostics, and social media analysis.

The performance gap between commercial and open-source large language models (LLMs) poses a critical challenge in achieving equitable access to advanced AI technologies, particularly for underfunded institutions. As commercial entities like OpenAI invest substantial resources into proprietary models, open-source alternatives struggle with limitations such as a lack of access to high-quality datasets and feedback, restricting opportunities for research and innovation. We propose strategies needed to democratize AI technology, emphasizing collaboration and knowledge sharing within the community. By fostering a more inclusive environment, we can develop versatile, user-focused models that empower diverse stakeholders and expand the horizons of AI research across various sectors. This paper calls for a holistic approach to bridging this gap through behavioral modeling, leveraging techniques such as reinforcement learning and scenario-based testing to enhance the capabilities of open-source LLMs.

Aspect-Based Sentiment Analysis (ABSA) comprises several subtasks: aspect term extraction (ATE), opinion term extraction (OTE), aspect term sentiment extraction (ATSE), aspect-opinion pair extraction (AOPE), and aspect sentiment triplet extraction (ASTE). Existing unified frameworks for ABSA rely heavily on large-scale annotated data, limiting scalability across domains. We propose UAOS, a double-layer unified span extraction framework that performs all five ABSA subtasks under weak supervision. Our approach first extracts aspect-opinion pairs using universal dependency-based rules from unannotated corpora. Sentiment labels for these pairs are generated via a novel zero-shot, domain-agnostic prompt-based method. The resulting weak labels train a unified span extraction architecture equipped with canonical correlation analysis for early stopping and a self-training mechanism to mitigate noise and bias in supervision. Extensive experiments on four ABSA benchmarks demonstrate that UAOS achieves competitive or superior performance compared to fully supervised baselines. It improves upon the state-of-the-art ODAO by +1.54 F1 for ATE, +0.56 for OTE, and +0.82 for AOPE. In ATSE and ASTE, where no weakly supervised baselines exist, UAOS outperforms several supervised models, setting new benchmarks. To assess domain generalizability, we evaluate UAOS on a psychology/education-domain dataset of student reflections spanning four instructional conditions. Without in-domain fine-tuning, it achieves macro F1 scores of 71.05 (ATE), 74.39 (OTE), 68.24 (AOPE), and 60.56 (ASTE). These results highlight the model’s ability to generalize to out-of-distribution, non-commercial text, underscoring its scalability for low-resource ABSA applications.

Large language models (LLMs) have been shown to struggle with complex logical reasoning tasks due to the inherent ambiguity and complexity of natural language. These challenges are further amplified when processing large and diverse datasets, increasing the likelihood of unfaithful reasoning and predictive hallucinations. However, LLMs can provide accurate responses when queries are clear and direct. Symbolic logic provides precise, well-defined rules that can help overcome ambiguity and support reasoning. In this work, we leverage symbolic logic’s precision to enhance LLMs’ logical reasoning capabilities by introducing the Graph of Logic (GoL) framework. GoL combines the power of graph structures with the strengths of LLMs and symbolic logic. GoL utilizes the precise rules of symbolic logic to infer new facts and detect LLM hallucinations effectively on complex datasets. Furthermore, GoL utilizes graph structures to support scalability for large datasets and tackle long dependencies, enabling efficient handling of complex reasoning tasks. We conduct extensive experiments across seven benchmark datasets, encompassing various types of reasoning. These include deductive, inductive, and abductive reasoning, each testing distinct aspects of logical inference. The experimental results demonstrate GoL’s advantage in improving the reasoning capabilities of LLMs. GoL outperforms the baselines with an average margin of 18.18% for the GPT-3.5 and GPT-4 models, outperforming the baselines for all datasets for the GPT-3.5 model and six out of seven datasets for the GPT-4 model1.

The aspect-opinion extraction tasks extract aspect terms and opinion terms from reviews. The supervised extraction methods achieve state-of-the-art performance but require large-scale human-annotated training data. Thus, they are restricted for open-domain tasks due to the lack of training data. This work addresses this challenge and simultaneously mines aspect terms, opinion terms, and their correspondence in a joint model. We propose an Open-Domain Aspect-Opinion Co-Mining (ODAO) method with a Double-Layer span extraction framework. Instead of acquiring human annotations, ODAO first generates weak labels for unannotated corpus by employing rules-based on universal dependency parsing. Then, ODAO utilizes this weak supervision to train a double-layer span extraction framework to extract aspect terms (ATE), opinion terms (OTE), and aspect-opinion pairs (AOPE). ODAO applies canonical correlation analysis as an early stopping indicator to avoid the model over-fitting to the noise to tackle the noisy weak supervision. ODAO applies a self-training process to gradually enrich the training data to tackle the weak supervision bias issue. We conduct extensive experiments and demonstrate the power of the proposed ODAO. The results on four benchmark datasets for aspect-opinion co-extraction and pair extraction tasks show that ODAO can achieve competitive or even better performance compared with the state-of-the-art fully supervised methods.

Diverse Natural Language Processing tasks employ constituency parsing to understand the syntactic structure of a sentence according to a phrase structure grammar. Many state-of-the-art constituency parsers are proposed, but they may provide different results for the same sentences, especially for corpora outside their training domains. This paper adopts the truth discovery idea to aggregate constituency parse trees from different parsers by estimating their reliability in the absence of ground truth. Our goal is to consistently obtain high-quality aggregated constituency parse trees. We formulate the constituency parse tree aggregation problem in two steps, structure aggregation and constituent label aggregation. Specifically, we propose the first truth discovery solution for tree structures by minimizing the weighted sum of Robinson-Foulds (RF) distances, a classic symmetric distance metric between two trees. Extensive experiments are conducted on benchmark datasets in different languages and domains. The experimental results show that our method, CPTAM, outperforms the state-of-the-art aggregation baselines. We also demonstrate that the weights estimated by CPTAM can adequately evaluate constituency parsers in the absence of ground truth.

Annotation quality and quantity positively affect the learning performance of sequence labeling, a vital task in Natural Language Processing. Hiring domain experts to annotate a corpus is very costly in terms of money and time. Crowdsourcing platforms, such as Amazon Mechanical Turk (AMT), have been deployed to assist in this purpose. However, the annotations collected this way are prone to human errors due to the lack of expertise of the crowd workers. Existing literature in annotation aggregation assumes that annotations are independent and thus faces challenges when handling the sequential label aggregation tasks with complex dependencies. To conquer the challenges, we propose an optimization-based method that infers the ground truth labels using annotations provided by workers for sequential labeling tasks. The proposed Aggregation method for Sequential Labels from Crowds (AggSLC) jointly considers the characteristics of sequential labeling tasks, workers’ reliabilities, and advanced machine learning techniques. Theoretical analysis on the algorithm’s convergence further demonstrates that the proposed AggSLC halts after a finite number of iterations. We evaluate AggSLC on different crowdsourced datasets for Named Entity Recognition (NER) tasks and Information Extraction tasks in biomedical (PICO), as well as a simulated dataset. Our results show that the proposed method outperforms the state-of-the-art aggregation methods. To achieve insights into the framework, we study the effectiveness of AggSLC’s components through ablation studies.