Publisher Copyright: © 2026 Owner/Author.

‘‘Is a Large Language Model (LLM) a good liar? Can we detect its lies for Turkish?” LLMs are capable of generating deceptive or misleading information, which can be considered as “lying”. In this article, we introduce a dataset called TQuADFake , consisting of correct and incorrect sentences generated by LLMs through the Question Answering (QA) task. We evaluate the quality of the dataset using both human and LLM evaluators. We explore linguistic features that may help distinguish incorrect sentences and build a lie detection model based on these features. Our analysis shows that while linguistic features are useful for identifying differences between correct and incorrect sentences, these features alone are insufficient to fully solve the lie detection problem. We also investigate whether LLMs are aware of their own inaccuracies and compare various models’ performance on this task, including SVM, pre-trained language models (PLMs), and LLMs. This study provides a foundation for future research on misinformation detection in Turkish and highlights the challenges in building effective lie detection systems. 1

Question Answering (QA) and Information Retrieval (IR) play a crucial role in information-seeking pipelines implemented in many emerging AI research assistant applications. Large Language Models (LLMs) have demonstrated exceptional effectiveness on QA tasks, with Retrieval Augmented Generation (RAG) techniques often boosting the results. However, in many of those emerging applications, the onus of conducting the actual literature search falls on the user, i.e. the user searches for the relevant literature and the LLM-based assistant extracts the solicited answers from each of the user-supplied documents. The interplay between the quality of the user-conducted search and the quality of the final results remains understudied.

Title generation is a crucial task in scientific writing, serving as the first point of contact for readers and impacting an article’s visibility and citation frequency. In this paper, we present a novel multi-domain and multilingual dataset, MoDeST ( M ulti- D omain S cientific T itle Generation), designed to advance research in scientific title generation. Unlike existing datasets, which are often limited to a single language or domain, MoDeST includes both English and Turkish titles across various academic disciplines, such as social sciences, medical science, science and engineering, and more. We explore the challenges of generating concise and informative titles using different sources (keywords, abstract, and full article) and evaluate the performance of large language models (LLMs) in zero-, few-shot and supervised fine-tuning (SFT) settings. We conduct human evaluation and calculate correlations between human judgement and automatic metrics, identifying the most effective evaluation metric for assessing title quality. Experiments show that fine-tuning significantly improves LLM performance for title generation, with LLaMA-3.1 8B, 70B, Aya-expanse 8B, and 32B achieving scores of 40.12, 45.21, 45.31, and 47.22 for Turkish, and 45.10, 49.10, 40.02, and 48.54 for English, respectively. Moreover, we find that abstract is the most effective input source for generating titles. Additionally, we analyse domain-specific challenges and the impact of cross-lingual generation, highlighting the need for tailored models for different domains. Our dataset, with its broad representation, ensures applicability across various academic disciplines, enhancing its utility for multi-domain and multilingual title generation while also benefiting the broader NLP community and related tasks.

Social media plays an important role in expressing the thoughts and sentiments of users. Irony is a way of stating a sentiment about something by expressing the opposite of the intended literal meaning. Irony detection is a recent emerging task in low-resource languages, although other tasks related to sentiment, such as sentiment analysis and emotion detection, have been widely tackled. In this study, we investigate Graph Neural Networks (GNNs) for irony detection in Turkish, a low-resource language in sentiment-related tasks. We incorporate semantic information into the GNNs using the Universal Conceptual Cognitive Annotation (UCCA) framework. Extensive experimental results and in-depth analysis show that our models outperform state-of-the-art irony detection models in Turkish. Our UCCA-GAT (UCCA-Graph Attention Network) model achieves an F 1 -score of 94.85% (7.362% gain over the state-of-the-art) on the Turkish-Irony-Dataset and an accuracy of 72.82% (4.39% gain over the state-of-the-art) on the IronyTR Dataset. We also provide a comprehensive analysis of the proposed models to understand their limitations. 1

Universal Conceptual Cognitive Annotation (UCCA) (Abend and Rappoport, 2013a) is a cross-lingual semantic annotation framework that provides an easy annotation without any requirement for linguistic background. UCCA-annotated datasets have been already released in English, French, and German. In this paper, we introduce the first UCCA-annotated Turkish dataset that currently involves 50 sentences obtained from the METU-Sabanci Turkish Treebank (Atalay et al., 2003; Oflazeret al., 2003). We followed a semi-automatic annotation approach, where an external semantic parser is utilised for an initial annotation of the dataset, which is partially accurate and requires refinement. We manually revised the annotations obtained from the semantic parser that are not in line with the UCCA rules that we defined for Turkish. We used the same external semantic parser for evaluation purposes and conducted experiments with both zero-shot and few-shot learning. While the parser cannot predict remote edges in zero-shot setting, using even a small subset of training data in few-shot setting increased the overall F-1 score including the remote edges. This is the initial version of the annotated dataset and we are currently extending the dataset. We will release the current Turkish UCCA annotation guideline along with the annotated dataset.

Part of speech (PoS) tagging is one of the fundamental syntactic tasks in Natural Language Processing, as it assigns a syntactic category to each word within a given sentence or context (such as noun, verb, adjective, etc.). Those syntactic categories could be used to further analyze the sentence-level syntax (e.g., dependency parsing) and thereby extract the meaning of the sentence (e.g., semantic parsing). Various methods have been proposed for learning PoS tags in an unsupervised setting without using any annotated corpora. One of the widely used methods for the tagging problem is log-linear models. Initialization of the parameters in a log-linear model is very crucial for the inference. Different initialization techniques have been used so far. In this work, we present a log-linear model for PoS tagging that uses another fully unsupervised Bayesian model to initialize the parameters of the model in a cascaded framework. Therefore, we transfer some knowledge between two different unsupervised models to leverage the PoS tagging results, where a log-linear model benefits from a Bayesian model’s expertise. We present results for Turkish as a morphologically rich language and for English as a comparably morphologically poor language in a fully unsupervised framework. The results show that our framework outperforms other unsupervised models proposed for PoS tagging.

The number of possible word forms is theoretically infinite in agglutinative languages. This brings up the out-of-vocabulary (OOV) issue for part-of-speech (PoS) tagging in agglutinative languages. Since inflectional morphology does not change the PoS tag of a word, we propose to learn stems along with PoS tags simultaneously. Therefore, we aim to overcome the sparsity problem by reducing word forms into their stems. We adopt a Bayesian model that is fully unsupervised. We build a Hidden Markov Model for PoS tagging where the stems are emitted through hidden states. Several versions of the model are introduced in order to observe the effects of different dependencies throughout the corpus, such as the dependency between stems and PoS tags or between PoS tags and affixes. Additionally, we use neural word embeddings to estimate the semantic similarity between the word form and stem. We use the semantic similarity as prior information to discover the actual stem of a word since inflection does not change the meaning of a word. We compare our models with other unsupervised stemming and PoS tagging models on Turkish, Hungarian, Finnish, Basque, and English. The results show that a joint model for PoS tagging and stemming improves on an independent PoS tagger and stemmer in agglutinative languages.