Abstract Abstractive summarization is an approach to document summarization that is not limited to selecting sentences from the document but can generate new sentences as well. We address the two main challenges in abstractive summarization: how to evaluate the performance of a summarization model and what is a good training objective. We first introduce new evaluation measures based on the semantic similarity of the input and corresponding summary. The similarity scores are obtained by the fine-tuned BERTurk model using either the cross-encoder or a bi-encoder architecture. The fine-tuning is done on the Turkish Natural Language Inference and Semantic Textual Similarity benchmark datasets. We show that these measures have better correlations with human evaluations compared to Recall-Oriented Understudy for Gisting Evaluation (ROUGE) scores and BERTScore. We then introduce a deep reinforcement learning algorithm that uses the proposed semantic similarity measures as rewards, together with a mixed training objective, in order to generate more natural summaries in terms of human readability. We show that training with a mixed training objective function compared to only the maximum-likelihood objective improves similarity scores.

Leonie Weissweiler, Valentin Hofmann, Anjali Kantharuban, Anna Cai, Ritam Dutt, Amey Hengle, Anubha Kabra, Atharva Kulkarni, Abhishek Vijayakumar, Haofei Yu, Hinrich Schuetze, Kemal Oflazer, David Mortensen. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

Abstract Sentiment analysis has attracted a lot of research interest in recent years, especially in the context of social media. While most of this research has focused on English, there is ample data and interest in the topic for many other languages, as well. In this article, we propose a comprehensive sentiment analysis system for Turkish. We cover different levels of sentiment analysis such as aspect, sentence, and document levels as well as some linguistic issues such as conjunction and intensification in Turkish sentiment analysis. Our system is evaluated on Turkish movie reviews and the obtained accuracies range from sixty per cent to seventy-nine per cent in ternary and binary classification tasks at different levels of analysis.

We present a human judgments dataset and an adapted metric for evaluation of Arabic machine translation.Our mediumscale dataset is the first of its kind for Arabic with high annotation quality.We use the dataset to adapt the BLEU score for Arabic.Our score (AL-BLEU) provides partial credits for stem and morphological matchings of hypothesis and reference words.We evaluate BLEU, METEOR and AL-BLEU on our human judgments corpus and show that AL-BLEU has the highest correlation with human judgments.We are releasing the dataset and software to the research community.

In the last decade or so, the ACM, the IEEE and other organizations have acknowledged that there is a problem with the way communication is taught in the Computer Science curriculum: the writing, speaking, and presentation skills students learn in the classroom do not match what is expected of them in the workplace. The proposed solution, adopted by many undergraduate colleges, was to add a technical communication course to the CS curriculum. This does not appear to be enough, as mainstream accreditation boards are still emphasizing the need for improvement of communication skills instruction in their recent reports and recommendations. For the last two years, we have experimented with a complementary transversal approach where many "traditional CS" courses in our program have added a communication component to their syllabus, while at the same time our technical communication course has been revamped to expose students to realistic practices as promoted by situated learning theory. The results, so far anecdotal, point to improved student performance and attitude across several communication dimensions, in particular writing and presentation. We plan to develop this experiment by spreading it across more classes and by starting to collect rigorous measurements of students' communication performance.

In this paper, we present the results of our work on the development of a phrase-based statistical machine translation prototype from English to Turkish—an agglutinative language with very productive inflectional and derivational morphology. We experiment with different morpheme-level representations for English–Turkish parallel texts. Additionally, to help with word alignment, we experiment with local word reordering on the English side, to bring the word order of specific English prepositional phrases and auxiliary verb complexes, in line with the morpheme order of the corresponding case-marked nouns and complex verbs, on the Turkish side. To alleviate the dearth of the parallel data available, we also augment the training data with sentences just with content word roots obtained from the original training data to bias root word alignment, and with highly reliable phrase-pairs from an earlier corpus alignment. We use a morpheme-based language model in decoding and a word-based language model in re-ranking the$n$-best lists generated by the decoder. Lastly, we present a scheme forrepairingthe decoder output bycorrectingwords which have incorrect morphological structure or which are out-of-vocabulary with respect to the training data and language model, to further improve the translations. We improve from 15.53 BLEU points for our word-based baseline model to 25.17 BLEU points for an improvement of 9.64 points or about 62% relative.

The suitability of different parsing methods for different languages is an important topic in syntactic parsing. Especially lesser-studied languages, typologically different from the languages for which methods have originally been developed, pose interesting challenges in this respect. This article presents an investigation of data-driven dependency parsing of Turkish, an agglutinative, free constituent order language that can be seen as the representative of a wider class of languages of similar type. Our investigations show that morphological structure plays an essential role in finding syntactic relations in such a language. In particular, we show that employing sublexical units called inflectional groups, rather than word forms, as the basic parsing units improves parsing accuracy. We test our claim on two different parsing methods, one based on a probabilistic model with beam search and the other based on discriminative classifiers and a deterministic parsing strategy, and show that the usefulness of sublexical units holds regardless of the parsing method. We examine the impact of morphological and lexical information in detail and show that, properly used, this kind of information can improve parsing accuracy substantially. Applying the techniques presented in this article, we achieve the highest reported accuracy for parsing the Turkish Treebank.

The suitability of different parsing methods for different languages is an important topic in syntactic parsing. Especially lesser-studied languages, typologically different from the languages for which methods have originally been developed, pose interesting challenges in this respect. This article presents an investigation of data-driven dependency parsing of Turkish, an agglutinative, free constituent order language that can be seen as the representative of a wider class of languages of similar type. Our investigations show that morphological structure plays an essential role in finding syntactic relations in such a language. In particular, we show that employing sublexical units called inflectional groups, rather than word forms, as the basic parsing units improves parsing accuracy. We test our claim on two different parsing methods, one based on a probabilistic model with beam search and the other based on discriminative classifiers and a deterministic parsing strategy, and show that the usefulness of sublexical units holds regardless of the parsing method. We examine the impact of morphological and lexical information in detail and show that, properly used, this kind of information can improve parsing accuracy substantially. Applying the techniques presented in this article, we achieve the highest reported accuracy for parsing the Turkish Treebank.

This paper investigates the use of sublexical units as a solution to handling the complex morphology with productive derivational processes, in the development of a lexical functional grammar for Turkish.Such sublexical units make it possible to expose the internal structure of words with multiple derivations to the grammar rules in a uniform manner.This in turn leads to more succinct and manageable rules.Further, the semantics of the derivations can also be systematically reflected in a compositional way by constructing PRED values on the fly.We illustrate how we use sublexical units for handling simple productive derivational morphology and more interesting cases such as causativization, etc., which change verb valency.Our priority is to handle several linguistic phenomena in order to observe the effects of our approach on both the c-structure and the f-structure representation, and grammar writing, leaving the coverage and evaluation issues aside for the moment.

This article presents a dependency parsing scheme using an extended finite-state approach. The parser augments input representation with “channels” so that links representing syntactic dependency relations among words can be accommodated and iterates on the input a number of times to arrive at a fixed point. Intermediate configurations violating various constraints of projective dependency representations such as no crossing links and no independent items except sentential head are filtered via finite-state filters. We have applied the parser to dependency parsing of Turkish.

This paper presents a scheme that allows one to relax the all-or-none nature of two-level constraints in two-level morphology in a controlled manner, so that word forms with violations of some of the two-level constraints can be analyzed and ranked. The problem has been motivated by a recent phenomenon in Turkish with imported words that violate a fundamental assumption of Turkish that pronunciation and orthography have almost a one-to-one correspondence, and by a problem in Basque words with differing amounts of competence errors. We present the formulation of our proposal, and provide details of implementations for both problems using the XRCE Finite State Toolkit.

This paper presents the results of a study on information extraction from unrestricted Turkish text using statistical language processing methods. In languages like English, there is a very small number of possible word forms with a given root word. However, languages like Turkish have very productive agglutinative morphology. Thus, it is an issue to build statistical models for specific tasks using the surface forms of the words, mainly because of the data sparseness problem. In order to alleviate this problem, we used additional syntactic information, i.e. the morphological structure of the words. We have successfully applied statistical methods using both the lexical and morphological information to sentence segmentation, topic segmentation, and name tagging tasks. For sentence segmentation, we have modeled the final inflectional groups of the words and combined it with the lexical model, and decreased the error rate to 4.34%, which is 21% better than the result obtained using only the surface forms of the words. For topic segmentation, stems of the words (especially nouns) have been found to be more effective than using the surface forms of the words and we have achieved 10.90% segmentation error rate on our test set according to the weighted TDT-2 segmentation cost metric. This is 32% better than the word-based baseline model. For name tagging, we used four different information sources to model names. Our first information source is based on the surface forms of the words. Then we combined the contextual cues with the lexical model, and obtained some improvement. After this, we modeled the morphological analyses of the words, and finally we modeled the tag sequence, and reached an F-Measure of 91.56%, according to the MUC evaluation criteria. Our results are important in the sense that, using linguistic information, i.e. morphological analyses of the words, and a corpus large enough to train a statistical model significantly improves these basic information extraction tasks for Turkish.

This paper presents a semiautomatic technique for developing broad-coverage finite-state morphological analyzers for use in natural language processing applications. It consists of three components—elicitation of linguistic information from humans, a machine learning bootstrapping scheme, and a testing environment. The three components are applied iteratively until a threshold of output quality is attained. The initial application of this technique is for the morphology of low-density languages in the context of the Expedition project at NMSU Computing Research Laboratory. This elicit-build-test technique compiles lexical and inØectional information elicited from a human into a finite-state transducer lexicon and combines this with a sequence of morphographemic rewrite rules that is induced using transformation-based learning from the elicited examples. The resulting morphological analyzer is then tested against a test set, and any corrections are fed back into the learning procedure, which then builds an improved analyzer.

Americanae nace como un proyecto conjunto que surge dentro de la Red Europea de Información y Documentación sobre América Latina (REDIAL), y que ha afrontado la Biblioteca de la Agencia Española de Cooperación Internacional para el Desarrollo (AECID). Esta nueva biblioteca virtual hace más accesibles los libros digitales de tema americanista a los investigadores y usuarios interesados de cualquier parte del mundo.

This paper presents a dependency parsing scheme using an extended finite state approach. The parser augments input representation with "channels" so that links representing syntactic dependency relations among words can be accommodated, and iterates on the input a number of times to arrive at a fixed point. Intermediate configurations violating various constraints of projective dependency representations such as no crossing links, no independent items except sentential head, etc, are filtered via finite state filters. We have applied the parser to dependency parsing of Turkish.

We present a constraint-based morphological disambiguation system in which individual constraints vote on matching morphological parses, and disambiguation of all the tokens in a sentence is performed at the end by selecting parses that receive the highest votes. This constraint application paradigm makes the outcome of the disambiguation independent of the rule sequence, and hence relieves the rule developer from worrying about potentially conflicting rule sequencing. Our results for disambiguating Turkish indicate that using about 500 constraint rules and some additional simple statistics, we can attain a recall of 95--96% and a precision of 94--95% with about 1.01 parses per token. Our system is implemented in Prolog and we are currently investigating an efficient implementation based on finite state transducers.

We present an efficient algorithm for retrieving from a database of trees, all trees that differ from a given query tree by a small number additional or missing leaves, or leaf label changes. It has natural language processing applications in searching for matches in example-based translation systems, and retrieval from lexical databases containing entries of complex feature structures. For large randomly generated synthetic tree databases (some having tens of thousands of trees), and on databases constructed from Wall Street Journal treebank, it can retrieve for trees with a small error, in a matter of tenths of a second to about a second.

This paper describes tactical generation in Turkish, a free con~stituent order language, in which the order of the constituents may change according to the information structure of the sentences to be generated.In the absence of any information regarding the information structure of a sentence (i.e., topic, focus, background, etc.), the constituents of the sentence obey a default order, but the order is almost freely changeable, depending on the constraints of the text flow or discourse.We have used a recursively structured finite state machine for handling the changes in constituent order, implemented as a right-linear grammar backbone.Our implementation environment is the GenKit system, developed at Carnegie Mellon University-Center for Machine Translation.Morphological realization has been implemented using an external morpholggical analysis/generation component which performs concrete morpheme selection and handles morphographemic processes.

We present a connectionist approach for solving Tangram puzzles. Tangram is an ancient Chinese puzzle where the object is to decompose a given figure into seven basic geometric figures. One connectionist approach models Tangram pieces and their possible placements and orientations as connectionist neuron units which receive excitatory connections from input units defining the puzzle and lateral inhibitory connections from competing or conflicting units. the network of these connectionist units, operating as a Boltzmann Machine, relaxes into a configuration in which units defining the solution receive no inhibitory input from other units. We present results from an implementation of our model using the Rochester Connectionist Simulator. © 1993 John Wiley & Sons, Inc.

Presents a hybrid pattern classification system which can classify patterns in a rotation, scaling, and translation invariant manner. The system is based on preprocessing the input image to map it into a rotation, scaling, and translation invariant canonical form, which is then classified by a multilayer feedforward neural net. Results from a number of classification problems are also presented in the paper.>