Engaging students in argument from evidence is an essential goal of science education. This is a complex skill to develop; recent research in science education proposed the use of simulated classrooms to facilitate the practice of the skill. We use data from one such simulated environment to explore whether automated analysis of the transcripts of the teacher’s interaction with the simulated students using Natural Language Processing techniques could yield an accurate evaluation of the teacher’s performance. We are especially interested in explainable models that could also support formative feedback. The results are encouraging: Not only can the models score the transcript as well as humans can, but they can also provide justifications for the scores comparable to those provided by human raters.

In this theme paper, we reflect on the progress of Automated Writing Evaluation (AWE), using Ellis Page's seminal 1966 paper to frame the presentation.We discuss some of the current frontiers in the field, and offer some thoughts on the emergent uses of this technology.

In a school context, learning is usually detected by repeated measurements of the skill of interest through a sequence of specially designed tests; in particular, this is the case with tracking improvement in oral reading fluency in elementary school children in the U.S. Results presented in this paper suggest that it is possible and feasible to detect improvement in oral reading fluency using data collected during children's independent reading of a book using the Relay Reader™ app. We are thus a step closer to the vision of having a child read for the story, not for a test, yet being able to unobtrusively assess their progress in oral reading fluency.

It is difficult to overstate the importance of literacy for adequate functioning in society, from educational attainment and employment opportunities to health outcomes. We created a reading app with the goal of helping readers improve their reading skill while reading for meaning and pleasure, and used it to collect unique data on children's extended reading. Analysis of the data reveals the importance of a behavioral factor in understanding observed reading performance.

Anastassia Loukina, Van Rynald T. Liceralde, Beata Beigman Klebanov. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

Many applications designed to assess and improve oral reading fluency use automated speech recognition (ASR) to provide feedback to students, teachers, and parents. Most such applications rely on a distributed architecture with the speech recognition component located in the cloud. For interactive applications, this approach requires a reliable Internet connection that may not always be available. We investigate whether on-device ASR can be used for a virtual reading companion using recordings obtained from children both in a controlled environment and in the field. Our limited evaluation makes us cautiously optimistic about the feasibility of using on-device ASR for our application.

Metaphors are intriguing.We know that George Lakoff and Mark Johnson, in their book Metaphors We Live By, pointed out that our daily language is full of metaphors (Lakoff and Johnson 1980).Metaphors are not rare at all as linguistic events and in general as a means of human communication (e.g., in visual art).Thus, more than an exception, they seem to be a necessity for our mind.People use metaphors as strategies to link concepts, to deal with situations, and sometimes to suggest solutions to problems.For example, you can see criminality as a monster or as an illness.What you have in mind to deal with it is probably to fight in the former case, and to cure or to plan prevention in the latter.Nonetheless, metaphors are extremely difficult to model in a computational framework.What is literal?What is metaphorical?Making this distinction has often proved to be a daunting task, even for a human judgment.Metaphors-so bound to our way of thinking and entangled with a huge quantity of knowledge and linguistics subtletiesconstitute an excellent research problem for computational linguistics and artificial intelligence in general.We could probably say that they belong to the AI-complete problems.The difficulty of these computational problems is equivalent to that of solving the central artificial intelligence problem-making computers as intelligent as people.Tony Veale, Ekaterina Shutova, and Beata B. Klebanov are experienced researchers in the field of figurative language processing.Their book is an excellent resource and a good reference for anyone who plans to tackle the subtleties of this complex topic.The book offers a comprehensive approach to the computational treatment of metaphors and of related figurative devices such as simile, analogy, and conceptual blending.The reader is introduced to multiple computational perspectives, from symbolic and statistical approaches to interpretation and paraphrase generation, without omitting contributions from philosophy on what constitutes the significance of a metaphor.The first three chapters introduce the reader to the concept of metaphor, particularly the theoretical foundations profitable for approaching the problem computationally.Particularly useful is the explanation of the related figurative devices: similes (the comparison of one thing with another of a different kind, used to make a description more emphatic or vivid, e.g., John is as brave as a lion); analogy (a comparison between one thing and another, typically for the purpose of explanation, e.g., marriage is slavery); and conceptual blending (a cognitive theory, originally developed by Gilles Fauconnier and Mark Turner [Fauconnier and Turner 2002], which refers to a set of cognitive operations for combining-or blending-words, images, and ideas in a network of "mental spaces" to create meaning, e.g., the painting in George Clooney's attic is a cue to create a conceptual

We demonstrate a method of improving a seed sentiment lexicon developed on essay data by using a pivot-based paraphrasing system for lexical expansion coupled with sentiment profile enrichment using crowdsourcing. Profile enrichment alone yields up to 15% improvement in the accuracy of the seed lexicon on 3-way sentence-level sentiment polarity classification of essay data. Using lexical expansion in addition to sentiment profiles provides a further 7% improvement in performance. Additional experiments show that the proposed method is also effective with other subjectivity lexicons and in a different domain of application (product reviews).

This article discusses the transition from annotated data to a gold standard, that is, a subset that is sufficiently noise-free with high confidence. Unless appropriately reinterpreted, agreement coefficients do not indicate the quality of the data set as a benchmarking resource: High overall agreement is neither sufficient nor necessary to distill some amount of highly reliable data from the annotated material. A mathematical framework is developed that allows estimation of the noise level of the agreed subset of annotated data, which helps promote cautious benchmarking.