Rashad Albo Jamara, Nico Herbig, Antonio Krüger, Josef van Genabith. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Compared to fully manual translation, postediting (PE) machine translation (MT) output can save time and reduce errors.Automatic word-level quality estimation (QE) aims to predict the correctness of words in MT output and holds great promise to aid PE by flagging problematic output.Quality of QE is crucial, as incorrect QE might lead to translators missing errors or wasting time on already correct MT output.Achieving accurate automatic word-level QE is very hard, and it is currently not known (i) at what quality threshold QE is actually beginning to be useful for human PE, and (ii), how to best present wordlevel QE information to translators.In particular, should word-level QE visualization indicate uncertainty of the QE model or not?In this paper, we address both research questions with real and simulated word-level QE, visualizations, and user studies, where time, subjective ratings, and quality of the final translations are assessed.Results show that current wordlevel QE models are not yet good enough to support PE. Instead, quality levels of ≥ 80% F1 are required.For helpful quality levels, a visualization reflecting the uncertainty of the QE model is preferred.Our analysis further shows that speed gains achieved through QE are not merely a result of blindly trusting the QE system, but that the quality of the final translations also improves.The threshold results from the paper establish a quality goal for future wordlevel QE research.

Nico Herbig, Tim Düwel, Santanu Pal, Kalliopi Meladaki, Mahsa Monshizadeh, Antonio Krüger, Josef van Genabith. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020.

In automatic post-editing (APE) it makes sense to condition post-editing (pe) decisions on both the source (src) and the machine translated text (mt) as input.This has led to multi-encoder based neural APE approaches.A research challenge now is the search for architectures that best support the capture, preparation and provision of src and mt information and its integration with pe decisions.In this paper we present an efficient multi-encoder based APE model, called transference.Unlike previous approaches, it (i) uses a transformer encoder block for src, (ii) followed by a decoder block, but without masking for self-attention on mt, which effectively acts as second encoder combining src → mt, and (iii) feeds this representation into a final decoder block generating pe.Our model outperforms the best performing systems by 1 BLEU point on the WMT 2016, 2017, and 2018 English-German APE shared tasks (PBSMT and NMT).Furthermore, the results of our model on the WMT 2019 APE task using NMT data shows performance at the level of the state-of-the-art.The inference time of our model is similar to the vanilla transformer-based NMT system although our model deals with two separate encoders.We further investigate the importance of our newly introduced second encoder and find that decreasing the number of layers hurts performance, while reducing the number of layers of the decoder does not matter much.

In this paper, we analyze a wide range of physiological, behavioral, performance, and subjective measures to estimate cognitive load (CL) during e-learning. To the best of our knowledge, the analyzed sensor measures comprise the most diverse set of features from a variety of modalities that have to date been investigated in the e-learning domain. Our focus lies on predicting the subjectively reported CL and difficulty as well as intrinsic content difficulty based on the explored features. A study with 21 participants, who learned through videos and quizzes in a Moodle environment, shows that classifying intrinsic content difficulty works better for quizzes than for videos, where participants actively solve problems instead of passively consuming videos. Regression analysis for predicting the subjectively reported level of CL and difficulty also works with very low error within content topics. Among the explored feature modalities, eye-based features yield the best results, followed by heart-based and then skin-based measures. Furthermore, combining multiple modalities results in better performance compared to using a single modality. The presented results can guide researchers and developers of cognition-aware e-learning environments by suggesting modalities and features that work particularly well for estimating difficulty and CL.

Current advances in machine translation increase the need for translators to switch from traditional translation to post-editing (PE) of machine-translated text, a process that saves time and improves quality. This affects the design of translation interfaces, as the task changes from mainly generating text to correcting errors within otherwise helpful translation proposals. Our results of an elicitation study with professional translators indicate that a combination of pen, touch, and speech could well support common PE tasks, and received high subjective ratings by our participants. Therefore, we argue that future translation environment research should focus more strongly on these modalities in addition to mouse- and keyboard-based approaches. On the other hand, eye tracking and gesture modalities seem less important. An additional interview regarding interface design revealed that most translators would also see value in automatically receiving additional resources when a high cognitive load is detected during PE.

The idea to enhance the learning experience of e-learning platforms by incorporating measures about the user's cognitive state, e.g. the cognitive load, boredom, or attention, has been proposed several times and shows promising results. However, the works have mostly dealt with conceptual implications and technical possibilities to detect the state, without considering the user acceptance. This paper therefore investigates whether users would actually be willing to provide access to sensor data such as heart or skin measurements for the sake of making e-learning systems cognition-aware. The results of an online survey with 50 participants show that people would provide access to behavioral data like keyboard input without major concerns; however, other sensors are considered more sensitive and would require strong learning experience improvements to make disclosure worthwhile. Participants also appear less concerned about sensors that are integrated into consumer devices than about less widespread ones. Furthermore, we report the general opinions regarding cognition-aware e-learning and discuss ideas on how best to adapt to the cognitive state.

While shoppers increasingly value sustainable products, considering sustainability can be difficult and time-consuming while shopping. In an expert workshop with 22 stakeholders, we gathered requirements for an assistance system supporting customers in identifying the sustainability of products at the point of sale. We integrated the resulting demands in a first mockup prototype, which was tested and discussed with a focus group. From the workshop and the focus group discussion, we deduced a set of ten guidelines for sustainability-oriented assistance systems. These guidelines were transferred into a prototypical mobile application, which allows customers to specify their personal understanding of multiple dimensions of sustainability. According to this profile, they receive easily understandable ratings for scanned products while shopping. A user study in a real supermarket strengthens the deduced guidelines and indicates that such a system can support customers to make more sustainable product choices.