Available training data for named entity recognition (NER) often contains a significant percentage of incorrect labels for entity types and entity boundaries.Such label noise poses challenges for supervised learning and may significantly deteriorate model quality.To address this, prior work proposed various noise-robust learning approaches capable of learning from data with partially incorrect labels.These approaches are typically evaluated using simulated noise where the labels in a clean dataset are automatically corrupted.However, as we show in this paper, this leads to unrealistic noise that is far easier to handle than real noise caused by human error or semi-automatic annotation.To enable the study of the impact of various types of real noise, we introduce NOISEBENCH, an NER benchmark consisting of clean training data corrupted with 6 types of real noise, including expert errors, crowdsourcing errors, automatic annotation errors and LLM errors.We present an analysis that shows that real noise is significantly more challenging than simulated noise, and show that current state-of-the-art models for noise-robust learning fall far short of their achievable upper bound.We release NOISEBENCH for both English and German to the research community 1 .

Crop type prediction is a very relevant and a very challenging task. The increasing availability of high-quality satellite imagery and machine learning have enabled the development of automatic crop type classification methods. In this paper, we present a crop type prediction data suite that consists of crop type information from three countries (Denmark, the Netherlands, and Slovenia) across three years (2017, 2018 and 2019). By considering the complex challenges contained by this data suite, we investigate the robustness of 7 deep learning methods used for crop type prediction (TempCNN, MSResNet, InceptionTime, OmniscaleCNN, LSTM, StarRNN, and Transformer networks). The comprehensive experiments reveal that the recurrence-based methods perform the best (with LSTM being the best performing). The methods can achieve very good predictive performance - up to a weighted F1 score of 0.8432.

Electrocardiogram (ECG) recordings are indicative for the state of the human heart. Automatic analysis of these recordings can be performed using various computational methods from the areas of signal processing and machine learning. In addition to the 12-lead ECG devices and the Holter monitor, as currently the most widely used ECG screening methods in clinical practice, ECG recordings are recently often acquired with small novel wireless ECG body sensors. These novel types of body sensors allow for ECG monitoring and analysis to be used for a much broader array of applications than only diagnosing cardiovascular disorders. The new types of ECG measuring devices, as well as their broader and more frequent use, pose new challenges in the processing and analysis of ECG, and furthermore, raise the need for automatic, low-cost, real-time, and efficient ECG monitoring that can be used at home or under ambulatory settings alike. This paper provides a comprehensive survey on the variety of both ECG data and computational methods in various applications: morphological and rhythmic arrhythmia detection, signal quality assessment, biometric identification, respiration estimation, fetal ECG extraction, and physical and emotional monitoring. It includes an extensive overview of 45 diverse ECG public databases and their analysis with state-of-the-art computational ECG methods. We highlight the most notable achievements in each of these ECG application areas in the recent years, and, furthermore, identify future trends in computational ECG analysis, especially analysis of ECG from mobile devices. The general conclusion is that ECG for medical diagnosis is successfully analyzed with the existing methods, while different applications during daily ECG monitoring are still open fields. Given how deep learning has been able to successfully address a lot of the most significant computational ECG problems, like arrhythmia classification, in future, it is expected for deep learning methods to be comprehensively tested in areas where they have not been yet applied, such as respiration estimation and fetal ECG extraction.