Multivariate time series classification is an ubiquitous and widely studied problem. Due to their strong generalization capability, neural networks are suitable for this problem, but their intrinsic black-box nature often limits their applicability. Temporal decision trees are a relevant alternative to neural networks for the same task regarding classification performances while attaining higher levels of transparency and interpretability. In this work, we approach the problem of hybridizing these two techniques, and present three independent, natural hybridization solutions to study if, and in what measure, both the ability of neural networks to capture complex temporal patterns and the transparency and flexibility of temporal decision trees can be leveraged. To this end, we provide initial experimental results for several tasks in a binary classification setting, showing that our proposed neural-symbolic hybridization schemata may be a step towards accurate and interpretable models.

Adverse Drug Event (ADE) extraction is one of the core tasks in digital pharmacovigilance, especially when applied to informal texts. This task has been addressed by the Natural Language Processing community using large pre-trained language models, such as BERT. Despite the great number of Transformer-based architectures used in the literature, it is unclear which of them has better performances and why. Therefore, in this paper we perform an extensive evaluation and analysis of 19 Transformer-based models for ADE extraction on informal texts. We compare the performance of all the considered models on two datasets with increasing levels of informality (forums posts and tweets). We also combine the purely Transformer-based models with two commonly-used additional processing layers (CRF and LSTM), and analyze their effect on the models performance. Furthermore, we use a well-established feature importance technique (SHAP) to correlate the performance of the models with a set of features that describe them: model category (AutoEncoding, AutoRegressive, Text-to-Text), pre-training domain, training from scratch, and model size in number of parameters. At the end of our analyses, we identify a list of take-home messages that can be derived from the experimental data.

Medical term normalization consists in mapping a piece of text to a large number of output classes.Given the small size of the annotated datasets and the extremely long tail distribution of the concepts, it is of utmost importance to develop models that are capable to generalize to scarce or unseen concepts.An important attribute of most target ontologies is their hierarchical structure.In this paper we introduce a simple and effective learning strategy that leverages such information to enhance the generalizability of both discriminative and generative models.The evaluation shows that the proposed strategy produces state-of-the-art performance on seen concepts and consistent improvements on unseen ones, allowing also for efficient zero-shot knowledge transfer across text typologies and datasets.