Growing parts of the population suffer from mental health problems and psychologists lack capacity to diagnose, let alone treat, all those in need of it. Given recent advancements in the field, deep learning-based NLP techniques could help by detecting those in need of help based on their written text. To this end, this work improves the current state-of-the-art Hierarchical Attention Network (HAN) model by incorporating contextual awareness through BERT-based word embeddings and a multi-head self-attention user-encoder yielding the Context-HAN model. When trained and tested on the eRisk data sets on Self-Harm, Anorexia, and Depression, Context-HAN outperformed the HAN model across all data sets based on various evaluation measures. Furthermore, we find and discuss some interesting insights from analysis of the attention scores, such as that longer and more recently written posts are more important for classification. This work shows the potential of attention mechanisms to leverage contextual information to improve the effectiveness of NLP methods at detecting mental health disorders from user-written text.

One of the problems that e-commerce users face is that the desired products are sometimes not available and Web shops fail to provide similar products due to their exclusive reliance on Boolean faceted search. User preferences are also often not taken into account. In order to address these problems, we present a novel framework specifically geared towards approximate faceted search within the product catalog of a Web shop. It is based on adaptations to the p-norm extended Boolean model, to account for the domain-specific characteristics of faceted search in an e-commerce environment. These e-commerce specific characteristics are, for example, the use of quantitative properties and the presence of user preferences. Our approach explores the concept of facet similarity functions in order to better match products to queries. In addition, the user preferences are used to assign importance weights to the query terms. Using a large-scale experimental setup based on real-world data, we conclude that the proposed algorithm outperforms the considered benchmark algorithms. Last, we have performed a user-based study in which we found that users who use our approach find more relevant products with less effort.

Every day millions of news articles and (micro)blogs that contain financial information are posted online. These documents often include insightful financial aspects with associated sentiments. In this paper, we predict financial aspect classes and their corresponding polarities (sentiment) within sentences. We use data from the Financial Question & Answering (FiQA) challenge, more precisely the aspect-based financial sentiment analysis task. We incorporate the hierarchical structure of the data by using the parent aspect class predictions to improve the child aspect class prediction (two-step model). Furthermore, we incorporate model output from the child aspect class prediction when predicting the polarity. We improve the F1 score by 7.6% using the two-step model for aspect classification over direct aspect classification in the test set. Furthermore, we improve the state-of-the-art test F1 score of the original aspect classification challenge from 0.46 to 0.70. The model that incorporates output from the child aspect classification performs up to par in polarity classification with our plain RoBERTa model. In addition, our plain RoBERTa model outperforms all the state-of-the-art models, lowering the MSE score by at least 28% and 33% for the cross-validation set and the test set, respectively.

E-commerce companies collaborate with retailers to sell products via their platforms, making it increasingly important to preserve platform quality. In this paper, we contribute by introducing a novel method to predict the quality of product orders shortly after they are placed. By doing so, platforms can act fast to resolve bad quality orders and potentially prevent them from happening. This introduces a trade-off between accuracy and timeliness, as the sooner we predict, the less we know about the status of a product order and, hence, the lower the reliability. To deal with this, we introduce the Hierarchical Classification Over Time (HCOT) algorithm, which dynamically classifies product orders using top-down, non-mandatory leaf-node prediction. We enforce a blocking approach by proposing the Certainty-based Automated Thresholds (CAT) algorithm, which automatically computes optimal thresholds at each node. The resulting CAT-HCOT algorithm has the ability to provide both accurate and timely predictions by classifying a product order's quality on a daily basis if the classification reaches a predefined certainty. CAT-HCOT obtains a predictive accuracy of 94%. Furthermore, CAT-HCOT classifies 40% of product orders on the order date itself, 80% within five days after the order date, and 100% of product orders after 10 days.

Attention is an important mechanism that can be employed for a variety of deep learning models across many different domains and tasks. This survey provides an overview of the most important attention mechanisms proposed in the literature. The various attention mechanisms are explained by means of a framework consisting of a general attention model, uniform notation, and a comprehensive taxonomy of attention mechanisms. Furthermore, the various measures for evaluating attention models are reviewed, and methods to characterize the structure of attention models based on the proposed framework are discussed. Last, future work in the field of attention models is considered.

Aspect-based sentiment analysis allows one to compute the sentiment for an aspect in a certain context. One problem in this analysis is that words possibly carry different sentiments for different aspects. Moreover, an aspect’s sentiment might be highly influenced by the domain-specific knowledge. In order to tackle these issues, in this paper, we propose a hybrid solution for sentence-level aspect-based sentiment analysis using A Lexicalized Domain Ontology and a Regularized Neural Attention model (ALDONAr). The bidirectional context attention mechanism is introduced to measure the influence of each word in a given sentence on an aspect’s sentiment value. The classification module is designed to handle the complex structure of a sentence. The manually created lexicalized domain ontology is integrated to utilize the field-specific knowledge. Compared to the existing ALDONA model, ALDONAr uses BERT word embeddings, regularization, the Adam optimizer, and different model initialization. Moreover, its classification module is enhanced with two 1D CNN layers providing superior results on standard datasets.

This work focuses on sentence-level aspect-based sentiment analysis for restaurant reviews. A two-stage sentiment analysis algorithm is proposed. In this method, first a lexicalized domain ontology is used to predict the sentiment and as a back-up algorithm a neural network with a rotatory attention mechanism (LCR-Rot) is utilized. Furthermore, two features are added to the backup algorithm. The first extension changes the order in which the rotatory attention mechanism operates (LCR-Rot-inv). The second extension runs over the rotatory attention mechanism for multiple iterations (LCR-Rot-hop). Using the SemEval-2015 and SemEval-2016 data, we conclude that the two-stage method outperforms the baseline methods, albeit with a small percentage. Moreover, we find that the method where we iterate multiple times over a rotatory attention mechanism has the best performance.

Currently many webshops rely on a fixed list of product facets to help users find the products of interest. Such a solution suffers from two problems: (1) it is difficult to devise a fixed list of facets that would satisfy all user interests, and (2) the top facets could become obsolete when all product results have these facets. To address these two problems we propose a novel algorithm for dynamic ordering of the product facets based on the query results. This algorithm relies on measures such as specificity and dispersion for qualitative and quantitative facets, respectively, to rank the properties associated with these facets so that users are able to find the products of interest with a minimum number of drill-down steps. Using a large-scale simulation study and a user-based evaluation, we show that our algorithm outperforms the expert-based fixed facets approach, a greedy baseline, and a state-of-the-art entropy-based solution. This paper is an extended abstract of our previous work [1].

Customers of a webshop are often presented large assortments, which can lead to customers struggling finding their desired product(s), an issue known as choice overload. In order to overcome this issue, recommender systems are used in webshops to provide personalized product recommendations to customers. Though, recommender systems using matrix factorization are not able to provide recommendations to new customers (i.e., cold users). To facilitate recommendations to cold users we investigate multiple active learning strategies, and subsequently evaluate which active learning strategy is able to optimally elicit the preferences from the cold users. Our model is empirically validated using a dataset from the webshop of de Bijenkorf, a Dutch department store. We find that the overall best-performing active learning strategy is PopGini, an active learning strategy which combines the popularity of an item with its Gini impurity score.

Faceted browsing is widely used in Web shops and product comparison sites. In these cases, a fixed ordered list of facets is often employed. This approach suffers from two main issues. First, one needs to invest a significant amount of time to devise an effective list. Second, with a fixed list of facets, it can happen that a facet becomes useless if all products that match the query are associated to that particular facet. In this work, we present a framework for dynamic facet ordering in e-commerce. Based on measures for specificity and dispersion of facet values, the fully automated algorithm ranks those properties and facets on top that lead to a quick drill-down for any possible target product. In contrast to existing solutions, the framework addresses e-commerce specific aspects, such as the possibility of multiple clicks, the grouping of facets by their corresponding properties, and the abundance of numeric facets. In a large-scale simulation and user study, our approach was, in general, favorably compared to a facet list created by domain experts, a greedy approach as baseline, and a state-of-the-art entropy-based solution.

The field of sentiment analysis, in which sentiment is gathered, analyzed, and aggregated from text, has seen a lot of attention in the last few years. The corresponding growth of the field has resulted in the emergence of various subareas, each addressing a different level of analysis or research question. This survey focuses on aspect-level sentiment analysis, where the goal is to find and aggregate sentiment on entities mentioned within documents or aspects of them. An in-depth overview of the current state-of-the-art is given, showing the tremendous progress that has already been made in finding both the target, which can be an entity as such, or some aspect of it, and the corresponding sentiment. Aspect-level sentiment analysis yields very fine-grained sentiment information which can be useful for applications in various domains. Current solutions are categorized based on whether they provide a method for aspect detection, sentiment analysis, or both. Furthermore, a breakdown based on the type of algorithm used is provided. For each discussed study, the reported performance is included. To facilitate the quantitative evaluation of the various proposed methods, a call is made for the standardization of the evaluation methodology that includes the use of shared data sets. Semanticallyrich concept-centric aspect-level sentiment analysis is discussed and identified as one of the most promising future research direction.

In this paper we present a framework for automatic exploitation of news in stock trading strategies. Events are extracted from news messages presented in free text without annotations. We test the introduced framework by deriving trading strategies based on technical indicators and impacts of the extracted events. The strategies take the form of rules that combine technical trading indicators with a news variable, and are revealed through the use of genetic programming. We find that the news variable is often included in the optimal trading rules, indicating the added value of news for predictive purposes and validating our proposed framework for automatically incorporating news in stock trading strategies.

Multifaceted search is a commonly used interaction paradigm in e-commerce applications, such as Web shops. Because of the large amount of possible product attributes, Web shops usually make use of static information to determine which facets should be displayed. Unfortunately, this approach does not take into account the user query, leading to a non-optimal facet drill down process. In this paper, we focus on automatic facet selection, with the goal of minimizing the number of steps needed to find the desired product. We propose several algorithms for facet selection, which we evaluate against the state-of-the-art algorithms from the literature. We implement our approach in a Web application called faccy.net. The evaluation is based on simulations employing 1000 queries, 980 products, 487 facets, and three drill down strategies. As evaluation metrics we use the average number of clicks, the average utility, and the top-10 promotion percentage. The results show that the Probabilistic Entropy algorithm significantly outperforms the other considered algorithms.

Building domain taxonomies is a crucial task in the domain of ontology construction. Domain taxonomy learning keeps getting more important as a form of automatically obtaining a knowledge representation of a certain domain. The alternative of manually developing domain taxonomies is not trivial. The main issues encountered when manually developing a taxonomy are the non-availability of a domain knowledge expert and the considerable amount of effort needed for this task. This paper proposes Taxo Learn, an approach to automatic construction of domain taxonomies. Taxo Learn is a new methodology that combines aspects from existing approaches, but also contains new steps in order to improve the quality of the resulted domain taxonomy. The contribution of this paper is threefold. First, we employ a word sense disambiguation step when detecting concepts in the text. Second, we show the use of semantics-based hierarchical clustering for the purpose of taxonomy learning. Third, we propose a novel dynamic labeling procedure for the concept clusters. We evaluate our approach by comparing the machine generated taxonomy with a manually constructed golden taxonomy. Based on a corpus of documents in the field of financial economics, Taxo Learn shows a high precision for the learned taxonomic concept relationships.

In order to effectively and efficiently disclose the ever-growing amount of widely distributed RDF data to demanding users in real-time environments, RDF query engines need to optimize the join order of partial query results. For this, a two-phase optimization (2PO) algorithm and a genetic algorithm (GA) have already been proposed. We propose an alternative approach - an ant colony system (ACS). On a large RDF data source, our approach significantly outperforms both 2PO and the GA in terms of execution time and solution quality for RDF chain queries consisting of up to about ten joins. For larger queries, our novel ACS delivers solutions of better quality than 2PO does, while realizing a solution quality that is comparable to the solution quality of the GA method. However, the GA approach offers the best trade-off between execution time and solution quality for such larger queries.

In this paper we evaluate several novelty control mechanisms for ranking Web news articles depicting a story. These mechanisms rank individual news items based on a novelty measure using as context the items which have been previously reviewed. An evaluation within the Hermes news personalization framework is performed for pair wise and non-pair wise novelty control mechanisms based on various distance measures and vector-based news representations. On average, the most effective distance measure is Kullback-Leibler and the best performing news representation vector uses named entities.

Sentiment analysis has applications in many areas and the exploration of its potential has only just begun. We propose Pathos, a framework which performs document sentiment analysis (partly) based on a document's discourse structure. We hypothesize that by splitting a text into important and less important text spans, and by subsequently making use of this information by weighting the sentiment conveyed by distinct text spans in accordance with their importance, we can improve the performance of a sentiment classifier. A document's discourse structure is obtained by applying Rhetorical Structure Theory on sentence level. When controlling for each considered method's structural bias towards positive classifications, weights optimized by a genetic algorithm yield an improvement in sentiment classification accuracy and macro-level F1 score on documents of 4.5% and 4.7%, respectively, in comparison to a baseline not taking into account discourse structure.

The emergence of Web 2.0 and the semantic Web as established technologies is fostering a whole new breed of Web applications and systems. These are often centered around knowledge engineering and context awareness. However, adequate temporal formalisms underlying context awareness are currently scarce. Our focus in this paper is two-fold. We first introduce a new OWL-based temporal formalism - TOWL - for the representation of time, change, and state transitions. Based hereon we present a financial Web-based application centered around the aggregation of stock recommendations and financial data.

The World Wide Web is a rapidly growing business environmenthosting a large number of business transactions. Methods fordesigning Web Information Systems (WIS) have adopted processmodels, typically as extensions of the navigation models they arebased on. We observe that the structure of business processes inWIS goes beyond the scope of the navigation structure and deservesto be a more prominent aspect of the design, abstracted from thenavigation specification. This paper explains the design of WISdriven by the specification of business processes of WIS that isused for the automatic generation of models describing theapplication logic including the navigation structure. Thespecification of processes is explained on a conference review andsubmission system and the generation process is demonstrated onthe Hera application model.

To make the WWW machine-understandable there is a strong demand both for languages describing metadata and for languages querying metadata. The Resource Description Framework (RDF), a language proposed by W3C, can be used for describing metadata about (Web) resources. The RDF schema (RDFS) extends RDF by providing means for creating application specific vocabularies (ontologies). While the above two languages are widely acknowledged as a standard means for describing Web metadata, a standardized language for querying RDF metadata is still an open issue. Research groups from both industry and academia are presently involved in proposing several RDF query languages. Due to the lack of an RDF algebra such query languages use APIs to describe their semantics and optimization issues are mostly neglected. This paper proposes RAL, an RDF algebra suitable for defining (and comparing) the semantics of different RDF query languages and (at a future stage) for performing algebraic optimizations. After a definition of the data model we present the operators with which the model can be manipulated. The operators come in three flavors: extraction operators retrieve needed resources from the input RDF model, loop operators support repetition, and construction operators build the resulting RDF model.