Abstract Ensemble of label ranking trees (LRTs) are currently the state‐of‐the‐art approaches to the label ranking problem. Recently, bagging, boosting, and random forest methods have been proposed, all based on the LRT algorithm, which adapts regression/classification trees to the label classification problem. The LRT algorithm uses theoretically grounded Mallows probability distribution to select the best split when growing the tree, and an EM‐type process to complete the rankings on the training data when they are incomplete. These two steps have proven to be accurate, but require a large computational effort. This article proposes two alternative methods that replace the use of the Mallows distribution with distance‐based criteria to select the best split at each inner node of the tree. Moreover, these distance‐based criteria allow dealing with incomplete rankings natively, so avoiding the completion process. We have carried out an extensive experimental evaluation, which shows that (1) the integration of the two proposed modifications to the LRT algorithm into ensemble methods (bagging and random forest) are an order of magnitude faster than using the original Mallows‐based LRT algorithm; (2) ensembles using the proposed LRT methods are significantly more accurate in the presence of incomplete rankings, while they are at least as accurate in the complete case; and (3) the two modified LRT algorithms are also an order of magnitude faster than the Mallows‐based LRT, while they are at least as accurate as the Mallows‐based LRT on both complete and incomplete rankings.

Label ranking studies the problem of learning a preference model that maps instances to rankings over a finite set of predefined class labels. The training data used to solve this problem consists of instances labeled with rankings. Since these rankings are often incomplete, models need to be able to deal with missing information in the class labels to be more useful in practice. Several decision tree models have been proposed to learn from incomplete rankings, mainly using axis-parallel decision nodes, which is the standard approach for decision tree induction. In contrast to this strategy, this present work introduces a method for learning oblique decision trees for the label ranking problem, as they have been shown to improve performance in the standard classification scenario. Our experimentation shows that this method offers several advantages over the current decision tree model. Not only does it generate more compact tree structures, but it is also shown to achieve outstandingly better results for complete rankings and in cases with a low percentage of missing labels. Moreover, the proposed method is faster in the largest datasets than the current decision tree model.

As the data sets increase in size, the process of manually labeling data becomes unfeasible by small groups of experts. Thus, it is common to rely on crowdsourcing platforms which provide inexpensive, but noisy, labels. Although implementations of algorithms to tackle this problem exist, none of them focus on scalability, limiting the area of application to relatively small data sets. In this paper, we present spark-crowd, an Apache Spark package for learning from crowdsourced data with scalability in mind.