Used car pricing is a critical aspect of the automotive industry, influenced by many economic factors and market dynamics. With the recent surge in online marketplaces and increased demand for used cars, accurate pricing would benefit both buyers and sellers by ensuring fair transactions. However, the transition towards automated pricing algorithms using machine learning necessitates the comprehension of model uncertainties, specifically the ability to flag predictions that the model is unsure about. Although recent literature proposes the use of boosting algorithms or nearest neighbor-based approaches for swift and precise price predictions, encapsulating model uncertainties with such algorithms presents a complex challenge. We introduce ProbSAINT, a model that offers a principled approach for uncertainty quantification of its price predictions, along with accurate point predictions that are comparable to state-of-the-art boosting techniques. Furthermore, acknowledging that the business prefers pricing used cars based on the number of days the vehicle was listed for sale, we show how ProbSAINT can be used as a dynamic forecasting model for predicting price probabilities for different expected offer durations. Our experiments further indicate that ProbSAINT is especially accurate in instances where it is highly certain. This proves the applicability of its probabilistic predictions in real-world scenarios where trustworthiness is crucial.

Sequential recommendation models are crucial for next-item prediction tasks in various online platforms, yet many focus on a single behavior, neglecting valuable implicit interactions. While multi-behavioral models address this using graph-based approaches, they often fail to capture sequential patterns simultaneously. Our proposed Multi-Behavioral Sequential Recommendation framework (MBSRec) captures the multi-behavior dependencies between the heterogeneous historical interactions via multi-head self-attention. Furthermore, we utilize a weighted binary cross-entropy loss for precise behavior control. Experimental results on four datasets demonstrate MBSRec’s significant outperformance of state-of-the-art approaches. The implementation code is available here 1.

Abstract Handling incomplete multivariate time series is an important and fundamental concern for a variety of domains. Existing time-series imputation approaches rely on basic assumptions regarding relationship information between sensors, posing significant challenges since inter-sensor interactions in the real world are often complex and unknown beforehand. Specifically, there is a lack of in-depth investigation into (1) the coexistence of relationships between sensors and (2) the incorporation of reciprocal impact between sensor properties and inter-sensor relationships for the time-series imputation problem. To fill this gap, we present the Structure-aware Decoupled imputation network (SaD), which is designed to model sensor characteristics and relationships between sensors in distinct latent spaces. Our approach is equipped with a two-step knowledge integration scheme that incorporates the influence between the sensor attribute information as well as sensor relationship information. The experimental results indicate that when compared to state-of-the-art models for time-series imputation tasks, our proposed method can reduce error by around 15%.

In this work, we develop machine learning algorithms for pricing used vehicles for B2B sales at Volkswagen Financial Services. The B2B pricing dataset is structured as Tabular data, however unlike commonly available tabular datasets, it is also time-indexed. To exploit the temporal component and improve prediction accuracy, we design a Sequential network that processes sequences of vehicles. We transform the data from a Tabular representation to a sequential representation by appending previously sold vehicles and their prices as additional features for a target vehicle to be priced. The sequential network, PriceNet, embeds each of the multiple vehicles through a dedicated Embedding module, and uses a series of Convolutional layers that learn sequential features relating to price trends and seasonalities. We show that PriceNet can improve performance over the state-of the-art deep-learning based Tabular baselines, Tab-Transformer and FT-Transformer.Additionally, this paper also covers related aspects such as Chronological Validation strategies for hyperparameter tuning models for time-indexed data. Notably, we also observed that Gradient Boosted Decision Tree algorithms, outperformed all the models in prediction accuracy. For this class of models, we therefore also designed a new Quantile level tuning approach that tunes the quantile level based on Out-of-sample chronological validation data. By tuning the quantile level, we can probabilistically determine whether to overshoot or undershoot in the case of temporal covariate shifts for out-of-sample testing data observed after the validation split.

Irregularly sampled time series data with missing values is a observed in many fields like healthcare, astronomy, and climate science. Interpolation of these types of time series is crucial for tasks such as root cause analysis and medical diagnosis, as well as for smoothing out irregular or noisy data. To address this challenge, we present a novel encoder-decoder architecture called “Tripletformer” for probabilistic interpolation of irregularly sampled time series with missing values. This attention-based model operates on sets of observations, where each element is composed of a triple of time, channel, and value. The encoder and decoder of the Tripletformer are designed with attention layers and fully connected layers, enabling the model to effectively process the presented set elements. We evaluate the Tripletformer against a range of baselines on multiple real-world and synthetic datasets and show that it produces more accurate and certain interpolations. Results indicate an improvement in negative loglikelihood error by up to 32% on real-world datasets and 85% on synthetic datasets when using the Tripletformer compared to the next best model.

Remarkable progress has been achieved in generative modeling for time-series data with the introduction of Generative Adversarial Networks (GANs) [1]. GANs are neural networks that are meant to generate synthetic instances of data utilizing two neural networks, a generator and a discriminator, that operate against each other at the same time [1]. The generator learns to generate fake data to get the discriminator to classify its generated samples as authentic. The discriminator, on the other hand, attempts to distinguish between authentic and produced data. Finally, the generator could generate realistic data. GANs have demonstrated their ability to generate realistic data and have made remarkable progress in various tasks, such as the generation of time-series [4], images [5], and videos [3]. Particularly, a significant amount of work has utilized GANs based on Recurrent Neural Networks (RNNs) for time-series generation [4]. However, by carefully examining the generated samples from these models, we can observe that RNN-based GANs, such as LSTM GANs and gated recurrent GANs, cannot handle long sequences. Although RNN-based GANs can generate many realistic samples, there is still a difficulty in training due to exploding vanishing gradients and mode collapse that limits their generation capability. In addition, these RNN-based GANs are typically designed for regular time-series data, and thus cannot maintain informative varying intervals properly, which is a major concern for generating time-series data.In this paper, we propose SparseGAN, a novel sparse self-attention-based GANs that allows for attention-driven, long-memory modeling for regular and irregular time-series generation through learned embedding space. This way, it can yield a more informative representation and capture long-range dependencies for time-series generation while using original data for supervision. SparseGAN comprises two essential sub-networks: the Supervision Network and the Generation Network. These networks collaborate in an end-to-end manner to generate realistic time-series data.The Supervision Network is an encoder-decoder network, employing Gated Recurrent Unit (GRU) cells for feature extraction. It takes input time-series data and produces a latent feature representation followed by a decoder that reconstructs the time-series data. This reconstruction is crucial for facilitating the generation task and reducing data complexity. Using reconstructed data, instead of actual data, helps the generation network learn underlying data dynamics more effectively. The Supervision Network aims to minimize a reconstruction loss defined as the expected difference between the input data and the network’s output.The Generation Network consists of a generator and a discriminator following the standard GAN architecture. The generator takes random noise as input and attempts to transform it into realistic time series, while the discriminator aims to distinguish between real and generated time series. The adversarial loss is used to train the generator and discriminator. This loss measures how well the discriminator can distinguish between real and generated data.To handle irregular and sparse time-series data, the proposed SparseGAN introduces a Sparse Self-Attention Module. Traditional self-attention mechanisms calculate dense dependencies between all pairs of time steps, which is inefficient and often fails to assign zero probability to less significant relationships. Sparse Self-Attention employs an $\alpha$-entmax transformation [2] to replace the standard softmax, enabling sparser probability distributions and reducing computation complexity.Sparse Self-Attention GANs incorporate the Sparse Self-Attention Module into the generator and discriminator architectures. The generator employs a stack of sparse self-attention layers, followed by a fully connected feed-forward network, while the discriminator also uses self-attention layers with $\alpha$-entmax transformation. This modification improves the generation of time-series data by accurately modeling the distribution of real data.The Supervision Network and Generation Network are jointly trained in an end-to-end manner. This approach ensures that the generated data aligns with the real data distribution. To enhance the quality of generated data, the Supervision Network’s output, C, is used to supervise the generator. This supervision loss, $L_{supervise}$, measures the difference between the reconstructed data and the generated data. The final objective function, L*, combines the reconstruction loss, adversarial loss, and supervision loss, with a hyperparameter $\lambda_{s}$ controlling the contribution of the supervision loss.For evaluation, we evaluate the effectiveness of the proposed model using synthetic and real-world datasets. We also conducted a series of experiments to evaluate the effectiveness of SparseGAN in addressing key challenges in time-series data generation. Firstly, we used a diverse range of time-series data types, including regular and irregular time series. These datasets included synthetic sine waves, daily Google stocks data, energy consumption data, power consumption data with irregular sampling, and air quality data with irregular sampling. To benchmark SparseGAN’s performance, we compared it against various baseline models, including TimeGAN [4] and several baseline models, to assess its data generation quality and diversity.The evaluation process comprised two key aspects: fidelity and diversity. Fidelity was assessed by conducting experiments to augment real-world datasets with SparseGAN-generated data and measuring the resulting improvement in time-series forecasting accuracy, especially in low-data scenarios. The results consistently showed that SparseGAN significantly improved forecasting accuracy compared to other models across various datasets.The diversity aspect was evaluated by examining how well SparseGAN-generated data preserved the diversity and patterns of the original data. The findings indicated that SparseGAN-generated data closely resembled the characteristics of real-world time series and was virtually indistinguishable from actual data.Furthermore, we conducted a sensitivity analysis to test the robustness of their findings. We investigated the effect of different attention mechanisms, with 1.5 entmax emerging as more beneficial for the model’s performance compared to softmax.In conclusion, SparseGAN proved to be a promising generative model for time-series data. It effectively addressed long-term dependencies, maintained data distribution characteristics, and outperformed existing models in terms of data quality and diversity. SparseGAN’s ability to improve time-series forecasting accuracy, particularly in low-data scenarios, underscores its potential for various practical applications.

We propose GQFormer, a probabilistic time series forecasting method that models the quantile function of the forecast distribution. Our methodology is rooted in the Implicit Quantile modeling approach, where samples from the Uniform distribution $\mathcal{U}\left( {0,1} \right)$ are reparameterized to quantile values of the target distribution. This allows implicit generative quantile modeling without any prior assumptions on the data distribution like Gaussianity, common in prior works. Our work is distinguished from prior quantile forecasting methods by novel methodological advances that relate to directly modeling the correlations among multiple quantile estimations at each forecasting horizon. To this end, we firstly develop a parameters haring architecture that implicitly models multiple quantile estimations efficiently and secondly regularize these through a novel multi-task loss function formulation that optimizes for quantile estimations to be sharper estimations individually and on the whole be spread maximally apart to capture the various modes of the underlying distribution. We experimentally validate the superiority of the method to state-of-the-art probabilistic forecasting baselines and ablations to the loss formulation.

Enterprise Resource Planning (ERP) systems are used to support and to control the business processes of a company or organization. Such systems integrate the data across the entire company into a complete system that is capable of enhancing the key operations in virtually any department. Commonly, running an ERP system helps companies operate more efficiently, however, this also leads to problems. Employees are able to enrich themselves through insider knowledge or by exploiting incomplete or incorrect permission settings. This is referred to as occupational fraud. Since ERP systems keep records of all executed business activities and log system events and permission checks, they provide a variety of different data sources that can be used to detect occupational fraud. This paper reviews existing literature in the area of fraud detection and fraud cases based on ERP data including business data and system data. Using the most widespread ERP system from SAP, the potential and the suitability of the various data sources with respect to fraud detection is examined.

Domain Adaptation methodologies have shown to effectively generalize from a labeled source domain to a label scarce target domain. Previous research has either focused on unlabeled domain adaptation without any target supervision or semi-supervised domain adaptation with few labeled target examples per class. On the other hand Positive-Unlabeled (PU-) Learning has attracted increasing interest in the weakly supervised learning literature since in quite some real world applications positive labels are much easier to obtain than negative ones. In this work we are the first to introduce the challenge of Positive-Unlabeled Domain Adaptation where we aim to generalise from a fully labeled source domain to a target domain where only positive and unlabeled data is available. We present a novel two-step learning approach to this problem by firstly identifying reliable positive and negative pseudo-labels in the target domain guided by source domain labels and a positive-unlabeled risk estimator. This enables us to use a standard classifier on the target domain in a second step. We validate our approach by running experiments on benchmark datasets for visual object recognition. Furthermore we propose real world examples for our setting and validate our superior performance on parking occupancy data.

Asynchronous Time Series is a multivariate time series where all the channels are observed asynchronously-independently, making the time series extremely sparse when aligning them. We often observe this effect in applications with complex observation processes, such as health care, climate science, and astronomy, to name a few. Because of the asynchronous nature, they pose a significant challenge to deep learning architectures, which presume that the time series presented to them are regularly sampled, fully observed, and aligned with respect to time. This paper proposes a novel framework, that we call Deep Convolutional Set Functions (DCSF), which is highly scalable and memory efficient, for the asynchronous time series classification task. With the recent advancements in deep set learning architectures, we introduce a model that is invariant to the order in which time series’ channels are presented to it. We explore convolutional neural networks, which are well researched for the closely related problem-classification of regularly sampled and fully observed time series, for encoding the set elements. We evaluate DCSF for AsTS classification, and online (per time point) AsTS classification. Our extensive experiments on multiple real world and synthetic datasets verify that the suggested model performs substantially better than a range of state-of-the-art models in terms of accuracy and run time. We increase the accuracy of the mini-Physionet dataset upto 2%; real datasets with synthetic setups of both AsTS, and TSMV upto 30%.

In sparse recommender settings, users’ context and item attributes play a crucial role in deciding which items to recommend next. Despite that, recent works in sequential and time-aware recommendations usually either ignore both aspects or only consider one of them, limiting their predictive performance. In this paper, we address these limitations by proposing a context and attribute-aware recommender model (CARCA) that can capture the dynamic nature of the user profiles in terms of contextual features and item attributes via dedicated multi-head self-attention blocks that extract profile-level features and predict item scores. Also, unlike many of the current state-of-the-art sequential item recommendation approaches that use a simple dot-product between the most recent item’s latent features and the target items embeddings for scoring, CARCA uses cross-attention between all profile items and the target items to predict their final scores. This cross-attention allows CARCA to harness the correlation between old and recent items in the user profile and their influence on deciding which item to recommend next. Experiments on four real-world recommender system datasets show that the proposed model significantly outperforms all state-of-the-art models in the task of item recommendation and achieving improvements of up to 53% in Normalized Discounted Cumulative Gain (NDCG) and Hit-Ratio. Results also show that CARCA outperformed several state-of-the-art dedicated image-based recommender systems by merely utilizing image attributes extracted from a pre-trained ResNet50 in a black-box fashion.

Normalized discounted cumulative gain (NDCG) is one of the popular evaluation metrics for recommender systems and learning-to-rank problems. As it is non-differentiable, it cannot be optimized by gradient-based optimization procedures. In the last twenty years, a plethora of surrogate losses have been engineered that aim to make learning recommendation and ranking models that optimize NDCG possible. However, binary relevance implicit feedback settings still pose a significant challenge for such surrogate losses as they are usually designed and evaluated only for multi-level relevance feedback. In this paper, we address the limitations of directly optimizing the NDCG measure by proposing a guided learning approach (GuidedRec) that adopts recent advances in parameterized surrogate losses for NDCG. Starting from the observation that jointly learning a surrogate loss for NDCG and the recommendation model is very unstable, we design a stepwise approach that can be seamlessly applied to any recommender system model that uses a point-wise logistic loss function. The proposed approach guides the models towards optimizing the NDCG using an independent surrogate-loss model trained to approximate the true NDCG measure while maintaining the original logistic loss function as a stabilizer for the guiding procedure. In experiments on three recommendation datasets, we show that our guided surrogate learning approach yields models better optimized for NDCG than recent state-of-the-art approaches using engineered surrogate losses.

Abstract Meta-learning, or learning to learn, is a machine learning approach that utilizes prior learning experiences to expedite the learning process on unseen tasks. As a data-driven approach, meta-learning requires meta-features that represent the primary learning tasks or datasets, and are estimated traditonally as engineered dataset statistics that require expert domain knowledge tailored for every meta-task. In this paper, first, we propose a meta-feature extractor called Dataset2Vec that combines the versatility of engineered dataset meta-features with the expressivity of meta-features learned by deep neural networks. Primary learning tasks or datasets are represented as hierarchical sets, i.e., as a set of sets, esp. as a set of predictor/target pairs, and then a DeepSet architecture is employed to regress meta-features on them. Second, we propose a novel auxiliary meta-learning task with abundant data called dataset similarity learning that aims to predict if two batches stem from the same dataset or different ones. In an experiment on a large-scale hyperparameter optimization task for 120 UCI datasets with varying schemas as a meta-learning task, we show that the meta-features of Dataset2Vec outperform the expert engineered meta-features and thus demonstrate the usefulness of learned meta-features for datasets with varying schemas for the first time.

In auction-based systems such as in used car centers and online auction websites, users usually bid on items, and those items get sold to their highest bidders. In these settings, every item is unique and can be sold only once, which means users’ purchase histories will be unique, and no common items will exist across them. On the other hand, items will not have any historical sales at all. Such extreme settings pose a significant challenge to the current recommender systems models that rely on historical user-item interactions. While some of those models will not be applicable altogether, such as the matrix factorization models, neighborhood models, and even the naive most-popular model, the rest will need to rely only on items’ attributes. In this paper, we address the challenges of auction-based item recommendation by proposing a simple multi-relational recommender model (MultiRec) that can seamlessly leverage user and item attributes along with auxiliary relational information such as the user’s bidding history. Experiments on one proprietary dataset from Volkswagen Financial Services used-cars center, and on a real-world publicly available eBay dataset show that the proposed model significantly outperforms multiple state-of-art models in the task of auction-based unique item recommendation.

The performance of gradient-based optimization strategies depends heavily on the initial weights of the parametric model. Recent works show that there exist weight initializations from which optimization procedures can find the task-specific parameters faster than from uniformly random initializations and that such a weight initialization can be learned by optimizing a specific model architecture across similar tasks via MAML (Model-Agnostic Meta-Learning). Current methods are limited to populations of classification tasks that share the same number of classes due to the static model architectures used during meta-learning. In this paper, we present HIDRA, a meta-learning approach that enables training and evaluating across tasks with any number of target variables. We show that Model-Agnostic Meta-Learning trains a distribution for all the neurons in the output layer and a specific weight initialization for the ones in the hidden layers. HIDRA explores this by learning one master neuron, which is used to initialize any number of output neurons for a new task. Extensive experiments on the Miniimagenet and Omniglot data sets demonstrate that HIDRA improves over standard approaches while generalizing to tasks with any number of target variables. Moreover, our approach is shown to robustify low-capacity models in learning across complex tasks with a high number of classes for which regular MAML fails to learn any feasible initialization.

Multi-task learning is an established approach for improving the generalization of a model. We explore multi-task learning in the context of time series forecasting. Specifically, we look into a multivariate setting where main and auxiliary series are to be forecasted for multi-step ahead. This results in an interesting multi-task learning problem formulation where the learning tasks come from future horizon of main and auxiliary series both. Our proposed method relies firstly on enumerating multiple Convolutional network architectures to balance the number of shared and non-shared layers between different time series tasks. Also, as multi-step strategies minimize forecast errors over the complete horizon, loss functions would be at different scales based on model uncertainty for near versus distant future. For this reason we propose a factorization of the weight vector for the learning tasks with respect to their categorization of belonging to main or auxiliary series and index in future. An optimal number of shared and non-shared layers together with a novel weighted loss, results in superior performance over 2 real-world datasets compared with several baselines.

The task of classifying multi-relational data spans a wide range of domains such as document classification in citation networks, classification of emails, and protein labeling in proteins interaction graphs. Current state-of-the-art classification models rely on learning per-entity latent representations by mining the whole structure of the relations' graph, however, they still face two major problems. Firstly, it is very challenging to generate expressive latent representations in sparse multi-relational settings with implicit feedback relations as there is very little information per-entity. Secondly, for entities with structured properties such as titles and abstracts (text) in documents, models have to be modified ad-hoc. In this paper, we aim to overcome these two main drawbacks by proposing a flexible nonlinear latent embedding model (BRNLE) for the classification of multi-relational data. The proposed model can be applied to entities with structured properties such as text by utilizing the numerical vector representations of those properties. To address the sparsity problem of implicit feedback relations, the model is optimized via a sparsely-regularized multi-relational pair-wise Bayesian personalized ranking loss (BPR). Experiments on four different real-world datasets show that the proposed model significantly outperforms state-of-the-art models for multi-relational classification.

In very sparse recommender data sets, attributes of users such as age, gender and home location and attributes of items such as, in the case of movies, genre, release year, and director can improve the recommendation accuracy, especially for users and items that have few ratings. While most recommendation models can be extended to take attributes of users and items into account, their architectures usually become more complicated. While attributes for items are often easy to be provided, attributes for users are often scarce for reasons of privacy or simply because they are not relevant to the operational process at hand. In this paper, we address these two problems for attribute-aware recommender systems by proposing a simple model that co-embeds users and items into a joint latent space in a similar way as a vanilla matrix factorization, but with non-linear latent features construction that seamlessly can ingest user or item attributes or both (GraphRec). To address the second problem, scarce attributes, the proposed model treats the user-item relation as a bipartite graph and constructs generic user and item attributes via the Laplacian of the user-item co-occurrence graph that requires no further external side information but the mere rating matrix. In experiments on three recommender datasets, we show that GraphRec significantly outperforms existing state-of-the-art attribute-aware and content-aware recommender systems even without using any side information.

Learning with pairwise ranking methods for implicit feedback datasets has shown promising results as compared to pointwise ranking methods for recommendation tasks. However, there is limited effort in scaling the pairwise ranking methods in a large scale distributed setting. In this paper we address the scalability aspect of a pairwise ranking method using Factorization Machines in distributed settings. Our proposed method is based on a block partitioning of the model parameters so that each distributed worker runs stochastic gradient updates on an independent block. We developed a dynamic block creation and exchange strategy by utilizing the frequency of occurrence of a feature in the local training data of a worker. Empirical evidence on publicly available benchmark datasets indicates that the proposed method scales better than the static block based methods and outperforms competing state-of-the-art methods.

Topic modeling is a widely used technique in knowledge discovery and data mining. However, finding the right number of topics in a given text source has remained a challenging issue. In this paper, we study the concept of conceptual stability via nonnegative matrix factorization. Based on this finding, we propose a method to identify the correct number of topics and offer empirical evidence in its favor in terms of classification accuracy and the number of topics that are naturally present in the text sources. Experiments on real-world text corpora demonstrate that the proposed method has outperformed state-of-the-art latent Dirichlet allocation and nonnegative matrix factorization models.

Automating machine learning by providing techniques that autonomously find the best algorithm, hyperparameter configuration and preprocessing is helpful for both researchers and practitioners. Therefore, it is not surprising that automated machine learning has become a very interesting field of research. While current research is mainly focusing on finding good pairs of algorithms and hyperparameter configurations, we will present an approach that automates the process of creating a top performing ensemble of several layers, different algorithms and hyperparameter configurations. These kinds of ensembles are called jokingly Frankenstein ensembles and proved their benefit on versatile data sets in many machine learning challenges. We compare our approach Automatic Frankensteining with the current state of the art for automated machine learning on 80 different data sets and can show that it outperforms them on the majority using the same training time. Furthermore, we compare Automatic Frankensteining on a large scale data set to more than 3,500 machine learning expert teams and are able to outperform more than 3,000 of them within 12 CPU hours.

Previous research on content-based geolocation in general has developed prediction methods via conducting pre-partitioning and applying classification methods. The input of these methods is the concatenation of individual tweets during a period of time. But unfortunately, these methods have some drawbacks. They discard the natural real-values properties of latitude and longitude as well as fail to capture geolocation in near real-time. In this work, we develop a novel generative content-based regression model via a matrix factorization technique to tackle the near real-time geolocation prediction problem. With this model, we aim to address a couple of un-answered questions. First, we prove that near real-time geolocation prediction can be accomplished if we leave out the concatenation. Second, we account the real-values properties of physical coordinates within a regression solution. We apply our model on Twitter datasets as an example to prove the effectiveness and generality. Our experimental results show that the proposed model, in the best scenario, outperforms a set of state-of-the-art regression models including Support Vector Machines and Factorization Machines by a reduction of the median localization error up to 79%.

Algorithm selection and hyperparameter tuning are omnipresent problems for researchers and practitioners. Hence, it is not surprising that the efforts in automatizing this process using various meta-learning approaches have been increased. Sequential model-based optimization (SMBO) is ne of the most popular frameworks for finding optimal hyperparameter configurations. Originally designed for black-box optimization, researchers have contributed different meta-learning approaches to speed up the optimization process. We create a generalized framework of SMBO and its recent additions which gives access to adaptive hyperparameter transfer learning with simple surrogates (AHT), a new class of hyperparameter optimization strategies. AHT provides less time-overhead for the optimization process by replacing time-and space-consuming transfer surrogate models with simple surrogates that employ adaptive transfer learning. In an empirical comparison on two different meta-data sets, we can show that AHT outperforms various instances of the SMBO framework in the scenarios of hyperparameter tuning and algorithm selection.

Motifs are the most repetitive/frequent patterns of a time-series. The discovery of motifs is crucial for practitioners in order to understand and interpret the phenomena occurring in sequential data. Currently, motifs are searched among series sub-sequences, aiming at selecting the most frequently occurring ones. Search-based methods, which try out series sub-sequence as motif candidates, are currently believed to be the best methods in finding the most frequent patterns. However, this paper proposes an entirely new perspective in finding motifs. We demonstrate that searching is non-optimal since the domain of motifs is restricted, and instead we propose a principled optimization approach able to find optimal motifs. We treat the occurrence frequency as a function and time-series motifs as its parameters, therefore we learn the optimal motifs that maximize the frequency function. In contrast to searching, our method is able to discover the most repetitive patterns (hence optimal), even in cases where they do not explicitly occur as sub-sequences. Experiments on several real-life time-series datasets show that the motifs found by our method are highly more frequent than the ones found through searching, for exactly the same distance threshold.

Hyperparameter optimization is often done manually or by using a grid search. However, recent research has shown that automatic optimization techniques are able to accelerate this optimization process and find hyperparameter configurations that lead to better models. Currently, transferring knowledge from previous experiments to a new experiment is of particular interest because it has been shown that it allows to further improve the hyperparameter optimization. We propose to transfer knowledge by means of an initialization strategy for hyperparameter optimization. In contrast to the current state of the art initialization strategies, our strategy is neither limited to hyperparameter configurations that have been evaluated on previous experiments nor does it need meta-features. The initial hyperparameter configurations are derived by optimizing for a meta-loss formally defined in this paper. This loss depends on the hyperparameter response function of the data sets that were investigated in past experiments. Since this function is unknown and only few observations are given, the meta-loss is not differentiable. We propose to approximate the response function by a differentiable plug-in estimator. Then, we are able to learn the initial hyperparameter configuration sequence by applying gradient-based optimization techniques. Extensive experiments are conducted on two meta-data sets. Our initialization strategy is compared to the state of the art for initialization strategies and further methods that are able to transfer knowledge between data sets. We give empirical evidence that our work provides an improvement over the state of the art.

Hyperparameter tuning is often done manually but current research has proven that automatic tuning yields effective hyperparameter configurations even faster and does not require any expertise. To further improve the search, recent publications propose transferring knowledge from previous experiments to new experiments. We adapt the sequential model-based optimization by replacing its surrogate model and acquisition function with one policy that is optimized for the task of hyperparameter tuning. This policy generalizes over previous experiments but neither uses a model nor uses meta-features, nevertheless, outperforms the state of the art. We show that a static ranking of hyperparameter combinations yields competitive results and substantially outperforms a random hyperparameter search. Thus, it is a fast and easy alternative to complex hyperparameter tuning strategies and allows practitioners to tune their hyperparameters by simply using a look-up table. We made look-up tables for two classifiers publicly available: SVM and AdaBoost. Furthermore, we propose a similarity measure for data sets that yields more comprehensible results than those using meta-features. We show how this similarity measure can be applied to surrogate models in the SMBO framework and empirically show that this change leads to better hyperparameter configurations in less trials.

Time-series classification has attracted considerable research attention due to the various domains where time-series data are observed, ranging from medicine to econometrics. Traditionally, the focus of time-series classification has been on short time-series data composed of a few patterns exhibiting variabilities, while recently there have been attempts to focus on longer series composed of multiple local patrepeating with an arbitrary irregularity. The primary contribution of this paper relies on presenting a method which can detect local patterns in repetitive time-series via fitting local polynomial functions of a specified degree. We capture the repetitiveness degrees of time-series datasets via a new measure. Furthermore, our method approximates local polynomials in linear time and ensures an overall linear running time complexity. The coefficients of the polynomial functions are converted to symbolic words via equi-area discretizations of the coefficients' distributions. The symbolic polynomial words enable the detection of similar local patterns by assigning the same word to similar polynomials. Moreover, a histogram of the frequencies of the words is constructed from each time-series' bag of words. Each row of the histogram enables a new representation for the series and symbolizes the occurrence of local patterns and their frequencies. In an experimental comparison against state-of-the-art baselines on repetitive datasets, our method demonstrates significant improvements in terms of prediction accuracy.

Multi-matrix factorization models provide a scalable and effective approach for multi-relational learning tasks such as link prediction, Linked Open Data (LOD) mining, recommender systems and social network analysis. Such models are learned by optimizing the sum of the losses on all relations in the data. Early models address the problem where there is only one target relation for which predictions should be made. More recent models address the multi-target variant of the problem and use the same set of parameters to make predictions for all target relations. In this paper, we argue that a model optimized for each target relation individually has better predictive performance than models optimized for a compromise on the performance on all target relations. We introduce specific parameters for each target but, instead of learning them independently from each other, we couple them through a set of shared auxiliary parameters, which has a regularizing effect on the target specific ones. Experiments on large Web datasets derived from DBpedia, Wikipedia and BlogCatalog show the performance improvement obtained by using target specific parameters and that our approach outperforms competitive state-of-the-art methods while being able to scale gracefully to big data.

Shapelets are discriminative sub-sequences of time series that best predict the target variable. For this reason, shapelet discovery has recently attracted considerable interest within the time-series research community. Currently shapelets are found by evaluating the prediction qualities of numerous candidates extracted from the series segments. In contrast to the state-of-the-art, this paper proposes a novel perspective in terms of learning shapelets. A new mathematical formalization of the task via a classification objective function is proposed and a tailored stochastic gradient learning algorithm is applied. The proposed method enables learning near-to-optimal shapelets directly without the need to try out lots of candidates. Furthermore, our method can learn true top-K shapelets by capturing their interaction. Extensive experimentation demonstrates statistically significant improvement in terms of wins and ranks against 13 baselines over 28 time-series datasets.

Users engaged in the Social Web increasingly rely upon continuous streams of Twitter messages (tweets) for real-time access to information and fresh knowledge about current affairs. However, given the deluge of tweets, it is a challenge for individuals to find relevant and appropriately ranked information. We propose to address this knowledge management problem by going beyond the general perspective of information finding in Twitter, that asks: "What is happening right now?", towards an individual user perspective, and ask: "What is interesting to me right now?" In this paper, we consider collaborative filtering as an online ranking problem and present RMFO, a method that creates, in real-time, user-specific rankings for a set of tweets based on individual preferences that are inferred from the user's past system interactions. Experiments on the 476 million Twitter tweets dataset show that our online approach largely outperforms recommendations based on Twitter's global trend and Weighted Regularized Matrix Factorization (WRMF), a highly competitive state-of-the-art Collaborative Filtering technique, demonstrating the efficacy of our approach.

The Social Web is successfully established, and steadily growing in terms of users, content and services. People generate and consume data in real-time within social networking services, such as Twitter, and increasingly rely upon continuous streams of messages for real-time access to fresh knowledge about current affairs. In this paper, we focus on analyzing social streams in real-time for personalized topic recommendation and discovery. We consider collaborative filtering as an online ranking problem and present Stream Ranking Matrix Factorization - RMFX -, which uses a pairwise approach to matrix factorization in order to optimize the personalized ranking of topics. Our novel approach follows a selective sampling strategy to perform online model updates based on active learning principles, that closely simulates the task of identifying relevant items from a pool of mostly uninteresting ones. RMFX is particularly suitable for large scale applications and experiments on the "476 million Twitter tweets" dataset show that our online approach largely outperforms recommendations based on Twitter's global trend, and it is also able to deliver highly competitive Top-N recommendations faster while using less space than Weighted Regularized Matrix Factorization (WRMF), a state-of-the-art matrix factorization technique for Collaborative Filtering, demonstrating the efficacy of our approach.

Recommender systems help web users to address information overload. However their performance depends on the number of provided ratings by users. This problem is amplified for a new user because he/she has not provided any rating. To address this problem, active learning methods have been proposed to acquire those ratings from users, that will help most in determining their interests. However, different from the classic active learning, users (the "oracle") are not always able to provide an answer for queries. The easiest way to solve this problem is to ask most popular items, i.e items which have received many ratings from training users. But it is static and presents the same items to all users regardless of the ratings they have provided so far. In this paper we propose a method that improves the most popular selection strategy using the characteristics of matrix factorization. It finds similar users to the new user in the latent space and then selects item which is most popular among the similar users. The experimental results show the proposed method outperforms the most popular method both in terms of error and the number of received ratings.

A key element of the social networks on the internet such as Facebook and Flickr is that they encourage users to create connections between themselves, other users and objects.

Due to its various applications, time-series classification is a prominent research topic in data mining and computational intelligence. The simple k-NN classifier using dynamic time warping (DTW) distance had been shown to be competitive to other state-of-the art time-series classifiers. In our research, however, we observed that a single fixed choice for the number of nearest neighbors k may lead to suboptimal performance. This is due to the complexity of time-series data, especially because the characteristic of the data may vary from region to region. Therefore, local adaptations of the classification algorithm is required. In order to address this problem in a principled way by, in this paper we introduce individual quality (IQ) estimation. This refers to estimating the expected classification accuracy for each time series and each k individually. Based on the IQ estimations we combine the classification results of several k-NN classifiers as final prediction. In our framework of IQ, we develop two time-series classification algorithms, IQ-MAX and IQ-WV. In our experiments on 35 commonly used benchmark data sets, we show that both IQ-MAX and IQ-WV outperform two baselines.

Recommender systems help Web users to address information overload. Their performance, however, depends on the amount of information that users provide about their preferences. Users are not willing to provide information for a large amount of items, thus the quality of recommendations is affected specially for new users. Active learning has been proposed in the past, to acquire preference information from users. Based on an underlying prediction model, these approaches determine the most informative item for querying the new user to provide a rating. In this paper, we propose a new active learning method which is developed specially based on aspect model features. There is a difference between classic active learning and active learning for recommender system. In the recommender system context, each item has already been rated by training users while in classic active learning there is not training user. We take into account this difference and develop a new method which competes with a complicated bayesian approach in accuracy while results in drastically reduced (one order of magnitude) user waiting times, i.e., the time that the users wait before being asked a new query.

MyMediaLite is a fast and scalable, multi-purpose library of recommender system algorithms, aimed both at recommender system researchers and practitioners. It addresses two common scenarios in collaborative filtering: rating prediction (e.g. on a scale of 1 to 5 stars) and item prediction from positive-only implicit feedback (e.g. from clicks or purchase actions). The library offers state-of-the-art algorithms for those two tasks. Programs that expose most of the library's functionality, plus a GUI demo, are included in the package. Efficient data structures and a common API are used by the implemented algorithms, and may be used to implement further algorithms. The API also contains methods for real-time updates and loading/storing of already trained recommender models.

The situation in which a choice is made is an important information for recommender systems. Context-aware recommenders take this information into account to make predictions. So far, the best performing method for context-aware rating prediction in terms of predictive accuracy is Multiverse Recommendation based on the Tucker tensor factorization model. However this method has two drawbacks: (1) its model complexity is exponential in the number of context variables and polynomial in the size of the factorization and (2) it only works for categorical context variables. On the other hand there is a large variety of fast but specialized recommender methods which lack the generality of context-aware methods.

The integration of OLAP with web-search technologies is a promising research topic. Recommender systems are popular web-search mechanisms, because they can address information overload and provide personalization of results. Nevertheless, the evaluation of recommender systems is a challenging task. In this paper, we propose a novel framework for evaluating recommender systems, which is multidimensional and takes into account for the multiple facets of the recommendation algorithms, data sets and performance measures. Emphasis is placed on supporting business applications of recommender systems, notably e-commerce, by allowing analysts to perform ad-hoc analysis and use popular online analytical processing (OLAP) operations. Combined with support for visual analysis, action such as drill-down or slice/dice allow assessment of the performance of recommendations in terms of business objectives. We describe a detailed methodology for designing and developing the proposed multidimensional framework, and provide insights about its applications. Our experimental results, using a research prototype, demonstrate the ability of the proposed framework to comprise an effective way for evaluating recommender systems.

Cold-start scenarios in recommender systems are situations in which no prior events, like ratings or clicks, are known for certain users or items. To compute predictions in such cases, additional information about users (user attributes, e.g. gender, age, geographical location, occupation) and items (item attributes, e.g. genres, product categories, keywords) must be used. We describe a method that maps such entity (e.g. user or item) attributes to the latent features of a matrix (or higher-dimensional) factorization model. With such mappings, the factors of a MF model trained by standard techniques can be applied to the new-user and the new-item problem, while retaining its advantages, in particular speed and predictive accuracy. We use the mapping concept to construct an attribute-aware matrix factorization model for item recommendation from implicit, positive-only feedback. Experiments on the new-item problem show that this approach provides good predictive accuracy, while the prediction time only grows by a constant factor.

No abstract available.

Tagging plays an important role in many recent websites. Recommender systems can help to suggest a user the tags he might want to use for tagging a specific item. Factorization models based on the Tucker Decomposition (TD) model have been shown to provide high quality tag recommendations outperforming other approaches like PageRank, FolkRank, collaborative filtering, etc. The problem with TD models is the cubic core tensor resulting in a cubic runtime in the factorization dimension for prediction and learning.

Recommender systems are an important component of many websites. Two of the most popular approaches are based on matrix factorization (MF) and Markov chains (MC). MF methods learn the general taste of a user by factorizing the matrix over observed user-item preferences. On the other hand, MC methods model sequential behavior by learning a transition graph over items that is used to predict the next action based on the recent actions of a user. In this paper, we present a method bringing both approaches together. Our method is based on personalized transition graphs over underlying Markov chains. That means for each user an own transition matrix is learned - thus in total the method uses a transition cube. As the observations for estimating the transitions are usually very limited, our method factorizes the transition cube with a pairwise interaction model which is a special case of the Tucker Decomposition. We show that our factorized personalized MC (FPMC) model subsumes both a common Markov chain and the normal matrix factorization model. For learning the model parameters, we introduce an adaption of the Bayesian Personalized Ranking (BPR) framework for sequential basket data. Empirically, we show that our FPMC model outperforms both the common matrix factorization and the unpersonalized MC model both learned with and without factorization.

Tag recommendation is the task of predicting a personalized list of tags for a user given an item. This is important for many websites with tagging capabilities like last.fm or delicious. In this paper, we propose a method for tag recommendation based on tensor factorization (TF). In contrast to other TF methods like higher order singular value decomposition (HOSVD), our method RTF ('ranking with tensor factorization') directly optimizes the factorization model for the best personalized ranking. RTF handles missing values and learns from pairwise ranking constraints. Our optimization criterion for TF is motivated by a detailed analysis of the problem and of interpretation schemes for the observed data in tagging systems. In all, RTF directly optimizes for the actual problem using a correct interpretation of the data. We provide a gradient descent algorithm to solve our optimization problem. We also provide an improved learning and prediction method with runtime complexity analysis for RTF. The prediction runtime of RTF is independent of the number of observations and only depends on the factorization dimensions. Besides the theoretical analysis, we empirically show that our method outperforms other state-of-the-art tag recommendation methods like FolkRank, PageRank and HOSVD both in quality and prediction runtime.

Selecting promising queries is the key to effective active learning. In this paper, we investigate selection techniques for the task of learning an equivalence relation where the queries are about pairs of objects. As the target relation satisfies the axioms of transitivity, from one queried pair additional constraints can be inferred. We derive both the upper and lower bound on the number of queries needed to converge to the optimal solution. Besides restricting the set of possible solutions, constraints can be used as training data for learning a similarity measure. For selecting queries that result in a large number of meaningful constraints, we present an approximative optimal selection technique that greedily minimizes the expected loss in each round of active learning. This technique makes use of inference of expected constraints. Besides the theoretical results, an extensive evaluation for the application of record linkage shows empirically that the proposed selection method leads to both interesting and a high number of constraints.

Regularized matrix factorization models are known to generate high quality rating predictions for recommender systems. One of the major drawbacks of matrix factorization is that once computed, the model is static. For real-world applications dynamic updating a model is one of the most important tasks. Especially when ratings on new users or new items come in, updating the feature matrices is crucial.

Relational classification aims at including relations among entities, for example taking relations between documents such as a common author or citations into account. However, considering more than one relation can further improve classification accuracy. In this paper we introduce a new approach to make use of several relations as well as both relations and attributes for classification using ensemble methods. To accomplish this, we present a generic relational ensemble model, that can use different relational and local classifiers as components. Furthermore, we discuss solutions for several problems concerning relational data such as heterogeneity, sparsity, and multiple relations. Finally, we provide empirical evidence, that our relational ensemble methods outperform existing relational classification methods, even rather complex models such as relational probability trees (RPTs), relational dependency networks (RDNs) and relational Bayesian classifiers (RBCs).

Object identification aims at identifying different representations of the same object based on noisy attributes such as descriptions of the same product in different online shops or references to the same paper in different publications. Numerous solutions have been proposed for solving this task, almost all of them based on similarity functions of a pair of objects. Although today the similarity functions are learned from a set of labeled training data, the structural information given by the labeled data is not used. By formulating a generic model for object identification we show how almost any proposed identification model can easily be extended for satisfying structural constraints. Therefore we propose a model that uses structural information given as pairwise constraints to guide collective decisions about object identification in addition to a learned similarity measure. We show with empirical experiments on public and on real-life data that combining both structural information and attribute-based similarity enormously increases the overall performance for object identification tasks.

Recommender systems recommend products to customers based on ratings or past customer behavior. Without any information about attributes of the products or customers involved, the problem has been tackled most successfully by a nearest neighbor method called collaborative filtering in the context, while additional efforts invested in building classification models did not pay off and did not increase the quality. Therefore, classification methods have mainly been used in conjunction with product or customer attributes. Starting from a view on the plain recommendation task without attributes as a multi-class classification problem, we investigate two particularities, its autocorrelation structure as well as the absence of re-occurring items (repeat buying). We adapt the standard generic reductions 1-vs-rest and 1-vs-l of multi-class problems to a set of binary classification problems to these particularities and thereby provide a generic compound classifier for recommender systems. We evaluate a particular specialization thereof using linear support vector machines as member classifiers on MovieLens data and show that it outperforms state-of-the-art methods, i.e., item-based collaborative filtering.

Recommender systems have been subject to an enormous rise in popularity and research interest over the last ten years. At the same time, very large taxonomies for product classification are becoming increasingly prominent among e-commerce systems for diverse domains, rendering detailed machine-readable content descriptions feasible. Amazon.com makes use of an entire plethora of hand-crafted taxonomies classifying books, movies, apparel, and various other goods. We exploit such taxonomic background knowledge for the computation of personalized recommendations. Hereby, relationships between super-concepts and sub-concepts constitute an important cornerstone of our novel approach, providing powerful inference opportunities for profile generation based upon the classification of products that customers have chosen. Ample empirical analysis, both offline and online, demonstrates our proposal's superiority over common existing approaches when user information is sparse and implicit ratings prevail.

While association rules for set data use and describe relations between parts of set valued objects completely, association rules for sequential data are restricted by specific interpretations of the subsequence relation: contiguous subsequences describe local features of a sequence valued object, noncontiguous subsequences its global features. We model both types of features with generalized subsequences that describe local deviations by wild cards, and present a new algorithm of a priori type for mining all generalized subsequences with prescribed minimum support from a given database of sequences. Furthermore we show that the given algorithm automatically takes into account an eventually underlying graph structure, i.e., is applicable to path data also.