The increasing scale and diversity of available content pose significant challenges for recommendation systems, including data sparsity, cold-start scenarios, and lack of personalization. We present a modular recommendation framework that addresses these issues by combining category-level sequence modeling with retrieval-augmented generation guided by structured prompts. By predicting semantic category trajectories rather than individual items, the system captures long-term user interests and improves generalization to unseen content. Retrieved documents and user context are dynamically injected into generation via prompt engineering strategies, enabling high-quality recommendations without model finetuning. Experiments demonstrate that incorporating category signals leads to substantial performance improvements, particularly in cold-start settings.

In the age of social media, customers have become opinion makers that share their experience. People interested in a product can reach for these reviews on the whole Internet, thus leading to heterogeneity of data sources. As the massive amounts of data are not systematically accessible through APIs, companies frequently scrap it from several platforms and marketplaces. Data acquisition processes include some amount of ETL, yet the variability of noise contained in the data and the heterogeneity induced by different existing sources create the need for ad-hoc tools. In other words, and even if large quantities of data are accessible for virtually free, customer insight extraction is an harduous task. To circumvent this issue, we apply the Embedded Dirichlet Process (EDP) and the Embedded Hierarchical Dirichlet Process (EHDP) in a realistic setting regarding industrial practices. These neural topic models simultaneously learn the number of topics, representations of documents, topic embeddings and word embeddings from data. These properties allow for semantic units' disambiguation and for deeper text exploration than other approaches. As such, they can also serve as a way to refine industrial ETL processes. When not higher, both the EDP and the EHDP models achieve similar likelihood than other methods in our experimental settings, while providing with more analytical levels. Last but not least, we could achieve our results without having to perform costly reruns to find the number of topics.

Abstract The ensemble method is a powerful data mining paradigm, which builds a classification model by integrating multiple diversified component learners. Bagging is one of the most successful ensemble methods. It is made of bootstrap‐inspired classifiers and uses these classifiers to get an aggregated classifier. However, in bagging, bootstrapped training sets become more and more similar as redundancy is increasing. Besides redundancy, any training set is usually subject to noise. Moreover, the training set might be imbalanced. Thus, each training instance has a different impact on the learning process. This paper explores some properties of the ensemble margin and its use in improving the performance of bagging. We introduce a new approach to measure the importance of training data in learning, based on the margin theory. Then, a new bagging method concentrating on critical instances is proposed. This method is more accurate than bagging and more robust than boosting. Compared to bagging, it reduces the bias while generally keeping the same variance. Our findings suggest that (a) examples with low margins tend to be more critical for the classifier performance; (b) examples with higher margins tend to be more redundant; (c) misclassified examples with high margins tend to be noisy examples. Our experimental results on 15 various data sets show that the generalization error of bagging can be reduced up to 2.5% and its resilience to noise strengthened by iteratively removing both typical and noisy training instances, reducing the training set size by up to 75%.

Efficiently extracting information from documents issued by their partners is crucial for companies that face huge daily document flows. Particularly, tables contain most valuable information of business documents. However, their contents are challenging to automatically parse as tables from industrial contexts may have complex and ambiguous physical structure. Bypassing their structure recognition, we propose a generic method for end-to-end table field extraction that starts with the sequence of document tokens segmented by an OCR engine and directly tags each token with one of the possible field types. Similar to the state-of-the-art methods for non-tabular field extraction, our approach resorts to a token level recurrent neural network combining spatial and textual features. We empirically assess the effectiveness of recurrent connections for our task by comparing our method with a baseline feedforward network having local context knowledge added to its inputs. We train and evaluate both approaches on a dataset of 28,570 purchase orders to retrieve the ID numbers and quantities of the ordered products. Our method outperforms the baseline with micro F1 score on unknown document layouts of 0.821 compared to 0.764.

We propose a hierarchical network architecture for context-aware dialogue systems, that chooses which parts of the past conversation to focus on through a two-layer attention mechanism. The model can encode the parts of the historical dialog that are relevant to the current question to reason about the required response. We first assess the performance of our model on the Dialog bAbI task that involves a restaurant reservation system, where the goal is to book a table at a restaurant. We then train our model on a new hand-crafted dialogue data set, consisting of 7500 dialogues, to inform prospective students about the Data Science master program at University of Lyon.

The benefit of exploiting label dependence in multi-label classification is known to be closely dependent on the type of loss to be minimized. In this paper, we show that the subsets of labels that appear as irreducible factors in the factorization of the conditional distribution of the label set given the input features play a pivotal role for multi-label classification in the context of subset Zero-One loss minimization, as they divide the learning task into simpler independent multi-class problems. We establish theoretical results to characterize and identify these irreducible label factors for any given probability distribution satisfying the Composition property. The analysis lays the foundation for generic multi-label classification and optimal feature subset selection procedures under this subclass of distributions. Our conclusions are supported by carefully designed experiments on synthetic and benchmark data.

Multi-label classification has attracted an increasing amount of attention in recent years. To this end, many ensemble-based algorithms have been developed to classify multi-label data in an effective manner. There are several factors that differentiate between the various ensembles methods to output a label set prediction for unseen instances: the method of combining the predictions of the base classifiers and the thresholding strategy to implement a decision function. In this paper, we present an extensive empirical study comparing several multi-label ensemble methods over ten benchmark data sets. We also examine the influence of two types of voting schemas and the effect of calibrating the final decision function via single and Multi thresholding strategies on each performance metric. The experimental results were analyzed using statistical test to assess the statistical differences in the predictions performance.

We consider the problem of using a large amount of unlabeled data to improve the efficiency of feature selection in high dimensional datasets, when only a small set of labeled examples is available. We propose a new semi-supervised feature importance evaluation method (SSFI for short), that combines ideas from co-training and random forests with a new permutation-based out-of-bag feature importance measure. We provide empirical results on several benchmark datasets indicating that SSFI can lead to significant improvement over state-of-the-art semi-supervised and supervised algorithms.

In this paper, we propose another extension of the Random Forests paradigm to unlabeled data, leading to localized unsupervised feature selection (FS). We show that the way internal estimates are used to measure variable importance in Random Forests are also applicable to FS in unsupervised learning. We first illustrate the clustering performance of the proposed method on various data sets based on widely used external criteria of clustering quality. We then assess the accuracy and the scalability of the FS procedure on UCI and real labeled data sets and compare its effectiveness against other FS methods.

BACKGROUND: The aim of this study was to provide a framework for the analysis of visceral obesity and its determinants in women, where complex inter-relationships are observed among lifestyle, nutritional and metabolic predictors. Thirty-four predictors related to lifestyle, adiposity, body fat distribution, blood lipids and adipocyte sizes have been considered as potential correlates of visceral obesity in women. To properly address the difficulties in managing such interactions given our limited sample of 150 women, bootstrapped Bayesian networks were constructed based on novel constraint-based learning methods that appeared recently in the statistical learning community. Statistical significance of edge strengths was evaluated and the less reliable edges were pruned to increase the network robustness. To allow accessible interpretation and integrate biological knowledge into the final network, several undirected edges were afterwards directed with physiological expertise according to relevant literature. RESULTS: Extensive experiments on synthetic data sampled from a known Bayesian network show that the algorithm, called Recursive Hybrid Parents and Children (RHPC), outperforms state-of-the-art algorithms that appeared in the recent literature. Regarding biological plausibility, we found that the inference results obtained with the proposed method were in excellent agreement with biological knowledge. For example, these analyses indicated that visceral adipose tissue accumulation is strongly related to blood lipid alterations independent of overall obesity level. CONCLUSIONS: Bayesian Networks are a useful tool for investigating and summarizing evidence when complex relationships exist among predictors, in particular, as in the case of multifactorial conditions like visceral obesity, when there is a concurrent incidence for several variables, interacting in a complex manner. The source code and the data sets used for the empirical tests are available at http://www710.univ-lyon1.fr/~aaussem/Software.html.

This paper introduces a novel conservative feature subset selection method with incomplete data sets. The method is conservative in the sense that it selects the minimal subset of features that renders the rest of the features independent of the target (the class variable) without making any assumption about the missing data mechanism. This is achieved in the context of determining the Markov blanket of the target that reflects the worst-case assumption about the missing data mechanism, including the case when data is not missing at random. An application of the method on synthetic incomplete data is carried out to illustrate its practical relevance. The method is compared against state-of-the-art approaches such as the expectation maximization (EM) algorithm and the available case technique.

We study an original scheme based on distributed feedforward neural networks, aimed at modelling several queueing systems in cascade fed with bursty traffic. For each queueing system, a neural network is trained to anticipate the average number of waiting packets, the packet loss rate and the coefficient of variation of the packet inter-departure time, given the mean rate, the peak rate and the coefficient of variation of the packet inter-arrival time. The latter serves for the calculation of the coefficient of variation of the cell inter-arrival time of the aggregated traffic which is fed as input to the next neural network along the path. The potential of this method is successfully illustrated on several single server FIFO (First In, First Out) queues and on small queueing networks made up from a combination of queues in tandem and in parallel fed by a superposition of ideal sources. Our long-term goal is the design of preventive control strategy in a multiservice communication network.

This article extends previous analysis of the gradient decay to a class of discrete-time fully recurrent networks, called dynamical recurrent neural networks, obtained by modeling synapses as finite impulse response (FIR) filters instead of multiplicative scalars. Using elementary matrix manipulations, we provide an upper bound on the norm of the weight matrix, ensuring that the gradient vector, when propagated in a reverse manner in time through the error-propagation network, decays exponentially to zero. This bound applies to all recurrent FIR architecture proposals, as well as fixed-point recurrent networks, regardless of delay and connectivity. In addition, we show that the computational overhead of the learning algorithm can be reduced drastically by taking advantage of the exponential decay of the gradient.

In this paper we propose an experimental forecasting strategy taking into account the long-range dependence of aggregate network traffic, and we apply it to provide one-minute-ahead World-Wide Web (Web) traffic demand forecasts in terms of average number of bytes transferred. Recently, statistical examination of Web traces have shown evidence that Web traffic arising from file transfers exhibits a behavior that is consistent with the notion of self-similarity. Essentially, self-similarity indicates that significant burstiness is present on a wide range of time scales (i.e., the process is long-range dependent). Hence the idea of exploiting this multiscale property with a view towards discovering and capturing regularities underlying the time series which may prove useful for short-term traffic load forecasting. We carry out a wavelet transform decomposition of the original series to decompose the traffic time series into varying scales of temporal resolution, with the aim of making the underlying temporal structures more tractable. In a second step, each individual wavelet series—supposed to capture some features of the series—is fitted with a dynamical recurrent neural network (DRNN) model to output the wavelet forecast. The latter are afterwards recombined to form the next-minute Web Traffic demand. The method is applied on a large set of HTTP logs and is shown to yield good results. © 2001 John Wiley & Sons, Inc.

This paper extends previous analysis of the gradient decay to a class of discrete-time fully recurrent networks, called dynamical recurrent neural networks, obtained by modelling synapses as finite impulse response (FIR) filters instead of multiplicative scalars. Using elementary matrix manipulations, we provide an upper bound on the norm of the weight matrix ensuring that the gradient vector, when propagated in a reverse manner in time through the error-propagation network, decays exponentially to zero. This bounds apply to all FIR architecture proposals as well as fixed point recurrent networks, regardless of delay and connectivity. In addition, we show that the computational overhead of the learning algorithm can be reduced drastically by taking advantage of the exponential decay of the gradient.

We discuss a scheme based on distributed feedforward neural networks (NN), aimed at modelling several queueing systems in cascade fed with bursty traffic. For each queueing system, a neural network is trained to anticipate the average number of waiting packets, the packet loss rate and the coefficient of variation of the packet inter-departure time, given the mean rate, the peak rate and the coefficient of variation of the packet inter-arrival time. The latter serves for the calculation of the coefficient of variation of the cell inter-arrival time of the aggregated traffic which is fed as input to the next NN along the path. The potential of this method was sucessfully illustrated on several single server FIFO queues in Aussem et al. (1999). We now apply this technique to model a small queueing network made up from a combination of queues in tandem and in parallel fed by a superimposition of OnOff sources. Our long-term goal is the design of preventive control strategy in a multiservice communication network.

We discuss a simple strategy aimed at improving neural network prediction accuracy, based on the combination of predictions at varying resolution levels of the domain under investigation (here: time series). First, a wavelet transform is used to decompose the time series into varying scales of temporal resolution. The latter provides a sensible decomposition of the data so that the underlying temporal structures of the original time series become more tractable. Then, a dynamical recurrent neural netork is trained on each resolution scale with the temporal-recurrent backpropagation algorithm. By virtue of its internal dynamic, this general class of dynamic connections network approximates the underlying law governing each resolution level by a system of non-linear difference equations. The individual wavelet scale forecasts are afterwards recombined to form the current estimate. The predictive ability of this strategy is assessed with the sunspot series.

Dynamical Recurrent Neural Networks (DRNN) (Aussem 1995a) are a class of fully recurrent networks obtained by modeling synapses as autoregressive filters. By virtue of their internal dynamic, these networks approximate the underlying law governing the time series by a system of nonlinear difference equations of internal variables. They therefore provide history-sensitive forecasts without having to be explicitly fed with external memory. The model is trained by a local and recursive error propagation algorithm called temporal-recurrent-backpropagation. The efficiency of the procedure benefits from the exponential decay of the gradient terms backpropagated through the adjoint network. We assess the predictive ability of the DRNN model with meterological and astronomical time series recorded around the candidate observation sites for the future VLT telescope. The hope is that reliable environmental forecasts provided with the model will allow the modern telescopes to be preset, a few hours in advance, in the most suited instrumental mode. In this perspective, the model is first appraised on precipitation measurements with traditional nonlinear AR and ARMA techniques using feedforward networks. Then we tackle a complex problem, namely the prediction of astronomical seeing, known to be a very erratic time series. A fuzzy coding approach is used to reduce the complexity of the underlying laws governing the seeing. Then, a fuzzy correspondence analysis is carried out to explore the internal relationships in the data. Based on a carefully selected set of meteorological variables at the same time-point, a nonlinear multiple regression, termed nowcasting (Murtagh et al. 1993, 1995), is carried out on the fuzzily coded seeing records. The DRNN is shown to outperform the fuzzy k-nearest neighbors method.