Lorena Calvo-Bartolomé, Valérie Aldana, Karla Cantarero, Alonso Madroñal de Mesa, Jerónimo Arenas-García, Jordan Lee Boyd-Graber. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Topic modeling has been extensively used across disciplines to extract knowledge from text collections for policy design, implementation, and monitoring. However, flat topic models are limited in their ability to group overly broad topics with more specific ones. Hierarchical topic models ( htm ) address this limitation by offering thematic analysis at different granularity levels, but existing htm s rely on complex implementations, complicating hyperparameter tuning and domain expert-guided topic curation. This paper introduces two novel algorithms for hierarchical topic modeling: htm-ws ( htm with word selection) and htm-ds ( htm with document selection). These expert-in-the-loop methods enable domain experts to refine topic hierarchies by selecting subtopics for further partitioning. They are model-agnostic—compatible with both Bayesian and neural topic models—and require minimal hyperparameter tuning, comparable to flat topic models. Both algorithms are integrated within a tool for the training and exploitation of topic models, developed in the context of the IntelComp project. Experiments on three scientific paper datasets demonstrate the effectiveness of the algorithms through quantitative evaluations with automatic metrics and qualitative assessments via human judgment, including comparisons both between the two methods and against existing hierarchical topic models from the literature.

Analyzing document collections in Science, Technology, and Innovation (STI) is essential for informed policy-making. Advances in topic modeling offer powerful tools to uncover key themes within large, heterogeneous STI datasets that can inform decision-making. However, challenges such as aligning model outputs with expert knowledge, determining the desired topic granularity, and addressing model variability, limit their widespread adoption. This paper presents the Topic Analysis and Search Engine ( tase ), a platform developed within the European IntelComp project to address these limitations. tase combines Bayesian and Neural topic modeling techniques within a unified framework that incorporates an expert-in-the-loop approach for training and curating models. This framework is further enhanced with a Solr-based exploitation tool, featuring an innovative indexing method and a novel criterion for document retrieval. These features enable efficient semantic similarity calculations and seamless integration into decision-support systems. We demonstrate tase ’s scalability and effectiveness through two real-world STI use cases, highlighting its potential for broader applications. The software is freely available as open-source under the MIT license.

In this paper, we propose a scheme to simplify the selection of kernel adaptive filters in a multikernel structure. By multiplying the output of each kernel filter by an adaptive biasing factor between zero and one, the degrading effects of poorly adjusted kernel filters can be minimized, increasing the robustness of the multikernel scheme. This approach is able to deal with the lack of the necessary statistical information for an optimal adjustment of the filter and its structure. The advantages of the proposed scheme with respect to other multi-kernel solutions are checked by means of numerical examples in the context of signal prediction and system identification.

Audio or visual data analysis tasks usually have to deal with high-dimensional and nonnegative signals. However, most data analysis methods suffer from overfitting and numerical problems when data have more than a few dimensions needing a dimensionality reduction preprocessing. Moreover, interpretability about how and why filters work for audio or visual applications is a desired property, especially when energy or spectral signals are involved. In these cases, due to the nature of these signals, the nonnegativity of the filter weights is a desired property to better understand its working. Because of these two necessities, we propose different methods to reduce the dimensionality of data while the nonnegativity and interpretability of the solution are assured. In particular, we propose a generalized methodology to design filter banks in a supervised way for applications dealing with nonnegative data, and we explore different ways of solving the proposed objective function consisting of a nonnegative version of the orthonormalized partial least-squares method. We analyze the discriminative power of the features obtained with the proposed methods for two different and widely studied applications: texture and music genre classification. Furthermore, we compare the filter banks achieved by our methods with other state-of-the-art methods specifically designed for feature extraction.

In-network distributed estimation of sparse parameter vectors via diffusion LMS strategies has been studied and investigated in recent years. In all the existing works, some convex regularization approach has been used at each node of the network in order to achieve an overall network performance superior to that of the simple diffusion LMS, albeit at the cost of increased computational overhead. In this paper, we provide analytical as well as experimental results which show that the convex regularization can be selectively applied only to some chosen nodes keeping rest of the nodes sparsity agnostic, while still enjoying the same optimum behavior as can be realized by deploying the convex regularization at all the nodes. Due to the incorporation of unregularized learning at a subset of nodes, less computational cost is needed in the proposed approach. We also provide a guideline for selection of the sparsity aware nodes and a closed form expression for the optimum regularization parameter.

We show how the convergence time of an adaptive network can be estimated in a distributed manner by the agents. Using this procedure, we propose a distributed mechanism for the nodes to switch from using fixed doubly-stochastic combination weights to adaptive combination weights. By doing so, and by knowing when to switch, the agents are able to enhance their steady-state mean-square-error performance without degrading the rate of convergence during the transient phase of the learning algorithm.

Recently, a new class of nonlinear adaptive filtering architectures has been introduced based on the functional link adaptive filter (FLAF) model. Here we focus specifically on the split FLAF (SFLAF) architecture, which separates the adaptation of linear and nonlinear coefficients using two different adaptive filters in parallel. This property makes the SFLAF a well-suited method for problems like nonlinear acoustic echo cancellation (NAEC), in which the separation of filtering tasks brings some performance improvement. Although flexibility is one of the main features of the SFLAF, some problem may occur when the nonlinearity degree of the input signal is not known a priori. This implies a non-optimal choice of the number of coefficients to be adapted in the nonlinear path of the SFLAF. In order to tackle this problem, we propose a proportionate FLAF (PFLAF), which is based on sparse representations of functional links, thus giving less importance to those coefficients that do not actively contribute to the nonlinear modeling. Experimental results show that the proposed PFLAF achieves performance improvement with respect to the SFLAF in several nonlinear scenarios.

Orthonormalized partial least squares (OPLS) is a popular multivariate analysis method to perform supervised feature extraction. In this paper, we propose a novel scheme to solve Orthonormalized Partial Least Squares (OPLS) that can be easily modified to include additional constraints over the input data projection vectors. This scheme is used to implement an OPLSmethod with sparsity constraints (SOPLS), which allows to obtain more interpretable projection vectors that depend only on a few of the original input variables. The discriminative power of the sparse features extracted by SOPLS is analyzed on a benchmark of classification problems, where the method shows very competitive performance in terms of classification error.

Combinations of adaptive filters have attracted attention as a simple way to improve filter performance, avoiding the compromise between fast adaptation and low excess mean-square error. However, the computational cost of the combined scheme is higher than that of a single filter, since two or more filters must be run in parallel. In this paper, we propose a new approach for reducing the computational cost of combinations of adaptive filters in custom and semi-custom hardware implementations. In the proposed scheme, the fast filter is adapted as usual, but instead of updating the slow filter, we adapt the difference between the slow and fast filters. We show that the coefficients of the difference filter have a lower dynamic range compared to the slow filter, and therefore can be updated using a smaller number of bits. The wordlength used for the difference filter can be chosen to make the performance of this low-cost scheme similar to that of an infinite-precision implementation.

This paper introduces a new class of nonlinear adaptive filters, whose structure is based on Hammerstein model. Such filters derive from the functional link adaptive filter (FLAF) model, defined by a nonlinear input expansion, which enhances the representation of the input signal through a projection in a higher dimensional space, and a subsequent adaptive filtering. In particular, two robust FLAF-based architectures are proposed and designed ad hoc to tackle nonlinearities in acoustic echo cancellation (AEC). The simplest architecture is the split FLAF, which separates the adaptation of linear and nonlinear elements using two different adaptive filters in parallel. In this way, the architecture can accomplish distinctly at best the linear and the nonlinear modeling. Moreover, in order to give robustness against different degrees of nonlinearity, a collaborative FLAF is proposed based on the adaptive combination of filters. Such architecture allows to achieve the best performance regardless of the nonlinearity degree in the echo path. Experimental results show the effectiveness of the proposed FLAF-based architectures in nonlinear AEC scenarios, thus resulting an important solution to the modeling of nonlinear acoustic channels.

Adaptive filters can improve their performance by exploiting the well known tradeoff between bias and variance of the estimated solution. In a previous work, a scheme for adaptively biasing the filter weights was introduced, multiplying the output of a filter of any kind by a shrinking factor a ∈ [0,1]. With an appropriate value a, such a scheme can reduce the steady-state error, especially for low signal-to-noise ra tio (SNR). Here, we extend such analysis for a tracking scenario in which the optimal solution follows a random walk-model. We briefly review a realizable scheme for learning a, based on recently proposed algorithms for adaptive filter combination. Our experiments validate the accurateness of the analysis, and illustrate the performance gains that can be expected from these biased configurations in stationary and tracking scenarios.

The combination of filters concept is a simple and flexible method to circumvent various compromises hampering the operation of adaptive linear filters. Recently, applications which require the identification of not only linear, but also nonlinear systems are widely studied. In this paper, we propose a combination of adaptive Volterra filters as the most versatile nonlinear models with memory. Moreover, we develop a novel approach that shows a similar behavior but significantly reduces the computational load by combining Volterra kernels rather than complete Volterra filters. Following an outline of the basic principles, the second part of the paper focuses on the application to nonlinear acoustic echo cancellation scenarios. As the ratio of the linear to nonlinear echo signal power is, in general, a priori unknown and time-variant, the performance of nonlinear echo cancellers may be inferior to a linear echo canceller if the nonlinear distortion is very low. Therefore, a modified version of the combination of kernels is developed obtaining a robust behavior regardless of the level of nonlinear distortion. Experiments with noise and speech signals demonstrate the desired behavior and the robustness of both the combination of Volterra filters and the combination of kernels approaches in different application scenarios.

This paper introduces a new support vector machine (SVM) formulation to obtain sparse solutions in the primal SVM parameters, providing a new method for feature selection based on SVMs. This new approach includes additional constraints to the classical ones that drop the weights associated to those features that are likely to be irrelevant. A ν-SVM formulation has been used, where ν indicates the fraction of features to be considered. This paper presents two versions of the proposed sparse classifier, a 2-norm SVM and a 1-norm SVM, the latter having a reduced computational burden with respect to the first one. Additionally, an explanation is provided about how the presented approach can be readily extended to multiclass classification or to problems where groups of features, rather than isolated features, need to be selected. The algorithms have been tested in a variety of synthetic and real data sets and they have been compared against other state of the art SVM-based linear feature selection methods, such as 1-norm SVM and doubly regularized SVM. The results show the good feature selection ability of the approaches.

Combinations of adaptive filters have attracted attention as a simple solution to improve filter performance, including tracking properties. In this paper, we consider combinations of LMS and RLS filters, and study their performance for tracking time-varying solutions. We show that a combination of two filters from the same family (i.e., two LMS or two RLS filters) cannot improve the performance over that of a single filter of the same type with optimal selection of the step size (or forgetting factor). However, combining LMS and RLS filters it is possible to simultaneously outperform the optimum LMS and RLS filters. In other words, combination schemes can achieve smaller errors than optimally adjusted individual filters. Experimental work in a plant identification setup corroborates the validity of our results.

This paper proposes an improved model for the transient of convex combinations of adaptive filters. A previous model, based on a first-order Taylor series approximation of the nonlinear functions that appear in convex combinations, tended to overestimate the variance of the auxiliary variable used to estimate the mixing parameter. In this paper, we apply a second-order Taylor approximation that improves these estimates, and obtains better agreement with simulations. In addition, we also extend the model to include a simple mechanism for the transfer of coefficients between the constituent filters, a procedure that greatly improves the convergence of the overall filter, and provide an expression to select the free parameter used in such a scheme.

This paper presents a method for estimating the optimum number of second-order kernel diagonals of an adaptive Volterra filter in system identification tasks. To this end, a recently proposed time-domain mechanism is carried over to the very efficient partitioned-block DFT-domain Volterra filtering technique. The size of the nonlinear memory is controlled by monitoring the performance of an adaptive combination scheme with two differently-sized quadratic kernels. Subsequently, an efficient version is derived, requiring only minor additional computations as compared to a single Volterra filter. The effectiveness of the outlined estimation procedure is demonstrated by various simulations with real nonlinear systems and both noise and speech inputs in an acoustic echo cancellation scenario.

Adaptive combinations of adaptive filters are gaining popularity as a flexible and versatile solution to improve adaptive filters performance. In the recent years, combination schemes have focused on two different approaches: Convex and affine combinations, developing principally practical implementations with just two component filters. However, combinations of a higher number of adaptive filters can offer additional advantages, mainly in tracking environments. In this paper, we introduce a practical adaptation scheme for the affine combination of an arbitrary number of filters, including a steady-state analysis where the proposed rule is compared with the optimal combination. Several experiments both in tracking and stationary scenarios serve to demonstrate the appropriate performance of this approach.

Nonlinear acoustic echo cancellers (NLAEC) are becoming increasingly important in hands-free applications. However, in some situations, an NLAEC is inferior to a linear AEC, especially when the channel generates a negligible (or no) nonlinear echo. In general, the ratio of the linear to nonlinear echo signal power is unknown a priori, and will vary over time, thus making it difficult to know if an NLAEC would improve or degrade the cancellation. In this paper, we present two novel solutions to this problem based on the adaptive combination of linear and nonlinear echo cancellers. Both solutions perform efficiently regardless of the level of nonlinear echo. The benefits and robustness of both schemes are illustrated by experiments using Laplacian colored noise and speech input signals.

Proportionate adaptive filters, such as those based on the improved proportionate normalized least-mean-square (IPNLMS) algorithm, have been proposed for echo cancellation as an interesting alternative to the normalized least-mean-square (NLMS) filter. Proportionate schemes offer improved performance when the echo path is sparse, but are still subject to some compromises regarding their convergence properties and steady-state error. In this paper, we study how combination schemes, where the outputs of two independent adaptive filters are adaptively mixed together, can be used to increase IPNLMS robustness to channels with different degrees of sparsity, as well as to alleviate the rate of convergence versus steady-state misadjustment tradeoff imposed by the selection of the step size. We also introduce a new block-based combination scheme which is specifically designed to further exploit the characteristics of the IPNLMS filter. The advantages of these combined filters are justified theoretically and illustrated in several echo cancellation scenarios.

Adaptive filtering schemes are subject to different tradeoffs regarding their steady-state misadjustment, speed of convergence, and tracking performance. To alleviate these compromises, a new approach has recently been proposed, in which two filters with complementary capabilities adaptively mix their outputs to get an overall filter of improved performance. Following this approach, we propose a new normalized rule for adapting the mixing parameter that controls the combination. The new update rule preserves the good features of the existing scheme and is more robust to changes in the filtering scenario, for instance when the signal-to-noise ratio (SNR) is time varying. The benefits of the normalized scheme are illustrated analytically and with a number of experiments in both stationary and tracking situations.

This paper studies the performance and applicability of a novel kernel partial least squares (KPLS) algorithm for nonlinear feature extraction in the context of remote sensing applications. The so-called kernel orthonormalized PLS algorithm with reduced complexity (rKOPLS) has the following two core parts: (1) a kernel version of OPLS (called KOPLS) and (2) a sparse approximation for large-scale data sets, which ultimately leads to the rKOPLS algorithm. The method is theoretically analyzed in terms of computational burden and memory requirements and is tested in common remote sensing applications: multi- and hyperspectral image classification and biophysical parameter estimation problems. The proposed method largely outperforms the traditional (linear) PLS algorithm and demonstrates good capabilities in terms of expressive power of the extracted nonlinear features, accuracy, and scalability as compared to the standard KPLS.

Progressively emphasizing samples that are difficult to classify correctly is the base for the recognized high performance of real Adaboost (RA) ensembles. The corresponding emphasis function can be written as a product of a factor that measures the quadratic error and a factor related to the proximity to the classification border; this fact opens the door to explore the potential advantages provided by using adjustable combined forms of these factors. In this paper, we introduce a principled procedure to select the combination parameter each time a new learner is added to the ensemble, just by maximizing the associated edge parameter, calling the resulting method the dynamically adapted weighted emphasis RA (DW-RA). A number of application examples illustrates the performance improvements obtained by DW-RA.

Automatic knowledge extraction from music signals is a key component for most music organization and music information retrieval systems. In this paper, we consider the problem of instrument modelling and instrument classification from the rough audio data. Existing systems for automatic instrument classification operate normally on a relatively large number of features, from which those related to the spectrum of the audio signal are particularly relevant. In this paper, we confront two different models about the spectral characterization of musical instruments. The first assumes a constant envelope of the spectrum (i.e., independent from the pitch), whereas the second assumes a constant relation among the amplitude of the harmonics. The first model is related to the Mel frequency cepstrum coefficients (MFCCs), while the second leads to what we will refer to as harmonic representation (HR). Experiments on a large database of real instrument recordings show that the first model offers a more satisfactory characterization, and therefore MFCCs should be preferred to HR for instrument modelling/classification.

This paper presents the study of a sparse kernel-based method for non-linear feature extraction in the context of remote sensing classification and regression problems. The so-called kernel orthonormalized PLS algorithm with reduced complexity (rKOPLS) has two core parts: (i) a kernel version of OPLS (called KOPLS), and (ii) a sparse (reduced) approximation for large scale data sets, which ultimately leads to rKOPLS. The method demonstrates good capabilities in terms of expressive power of the extracted features and scalability.

Adaptive blind equalization plays an essential role in modern communication systems. We propose to improve the performance of constant modulus algorithm (CMA) based equalizers by using an adaptive convex combination of two CMA filters with a large and small step size, respectively, in order to simultaneously obtain fast convergence with low misadjustment during stationary periods. Some experiments show the effectiveness of the new algorithm and suggest that it is a reasonable alternative to blind equalizers that commute between the CMA and the (decision-directed) least mean square filter

In this paper we are presenting a novel multivariate analysis method. Our scheme is based on a novel kernel orthonormalized partial least squares (PLS) variant for feature extraction, imposing sparsity constrains in the solution to improve scalability. The algorithm is tested on a benchmark of UCI data sets, and on the analysis of integrated short-time music features for genre prediction. The upshot is that the method has strong expressive power even with rather few features, is clearly outperforming the ordinary kernel PLS, and therefore is an appealing method for feature extraction of labelled data.

Combination approaches can improve the performance of adaptive schemes. In this paper, we study the steady-state performance of an adaptive convex combination of transversal filters and show its universality in the sense that the combination performs, in steady-state, at least as well as its best component. We specialize the results to a convex combination of LMS filters using energy conservation arguments.

The recent interest in combining neural networks has produced a variety of techniques. This paper deals with boosting methods, in particular, real AdaBoost schemes built up with radial basis function networks. Real Adaboost emphasis function can be divided into two different terms, the first only focus on the quadratic error of each pattern and the second only takes into account its "proximity" to the boundary. Incorporating to this fixed emphasis function an additional degree of freedom, that allows us to weight these two terms, and also to select the radial basis functions centroids according to the emphasized regions, we show performance improvements: an error rate reduction, a faster convergence, and overfitting robustness.

We propose a new approach for solving multiclass problems with support vector machines. We modify the existing technique to properly reduce the empirical error, therefore we will be ideally able to outperform the previously proposed scheme for multi-class SVMs. The proposed approach also provides solutions with a significant reduction in the number of support vectors, which is an important feature for fast systems.