The ability of large language models (LLMs) to reason effectively is crucial for a wide range of applications, from complex decision-making to scientific research. However, it remains unclear how well reasoning capabilities are transferred or preserved when LLMs undergo Knowledge Distillation (KD), a process that typically reduces model size while attempting to retain performance. In this study, we explore the effects of model distillation on the reasoning abilities of various reasoning language models (RLMs). We introduce Cognitive Flow, a novel framework that systematically extracts meaning and map states in Chain-of-Thought (CoT) processes, offering new insights on model reasoning and enabling quantitative comparisons across RLMs. Using this framework, we investigate the impact of KD on CoTs produced by RLMs. We target DeepSeek-R1-671B and its distilled 70B, 32B and 14B versions, as well as QwenQwQ-32B from the Qwen series. We evaluate the models on three subsets of mathematical reasoning tasks with varying complexity from the MMLU benchmark. Our findings demonstrate that while distillation can effectively replicate a similar reasoning style under specific conditions, it struggles with simpler problems, revealing a significant divergence in the observable thought process and a potential limitation in the transfer of a robust and adaptable problem-solving capability.

ABSTRACT Nowadays, automated intelligent systems play an increasingly vital role in aiding decision‐making processes across various fields. Firefighting represents a crucial area where accurate information gathering is paramount for efficient resource allocation. Social media platforms as Twitter (or X) have emerged as valuable sources of real‐time data, often referred to as ‘citizen science’, offering additional insights alongside traditional data sources. In this work, we introduce a novel pipeline that leverages Natural Language Processing (NLP) techniques and Twitter data, utilising transformer models to identify and monitor wildfire incidents. Expanding on this approach, we incorporate sentiment analysis to provide deeper insights into public perceptions and emotions related to fire events. Additionally, we present visual representations of geographic data through heat mapping, potentially aiding firefighters in making informed decisions. By integrating advanced NLP techniques with social media data, our approach presents a promising strategy for enhancing wildfire management efforts.

Customer-support services increasingly rely on automation, whether full or with human intervention.Despite optimising resources, this may result in mechanical protocols and lack of human interaction, thus reducing customer loyalty.Our goal is to enhance interpretability and provide guidance in communication through novel tools for easier analysis of message trends and sentiment variations.Monitoring these contributes to more informed decision-making, enabling proactive mitigation of potential issues, such as protocol deviations or customer dissatisfaction.We propose a generic approach for dialogue flow discovery that leverages clustering techniques to identify dialogue states, represented by related utterances.State transitions are further analyzed to detect prevailing sentiments.Hence, we discover sentimentaware dialogue flows that offer an interpretability layer to artificial agents, even those based on black-boxes, ultimately increasing trustworthiness.Experimental results demonstrate the effectiveness of our approach across different dialogue datasets, covering both human-human and human-machine exchanges, applicable in task-oriented contexts but also to social media, highlighting its potential impact across various customer-support settings.

Several dialogue corpora are currently available for research purposes, but they still fall short for the growing interest in the development of dialogue systems with their own specific requirements. In order to help those requiring such a corpus, this paper surveys a range of available options, in terms of aspects like speakers, size, languages, collection, annotations, and domains. Some trends are identified and possible approaches for the creation of new corpora are also discussed.

We propose two novel Federated Learning (FL) algorithms based on decaying momentum (Demon): Federated Demon (FedDemon) and Federated Demon Adam (FedDemonAdam).In particular, we apply Demon to Momentum Stochastic Gradient Descent (SGD) and Adam in a Federated setting, which has shown to improve results in a centralized environment.We empirically show that FedDemon and FedDemonAdam have a faster convergence rate and performance improvements compared to state-of-the-art algorithms including FedAvg, FedAvgM and FedAdam.17

Current challenges in machine learning include dealing with temporal data streams, drift and non-stationary scenarios, often with text data, whether in social networks or in business systems. This dynamic nature tends to limit the performance of traditional static learning models and dynamic learning strategies must be put forward. However, acquiring the performance of those strategies is not a straightforward issue, as sample's dependency undermines the use of validation techniques, like crossvalidation. In this paper we propose to use the McNemar's test to compare two distinct approaches that tackle adaptive learning in dynamic environments, namely DARK (Drift Adaptive Retain Knowledge) and Learn++. NSE (Learn++ for Non-Stationary Environments). The validation is based on a Twitter case study benchmark constructed using the DOTS (Drift Oriented Tool System) dataset generator. The results obtained demonstrate the usefulness and adequacy of using McNemar's statistical test in dynamic environments where time is crucial for the learning algorithm.

This paper describes an Erasmus Intensive Programme that used international collaboration as a novel pedagogical approach to teaching programming skills to first-year students in a blended learning context using a mixture of virtual environment and intensive teaching. The experience and outcomes of the Programme are evaluated from the viewpoints of the students and instructors and conclusions are drawn on the value and conduct of international student collaborations.

Social networks are making part of the daily routine of millions of users. Twitter is among Facebook and Instagram one of the most used, and can be seen as a relevant source of information as users share not only daily status, but rapidly propagate news and events that occur worldwide. Considering the dynamic nature of social networks, and their potential in information spread, it is imperative to find learning strategies able to learn in these environments and cope with their dynamic nature. Time plays an important role by easily out-dating information, being crucial to understand how informative can past events be to current learning models and for how long it is relevant to store previously seen information, to avoid the computation burden associated with the amount of data produced. In this paper we study the impact of longstanding messages in micro-blogging classification by using different training time-window sizes in the learning process. Since there are few studies dealing with drift in Twitter and thus little is known about the types of drift that may occur, we simulate different types of drift in an artificial dataset to evaluate and validate our strategy. Results shed light on the relevance of previously seen examples according to different types of drift.

Handwritten signature recognition is one important component of biometric authentication. This is a central process in a broad range of areas requiring personal identification, such as security, legal contracts and bank transactions. Extensive efforts have been put into the research towards the verification of handwritten signatures, which contain biometric information. Although many successful methods have been used, they often disregard the size of databases, which can be very large, posing scalability problems to their application in real-world scenarios. To overcome this problem, in this paper, we use binary embeddings of high-dimensional data which is an efficient tool for indexing big datasets of biometric images. The rationale is to find a good hash function such that similar data points in Euclidean space preserve their similarities in the resulting Hamming space for fast data retrieval and state-of-the-art classification performance. In the settings of an handwritten signature retrieval system, an indexing hashing-based scheme is presented. We propose to learn k-bits hash code with a generalised regression neural network (GRNN), which yielded competitive results in the GPDS database.

Learning in non-stationary environments is not an easy task and requires a distinctive approach. The learning model must not only have the ability to continuously learn, but also the ability to acquired new concepts and forget the old ones. Additionally, given the significant importance that social networks gained as information networks, there is an ever-growing interest in the extraction of complex information used for trend detection, promoting services or market sensing. This dynamic nature tends to limit the performance of traditional static learning models and dynamic learning strategies must be put forward. In this paper we present a learning strategy to learn with drift in the occurrence of concepts in Twitter. We propose three different models: a time-window model, an ensemble-based model and an incremental model. Since little is known about the types of drift that can occur in Twitter, we simulate different types of drift by artificially time stamping real Twitter messages in order to evaluate and validate our strategy. Results are so far encouraging regarding learning in the presence of drift, along with classifying messages in Twitter streams.

Nowadays health concerns are effectively becoming ubiquitous. Most people have the need to effectively control their nutritional consumptions, mostly due to health issues. Personal computational devices may assist this control with a solution that allows an efficient management of each individual nutritional profile. In this work we propose a mobile service architecture that allows users to manage their nutritional information, using a profile-based system and build shopping lists based on the user's profiles. This application may contribute to the improvement of the lifestyle of the population through the recommendation of food and drinks that fit their profile of restrictions and/or nutritional options (for instance, due to hypertension or obesity, among others). The person's profile can be accessed and configured on a mobile device. A set of predefined templates provides the initial rules that may be customized to represent specific individual nutrition rules. The rules defined in the profile can later be used to filter the food presented to each user. The paper includes preliminary usability results from experiments using real data to validate the approach. These results suggest it could be used in real scenarios, although it may require more than simply filtering results.

Epilepsy diagnosis can be an extremely complex process, demanding considerable time and effort from physicians and healthcare infrastructures. Physicians need to classify each specific type of epilepsy based on different data, e.g., types of seizures, events and exams' results. This work presents a text mining approach to support medical decisions relating to epilepsy diagnosis and classification in children. We propose a text mining process that, using patient medical records, applies ontologies and named entities recognition as preprocessing steps, then applying K-Nearest Neighbors as a white-box lazy method to classify each instance. Results on real medical records suggest that the proposed framework shows good performance and clear interpretations, albeit the reduced volume of available training data.

Crowdsourcing is an emergent trend for general-purpose classification problem solving. Over the past decade, this notion has been embodied by enlisting a crowd of humans to help solve problems. There are a growing number of real-world problems that take advantage of this technique, such as Wikipedia, Linux or Amazon Mechanical Turk. In this paper, we evaluate its suitability for classification, namely if it can outperform state-of-the-art models by combining it with active learning techniques. We propose two approaches based on crowdsourcing and active learning and empirically evaluate the performance of a baseline Support Vector Machine when active learning examples are chosen and made available for classification to a crowd in a web-based scenario. The proposed crowdsourcing active learning approach was tested with Jester data set, a text humour classification benchmark, resulting in promising improvements over baseline results.

In recent years the the potential and programmability of Graphics Processing Units (GPU) has raised a note-worthy interest in the research community for applications that demand high-computational power. In particular, in financial applications containing thousands of high-dimensional samples, machine learning techniques such as neural networks are often used. One of their main limitations is that the learning phase can be extremely consuming due to the long training times required which constitute a hard bottleneck for their use in practice. Thus their implementation in graphics hardware is highly desirable as a way to speed up the training process. In this paper we present a bankruptcy prediction model based on the parallel implementation of the Multiple BackPropagation (MBP) algorithm which is tested on a real data set of French companies (healthy and bankrupt). Results by running the MBP algorithm in a sequential processing CPU version and in a parallel GPU implementation show reduced computational costs with respect to the latter while yielding very competitive performance.

Financial distress prediction is of great importance to all stakeholders in order to enable better decision-making in evaluating firms. In recent years, the rate of bankruptcy has risen and it is becoming harder to estimate as companies become more complex and the asymmetric information between banks and firms increases. Although a great variety of techniques have been applied along the years, no comprehensive method incorporating an holistic perspective had hitherto been considered. Recently, SVM+ a technique proposed by Vapnik [17] provides a formal way to incorporate privileged information onto the learning models improving generalization. By exploiting additional information to improve traditional inductive learning we propose a prediction model where data is naturally separated into several groups according to the size of the firm. Experimental results in the setting of a heterogeneous data set of French companies demonstrated that the proposed model showed superior performance in terms of prediction accuracy in bankruptcy prediction and misclassification cost.

Cancer diagnosis is generally the process of using some form of physical or genetic tests or exams, usually referred as patient data, to detect the disease. One of the main problems with cancer diagnosis systems is the lack of labeled data, as well as the difficulties of labeling pre-existing unlabeled data. Thus, there is a growing interest in exploring the use of unlabeled data as a way to improve classification performance in cancer diagnosis. The possible availability of this kind of data for some applications makes it an appealing source of information. In this work we explore an Incremental Background Knowledge (IBK) technique to introduce unlabeled data into the training set by expanding it using initial classifiers to better aid decisions, namely by improving recall values. The defined incremental SVM margin-based method was tested in the Wisconsin-Madison breast cancer diagnosis problem to examine the effectiveness of such techniques in supporting diagnosis.

Constructing a single text classifier that excels in any given application is a rather inviable goal. As a result, ensemble systems are becoming an important resource, since they permit the use of simpler classifiers and the integration of different knowledge in the learning process. However, many text-classification ensemble approaches have an extremely high computational burden, which poses limitations in applications in real environments. Moreover, state-of-the-art kernel-based classifiers, such as support vector machines and relevance vector machines, demand large resources when applied to large databases. Therefore, we propose the use of a new systematic distributed ensemble framework to tackle these challenges, based on a generic deployment strategy in a cluster distributed environment. We employ a combination of both task and data decomposition of the text-classification system, based on partitioning, communication, agglomeration, and mapping to define and optimize a graph of dependent tasks. Additionally, the framework includes an ensemble system where we exploit diverse patterns of errors and gain from the synergies between the ensemble classifiers. The ensemble data partitioning strategy used is shown to improve the performance of baseline state-of-the-art kernel-based machines. The experimental results show that the performance of the proposed framework outperforms standard methods both in speed and classification.

Nowadays machine learning are faced with an overload of data, both in terms of examples and features. Although recent algorithms, like support vector machines, can handle high dimensionality, it remains valuable to find smaller and more fitted spaces to perform learning tasks. We propose a twofold approach to tackle these high dimensionality issues in a text classification setting. First we use manifold learning as a pre-processing step to nonlinearly reduce the feature space. Second we use support vector machines to implement an active learning strategy, where the kernel trick is used to define the active examples. This approach deals with the high dimensionality both reducing the features and the number of examples needed to reach a desired performance.Results on a real-world benchmark corpus from Reuters and also on a reduced realistic version of the corpus show first the visualization capabilities of manifold learning and the performance improvement achieved with the active learning strategy.

One of the Internet’s hallmark is the rapid spread of the use of information and communication technology. This has boosted methods for hiding stego information inside digital cover content images which is a concerning issue in information security. On the other hand, attack of steganographic schemes has leveraged methods for steganalysis which is a challenging problem. In this paper, first we look at the design of classifiers, such as, Support Vector Machines (SVM) and neural networks (RBF and MLP) which are able to detect the presence of Least Significant Bit (LSB) matching steganography of gray scale images. Second, by combining with feature ranking methods (SVM-Recursive Feature Elimination, Kruskal Wallis) and reduction techniques (PCA) pattern classification of stego is successfully achieved. It is of utmost importance to look at the large set of features extracted from images and find ranking methods able, namely, to exclude correlated and redundant features, avoid the curse of dimensionality or circumvent the need of the steganalyzer to be re-designed. Results show that desirable properties of robustness and resilience are attained by designing classifiers able to deal with redundancy and noise. Moreover, comparison of classifiers performance emphasizes the chosen model for the steganalyser.

In this paper we develop and analyze methods for expanding automated learning of Relevance Vector Machines (RVM) to large scale text sets. RVM rely on Bayesian inference learning and while maintaining state-of-the-art performance, offer sparse and probabilistic solutions. However, efforts towards applying RVM to large scale sets have met with limited success in the past, due to computational constraints. We propose a diversified set of divide-and-conquer approaches where decomposition techniques promote the definition of smaller working sets that permit the use of all training examples. The rationale is that by exploring incremental, ensemble and boosting strategies, it is possible to improve classification performance, taking advantage of the large training set available. Results on Reuters-21578 and RCV1 are presented, showing performance gains and maintaining sparse solutions that can be deployed in distributed environments.

Relevance vector machines (RVM) have proven successful in many learning tasks. However, in large applications, they scale poorly. In many settings there is a large amount of unlabeled data which could be actively chosen by a learner and integrated in the learning procedure. The idea is to improve performance meanwhile reducing costs from data categorization. In this paper we propose an active learning RVM method based on the kernel trick. The underpinning idea is to define a working space between the relevance vectors (RV) initially obtained in a small labeled data set and the new unlabeled examples, where the most informative instances are chosen. By using kernel distance metrics, such a space can be defined and more informative examples can be added to the training set, increasing performance even though the problem dimension is not significantly affected. We detail the proposed method giving illustrative examples in the Reuters-21578 benchmark. Results show performance improvement and scalability.

Support vector machines (SVM) and relevance vector machines (RVM) constitute two state-of-the-art learning machines that are currently focus of cutting-edge research. SVM present accuracy and complexity preponderance, but are surpassed by RVM when probabilistic outputs or kernel selection come to discussion. We propose a two-level hierarchical hybrid SVM-RVM model to combine the best of both learning machines. The proposed model first level uses an RVM to determine the less confident classified examples and the second level then makes use of an SVM to learn and classify the tougher examples. We show the benefits of the hierarchical approach on a text classification task, where the two-levels outperform both learning machines

Text classification (TC) is a complex ubiquitous task that handles a huge amount of data. Current research has recently proved that kernel learning based methods are quite effective in this problem. As opposed to support vector machines (SVM), the relevance vector machine (RVM) in particular yields a probabilistic output while preserving its accuracy. However, few research efforts have addressed the issue of scalability that arises when applying RVM to large scale problems like TC. We propose a new model which consists of a two-step RVM classifier able to (i) be competitive regarding processing time, (ii) use all available training elements and (iii) improve RVM classification performance. The paper also shows that a convenient similitude measure among documents can be defined on all the collection data, which does not only make the process swifter but also parallelizable. Using REUTERS-21578, we show that deployment of successful real-time applications is possible through reduction of the computational complexity and improvement of overall performance, obtained by the proposed model.

The amount of texts available in digital form has dramatically increased, giving rise to the need of fast text classifiers. The tasks involved can be parallelized and distributed in a grid environment. This paper reports a study conducted on Reuters-21578 corpus, using a SVM learning machine. The task of text categorization is distributed in several platforms. The results achieved are very promising for speeding-up text categorization tasks and are valid independently of the learning machine.

Text mining, also known as intelligent text analysis, text data mining or knowledge-discovery in text, refers generally to the process of extracting interesting and nontrivial information and knowledge from text. One of the main problems with text mining and classification systems is the lack of labeled data, as well as the cost of labeling unlabeled data (Kiritchenko and Matwin 2001). Thus, there is a growing interest in exploring the use of unlabeled data as a way to improve classification performance in text classification. The ready availability of this kind of data in most applications makes it an appealing source of information. In this work we evaluate the benefits of introducing unlabeled data in a support vector machine automatic text classifier. We further evaluate the possibility of learning actively and propose a method for choosing the samples to be learned.

There is a growing interest in exploring the use of unlabeled data as a way to improve classification performance in text categorization. The ready availability of this kind of data in most applications makes it an appealing source of information. This work reports a study carried out on the Reuters-21578 corpus to evaluate the performance of support vector machines when unlabeled examples are introduced in the learning process. The improvement achieved, especially in false negative values and therefore in recall values, demonstrates that the use of unlabeled examples can be very important for small data sets.

A technique is proposed to detect and avoid obstacles for a mobile robot in an unknown environment. The usual problem of having too much sensorial information is dealt with by using several neural networks that cooperate in the guidance of the robot. Several unknown obstacle configurations were presented to the modular networks, proving that the MONODA architecture is very effective for obstacle avoidance when there is neither a priori nor a posteriori maps of the environment.