The Lottery Ticket Hypothesis (LTH) suggests that dense networks contain sparse, trainable subnetworks, "winning tickets", that can match the original model's performance when trained in isolation.These subnetworks are usually found through iterative pruning, but at high sparsity many potentially effective subnetworks fail to converge under standard training.Knowledge distillation (KD) mitigates this issue by providing richer supervision from a teacher model.We propose the Knowledge-Distilled Lottery Ticket (KDLT) procedure, a dual-phase method that applies KD during pruning and retraining to recover stronger sparse subnetworks.Experiments on MNIST, CIFAR-10/100, and Tiny-ImageNet show that KDLT delivers higher accuracy at fixed sparsity or comparable accuracy at higher sparsity.

To effectively monitor industrial applications, an accurate estimate of their condition, or a Condition Indicator (CI), is required.Recently, a CI estimation method called Monotonically Constraint Guided Autoencoders (MCGAE) was introduced, which constrains the CI to remain within predefined ranges for both normal and anomalous data while also enforcing monotonic behavior over time.However, that work employed a Convolutional AutoEncoder (CAE) architecture that did not capture longer-term temporal dependencies.In this study, we evaluate a recurrent CAE variant that incorporates a Long Short-Term Memory (LSTM) layer and compare its performance against alternative architectures without a recurrent component.Experimental results on a bearing run-to-failure dataset, indicate that the addition of LSTM improves the monotonic behavior of the estimated CI.* This research received funding from

Computerized detection of relevant lung sound events has the potential to assist physicians during auscultation and to monitor the severity of pulmonary diseases in ambulatory settings.In some cases, realtime detection of adventitious lung sounds is required to provide instant feedback to physicians, e.g. during autogenic drainage therapy.Stateof-the-art solutions for this task leverage deep learning models, which vary significantly in complexity.For real-time applications on resourceconstrained devices, such as stethoscope-integrated hardware, both detection accuracy and model complexity are important to consider.While most existing research focusses primarily on accuracy, this work evaluates both accuracy and computational complexity.The contributions of this work are threefold.First, the effect of using a full breathing cycle as input is studied to assess its impact on event detection performance.This approach introduces a computational cost due to the required segmentation process.Second, a transformer-based architecture is compared with two relatively simple convolutional models, each utilizing different input horizons.Evaluations are conducted on both public and in-house lung sound datasets.Third, recognizing that the event detection task aligns better with a multi-label setting than the commonly used multi-class setup, this study compares both approaches.We conclude that a multi-label output outperforms a multi-class approach, that inputs segmented per breathing cycle are preferred, and that the high complexity models have similar performance to the models with low complexity on unseen data.The source code is available through this GitHub repository.

The success of deep learning has been based on smooth loss functions that can easily be optimized using gradient descent and an off-the-shelf optimizer.However, training a neural network for a new application is not trivial as it requires many hyperparameters to be tuned.Several issues exist such as overfitting and underfitting.Many applications allow for some errors to be made, although, traditional learning objectives will influence the training in all cases except the one perfect prediction is made.In this work, constraints are proposed to replace the cross-entropy or the mean squared error to allow the neural network to make some errors.These errors can be set in advance to reflect how accurate the predictions of the neural network need to be.For each loss function, it is shown on two different data sets that the proposed constraint based learning performs similarly or even outperforms the standard loss functions.Moreover, in the case of classification problems, the constraints can result in predictions with significantly higher probability on a test set.

Lately, the monitoring of the respiratory health status using sound signals for pig cough classification has caught the attention of the research community. In this paper, existing data augmentation techniques for Sound Event Classification (SEC) and Transfer Learning (TL) schemes are evaluated in the scenario of pig cough classification. Specifically, we talk about a) Deep Learning (DL) and TL as a popular choice for SEC and b) Enhancing model robustness using data augmentation which adds (realistic) acoustic variability to the data without requiring additional annotations. The pig cough dataset recorded in commercial farm environments is used and divided into two categories (typel and type2) based on acoustical properties of the farms. Transfer learning with data augmentation strategies are used to explore the generalization of the pig cough classifier to changing conditions. Overall, data augmentation methods improved the performance when the model was trained on typel data and tested on type2 data. A maximum improvement in Fl-score of 3.69 percentage points and reduction of 1.66 percentage points in the Fl-score standard deviation (FI-SD) was achieved with a OpenL3 based model using a hybrid data augmentation. OpenL3 based models emerge as a light weight alternative to ResNet based models for this task. In conclusion, the findings of this study call for domain specific methods to achieve classifier adaptability to changing environmental acoustical conditions in farms.

sponsorship: This research received funding from the Flemish Government (AI Research Program). (Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen)

In this paper two different convolutional neural network (CNN) architectures are investigated for the purpose of real-time on-edge domestic acoustic event classification.For training and evaluation of the models, a real-life acoustical dataset was recorded in 72 different home environments.A quantization-aware training scheme was applied that takes into account that the models need to run on 8-bit fixed-point processing hardware.Once trained, the models were successfully deployed on an ARM cortex-M7 microcontroller unit (i.MX RT1064).This study indicates that the used procedure can lead to an efficient and real-time embedded on-edge implementation of a domestic sound event classifier that does not sacrifice classification performance compared to its floating-point counterpart.

In this paper a multi-channel speech enhancement framework for distant speech acquisition in noisy and reverberant environments for Non-negative Matrix Factorization (NMF)-based Automatic Speech Recognition (ASR) is proposed. The system is evaluated for its use in an assistive vocal interface for physically impaired and speech-impaired users. The framework utilises the Spatially Pre-processed Speech Distortion Weighted Multi-channel Wiener Filter (SP-SDW-MWF) in combination with a postfilter to reduce noise and reverberation. Additionally, the estimation uncertainty of the speech enhancement framework is propagated through the Mel-Frequency Cepstrum Coefficients (MFCC) feature extraction to allow for feature compensation in a later stage. Results indicate that a) using a trade-off parameter between noise reduction and speech distortion has a positive effect on the recognition performance with respect to the well-known GSC and MWF and b) the addition of a postfilter and the feature compensation increases performance with respect to several baselines for a non-pathological and pathological speaker.

Telehealth systems and applications are extensively investigated nowadays to enhance the quality-of-care and, in particular, to detect emergency situations and to monitor the well-being of elderly people, allowing them to stay at home independently as long as possible. In this paper, an embedded telehealth system for continuous, automatic, and remote monitoring of real-time fall emergencies is presented and discussed. The system, consisting of a radar sensor and base station, represents a cost-effective and efficient healthcare solution. The implementation of the fall detection data processing technique, based on the least-square support vector machines, through a digital signal processor and the management of the communication between radar sensor and base station are detailed. Experimental tests, for a total of 65 mimicked fall incidents, recorded with 16 human subjects (14 men and two women) that have been monitored for 320 min, have been used to validate the proposed system under real circumstances. The subjects' weight is between 55 and 90 kg with heights between 1.65 and 1.82 m, while their age is between 25 and 39 years. The experimental results have shown a sensitivity to detect the fall events in real time of 100% without reporting false positives. The tests have been performed in an area where the radar's operation was not limited by practical situations, namely, signal power, coverage of the antennas, and presence of obstacles between the subject and the antennas.

Anomaly detection starts from a model of normal behavior and classifies departures from this model as anomalies. This paper introduces a statistical non-parametric approach for anomaly detection that is based on a multivariate extension of the Poisson point process model for univariate extremes. The method is demonstrated on both a synthetic and a real-world data set, the latter being an unbalanced data set of acceleration data collected from movements of 7 pediatric patients suffering from epilepsy that is previously studied in [1]. The positive predictive values could be improved with an increase up to 12.9% (and a mean of 7%) while the sensitivity scores stayed unaltered. The proposed method was also shown to outperform an one-class SVM classifier. Because the Poisson point process model of extremes is able to combine information on the number of excesses over a fixed threshold with that on the excess values, a powerful model to detect anomalies is obtained that can be of high value in many applications.

Nocturnal home monitoring of epileptic children is often not feasible due to the cumbersome manner of seizure monitoring with the standard method of video/EEG-monitoring. We propose a method for hypermotor seizure detection based on accelerometers attached to the extremities. From the acceleration signals, multiple temporal, frequency, and wavelet-based features are extracted. After determining the features with the highest discriminative power, we classify movement events in epileptic and nonepileptic movements. This classification is only based on a nonparametric estimate of the probability density function of normal movements. Such approach allows us to build patient-specific models to classify movement data without the need for seizure data that are rarely available. If, in the test phase, the probability of a data point (event) is lower than a threshold, this event is considered to be an epileptic seizure; otherwise, it is considered as a normal nocturnal movement event. The mean performance over seven patients gives a sensitivity of 95.24% and a positive predictive value of 60.04%. However, there is a noticeable interpatient difference.

This paper gives an overview of research within the ALADIN project, which aims to develop an assistive vocal interface for people with a physical impairment. In contrast to existing ap-proaches, the vocal interface is trained by the end-user himself, which means it can be used with any vocabulary and grammar, and that it is maximally adapted to the — possibly dysarthric — speech of the user. This paper describes the overall learn-ing framework, the user-centred design and evaluation aspects, database collection and approaches taken to combat problems such as noise and erroneous input. Index Terms: vocal user interface, user-centred design, self-taught learning, speech database, dysarthric speech

This work studies an optimization scheme for computing sparse approximate solutions of over-determined linear systems. Sparse Conjugate Directions Pursuit (SCDP) aims to construct a solution using only a small number of nonzero (i.e. nonsparse) coefficients. Motivations of this work can be found in a setting of machine learning where sparse models typically exhibit better generalization performance, lead to fast evaluations, and might be exploited to define scalable algorithms. The main idea is to build up iteratively a conjugate set of vectors of increasing cardinality, in each iteration solving a small linear subsystem. By exploiting the structure of this conjugate basis, an algorithm is found (i) converging in at most D iterations for D -dimensional systems, (ii) with computational complexity close to the classical conjugate gradient algorithm, and (iii) which is especially efficient when a few iterations suffice to produce a good approximation. As an example, the application of SCDP to Fixed-Size Least Squares Support Vector Machines (FS-LSSVM) is discussed resulting in a scheme which efficiently finds a good model size for the FS-LSSVM setting, and is scalable to large-scale machine learning tasks. The algorithm is empirically verified in a classification context. Further discussion includes algorithmic issues such as component selection criteria, computational analysis, influence of additional hyper-parameters, and determination of a suitable stopping criterion.

Acoustic differences are so subtle in a native accent identification (AID) task that a brute force frame-based Gaussian Mixture Model (GMM) fails to discover the tiny distinctions [1]. Apart from the frame-based framework, in this paper we propose a vector-based speaker modeling method, to which common support vector machine (SVM) kernels can be applied. The vector-based speaker model is composed of the concatenation of the average acoustic representations of all phonemes. SVM and GMM classifiers are compared on the speaker models. Moreover, based on the observation that accents only differ in a limited number of phonemes, a variable selection framework is indispensable to select accent relevant features. We investigate a forward selection method, Analysis of Variance (ANOVA) , and a backward selection method, SVM- Recursive Feature Elimination (SVM-RFE). We find that the multiclass SVM-RFE achieves comparable performance with the ANOVA on optimally selected variable sets, while it obtains excellent performance with very few features in low dimensions. Results demonstrate the effectiveness of the proposed speaker models together with the SVM classifier both in low dimensions and in high dimensions as well as the necessity of variable selection. Index Terms: variable selection, native accent identification, support vector machines, recursive feature elimination, cross

This research studies a practical iterative algorithm for multi-class kernel logistic regression (KLR). Starting from the negative penalized log likelihood criterium we show that the optimization problem in each iteration can be solved by a weighted version of least squares support vector machines (LS-SVMs). In this derivation it turns out that the global regularization term is reflected as a usual regularization in each separate step. In the LS-SVM framework, fixed-size LS-SVM is known to perform well on large data sets. We therefore implement this model to solve large scale multi-class KLR problems with estimation in the primal space. To reduce the size of the Hessian, an alternating descent version of Newton's method is used which has the extra advantage that it can be easily used in a distributed computing environment. It is investigated how a multi-class kernel logistic regression model compares to a one-versus-all coding scheme.

Kernel logistic regression (KLR) is a popular non-linear classification technique. Unlike an empirical risk minimization approach such as employed by Support Vector Machines (SVMs), KLR yields probabilistic outcomes based on a maximum likelihood argument which are particularly important in speech recognition. Different from other KLR implementations we use a Nyström approximation to solve large scale problems with estimation in the primal space such as done in fixed-size Least Squares Support Vector Machines (LS-SVMs). In the speech experiments it is investigated how a natural KLR extension to multi-class classification compares to binary KLR models coupled via a one-versus-one coding scheme. Moreover, a comparison to SVMs is made. Index Terms: phoneme classification, kernel logistic regression, large-scale, multi-class