Understanding complex physical systems often requires integrating data from multiple diagnostics, each with limited resolution or coverage. We present a machine learning framework that reconstructs synthetic high-temporal-resolution data for a target diagnostic using information from other diagnostics, without direct target measurements during the inference. This multimodal super-resolution technique improves diagnostic robustness and enables monitoring even in case of measurement failures or degradation. Applied to fusion plasmas, our method targets edge-localized modes (ELMs), which can damage plasma-facing materials. By reconstructing super-resolution Thomson Scattering data from complementary diagnostics, we uncover fine-scale plasma dynamics and validate the role of resonant magnetic perturbations (RMPs) in ELM suppression through magnetic island formation. The approach provides new observation supporting the plasma profile flattening due to these islands. Our results demonstrate the framework’s ability to generate high-fidelity synthetic diagnostics, offering a powerful tool for ELM control development in future reactors like ITER. The approach is broadly transferable to other domains facing sparse, incomplete, or degraded diagnostic data, opening new avenues for discovery.

In this paper, a fully model-free architecture for vertical stabilization of thermonuclear plasmas in tokamak experimental reactors is presented. For the first time, an Extremum Seeking control algorithm is combined with neural networks to estimate the Lyapunov function to be minimized, resulting in a fully data-driven control architecture. The performance of different neural networks are compared. Specifically, Multilayer Perceptrons and Extreme Learning Machines are considered. The proposed architecture is tested in simulation to show that it can counteract relevant plasma disturbances, resulting in a significant improvement in terms of the achievable operative space compared to the Extremum Seeking algorithm, which still relies on model-based cost estimator. • An innovative fully model-free approach to the Vertical Stabilization problem. • Extremum Seeking combined with a neural network estimation of the cost function. • Control gain event-driven adaptive logic for efficient power supply management. • Comparison of performance and computational complexity of neural networks. • Enlarged operative space without tailoring the gains for specific scenarios.

The successful steady-state operation of burning fusion plasmas in planned future devices such as the ITER tokamak requires understanding of fast-ion physics. Alfven eigenmodes are special cases of plasma waves driven by fast ions that are important to identify and control since they can lead to loss of confinement and potential damage to the inner walls of a plasma device. The goal of this work is to compare machine learning-based systems trained to classify Alfven eigenmodes using CO2interferometer data from a labelled database on the DIII-D tokamak. A Long-Short Term Memory (LSTM) network is trained from scratch using simple spectrogram representations of the CO2phase data. The model is trained using a single chord (sequence) per training step. Results show a total true positive rate of = 90% and a false positive rate of = 18%. This paper demonstrates the potential of applying machine learning models to detect and identify different classes of Alfven eigenmodes for real-time applications in steady-state plasma operations that could potentially drive actuators to mitigate Alfven eigenmode impacts.

Beam emission spectroscopy (BES) is an active plasma diagnostic utilized for plasma density measurements. BES synthetic diagnostics are computationally expensive and comprehensive modelling suites designed to provide a better understanding of the diagnostics' perception of underlying plasma phenomena. RENATE-OD is an advanced BES synthetic diagnostic relying on a rate-equation solver to derive the beam emission for given input plasma profiles. In this work, linear regression, multi-layer perceptron and extreme learning machines were explored as a substitute for the rate-equation solver to predict the beam emission. Our experiments show that extreme learning machines are suitable for the task of predicting the arising emission profiles with very high accuracy and about 8000 times faster than RENATE-OD.

Tokamak is a torus-shaped nuclear fusion device that uses magnetic fields to confine fusion fuel in the form of plasma. Tearing instability in plasma is a major issue in which the magnetic field breaks and recombines in tokamak. This instability can lead to plasma disruption that terminates the fusion power generation and damages the plasma-facing wall materials. For a successful steady operation of a large-scale tokamak without disruption, it is required to predict and alarm the tearing instabilities well in advance to avoid them. In this work, we develop and validate a deep neural network-based multimodal prediction system that estimates the future tearing instability likelihood from multi-diagnostics signals in the DIII-D tokamak.

In recent years, Echo State Networks (ESNs), a special type of Recurrent Neural Networks (RNNs), have become increasingly established in the Machine Learning (ML) community due to their relatively simple initialization and training methods. Traditionally, the input and recurrent weights are generated randomly, with only the output weights being trained, typically using linear regression. However, recent publications have proposed alternative ways to initialize the weight matrices, e.g., by using more deterministic methods or data-driven approaches. This is the first work comparing different simple reservoir structures and an ESN with pre-trained input weights for the task of monitoring the state of a door using a surveillance camera in real-time. The results show that deterministic ESN structures perform better than the randomly initialized baseline, achieving a frame error rate of 2.62% vs. 2.93%.

Abstract Echo State Networks (ESNs) are a special type of Recurrent Neural Networks (RNNs), in which the input and recurrent connections are traditionally generated randomly, and only the output weights are trained. However, recent publications have addressed the problem that a purely random initialization may not be ideal. Instead, a completely deterministic or data-driven initialized ESN structure was proposed. In this work, an unsupervised training methodology for the hidden components of an ESN is proposed. Motivated by traditional Hidden Markov Models (HMMs), which have been widely used for speech recognition for decades, we present an unsupervised pre-training method for the recurrent weights and bias weights of ESNs. This approach allows for using unlabeled data during the training procedure and shows superior results for continuous spoken phoneme recognition, as well as for a large variety of time-series classification datasets.

Echo state networks (ESNs) are a special type of recurrent neural networks (RNNs), in which the input and recurrent connections are traditionally generated randomly, and only the output weights are trained. Despite the recent success of ESNs in various tasks of audio, image, and radar recognition, we postulate that a purely random initialization is not the ideal way of initializing ESNs. The aim of this work is to propose an unsupervised initialization of the input connections using the K -means algorithm on the training data. We show that for a large variety of datasets, this initialization performs equivalently or superior than a randomly initialized ESN while needing significantly less reservoir neurons. Furthermore, we discuss that this approach provides the opportunity to estimate a suitable size of the reservoir based on prior knowledge about the data.

Nuclear fusion is a promising alternative to address the problem of sustainable energy production. The tokamak is an approach to fusion based on magnetic plasma confinement, constituting a complex physical system with many control challenges. We study the characteristics and optimization of reservoir computing (RC) for real-time and adaptive prediction of plasma profiles in the DIII-D tokamak. Our experiments demonstrate that RC achieves comparable results to state-of-the-art (deep) convolutional neural networks (CNNs) and long short-term memory (LSTM) models, with a significantly easier and faster training procedure. This efficient approach allows for fast and frequent adaptation of the model to new situations, such as changing plasma conditions or different fusion devices.

Given their substantial success in addressing a wide range of computer vision challenges, Convolutional Neural Networks (CNNs) are increasingly being used in smart home applications, with many of these applications relying on the automatic recognition of human activities. In this context, low-power radar devices have recently gained in popularity as recording sensors, given that the usage of these devices allows mitigating a number of privacy concerns, a key issue when making use of conventional video cameras. Another concern that is often cited when designing smart home applications is the resilience of these applications against cyberattacks. It is, for instance, well-known that the combination of images and CNNs is vulnerable against adversarial examples, mischievous data points that force machine learning models to generate wrong classifications during testing time. In this paper, we investigate the vulnerability of radar-based CNNs to adversarial attacks, and where these radar-based CNNs have been designed to recognize human gestures. Through experiments with four unique threat models, we show that radar-based CNNs are susceptible to both white- and black-box adversarial attacks. We also expose the existence of an extreme adversarial attack case, where it is possible to change the prediction made by the radar-based CNNs by only perturbing the padding of the inputs, without touching the frames where the action itself occurs. Moreover, we observe that gradient-based attacks exercise perturbation not randomly, but on important features of the input data. We highlight these important features by making use of Grad-CAM, a popular neural network interpretability method, hereby showing the connection between adversarial perturbation and prediction interpretability.

In music analysis, one of the most fundamental tasks is note onset detection - detecting the beginning of new note events. As the target function of onset detection is related to other tasks, such as beat tracking or tempo estimation, onset detection is the basis for such related tasks. Furthermore, it can help to improve Automatic Music Transcription (AMT). Typically, different approaches for onset detection follow a similar outline: An audio signal is transformed into an Onset Detection Function (ODF), which should have rather low values (i.e. close to zero) for most of the time but with pronounced peaks at onset times, which can then be extracted by applying peak picking algorithms on the ODF. In the recent years, several kinds of neural networks were used successfully to compute the ODF from feature vectors. Currently, Convolutional Neural Networks (CNNs) define the state of the art. In this paper, we build up on an alternative approach to obtain a ODF by Echo State Networks (ESNs), which have achieved comparable results to CNNs in several tasks, such as speech and image recognition. In contrast to the typical iterative training procedures of deep learning architectures, such as CNNs or networks consisting of Long-Short-Term Memory Cells (LSTMs), in ESNs only a very small part of the weights is easily trained in one shot using linear regression. By comparing the performance of several feature extraction methods, pre-processing steps and introducing a new way to stack ESNs, we expand our previous approach to achieve results that fall between a bidirectional LSTM network and a CNN with relative improvements of 1.8 % and -1.4 %, respectively. For the evaluation, we used exactly the same 8-fold cross validation setup as for the reference results.

Industrial processes are heavily instrumented by employing a large number of sensors, generating huge amounts of data. One goal of the Industry 4.0 era is to apply data-driven approaches to optimize such processes. At the basic oxygen furnace (BOF), molten iron is transformed into steel by lowering its carbon content and achieving a certain chemical endpoint. In this work, we propose a data-driven approach to predict the endpoint temperature and chemical concentration of phosphorus, manganese, sulfur and carbon at the basic oxygen furnace. The prediction is based on two distinct datasets. First, a collection of static features is used which represent a more classic data-driven solution. The second approach includes time-series data that provide a better estimate of the final endpoint and enable further tuning of the process parameters, if necessary. For both approaches, model-based feature selection is used to filter the most relevant information. Results obtained by both models are compared in order to estimate the added value of including the time series data analysis on the performance of the BOF process. Results show that a simple feature extraction approach can enhance the prediction for phosphorus, manganese and temperature.

Contemporary surveillance systems mainly use video cameras as their primary sensor. However, video cameras possess fundamental deficiencies, such as the inability to handle low-light environments, poor weather conditions, and concealing clothing. In contrast, radar devices are able to sense in pitch-dark environments and to see through walls. In this paper, we investigate the use of micro-Doppler (MD) signatures retrieved from a low-power radar device to identify a set of persons based on their gait characteristics. To that end, we propose a robust feature learning approach based on deep convolutional neural networks. Given that we aim at providing a solution for a real-world problem, people are allowed to walk around freely in two different rooms. In this setting, the IDentification with Radar data data set is constructed and published, consisting of 150 min of annotated MD data equally spread over five targets. Through experiments, we investigate the effectiveness of both the Doppler and time dimension, showing that our approach achieves a classification error rate of 24.70% on the validation set and 21.54% on the test set for the five targets used. When experimenting with larger time windows, we are able to further lower the error rate.

With the increasing device scaling in the semiconductor technology, the necessity for designing robust and efficient Networks-on-Chip (NoCs) is more pronounced. The rerouting approach which is employed in most of the fault-tolerant methods causes the network performance to degrade considerably due to taking longer paths and creating hotspots around the faults. In this paper, a dynamically reconfigurable technique is proposed to target fault-tolerance and minimal routing in a unified manner. To accomplish this goal, the router architecture is modified to enable the frequently communicating nodes to bypass the faulty router and communicate through shorter paths. Thus, not only the rerouting is minimized, the connectivity of the network is maintained in the vicinity of faults. The experimental results validate the performance and reliability of the proposed technique with a small hardware overhead.

Reservoir Computing Network (RCN) is a special type of the single layer recurrent neural networks, in which the input and the recurrent connections are randomly generated and only the output weights are trained. Besides the ability to process temporal information, the key points of RCN are easy training and robustness against noise. Recently, we introduced a simple strategy to tune the parameters of RCN resulted in an effective and noise-robust RCN-based model for speech recognition. The aim of this work is to extend that study to the field of image processing. In particular, we investigate the potential of RCNs in achieving a competitive performance on the well-known MNIST dataset by following the aforementioned parameter optimizing strategy. Moreover, we achieve good noise robust recognition by utilizing such a network to denoise images and supplying them to a recognizer that is solely trained on clean images. The conducted experiments demonstrate that the proposed RCN-based handwritten digit recognizer achieves an error rate of 0.81 percent on the clean test data of the MNIST benchmark and that the proposed RCN-based denoiser can effectively reduce the error rate on the various types of noise.

The searchability of video content is often limited to the descriptions authors and/or annotators care to provide. The level of description can range from absolutely nothing to fine-grained annotations at the level of frames. Based on these annotations, certain parts of the video content are more searchable than others.

Reservoir Computing (RC) has recently been introduced as an interesting alternative for acoustic modeling. For phone and continuous digit recognition, the reservoir approach obtained quite promising results. In this work, we further elaborate this concept by porting some well-known techniques used to enhance recognition rates of GMM-based models to Reservoir Computing. In particular, we introduce context-dependent (CD) triphone states to model co-articulation and pronunciation mismatches arising from an imperfect lexicon. We also propose to incorporate two speaker normalization methods in the feature space, namely mean \& variance normalization and vocal tract length normalization. The impact of the investigated techniques is studied in the context of phone recognition on the TIMIT corpus. Our CD-RC-HMM hybrid yields a speaker-independent phone error rate (PER) of 22\% and a speaker-dependent PER of 20.5\%. By combining GMM and RC-based likelihoods at the state level, these scores can be reduced further.

Accurate acoustic modeling is an essential requirement of a state-of-the-art continuous speech recognizer. The Acoustic Model (AM) describes the relation between the observed speech signal and the non-observable sequence of phonetic units uttered by the speaker. Nowadays, most recognizers use Hidden Markov Models (HMMs) in combination with Gaussian Mixture Models (GMMs) to model the acoustics, but neural-based architectures are on the rise again. In this work, the recently introduced Reservoir Computing (RC) paradigm is used for acoustic modeling. A reservoir is a fixed - and thus non-trained - Recurrent Neural Network (RNN) that is combined with a trained linear model. This approach combines the ability of an RNN to model the recent past of the input sequence with a simple and reliable training procedure. It is shown here that simple reservoir-based AMs achieve reasonable phone recognition and that deep hierarchical and bi-directional reservoir architectures lead to a very competitive Phone Error Rate (PER) of 23.1% on the well-known TIMIT task.

In this paper a formerly proposed continuous digit recognition system based on Reservoir Computing (RC) is improved in two respects: (1)the single reservoir is substituted by a stack of reservoirs, and (2)the straightforward mapping of reservoir outputs to state likelihoods is replaced by a trained non-parametric mapping. Furthermore, it is shown that a reservoir-based method can improve a model trained on clean speech to work better in a noisy condition from which it has a number of unknown digit string recordings available. The first two improvements have lead to a system that outperforms a HMM-based system with the same noise robust features as input. The model adaptation offers a promising supplementary gain at modest noise levels.

Most automatic speech recognition systems employ Hidden Markov Models with Gaussian mixture emission distributions to model the acoustics. There have been several attempts however to challenge this approach, e.g. by introducing a neural network (NN) as an alternative acoustic model. Although the performance of these so-called hybrid systems is actually quite good, their training is often problematic and time consuming. By using a reservoir – this is a recurrent NN with only the output weights being trainable – we can overcome this disadvantage and yet obtain good accuracy. In this paper, we propose the first reservoir-based connected digit recognition system, and we demonstrate good performance on the Aurora-2 testbed. Since RC is a new technology, we anticipate that our present system is still sub-optimal, and further improvements are possible.

Automatic speech recognition has gradually improved over the years, but the reliable recognition of unconstrained speech is still not within reach. In order to achieve a breakthrough, many research groups are now investigating new methodologies that have potential to outperform the Hidden Markov Model technology that is at the core of all present commercial systems. In this paper, it is shown that the recently introduced concept of Reservoir Computing might form the basis of such a methodology. In a limited amount of time, a reservoir system that can recognize the elementary sounds of continuous speech has been built. The system already achieves a state-of-the-art performance, and there is evidence that the margin for further improvements is still significant.