Computational sleep scoring from multimodal neurophysiological time-series (polysomnography PSG) has achieved impressive clinical success. Models that use only a single electroencephalographic (EEG) channel from PSG have not yet received the same clinical recognition, since they lack Rapid Eye Movement (REM) scoring quality. The question whether this lack can be remedied at all remains an important one. We conjecture that predominant Long Short-Term Memory (LSTM) models do not adequately represent distant REM EEG segments (termed epochs), since LSTMs compress these to a fixed-size vector from separate past and future sequences. To this end, we introduce the EEG representation model ENGELBERT (electroEncephaloGraphic Epoch Local Bidirectional Encoder Representations from Transformer). It jointly attends to multiple EEG epochs from both past and future. Compared to typical token sequences in language, for which attention models have originally been conceived, overnight EEG sequences easily span more than 1000 30 s epochs. Local attention on overlapping windows reduces the critical quadratic computational complexity to linear, enabling versatile sub-one-hour to all-day scoring. ENGELBERT is at least one order of magnitude smaller than established LSTM models and is easy to train from scratch in a single phase. It surpassed state-of-the-art macro F1-scores in 3 single-EEG sleep scoring experiments. REM F1-scores were pushed to at least 86%. ENGELBERT virtually closed the gap to PSG-based methods from 4-5 percentage points (pp) to less than 1 pp F1-score.

Skin cancer is among the most common cancer types. Dermoscopic image analysis improves the diagnostic accuracy for detection of malignant melanoma and other pigmented skin lesions when compared to unaided visual inspection. Hence, computer-based methods to support medical experts in the diagnostic procedure are of great interest. Fine-tuning pre-trained convolutional neural networks (CNNs) has been shown to work well for skin lesion classification. Pre-trained CNNs are typically trained with natural images of a fixed image size significantly smaller than captured skin lesion images and consequently dermoscopic images are downsampled for fine-tuning. However, useful medical information may be lost during this transformation. In this paper, we explore the effect of input image size on skin lesion classification performance of fine-tuned CNNs. For this, we resize dermoscopic images to different resolutions, ranging from 64 × 64 to 768 × 768 pixels and investigate the resulting classification performance of three well-established CNNs, namely DenseNet-121, ResNet-18, and ResNet-50. Our results show that using very small images (of size 64 × 64 pixels) degrades the classification performance, while images of size 128 × 128 pixels and above support good performance with larger image sizes leading to slightly improved classification. We further propose a novel fusion approach based on a three-level ensemble strategy that exploits multiple fine-tuned networks trained with dermoscopic images at various sizes. When applied on the ISIC 2017 skin lesion classification challenge, our fusion approach yields an area under the receiver operating characteristic curve of 89.2% and 96.6% for melanoma classification and seborrheic keratosis classification, respectively, outperforming state-of-the-art algorithms.

We use neural networks (NN) as a tool for a nonlinear autoregression to predict the second moment of the conditional density of return series. The NN models are compared to the popular econometric GARCH(1,1) model. We estimate the models in a Bayesian framework using Markov chain Monte Carlo posterior simulations. The interlinked aspects of the proposed Bayesian methodology are identification of NN hidden units and treatment of NN complexity based on model evidence. The empirical study includes the application of the designed strategy to market data, where we found a strong support for a nonlinear multilayer perceptron model with two hidden units.

This work is about the relevance of Gibson's concept of affordances for visual perception in interactive and autonomous robotic systems. In extension to existing functional views on visual feature representations, we identify the importance of learning in perceptual cueing for the anticipation of opportunities for interaction of robotic agents. We investigate how the originally defined representational concept for the perception of affordances - in terms of using either optical flow or heuristically determined 3D features of perceptual entities should be generalized to using arbitrary visual feature representations. In this context we demonstrate the learning of causal relationships between visual cues and predictable interactions, and emphasize on a novel framework for cueing and hypothesis verification of affordances that could play an important role in future robot control architectures. We argue that affordance based perception should enable systems to react to environment stimuli both more efficient and autonomous, and provide a potential to plan on the basis of responses to more complex perceptual configurations. We verify the concept with a concrete implementation applying state-of-the-art visual descriptors and regions of interest within a simulated robot scenario and prove that these features were successfully selected for predicting opportunities of robot interaction

One of the central goals in finance is to find better models for pricing and hedging financial derivatives such as call and put options. We present a new semi-nonparametric approach to risk-neutral density extraction from option prices, which is based on an extension of the concept of mixture density networks. The central idea is to model the shape of the risk-neutral density in a flexible, nonlinear way as a function of the time horizon. Thereby, stylized facts such as negative skewness and excess kurtosis are captured. The approach is applied to a very large set of intraday options data on the FTSE 100 recorded at LIFFE. It is shown to yield significantly better results in terms of out-of-sample pricing accuracy in comparison to the basic and an extended Black-Scholes model. It is also significantly better than a more elaborate GARCH option pricing model which includes a time-dependent volatility process. From the perspective of risk management, the extracted risk-neutral densities provide valuable information for value-at-risk estimations.

We simulate daily trading of straddles on financial indexes. The straddles are traded based on predictions of daily volatility differences in the indexes. The main predictive models studied are recurrent neural nets (RNN). Such applications have often been studied in isolation. However, due to the special character of daily financial time-series, it is difficult to make full use of RNN representational power. Recurrent networks either tend to overestimate noisy data, or behave like finite-memory sources with shallow memory; they hardly beat classical fixed-order Markov models. To overcome data nonstationarity, we use a special technique that combines sophisticated models fitted on a larger data set, with a fixed set of simple-minded symbolic predictors using only recent inputs. Finally, we compare our predictors with the GARCH family of econometric models designed to capture time-dependent volatility structure in financial returns. GARCH models have been used to trade volatility. Experimental results show that while GARCH models cannot generate any significantly positive profit, by careful use of recurrent networks or Markov models, the market makers can generate a statistically significant excess profit, but then there is no reason to prefer RNN over much more simple and straightforward Markov models. We argue that any report containing RNN results on financial tasks should be accompanied by results achieved by simple finite-memory sources combined with simple techniques to fight nonstationarity in the data.

There are two notions of volatility in literature: historical volatility and implied volatility. We concentrate on the latter by analyzing the profitability of a pure volatility trading strategy which is delta-neutral and independent of an option pricing model, for the German stock index DAX. Several very different methods ranging from linear and nonlinear, real-valued models to symbolic models of volatility changes are applied to predict the change in volatility to the next trading day and to gain profits by buying or selling straddles accordingly. The trading performance is evaluated for one historical and one implied volatility measure. The results are carefully evaluated concerning transaction costs, stationarity issues, and statistical significance. The main contribution of the paper is that, for the first time, the trading performance of models based on different modelling paradigms is compared.

We report about an automatic continuous sleep stager which is based on probabilistic principles employing hidden Markov models (HMMs). Our sleep stager offers the advantage of being objective by not relying on human scorers, having much finer temporal resolution (1 second instead of 30 second): and being based on solid probabilistic principles rather than a predefined set of rules. Results obtained for nine whole night sleep recordings are reported.

Since the seminal works of R.F. Engle (1982) and T. Bollerslev (1986) about heteroskedastic return series models, many extensions of their (G)ARCH models have been proposed in the literature. In particular, the functional dependence of conditional variances and the shape of the conditional distribution of returns have been varied in several ways (A.K. Bera and M.L. Higgins, 1993; T. Bollerslev et al., 1992). These two issues have been addressed by the neural network community using multi-layer perceptrons (MLPs) and mixture density networks (MDNs). We extend the concept of MDNs in a recurrent way to allow for "GARCH effects". These recurrent MDNs (RMDNs) offer a consistent framework to analyze the impact of nonlinearity and of non Gaussian (leptokurtic) conditional distributions on the explanatory power of volatility models. We present numerical experiments on a very large return data set, the size of which allows one to perform detailed statistical tests to compare the obtained results. In summary, conditional non Gaussian distributions (fat tails in the conditional distributions) tend to be more important than nonlinear specifications for conditional means and variances in the likelihood framework. With respect to other error measures however, the application of nonlinear neural networks seems to be promising. We think that the choice of a particular model for predicting volatility is closely related to the question of how to measure the prediction performance of a model.

In this paper we report the application of multilayer perceptrons MLPs to three important tasks in the control of rotary blood pumps, namely the estimation of left atrium pressure and the indication of suction as well as danger of suction, in the left atrium. In order to select those tasks for which MLPs as nonlinear classifiers are truly appropriate, conventional techniques for statistical data analysis, such as principal component analysis PCA , were applied. Eleven parameters derived from actual measurements during the performance of the pump were available as input for the three tasks.

Abstract

The text-to-speech system GRAPHON synthesizes speech from German text in two steps: first, a grapheme-to-phoneme conversion program derives the correct pronunciation by applying a set of about 600 rules. A currently developed morphological analysis component is expected to yield proper decomposition and pronunciation of compounds, assignment of word stress and extraction of syntax related information required to generate sentence intonation. The second step is phonetic-acoustic synthesis. A simplified articulatory model is used to control a refined digital vocal tract synthesizer. Operation of the whole text-to-speech system should be real-time.