This paper develops a novel mathematical framework for collaborative learning by means of geometrically inspired kernel machines which includes statements on the bounds of generalisation and approximation errors, and sample complexity. For classification problems, this approach allows us to learn bounded geometric structures around given data points and hence solve the global model learning problem in an efficient way by exploiting convexity properties of the related optimisation problem in a Reproducing Kernel Hilbert Space (RKHS). In this way, we can reduce classification problems to determining the closest bounded geometric structure from a given data point. Further advantages that come with our solution is that our approach does not require clients to perform multiple epochs of local optimisation using stochastic gradient descent, nor require rounds of communication between client/server for optimising the global model. We highlight that numerous experiments have shown that the proposed method is a competitive alternative to the state-of-the-art.

This paper develops a novel mathematical framework for collaborative learning by means of geometrically inspired kernel machines which includes statements on the bounds of generalisation and approximation errors, and sample complexity. For classification problems, this approach allows us to learn bounded geometric structures around given data points and hence solve the global model learning problem in an efficient way by exploiting convexity properties of the related optimisation problem in a Reproducing Kernel Hilbert Space (RKHS). In this way, we can reduce classification problems to determining the closest bounded geometric structure from a given data point. Further advantages that come with our solution is that our approach does not require clients to perform multiple epochs of local optimisation using stochastic gradient descent, nor require rounds of communication between client/server for optimising the global model. We highlight that numerous experiments have shown that the proposed method is a competitive alternative to the state-of-the-art.

Privacy-utility tradeoff remains as one of the fundamental issues of differentially private machine learning. This paper introduces a geometrically inspired kernel-based approach to mitigate the accuracy-loss issue in classification. In this approach, a representation of the affine hull of given data points is learned in Reproducing Kernel Hilbert Spaces (RKHS). This leads to a novel distance measure that hides privacy-sensitive information about individual data points and improves the privacy-utility tradeoff via significantly reducing the risk of membership inference attacks. The effectiveness of the approach is demonstrated through experiments on MNIST dataset, Freiburg groceries dataset, and a real biomedical dataset. It is verified that the approach remains computationally practical. The application of the approach to federated learning is considered and it is observed that the accuracy-loss due to data being distributed is either marginal or not significantly high.

The concept of Kernel Affine Hull Machine (KAHM) was recently introduced for representing data via learning in Reproducing Kernel Hilbert Spaces.KAHM defines a bounded geometric body in data space such that a distance measure from the geometric body can be used to aggregate local KAHM-based models to build a global model.This study leverages KAHMs for secure federated learning where data is protected from an aggressive aggregator by fully homomorphic encryption.An accurate and computationally efficient federated learning architecture, that combines local KAHMs-based classifiers in a robust and flexible manner such that the global model can be homomorphically evaluated in an efficient manner, is provided. * The research reported in this paper has been supported by the Austrian Research Promotion Agency (FFG) COMET-Modul S3AI; FFG Grant SMiLe; BMK, BMAW, and the State of Upper

In this article, we consider the problem of privacy-preserving distributed deep learning where data privacy is protected by fully homomorphic encryption. The aim is to develop a method for practical and scalable distributed deep learning with fully homomorphic encrypted data. The method must address the issue arising from the large computational cost associated with fully homomorphic encrypted data to offer a practical and scalable solution. An approach that leverages fuzzy-based membership mappings for data representation learning is considered for distributed deep learning with fully homomorphic encrypted data. The method introduces globally convergent and robust variational membership mappings to build local deep models. The local models are combined in a robust and flexible manner by means of fuzzy attributes to build a global model such that the global model can be homomorphically evaluated in an efficient manner. The membership-mappings-based privacy-preserving distributed deep learning method is accurate, practical, and scalable. This is verified through numerous experiments that include demonstrations using MNIST and Freiburg Groceries datasets, and a biomedical application related to the detection of mental stress on individuals. This study develops globally convergent and robust variational membership mappings for their application to accurate, practical, and scalable privacy-preserving distributed deep learning.

This article presents an explainable fuzzy theoretic nonparametric deep model for an analysis of heart rate variability in application to stress assessment. We are concerned with the development of a model that evaluates and explains a short-time (3–5 min long) heartbeat interval sequence of an individual to estimate the level of acute perceived stress on a numerical scale from 0 to 100 via monitoring the functioning of the autonomic nervous system. The salient features of the approach are the following. 1) A deep model, consisting of a nested composition of mappings, discovers layers of increasingly abstract heartbeat interval data representation. 2) An analytical solution of the deep model's learning problem facilitates inducing a mapping from the noninterpretable heartbeat-interval-data-space onto anotherinterpretabledomain spanned by a stress index. A given noninterpretable R– R interval feature vector is explained by: 1) estimating the corresponding stress value; 2) providing the weights which must be assigned to the subjective ratings of stress; and 3) providing various information about the sympathetic and parasympathetic activities of autonomic nervous system by analyzing R-R interval sequence in frequency domain at different abstraction levels. The proof-of-concept is provided by experimentation on a previously studied dataset of 50 subjects and a new dataset of 100 subjects.

The application of fuzzy theory to deep learning is limited,1) under the realm of deep neural networks, 2) to the parametric form of modeling, and 3) relying on gradient-descent-based numerical algorithms for optimization because of lack of analytical solutions. This article fills this gap by providing an analytical nonparametric deep modeling solution based on the mathematical analysis of membership functions assigned to model variables. The nonparametric approach is based on the concept of representing the unknown mappings (between input and output variables) through a fuzzy set with Student-t type membership function, such that the dimension of membership function increases with an increasing data size. This concept of function representation is referred to as Student-t fuzzy-mapping in this article. The most significant feature of this article is to analytically derive the mathematical expressions for membership functions (which quantify uncertainties regarding the values of variables) using variational optimization such that the degree-of-belongingness of given data to the considered datamodel is maximized. This article focuses on the modeling of image features, where a layer of the deep-model first projects the feature vector onto a lower dimensional subspace, and then construct the output feature vector through Student-t fuzzy-mappings. Numerous image classification experiments are provided to support the proposed approach.

There are numerous applications across all the spectrum of scientific areas that demand the mathematical study of signals/data. The two typical study areas of theoretical research on signal/data processing are of modeling (i.e., understanding of signal's behavior) and of analysis (i.e., evaluation of given signal for finding its association to existing signal models). The objective of this paper is to provide a stochastic framework to design both fuzzy filtering and analysis algorithms in a unified manner. The signals are modeled via linear-in-parameters models (e.g., a type of Takagi-Sugeno fuzzy model) based on variational Bayes (VB) methodology. This gives rise to the “negative free energy maximizing” filtering algorithm. The issue of intractability was handled first by carefully choosing the priors as conjugate to the likelihood and then by using Stirling approximation for the Gamma function. This paper highlighted that it was analytically possible to maximize the information theoretic quantity, “mutual information,” exactly in the same manner as maximizing “negative free energy” in VB methodology. This gives rise to the “variational information maximizing” analysis algorithm. The robustness of the methodology against data outliers is achieved by modeling the noises with Student-t distributions. The framework takes into account the inputs noises as well apart from the usually considered output noise. The robustness of the adaptive filtering algorithm against noise is shown by a deterministic analysis where an upper bound on the magnitude of estimation errors is derived.

The fuzzy-membership-function-based local image descriptors are introduced as a competing alternative to widely accepted histogram-based image descriptors. The fuzzy membership descriptors are highly distinctive and, thus, facilitate an accurate image matching. This study utilizes fuzzy membership descriptors to design a method meant for image matching. The method finds the correspondence between the two images. The study also introduces a Gamma mixture fuzzy model to detect geometrically consistent correspondence between the two images. The Gamma mixture fuzzy model combines a finite number of Gamma distributions through a fuzzy model. The parameters of the Gamma mixture fuzzy model are inferred by a method similar to the variational Bayes. The experimental studies support the claim of fuzzy membership descriptors being highly distinctive. The method was also applied to 2-D ear images for an automated human identification. The experimental results achieved the rank-1 recognition accuracy of 97.5659% on a database of 125 subjects containing 493 ear images. The motivation of this study is derived from the application potential of fuzzy membership functions in characterizing the local image features.

The unique identification of children is crucial for information technology supported vaccine delivery to the unprivileged population of third world countries. New robust image matching algorithms are required to match two ear photographs taken under nonstandard real-world conditions such as the presence of unwanted background objects in the photographs. This paper applies stochastic fuzzy models to the robust matching of ear images. The local features of the image regions are extracted using a “force-field-like” transformation. The extracted features of an image region are modeled by a stochastic fuzzy system. A region of an image is matched to a region of another image by matching the features of an image's region with the model of another image's region. As the model is fuzzy as well as stochastic, a robust matching of features' data to a model is facilitated by handling any uncertainties arising from fuzziness and randomness of the image features. The study introduces an information-theoretic index for measuring the degree of matching between image features and a model of the features. Several experiments are performed on a database of 750 ear-photographs of children (0-6 years) to justify the novel stochastic fuzzy image matching method.

There are numerous applications across all the spectrum of scientific areas that demand the mathematical study of signals/data. The two typical study areas of theoretical research on signal/data processing are of modeling (i.e., understanding of signal's behavior) and of analysis (i.e., evaluation of given signal for finding its association to existing signal models). The objective of this paper is to provide a stochastic framework to design both fuzzy filtering and analysis algorithms in a unified manner. The signals are modeled via linear-in-parameters models (e.g., a type of Takagi-Sugeno fuzzy model) based on variational Bayes (VB) methodology. This gives rise to the "negative free energy maximizing" filtering algorithm. The issue of intractability was handled first by carefully choosing the priors as conjugate to the likelihood and then by using Stirling approximation for the Gamma function. This paper highlighted that it was analytically possible to maximize the information theoretic quantity, "mutual information," exactly in the same manner as maximizing "negative free energy" in VB methodology. This gives rise to the "variational information maximizing" analysis algorithm. The robustness of the methodology against data outliers is achieved by modeling the noises with Student- t distributions. The framework takes into account the inputs noises as well apart from the usually considered output noise. The robustness of the adaptive filtering algorithm against noise is shown by a deterministic analysis where an upper bound on the magnitude of estimation errors is derived.

Stationary fuzzy Fokker-Planck learning (SFFPL) is a recently introduced computational method that applies fuzzy modeling to solve optimization problems. This study develops a concept of applying SFFPL-based computations for nonlinear constrained optimization. We consider the development of SFFPL-based optimization algorithms which do not require derivatives of the objective function and of the constraints. The sequential penalty approach was used to handle the inequality constraints. It was proved under some standard assumptions that the carefully designed SFFPL-based algorithms converge asymptotically to the stationary points. The convergence proofs follow a simple mathematical approach and invoke mean-value theorem. The algorithms were evaluated on the test problems with the number of variables up to 50. The performance comparison of the proposed algorithms with some of the standard optimization algorithms further justifies our approach. The SFFPL-based optimization approach, due to its novelty, could possibly be extended to several research directions.

Quantifying stress levels of an individual based on a mathematical analysis of real-time physiological data measurements is challenging. This study suggests a stochastic fuzzy analysis method to evaluate the short time series of R-R intervals (time intervals between consecutive heart beats) for a quantification of the stress level. The 5-min-long series of R-R intervals recorded under a given stress level are modeled by a stochastic fuzzy system. The stochastic model of heartbeat intervals is individual specific and corresponds to a particular stress level. Once the different heartbeat interval models are available for an individual, an analysis of the given R-R interval series generated under an unknown stress level is performed by a stochastic interpolation of the models. The stress estimation method has been implemented in a mobile telemedical application employing an e-health system for an efficient and cost-effective monitoring of patients while at home or at work. The experiments involve 50 individuals whose stress scores were assessed at different times of the day. The subjective rating scores showed a high correlation with the values predicted by the proposed analysis method.

The application of nonlinear optimization to the estimation of fuzzy model parameters is well known. To do the reverse of this, the concept of stationary fuzzy Fokker-Planck learning (SFFPL) is introduced, i.e., SFFPL applies the fuzzy modeling technique in nonlinear optimization problems. SFFPL is based on the fuzzy approximation of the stationary cumulative distribution function of a stochastic search process associated with the nonlinear optimization problem. A carefully designed algorithm is suggested for SFFPL to locate the optimum point. This paper also considers the variational Bayes (VB)-based inference of a stochastic fuzzy filter whose consequents, as well as antecedents, are random variables. The problem of VB inference of stochastic antecedents, because of the nonlinearity of the likelihood function, is analytically intractable. The SFFPL algorithm for high-dimensional nonlinear optimization that does not require the derivative of the objective function can be used to numerically solve the stochastic fuzzy filtering problem.

This study, under the variational Bayes (VB) framework, infers the parameters of a Takagi-Sugeno fuzzy filter having deterministic antecedents and stochastic consequents. The aim of this study is to take advantages of the VB framework to design fuzzy-filtering algorithms, which include an automated regularization, incorporation of statistical noise models, and model-comparison capability. The VB method can be easily applied to the linear-in-parameters models. This paper applies the VB method to the nonlinear fuzzy filters without using Taylor expansion for a linear approximation of some nonlinear function. It is assumed that the nonlinear parameters (i.e., antecedents) of the fuzzy filter are deterministic, while linear parameters are stochastic. The VB algorithm, by maximizing a strict lower bound on the data evidence, makes the approximate posterior of linear parameters as close to the true posterior as possible. The nonlinear deterministic parameters are tuned in a way to further increase the lower bound on data evidence. The VB paradigm can be used to design an algorithm that automatically selects the most-suitable fuzzy filter out of the considered finite set of fuzzy filters. This is done by fitting the observed data as a stochastic combination of the different Takagi-Sugeno fuzzy filters such that the individual filters compete with one another to model the data.

This study suggests the use of fuzzy-filtering algorithms to deal with the uncertainties associated to the analysis of physiological signals. The signal characteristics, for a given situation or physiological state, vary for an individual over time and also vary among the individuals with the same state. These random variations are due to the several factors related to the physiological behavior of individuals, which cannot be taken into account in the interpretation of signal characteristics. Our approach is to reduce the effect of random variations on the analysis of signal characteristics via filtering out randomness or uncertainty from the signal using a nonlinear fuzzy filter. A fuzzy-filtering algorithm, which is based on a modification of filtering algorithm of Kumaret al.[M. Kumar, N. Stoll, and R. Stoll,IEEE Trans. Fuzzy Syst., vol. 17, no. 1, pp. 150-166, Feb. 2009], is proposed for an improved performance. The method is illustrated by studying the effect of head-up tilting on the heart-rate signal of 40 healthy subjects.

This study derives a class of filtering algorithms for Takagi-Sugeno fuzzy models via solving a nonlinear parameters estimation problem. The considered estimation problem is related to the problem of minimizing the expected value of the exponential of filtering errors energy. Under some stochastic assumptions, the filtering criteria (which involve an expectation operator) are replaced by the deterministic quadratic optimization problems whose solutions provide a class of fuzzy filtering algorithms. From a viewpoint of errors in the estimation of linear parameters of the fuzzy filter, the derived filtering algorithms were analyzed with emphasis on stability, robustness, and steady-state error issues. The stability and robustness analyses have been made deterministically without making any assumption.

Many real-world applications involve the filtering and estimation of process variables. This study considers the use of interpretable Sugeno-type fuzzy models for adaptive filtering. Our aim in this study is to provide different adaptive fuzzy filtering algorithms in a deterministic setting. The algorithms are derived and studied in a unified way without making any assumptions on the nature of signals (i.e., process variables). The study extends, in a common framework, the adaptive filtering algorithms (usually studied in signal processing literature) andp-norm algorithms (usually studied in machine learning literature) to semilinear fuzzy models. A mathematical framework is provided that allows the development and an analysis of the adaptive fuzzy filtering algorithms. We study a class of nonlinear LMS-like algorithms for the online estimation of fuzzy model parameters. A generalization of the algorithms to thep-norm is provided using Bregman divergences (a standard tool for online machine learning algorithms).

This paper deals with the development of a computer model to estimate the subjective workload score of individuals by evaluating their heart-rate (HR) signals. The identification of a model to estimate the subjective workload score of individuals under different workload situations is too ambitious a task because different individuals (due to different body conditions, emotional states, age, gender, etc.) show different physiological responses (assessed by evaluating the HR signal) under different workload situations. This is equivalent to saying that the mathematical mappings between physiological parameters and the workload score are uncertain. Our approach to deal with the uncertainties in a workload-modeling problem consists of the following steps: 1) The uncertainties arising due the individual variations in identifying a common model valid for all the individuals are filtered out using a fuzzy filter; 2) stochastic modeling of the uncertainties (provided by the fuzzy filter) use finite-mixture models and utilize this information regarding uncertainties for identifying the structure and initial parameters of a workload model; and 3) finally, the workload model parameters for an individual are identified in an online scenario using machine learning algorithms. The contribution of this paper is to propose, with a mathematical analysis, a fuzzy-based modeling technique that first filters out the uncertainties from the modeling problem, analyzes the uncertainties statistically using finite-mixture modeling, and, finally, utilizes the information about uncertainties for adapting the workload model to an individual's physiological conditions. The approach of this paper, demonstrated with the real-world medical data of 11 subjects, provides a fuzzy-based tool useful for modeling in the presence of uncertainties.

Mental stress is accompanied by dynamic changes in autonomic nervous system (ANS) activity. Heart rate variability (HRV) analysis is a popular tool for assessing the activities of autonomic nervous system. This paper presents a novel method of HRV analysis for mental stress assessment using fuzzy clustering and robust identification techniques. The approach consists of 1) online monitoring of heart rate signals, 2) signal processing (e.g., using the continuous wavelet transform to extract the local features of HRV in time-frequency domain), 3) exploiting fuzzy clustering and fuzzy identification techniques to render robustness in HRV analysis against uncertainties due to individual variations, and 4) monitoring the functioning of autonomic nervous system under different stress conditions. Our experiments involved 38 physically fit subjects (26 male, 12 female, aged 18-29 years) in air traffic control task simulations. The subjective rating scores of mental workload were assessed using NASA task load index. Fuzzy clustering methods have been used to model the experimental data. Further, a robust fuzzy identification technique has been used to handle the uncertainties due to individual variations for the assessment of mental stress.

This study, for any unknown physical process y=f(x1,...,xn), is concerned with the: 1) fuzzy partition of n-dimensional input space X=X1timesmiddotmiddotmiddottimesXninto K different clusters, 2) estimating the process behavior ycirc=f(xcirc) for a given input xcirc=(xcirc1,middotmiddotmiddot,xcircn)isinX, and 3) fuzzy approximation of the process, with uncertain input-output identification data {(x(k)plusmndeltaxk),(y(k)plusmnvk)}k=1,..., using a Sugeno type fuzzy inference system. A unified min-max approach (that attempts to minimize the worst-case effect of data uncertainties and modeling errors on estimation performance), is suggested to provide robustness against data uncertainties and modeling errors. The proposed method of min-max fuzzy parameters estimation does not make any assumption and does not require a priori knowledge of upper bounds, statistics, and distribution of data uncertainties and modeling errors. To show the feasibility of the approach, simulation studies and a real-world application of physical fitness classification based on the fuzzy interpretation of physiological parameters, have been provided

We believe that nonlinear fuzzy filtering techniques may be turned out to give better robustness performance than the existing linear methods of estimation (H/sup 2/ and H/sup /spl infin// filtering techniques), because of the fact that not only linear parameters (consequents), but also the nonlinear parameters (membership functions) attempt to identify the uncertain behavior of the unknown system. However, the fuzzy identification methods must be robust to data uncertainties and modeling errors to ensure that the fuzzy approximation of unknown system's behavior is optimal in some sense. This study presents a deterministic approach to the robust design of fuzzy models in the presence of unknown but finite uncertainties in the identification data. We consider online identification of an interpretable fuzzy model, based on the robust solution of a regularized least-squares fuzzy parameters estimation problem. The aim is to resolve the difficulties associated with the robust fuzzy identification method due to lack of a priori knowledge about upper bounds on the data uncertainties. The study derives an optimal level of regularization that should be provided to ensure the robustness of fuzzy identification strategy by achieving an upper bound on the value of energy gain from data uncertainties and modeling errors to the estimation errors. A time-domain feedback analysis of the proposed identification approach is carried out with emphasis on stability, robustness, and steady-state issues. The simulation studies are provided to show the superiority of the proposed fuzzy estimation over the classical estimation methods.

This study is concerned with the adaptive learning of an interpretable Sugeno-type fuzzy inference system, in a deterministic framework, in the presence of data uncertainties and modeling errors. The authors explore the use of Hinfinity estimation theory and least squares estimation for online learning of membership functions and consequent parameters without making any assumption and requiring a priori knowledge of upper bounds, statistics, and distribution of data uncertainties and modeling errors. The issues of data uncertainties, modeling errors, and time variations have been considered mathematically in a sensible way. The proposed robust approach to the adaptive learning of fuzzy models has been illustrated through the examples of adaptive system identification, time-series prediction, and estimation of an uncertain process.