This paper develops a novel mathematical framework for collaborative learning by means of geometrically inspired kernel machines which includes statements on the bounds of generalisation and approximation errors, and sample complexity. For classification problems, this approach allows us to learn bounded geometric structures around given data points and hence solve the global model learning problem in an efficient way by exploiting convexity properties of the related optimisation problem in a Reproducing Kernel Hilbert Space (RKHS). In this way, we can reduce classification problems to determining the closest bounded geometric structure from a given data point. Further advantages that come with our solution is that our approach does not require clients to perform multiple epochs of local optimisation using stochastic gradient descent, nor require rounds of communication between client/server for optimising the global model. We highlight that numerous experiments have shown that the proposed method is a competitive alternative to the state-of-the-art.

Recent findings suggest that consecutive layers of neural networks with the ReLU activation function fold the input space during the learning process. While many works hint at this phenomenon, an approach to quantify the folding was only recently proposed by means of a space folding measure based on the Hamming distance in the discrete activation space. We generalize the space folding measure to a wider class of activation functions through the introduction of equivalence classes of input data. We then analyze its mathematical and computational properties. Lastly, we link the folding to geometry of adversarial attacks. We underpin our claims with an experimental evaluation.

Rapid development of the Industrial Internet of Things (IIoT) is turning the management of machinery and decision making into a multitude of operational data. A key issue in such an environment would be the ability to predict the failure of assets well to reduce downtimes and maximize on performance. The role of Predictive Maintenance (PdM) is significant here, although the current literature usually focuses on the development of the algorithms and ignores the consideration of technological, organizational, and industrial aspects. The paper will provide an extensive overview of AI-driven PdM in the framework of Industry 4.0 and suggest a hybrid taxonomy that will combine AI paradigms, stages of maintenance lifecycle, and industrial deployment scenarios. The taxonomy offers a multidimensional viewpoint connecting data analytics, machine learning and operational readiness as a conceptual framework to tie together scholarly research and industrial practice. The paper critically examines the existing issues such as data heterogeneity, model interpretability, and scalability and presents research gaps that impede the adoption of PdM, in particular by small and medium-sized enterprises (SMEs). This work provides a systematic basis to the development of explainable, adaptive, and interoperable PdM systems by providing engineering and computer science viewpoints. The results highlight the necessity of multidisciplinary solutions that can make the AI innovation stay relevant to the real-world maintenance plans and create resilient and smart industrial ecosystems.

Recently, Weyl’s discrepancy has been shown to be the optimum metric for threshold-based sampling. Based on this discrepancy, threshold-adaption strategies relying on the local discrepancy in the spike domain have been developed. This work proposes a low-complexity architecture allowing for a singlecycle calculation of the local discrepancy in digital hardware. By introducing a thermometer code representation, the local discrepancy can not only be calculated with low complexity tailored for a digital hardware implementation but also features inherent overflow robustness. We describe that, even when using a simple PWM-based DAC, SNDR values above 58 dB and SNR values above 40 dB for sinusoidal test cases and more than 30 dB for ECG signals can be achieved while requiring significantly less spikes compared to recent state-of-the-art works.

Spiking Neural Networks (SNNs) have garnered attention over recent years due to their increased energy efficiency and advantages in terms of operational complexity compared to traditional Artificial Neural Networks (ANNs). Two important questions when implementing SNNs are how to best encode existing data into spike trains and how to efficiently process these spike trains in hardware. This paper addresses both of these problems by incorporating the encoding into the learning process, thus allowing the network to learn the spike encoding alongside the weights. Furthermore, this paper proposes a hardware architecture based on a recently introduced differential-time representation for spike trains allowing decoupling of spike time and processing time. Together these contributions lead to a feedforward SNN using only Leaky-Integrate and Fire (LIF) neurons that surpasses 99% accuracy on the MNIST dataset while still being implementable on medium-sized FPGAs with inference times of less than 295µs.

This paper develops a novel mathematical framework for collaborative learning by means of geometrically inspired kernel machines which includes statements on the bounds of generalisation and approximation errors, and sample complexity. For classification problems, this approach allows us to learn bounded geometric structures around given data points and hence solve the global model learning problem in an efficient way by exploiting convexity properties of the related optimisation problem in a Reproducing Kernel Hilbert Space (RKHS). In this way, we can reduce classification problems to determining the closest bounded geometric structure from a given data point. Further advantages that come with our solution is that our approach does not require clients to perform multiple epochs of local optimisation using stochastic gradient descent, nor require rounds of communication between client/server for optimising the global model. We highlight that numerous experiments have shown that the proposed method is a competitive alternative to the state-of-the-art.

In order to ease the analysis of error propagation in neuromorphic computing and to get a better understanding of spiking neural networks (SNN), we address the problem of mathematical analysis of SNNs as endomorphisms that map spike trains to spike trains. A central question is the adequate structure for a space of spike trains and its implication for the design of error measurements of SNNs including time delay, threshold deviations, and the design of the reinitialization mode of the leaky-integrate-and-fire (LIF) neuron model. First, we identify the underlying topology by analyzing the closure of all sub-threshold signals of a LIF model. For zero leakage this approach yields the Alexiewicz topology, which we adopt to LIF neurons with arbitrary positive leakage. As a result, LIF can be understood as spike train quantization in the corresponding norm. This way we obtain various error bounds and inequalities such as a quasi-isometry relation between incoming and outgoing spike trains. Another result is a Lipschitz-style global upper bound for the error propagation and a related resonance-type phenomenon.

Privacy-utility tradeoff remains as one of the fundamental issues of differentially private machine learning. This paper introduces a geometrically inspired kernel-based approach to mitigate the accuracy-loss issue in classification. In this approach, a representation of the affine hull of given data points is learned in Reproducing Kernel Hilbert Spaces (RKHS). This leads to a novel distance measure that hides privacy-sensitive information about individual data points and improves the privacy-utility tradeoff via significantly reducing the risk of membership inference attacks. The effectiveness of the approach is demonstrated through experiments on MNIST dataset, Freiburg groceries dataset, and a real biomedical dataset. It is verified that the approach remains computationally practical. The application of the approach to federated learning is considered and it is observed that the accuracy-loss due to data being distributed is either marginal or not significantly high.

The concept of Kernel Affine Hull Machine (KAHM) was recently introduced for representing data via learning in Reproducing Kernel Hilbert Spaces.KAHM defines a bounded geometric body in data space such that a distance measure from the geometric body can be used to aggregate local KAHM-based models to build a global model.This study leverages KAHMs for secure federated learning where data is protected from an aggressive aggregator by fully homomorphic encryption.An accurate and computationally efficient federated learning architecture, that combines local KAHMs-based classifiers in a robust and flexible manner such that the global model can be homomorphically evaluated in an efficient manner, is provided. * The research reported in this paper has been supported by the Austrian Research Promotion Agency (FFG) COMET-Modul S3AI; FFG Grant SMiLe; BMK, BMAW, and the State of Upper

We identify tessellation-filtering ReLU neural networks that, when composed with another ReLU network, keep its non-redundant tessellation unchanged or reduce it.The additional network complexity modifies the shape of the decision surface without increasing the number of linear regions. We provide a mathematical understanding of the related additional expressiveness by means of a novel measure of shape complexity by counting deviations from convexity which results in a Boolean algebraic characterization of this special class. A local representation theorem gives rise to novel approaches for pruning and decision surface analysis.

We address the unsolved algorithm design problem of choosing a justified regularization parameter in unsupervised domain adaptation. This problem is intriguing as no labels are available in the target domain. Our approach starts with the observation that the widely-used method of minimizing the source error, penalized by a distance measure between source and target feature representations, shares characteristics with regularized ill-posed inverse problems. Regularization parameters in inverse problems are optimally chosen by the fundamental principle of balancing approximation and sampling errors. We use this principle to balance learning errors and domain distance in a target error bound. As a result, we obtain a theoretically justified rule for the choice of the regularization parameter. In contrast to the state of the art, our approach allows source and target distributions with disjoint supports. An empirical comparative study on benchmark datasets underpins the performance of our approach.

Deep learning (DL) algorithms have already proved their effectiveness on a wide variety of application domains, including speech recognition, natural language processing, and image classification. To foster their pervasive adoption in applications where low latency, privacy issues and data bandwidth are paramount, the current trend is to perform inference tasks at the edge. This requires deployment of DL algorithms on low-energy and resource-constrained computing nodes, often heterogenous and parallel, that are usually more complex to program and to manage without adequate support and experience. In this paper, we present ALOHA, an integrated tool flow that tries to facilitate the design of DL applications and their porting on embedded heterogenous architectures. The proposed tool flow aims at automating different design steps and reducing development costs. ALOHA considers hardware-related variables and security, power efficiency, and adaptivity aspects during the whole development process, from pre-training hyperparameter optimization and algorithm configuration to deployment.

We consider the problem of unsupervised domain adaptation (DA) in regression under the assumption of linear hypotheses (e.g. Beer-Lambert's law) - a task recurrently encountered in analytical chemistry. Following the ideas from the non-linear iterative partial least squares (NIPALS) method, we propose a novel algorithm that identifies a low-dimensional subspace aiming at the following two objectives: i) the projections of the source domain samples are informative w.r.t. the output variable and ii) the projected domain-specific input samples have a small covariance difference. In particular, the latent variable vectors that span this subspace are derived in closed-form by solving a constrained optimization problem for each subspace dimension adding flexibility for balancing the two objectives. We demonstrate the superiority of our approach over several state-of-the-art (SoA) methods on two typical DA scenarios involving unsupervised adaptation of multivariate calibration models between different process lines in melamine production and equality to SoA on a well-known benchmark dataset from analytical chemistry involving (unsupervised) model adaptation between different spectrometers. The former data set is provided along with this paper.

The paper focuses on similarity measures for translationally misaligned image and volumetric patterns. For measures based on standard concepts such as cross-correlation, L(p)-norm, and mutual information, monotonicity with respect to the extent of misalignment cannot be guaranteed. In this paper, we introduce a novel distance measure based on Hermann Weyl's discrepancy concept that relies on the evaluation of partial sums. In contrast to standard concepts, in this case, monotonicity, positive-definiteness, and a homogenously linear upper bound with respect to the extent of misalignment can be proven. We show that this monotonicity property is not influenced by the image's frequencies or other characteristics, which makes this new similarity measure useful for similarity-based registration, tracking, and segmentation.

Hermann Weyl's concept of a discrepancy measure is discussed in the context of time series analysis. A concept for autocorrelation based on this discrepancy notion is introduced. It is shown that in particular for high frequent signals as they, for example, are typically encountered in a financial context, the introduced autocorrelation concept stands out by a better discriminative power than its classical counterpart. While the computational complexity of this novel autocorrelation is of quadratic order in terms of the number of given time steps an approximation based on Lp-norms is introduced which can be computed by convolution, and therefore reduces the order of complexity to that of its classical counterpart. It is shown that the proposed approximation can be tuned to be arbitrarily close to the original discrepancy based version, and that it shows similar desirable behavior.

Kernels have proven useful for machine learning, data mining, and computer vision as they provide a means to derive non-linear variants of learning, optimization or classification strategies from linear ones. A central question when applying a kernel-based method is the choice and the design of the kernel function. This paper provides a novel view on kernels based on fuzzy logical concepts that allows to incorporate prior knowledge in the design process. It is demonstrated that kernels that map to the unit interval and have constantly 1 in their diagonals can be represented by a commonly used fuzzy-logical formula for representing fuzzy relations. This means that a large and important class of kernels can be represented by fuzzy logical concepts. Beside this result which only guarantees the existence of such a representation, constructive examples are presented.

Kernels are two-placed functions that can be interpreted as inner products in some Hilbert space. It is this property which makes kernels predestinated to carry linear models of learning, optimization or classification strategies over to non-linear variants. Following this idea, various kernel-based methods like support vector machines or kernel principal component analysis have been conceived which prove to be successful for machine learning, data mining and computer vision applications. When applying a kernel-based method a central question is the choice and the design of the kernel function. This paper provides a novel view on kernels based on fuzzy-logical concepts which allows to incorporate prior knowledge in the design process. It is demonstrated that kernels mapping to the unit interval with constant one in its diagonal can be represented by a commonly used fuzzy-logical formula for representing fuzzy rule bases. This means that a great class of kernels can be represented by fuzzy-logical concepts. Apart from this result, which only guarantees the existence of such a representation, constructive examples are presented and the relation to unlabeled learning is pointed out.

Mamdani (1975) controller was successfully used in many applications. One of its interpretations is that it uses a fuzzy relation as an approximation of the desirable input-output correspondence. We analyze mathematical properties of Mamdani controller and notice that it has lower computational complexity when compared to the residuum-based controller. However, we show that in standard situations, both these fuzzy controllers do not represent the rule base properly in the sense of finding a solution to the related system of fuzzy relational equations. First, we consider the premises and consequents as typical inputs and outputs, and we want their correspondence to be kept. Next, we require that each normal input produces an output that bears nontrivial information. These two conditions appear to be almost contradictory to the previous controllers. We suggest a generalization of Mamdani controller which allows us to satisfy these requirements. The theory and experiments suggest that it performs better without any change of rule base and without a substantial increase of complexity.