To tackle the technical challenges of precisely aligning and inserting test tubes into sampling needles, this paper proposes an autonomous operation method for blood gas analyzers based on reinforcement learning. To simplify the complexity of the alignment and insertion task, it is decomposed into two independent subtasks, which are learned in a distributed manner and executed sequentially to complete the overall needle insertion process. To enable accurate perception of dynamic environmental states, a reinforcement learning model is developed that defines the state space, action space, and multi-task reward functions. Building on this framework, we enhance the Actor-Critic structure of the Proximal Policy Optimization (PPO) algorithm by introducing a dual-network architecture that integrates Long Short-Term Memory (LSTM) and Wavelet Transform Convolution (WTC) networks. This enhancement significantly improves both learning efficiency and policy stability. Extensive comparative experiments demonstrate that the proposed LSTM-WTC-PPO algorithm achieves a high success rate, stable convergence, and efficient policy optimization.

Semantic representation of actions is essential for the development of mutual cognition towards efficient human-robot collaboration. As such, it has become an exciting emerging venue for robots to understand human intentions with Transformers, as a family of highly expressive models. However, due to high computational complexity, local redundancy in video frames can add up to inference latency of Transformers. To address this issue, we propose a prototype-aware token pruning method, namely ProtoPrune, to select important features for efficient action recognition. Specifically, for a video sequence, we first encode knowledge encapsulated in keyframes as prototype representations with a pretrained Transformer. Next, these prototypes teach the pretrained Transformer to preserve important visual features, pruning task-irrelevant tokens for improved throughput. In our experiments, the probabilistic token pruning method reduces over 37% of GFLOPs, with a performance retention rate of 92.9% without retraining ViT backbones. Our visualization showcases the improved robustness of learned action representations.

Leveraging large language models (LLMs) to fuse heterogeneous knowledge is an exciting emerging field. However, with billions of parameters, these pretrained language models are prohibitively computationally expensive at inference time. Token sparsification methods can proactively accelerate inference by selecting important features from the sequence but often require task-dependent retraining. To address this, we propose Bilevel Token prUniNg wiTh Infused kNowledGe (Bunting), an interpretable token pruning method that leverages task-level knowledge encoded in prefixes to guide token sparsification, eliminating the need for task-specific retraining. Bunting performs Bayesian Token Sparsification, where the inner loop learns a joint representation to perform the task, and the outer loop learns adaptive attention masks for sparse representations, thus pruning redundant tokens layer-by-layer without compromising the pretrained abilities of LLMs. Additionally, we introduce an innovative antiphrasis evaluation protocol to test model adaptivity on rhetorical relations. Furthermore, we demonstrate that precomputed prefixes can effectively guide token sparsification in different knowledge-intensive tasks, maintaining task-level knowledge to identify important tokens and reduce the finetuning burden. Experimental results demonstrate that our method achieves over $0.3 x$ wall-clock speed-up with only $0.14 \times$ learnable parameters in knowledge-intensive tasks. Our findings suggest that token pruning can improve out-of-distribution detection, with sarcasm being more challenging to detect than immorality.

This paper addresses the problem of reconstructing image sequences from a rolling shutter camera-based thermal image acquisition system that integrates the image field over an exposure time. It proposes a novel approach to extend the distributed Kalman filtering framework for intra-frame reconstruction. This accounts for the exposure effect through a state-augmentation model, while respecting the row-byrow image acquisition process. Additionally, two alternative rolling shutter scan strategies: interlaced-x and random, are explored to mitigate delays in observing abrupt changes inherent in sequential rolling shutter scans. Simulation results demonstrate that the proposed approach effectively accommodates exposure and achieves reliable intra-frame reconstruction quality. The interlaced-x scan strategy, with x equal to the size of the image partition block, emerges as the preferred choice, highlighting improved performance in recovering from sudden events. The augmented distributed Kalman filter offers a scalable solution to enhance temporal resolution and overall reliability of thermal imaging of dynamic thermal processes.

Dynamic Bayesian Networks (DBNs) are useful tools for modelling complex systems whose network representations can be elicited a priori or learnt from data. In this paper, a maximum likelihood Doubly-Iterative Expectation Maximization (DI-EM) Algorithm is developed for the identification of grey-box ARMAX state-space model representations of DBNs involving known, noisy measurement processes. The grey-box model incorporates network dependencies among time series variables and exploits time series data of low longitudinal and high cross-sectional dimensions. A network learning procedure is developed using a score-based structure-selection method to find the underlying network of an input-driven dynamical system. By computing a finite data version of the Bayesian Information Criterion (BIC) for small sample sizes, the proposed method's performance is investigated on simulated and real-world data. The algorithm recovers the underlying ground-truth networks of simulated systems under finite data criteria with Jaccard Coefficient values of up to 0.84, and selects structures with improved weighted mean-squared error loss over a baseline black-box model fit on real-world data.

Aircraft assembly entails drilling numerous holes, often in multi-material stacks, then joining parts with fasteners fitted through the holes. There are stringent quality requirements on the holes, and assessment of hole quality is crucial to ensuring the integrity of the joints. Carbon fibre reinforced polymer (CFRP) is a commonly used material in aircraft structures due to its desirable properties. However, it is susceptible to defects not associated with metals, such as delamination and uncut fibres. While there have been multiple metrics for assessment of these defects proposed in literature, relatively few attempts to consolidate them have been seen. Furthermore, common measurement methods used for assessing these defects (e.g. 3D-microscopy) are well established, but can be time-consuming; manual interrogation of raw inspection data can also be highly subjective. To address these challenges, this paper proposes a set of combined metrics along with an automatic image processing framework for fast assessment of delamination and uncut fibre defects. The combined metric for the delamination region is aggregated from across six factors and uncut fibre uniform metrics from across five factors. The image processing framework receives grayscale images as inputs, taken from an optical coordinate measuring machine, and outputs the combined metrics along with eleven separate metrics. Experimental results, using a preexisting dataset from twenty-four holes on two workpieces from a real robotic drilling operation, are given to demonstrate the effectiveness of the proposed combined metrics and the corresponding image processing framework.

Denoising plays a vital role in freeform structured surface metrology. Traditional techniques, such as Gaussian and partial differential equation-based diffusion filters, often involve a time-consuming calibration process, particularly for complex surfaces. The main challenge lies in automating the denoising operation while accurately preserving features for varied surface textures. To address this challenge, an automatic approach PI-DnCNN based on small sample learning is presented in this paper. Denoising convolutional neural network (DnCNN) is employed as the basic architecture of this approach, due to its effectiveness in tackling mix-level Gaussian noise and adapting to small training datasets. Acknowledging the constraints of limited datasets, a novel physics-informed denoising loss function marrying filtering techniques is proposed to improve model performance. Additionally, a hybrid data augmentation strategy is developed to enhance the recognition of complex components. The paper also reports a set of experiments to demonstrate the presented approach in terms of performance over conventional techniques, enhancements with limited sample sizes, and applicability in general image denoising. The experiment results suggest that the presented approach consistently achieves higher average scores compared to traditional filters and emerges superior compared to the conventional DnCNN loss across different dataset sizes. In addition, the proposed loss also shows effectiveness in general image denoising, which suggests the robustness and universality of the approach.

Abstract This paper proposes a novel modelling framework for estimating the global potential field from trajectories of multiple sensing agents whose perception of the unknown field is subject to abrupt changes. We derive a parametrised formulation of the estimation problem by combining the jump Markov non‐linear system (JMNLS) model of agent dynamics with a basis function decomposition of the environmental field. An approximate expectation‐maximisation algorithm is employed for joint estimation of the global field and of the agent behavioural modes from observed agent trajectories. To avoid prohibitive computational costs associated with the state estimation of JMNLS, we utilise two approximation steps. First, an interacting multiple model smoother is used to account for the hybrid structure that emerges in this problem. Second, we propose two approaches to approximating the non‐linear sufficient statistics during the expectation step. This results in the maximization step being exact. The performance of the developed framework is tested on simulation examples and demonstrated on an application study in which the observed movement patterns of immune cells are utilised in quantifying the underlying chemical concentration field that governs their migration. The results showcase that the proposed framework can be readily applied to problems where agents assume several behavioural modes.

Survival analysis is a critical tool for the modeling of time-to-event data, such as life expectancy after a cancer diagnosis or optimal maintenance scheduling for complex machinery. However, current neural network models provide an imperfect solution for survival analysis as they either restrict the shape of the target probability distribution or restrict the estimation to predetermined times. As a consequence, current survival neural networks lack the ability to estimate a generic function without prior knowledge of its structure. In this article, we present the metaparametric neural network framework that encompasses the existing survival analysis methods and enables their extension to solve the aforementioned issues. This framework allows survival neural networks to satisfy the same independence of generic function estimation from the underlying data structure that characterizes their regression and classification counterparts. Furthermore, we demonstrate the application of the metaparametric framework using both simulated and large real-world datasets and show that it outperforms the current state-of-the-art methods in: 1) capturing nonlinearities and 2) identifying temporal patterns, leading to more accurate overall estimations while placing no restrictions on the underlying function structure.

In this paper, we propose two nonlinear controllers for a three-phase rectifier, and a bidirectional DC/DC boost converter respectively, to ensure voltage regulation, reactive power control and load power sharing with an inherent current-limiting capability, independently from system parameters. In contrast to the traditional approaches that use small-signal modelling, this approach takes into account the nonlinear model of the rectifier by considering the generic dq transformation, and the accurate nonlinear model of the dc/dc bidirectional converter, to demonstrate the boundedness and the current-limiting capability using Lyapunov methods and the input-to-state stability theory. This new method is based on the concept of introducing a bounded dynamic virtual resistance at the input of the rectifier, and a constant virtual resistance with a bounded dynamic virtual controllable voltage for the bidirectional converter that can be both positive and negative leading to a bidirectional power flow. Simulation results of a DC micro-grid consisting of a three-phase rectifier in parallel with a bidirectional dc/dc boost converter feeding a common load are presented to verify the effectiveness of the proposed control strategy.

MOTIVATION: Advances in analytical instrumentation towards acquiring high-resolution images of mass spectrometry constantly demand efficient approaches for data analysis. This is particularly true of time-of-flight secondary ion mass spectrometry imaging where recent advances enable acquisition of high-resolution data in multiple dimensions. In many applications, the distribution of different species from a sampled surface is spatially continuous in nature and a model that incorporates the spatial correlation across the surface would be preferable to estimations at discrete spatial locations. A key challenge here is the capability to analyse the high-resolution multidimensional data to extract relevant information reliably and efficiently. RESULTS: We propose a framework based on alternating non-negativity-constrained least squares which accounts for the spatial correlation across the sample surface. The proposed method also decouples the computational complexity of the estimation procedure from the image resolution, which significantly reduces the processing time. We evaluate the performance of the algorithm with biochemical image datasets generated from mixture of metabolites.

We present a framework for the identification of spatiotemporal linear dynamical systems. We use a state-space model representation that has the following attributes: 1) the number of spatial observation locations are decoupled from the model order; 2) the model allows for spatial heterogeneity; 3) the model representation is continuous over space; and 4) the model parameters can be identified in a simple and sparse estimation procedure. The model identification procedure we propose has four steps: 1) decomposition of the continuous spatial field using a finite set of basis functions where spatial frequency analysis is used to determine basis function width and spacing, such that the main spatial frequency contents of the underlying field can be captured; 2) initialization of states in closed form; 3) initialization of state-transition and input matrix model parameters using sparse regression-the least absolute shrinkage and selection operator method; and 4) joint state and parameter estimation using an iterative Kalman-filter/sparse-regression algorithm. To investigate the performance of the proposed algorithm we use data generated by the Kuramoto model of spatiotemporal cortical dynamics. The identification algorithm performs successfully, predicting the spatiotemporal field with high accuracy, whilst the sparse regression leads to a compact model.

We present a variational Bayesian (VB) approach for the state and parameter inference of a state-space model with point-process observations, a physiologically plausible model for signal processing of spike data. We also give the derivation of a variational smoother, as well as an efficient online filtering algorithm, which can also be used to track changes in physiological parameters. The methods are assessed on simulated data, and results are compared to expectation-maximization, as well as Monte Carlo estimation techniques, in order to evaluate the accuracy of the proposed approach. The VB filter is further assessed on a data set of taste-response neural cells, showing that the proposed approach can effectively capture dynamical changes in neural responses in real time.

MOTIVATION: Human pluripotent stem cell lines persist in culture as a heterogeneous population of SSEA3 positive and SSEA3 negative cells. Tracking individual stem cells in real time can elucidate the kinetics of cells switching between the SSEA3 positive and negative substates. However, identifying a cell's substate at all time points within a cell lineage tree is technically difficult. RESULTS: A variational Bayesian Expectation Maximization (EM) with smoothed probabilities (VBEMS) algorithm for hidden Markov trees (HMT) is proposed for incomplete tree structured data. The full posterior of the HMT parameters is determined and the underflow problems associated with previous algorithms are eliminated. Example results for the prediction of the types of cells in synthetic and real stem cell lineage trees are presented. AVAILABILITY: The Matlab code for the VBEMS algorithm is freely available at http://www.acse.dept.shef.ac.uk/repository/vbems_lineage_tree/VBEMS.ZIP CONTACT: [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

In this work, the problem of metabolic flux estimation is formulated as a problem of parameter estimation from incomplete labelling data. The expectation/conditional maximisation (ECM) algorithm is used to determined a maximum-likelihood (ML) estimate because of its simplicity and stable convergence. We propose to simplify a nonlinear inverse problem, generally numerically solved by an iterative optimisation algorithm, to a linear regression problem which is arrived at from a linear-in-the-parameter formulation during a partial optimisation process of the ECM algorithm. Three linear least square algorithms, the ordinary least squares (LS), the total least squares (TLS) and the constrained least squares (CLS), have been tested to solve the linear regression in this step. Using simulations, resulting parameter estimates and errors in flux estimation are compared and evaluated. The performance of the algorithms are investigated under two scenarios; when the labelling data are corrupted by a wide range of noise and when the labelling data are incompletely observed. Results suggest that the estimates from the ECM algorithm using CLS produce results superior to other combinations and have potential to be refined to improve its performance in metabolic flux estimation

The quantitative comprehension of a metabolic system in its dynamic state is a prerequisite for purposed strain improvement and enzymatic regulation. It is therefore crucial to accurately obtain the extracellular and intracellular metabolite concentrations in vivo in the time scale faster than typical metabolite turn-over rate. Though intracellular metabolite dynamics are addressable by latest rapid sampling technology, the measurements are not satisfactory due to the low percentage of intracellular volume to the total sample volume and often the low concentration levels of most intracellular metabolites. When the examined system is observable, a possible solution to this problem is by means of available statistical estimation approach. Hence, in this paper, the sequential Monte Carlo filter is applied to estimate the intracellular metabolite concentrations with the knowledge of extracellular metabolite concentrations. The application of this algorithm in a synthetic system with simulated data illustrates the applicability of this approach. All the intracellular metabolite concentrations are accurately estimated and the extracellular states are reconstructed from their noisy measurements. The dynamic flux distributions are also obtained and their underlying biological meanings are described

The Gaussian process latent variable model (GPLVM) is a novel unsupervised approach to nonlinear low dimensional embedding proposed by Lawrence (2005). This paper presents the development of a framework for the implementation of the GPLVM for fault detection. A series of experiments have been carried out comparing and combining the GPLVM to the conventional and widely used linear dimension reduction technique of principal component analysis (PCA). The inclusion of the GPLVM for the visualisation and data analysis, led to a considerable improvement in the classification results

Using historical data to forecast future trends in time series is a key application of data mining. This paper deals with the problem of time series forecasting using the non-parametric Gaussian process model. The time series forecasting is accomplished by using multiple Gaussian process models of each step ahead predictor in accordance with the direct approach. The separable least-squares approach is applied to train these Gaussian process models. Hyperparameters of the covariance function are coded into binary bit strings and candidate weighting parameters of the mean function corresponding to each candidate of hyperparameters are estimated by the linear least-squares method. The genetic algorithm is utilized to determine these unknown hyperparameters by minimizing the negative log marginal likelihood of the training data. Simulation results are shown to illustrate the proposed forecasting method and compared with the iterated prediction method

Abstract For industrial fault diagnostics, many model‐based fault diagnosis approaches have been proposed so far and some of them have been put into practice. However, for modern complex processes, owing to the variable nature of faults and model uncertainty, no single method can diagnose all faults and meet different contradictory criteria. In this paper, the importance of integration of different fault detection and isolation schemes in a generic problem‐solving environment is emphasized. A service‐oriented architecture for the integration is proposed, based on Grid technologies. As an engineering implementation, a decision support system for the gas turbine engine fault diagnosis is presented and some deployed services are discussed. Copyright © 2006 John Wiley & Sons, Ltd.

Metabolic flux analysis is important for metabolic system regulation and intracellular pathway identification. A popular approach for intracellular flux estimation involves using 13C tracer experiments to label states that can be measured by nuclear magnetic resonance spectrometry or gas chromatography mass spectrometry. However, the bilinear balance equations derived from 13C tracer experiments and the noisy measurements require a nonlinear optimization approach to obtain the optimal solution. In this paper, the flux quantification problem is formulated as an error-minimization problem with equality and inequality constraints through the 13C balance and stoichiometric equations. The stoichiometric constraints are transformed to a null space by singular value decomposition. Self-adaptive evolutionary algorithms are then introduced for flux quantification. The performance of the evolutionary algorithm is compared with ordinary least squares estimation by the simulation of the central pentose phosphate pathway. The proposed algorithm is also applied to the central metabolism of Corynebacterium glutamicum under lysine-producing conditions. A comparison between the results from the proposed algorithm and data from the literature is given. The complexity of a metabolic system with bidirectional reactions is also investigated by analyzing the fluctuations in the flux estimates when available measurements are varied.

MOTIVATION: Metabolic flux analysis via a (13)C tracer experiment has been achieved using a Monte Carlo method with the assumption of system noise as Gaussian noise. However, an unbiased flux analysis requires the estimation of fluxes and metabolites jointly without the restriction on the assumption of Gaussian noise. The flux distributions under such a framework can be freely obtained with various system noise and uncertainty models. RESULTS: In this paper, a stochastic generative model of the metabolic system is developed. Following this, the Markov Chain Monte Carlo (MCMC) approach is applied to flux distribution analysis. The disturbances and uncertainties in the system are simplified as truncated Gaussian multiplicative models. The performance in a real metabolic system is illustrated by the application to the central metabolism of Corynebacterium glutamicum. The flux distributions are illustrated and analyzed in order to understand the underlying flux activities in the system. AVAILABILITY: Algorithms are available upon request.

A novel and more robust implementation is proposed for sequence estimation in uncertain environments with additive non-Gaussian ambient noise and intersymbol interference. This is based on a deterministic performance index that minimizes the effect of worst-case disturbances on the estimation error. The decoder has multiple$H^infty$filters and is in the fashion of per-survivor processing with a Viterbi trellis for decoding. There is a substantial performance improvement over maximum-likelihood sequence estimation, as shown by simulation results obtained for joint channel estimation and symbol decoding in non-Gaussian channels.

Previous stability analysis of the particle swarm optimizer was restricted to the assumption that all parameters are nonrandom, in effect a deterministic particle swarm optimizer. We analyze the stability of the particle dynamics without this restrictive assumption using Lyapunov stability analysis and the concept of passive systems. Sufficient conditions for stability are derived, and an illustrative example is given. Simulation results confirm the prediction from theory that stability of the particle dynamics requires increasing the maximum value of the random parameter when the inertia factor is reduced.

Metabolic fluxes have been regarded as an important quantity for metabolic engineering as they reveal cause-effect relationships between genetic modifications and resulting changes in metabolic activity and are used as a prerequisite for the design of optimal whole cell biocatalysts. The intracellular fluxes must be estimated due to the inability to measure them directly. A particular useful technique involves the use of /sup 13/C-enriched substrates and the measurement of label distribution generated for each intermediate to uncover all unmeasured fluxes by solving the label balance equations, e.g. isotopomer balances, at steady state. However, the formation of these equations typically requires tedious algebraic manipulation and in many cases the resulting equations must be solved numerically, due to the nonlinearity and high dimensionality. Here we present three different evolutionary algorithm (EA) based approaches in combination with the least squares algorithm to show the applicability of EAs in metabolic flux quantification. The performance of the algorithms are illustrated and discussed through the simulation of the cyclic pentose phosphate network in a noisy environment and the identifiability problem is also considered.

Model-based methods are commonly used for fault diagnosis. Many model-based fault diagnosis approaches have been proposed so far. But for modern complex processes, due to the variable nature of faults and model uncertainty, no single approach can diagnose all faults and meet different contradictory criteria. In this paper, the importance of integration of different fault diagnosis schemes in a common framework is emphasised. A service-oriented architecture for the integration is proposed based on grid technologies. The preliminary implementation of this integration for the gas turbine engine fault diagnosis is discussed.

A blind adaptive multiuser detection in a MIMO system with time-varying channel gain is considered. Alamouti encoding is used for data transmission with space-time diversity and a suboptimal multiple model based adaptive algorithm is used for simultaneously tracking the channel gain and detecting the data symbols. The results obtained show the robustness of our approach in real-time detection for Rayleigh fading channels varying with time.

This paper presents an adaptive code-aided technique for the suppression of narrowband interference (NBI) in direct-sequence code-division multiple access (DS-CDMA) systems. This technique uses a multiuser detector based on the interacting multiple model (BOA) algorithm. This detector is based on the concept that the effective model of the received signal at a time instance can be approximated by one of the models in the IMM algorithm. Simulations are used to compare the performance of the proposed technique with that of the recursive least squares (RLS) version of the minimum mean-square error (MMSE) for multiuser detection.

An adaptive multiuser detector based on an interacting multiple model (IMM) is introduced to estimate the transmitted sequence corrupted by MAI (multiple access interference), multipath fading and noise. The proposed algorithm presents a novel multipath combining scheme based on a multiple model concepts and used in frequency selective Raleigh fading channels. The performance of the IMM multiuser detector is studied and compared with the adaptive per-survivor detector of X. Wang and H. Poor (see Wireless Networks, vol.4, p.453-70, 1998).

A blind sequence estimation algorithm based on an interacting multiple model (IMM) is introduced to estimate the channel and the transmitted sequence corrupted by ISI (intersymbol interference) and noise. The proposed algorithm avoids the exponential growth complexity caused by increasing channel memory length. The performance of the IMM based equalizer is studied and compared with a well known algorithm, DDFSE (delayed decision-feedback sequence estimation).

This paper presents the development of a particle filtering (PF) based method for fault detection and isolation (FDI) in stochastic nonlinear dynamic systems. The FDI problem is formulated in the multiple model (MM) environment, then by combining the likelihood ratio (LR) test with the PF, a new FDI scheme is developed. The simulation results on a highly nonlinear system are provided which demonstrate the effectiveness of the proposed method.

This paper is concerned with the adaptive control of continuous-time nonlinear dynamical systems using neural networks. A novel neural network architecture, referred to as a variable neural network, is proposed and shown to be useful in approximating the unknown nonlinearities of dynamical systems. In the variable neural networks, the number of basis functions can be either increased or decreased with time, according to specified design strategies, so that the network will not overfit or underfit the data set. Based on the Gaussian radial basis function (GRBF) variable neural network, an adaptive control scheme is presented. The location of the centers and the determination of the widths of the GRBFs in the variable neural network are analyzed to make a compromise between orthogonality and smoothness. The weight-adaptive laws developed using the Lyapunov synthesis approach guarantee the stability of the overall control scheme, even in the presence of modeling error(s). The tracking errors converge to the required accuracy through the adaptive control algorithm derived by combining the variable neural network and Lyapunov synthesis techniques. The operation of an adaptive control scheme using the variable neural network is demonstrated using two simulated examples.

An adaptive control technique, using dynamic structure Gaussian radial basis function neural networks, that grow in time according to the location of the system's state in space is presented for the affine class of nonlinear systems having unknown or partially known dynamics. The method results in a network that is "economic" in terms of network size, for cases where the state spans only a small subset of state space, by utilizing less basis functions than would have been the case if basis functions were centered on discrete locations covering the whole, relevant region of state space. Additionally, the system is augmented with sliding control so as to ensure global stability if and when the state moves outside the region of state space spanned by the basis functions, and to ensure robustness to disturbances that arise due to the network inherent approximation errors and to the fact that for limiting the network size, a minimal number of basis functions are actually being used. Adaptation laws and sliding control gains that ensure system stability in a Lyapunov sense are presented, together with techniques for determining which basis functions are to form part of the network structure. The effectiveness of the method is demonstrated by experiment simulations.

In this paper, we investigate the problem of optimal sequential learning, viewed as a problem of estimating an underlying function sequentially rather than estimating a set of parameters of the neural network. First, we arrive at a suboptimal solution to the sequential estimate that can be mapped by a growing gaussian radial basis function (GaRBF) network. This network adds hidden units for each observation. The function space approach in which the estimates are represented as vectors in a function space is used in developing a growth criterion to limit its growth. A simplification of the criterion leads to two joint criteria on the distance of the present pattern and the existing unit centers in the input space and on the approximation error of the network for the given observation to be satisfied together. This network is similar to the resource allocating network (RAN) (Platt 1991a) and hence RAN can be interpreted from a function space approach to sequential learning. Second, we present an enhancement to the RAN. The RAN either allocates a new unit based on the novelty of an observation or adapts the network parameters by the LMS algorithm. The function space interpretation of the RAN lends itself to an enhancement of the RAN in which the extended Kalman filter (EKF) algorithm is used in place of the LMS algorithm. The performance of the RAN and the enhanced network are compared in the experimental tasks of function approximation and time-series prediction demonstrating the superior performance of the enhanced network with fewer number of hidden units. The approach adopted here has led us toward the minimal network required for a sequential learning problem.

A model of dynamic complexity, a growing Gaussian radial basis function (GRBF) network, is developed by analyzing sequential learning in the function space. The criteria for adding a new basis function to the model are based on the angle formed between a new basis function and the existing basis functions and also on the prediction error. When a new basis function is not added the model parameters are adapted by the extended Kalman filter (EKF) algorithm. This model is similar to the resource allocating network (RAN) and hence this work provides an alternative interpretation to the RAN. An enhancement to the RAN is suggested where RAN is combined with EKF. The RAN and its variants are applied to the task of predicting the logistic map and the Mackey-Glass chaotic time-series, and the advantages of the enhanced model are demonstrated.>

The relationship of the F-projections adaptive algorithm to the LMS (least mean square), RLS (recursive least squares), and Kalman algorithms is investigated. A recursive form of nonlinear least squares is developed, and the conditions under which the F-projections algorithm becomes equivalent to it are established. A radial basis function neural network is used as a nonlinear model in analyzing time series under nonstationary environments. The performances of the F-projections and the extended Kalman algorithms for this nonlinear model in predicting a chaotic series and in tracking a time-varying system are compared.>

The approach is an extension of the method of radial basis functions. Parameter estimation for the nonlinear predictor is performed by a gradient descent over a mean squared error measure, starting from a random initialization of the parameters. Results on predicting segments of speech data and the sunspot series are presented and compared to a linear predictor. An approach to adaptive estimation of the model by means of an extended Kalman filter is presented. In terms of prediction residual, the nonlinear predictor is found to perform significantly better than a linear model with the same number of parameters. Difficulties in applying this model in speech processing are discussed.>