This article delves into the exploration of state-space methods applied to the modeling and estimation of systems with time-varying parameters. While typically existing approaches rely on the assumption that the parameters satisfy the Markov evolution and require the knowledge of the transfer matrix, this article develops an explicit autoregressive (AR) model for time-varying parameters in which the invariant matrix represents the dynamic changes in the parameters. Unlike the previous work, the state-space model is constructed by stacking the invariant matrix and time-varying parameters into the unknown state vector. Then, the joint state estimation (JSE) algorithm is deduced based on the Kalman filtering principle, aiming to reduce the dependence on the prior knowledge of the invariant matrix. Through the numerical simulation and Monte Carlo test, it is indicated that the developed algorithm maintains reliability under various random white noises. In addition, the practical estimation results with the real-time series also verify the validity.

Fault detection (process monitoring) for dynamic and nonlinear industrial processes, which can be viewed as complex systems, has become increasingly important in recent years. Under complex and dynamic operating conditions, traditional multivariate statistical process monitoring (MSPM) methods struggle to effectively capture the evolving behaviors of quality indicators at the system level. In this article, a novel variable selection-based adaptive MSPM method is proposed for quality-related process monitoring. First, an improved part mutual information (PMI) method is proposed for variable selection, categorizing process variables into quality-related and quality-unrelated groups, addressing the shortcomings of the original PMI being able to measure the correlation between only two variables. Second, a novel recursive kernel principal component regression (RKPCR) incorporating the hierarchical model order-reduction strategy is proposed to effectively track quality indicators. This method screens high-quality data for model updates based on distance-based classification and linear approximation, which mitigates model degradation caused by faulty data. In addition, a recursive kernel principal component analysis (RKPCA) method similar to RKPCR is proposed to monitor quality-unrelated faults. Finally, a numerical example and two industrial processes are used to demonstrate the performance of the proposed method.

This letter presents a state estimation algorithm for linear discrete-time systems with state-delay. In order to overcome the difficulty that the traditional Kalman filter cannot estimate the states of the systems with state-delay, a state estimation strategy is developed by combining the auxiliary model with the delta operator. Then, by constructing and minimizing the covariance matrix of the state reconstruction errors, a delta operator state estimation algorithm is derived and it can fulfill effective state estimation for the linear discrete-time system with state-delay. Moreover, the convergence proof is provided by means of stochastic stability theory. Finally, the experimental results demonstrate that the developed state estimation method is effective.

To identify a reliable model for a dynamic system with nonideal measurements, this article develops a novel parameter transfer identification (PTI) algorithm that leverages the knowledge from a heterogeneous source system. Specifically, a mapping matrix is proposed to transform source parameters into intermediate parameters with dimensions matching the target parameters. By treating the intermediate parameter and mapping matrix as latent variables, variational Bayesian (VB) inference is introduced to efficiently approximate intractable posterior distributions of all unknown parameters, with variances reflecting their uncertainty levels. A probabilistic PTI is then proposed to derive the transfer posterior conditioned on the intermediate parameters, whose analytical form is vital for carrying out VB. Based on this, a heterogeneous PTI is established under the VB framework such that variational posterior distributions for all unknown parameters can be updated iteratively. Finally, an atmospheric fermenter example verifies that the proposed algorithm can bring in model accuracy improvement as high as 60% compared with the nontransfer identification approach, when dealing with nonideal measurements.

This letter aims to facilitate the identification proce-dure for dynamic systems by utilizing knowledge from different but related systems. By introducing the transfer gain matrix and constructing the transfer identification criterion, a novel parameters-transfer identification method is developed for the system with low-quality measurements. Meanwhile, the condi-tion for avoiding negative transfer is exploited to theoretically guarantee the effectiveness of knowledge transfer. Moreover, the size and elements of the transfer gain matrix depend on all measurements, a recursive form of the proposed method is derived to overcome the curse of dimensionality. Finally, a mass-spring-damper example and a continuous fermentation reactor example are simulated to demonstrate the advantages and capabilities of the proposed methods.

To extract important information about the nonlinear signals, this letter makes the utmost of the fitting advantages of Gaussian and polynomial functions, and proposes a nonlinear signal model with broader applications. Then we focus on the parameter estimation issues of the proposed models in the presence of noises. The stability factor recursive algorithm is devised based on the increasing noisy data, which makes full use of the information from the nonlinear signals. Applying the hierarchical identification principle, a two-stage recursive algorithm with higher computational efficiency is developed for the nonlinear signals. The simulation results test the effectiveness of the proposed algorithms from the aspects of estimation accuracy and prediction effect.

Nonlinear time-series modeling is fundamental to a wide variety of control and prediction problems. This letter focuses on the joint parameter and time-delay estimation for an extended version of the nonlinear exponential autoregressive (ExpAR) time-series model. To address the difficulties posed by the unknown time-delay and improve the estimation accuracy, we first employ the redundant rule to transform the ExpAR model into an augmented identification model. Then we invoke the multi-innovation theory to enhance data utilization and propose a new algorithm that combines stochastic gradient descent with discrete search for estimating the unknown model parameters and time-delay. The simulation results show that by properly adjusting the innovation length, the estimation accuracy of the proposed multi-innovation algorithm can significantly exceed that of the single-innovation algorithm.

The previous work for the filter design considers uncorrelated white measurement noise disturbance. For more complex correlated noise disturbance, the conventional adaptive filter results in biased estimates. To overcome this problem, we introduce a linear prefilter to whiten the correlated noise (i.e., colored noise) for obtaining the unbiased estimate of the filter weight. Moreover, the design of some adaptive filters mainly focuses on the integer-order optimization methods. However, compared with the integer-order-based adaptive algorithms, the fractional-order-based algorithms show better performance. Thus, this letter develops a new gradient approach for the adaptive filter design based on the fractional-order derivative and a linear filter. Finally, the simulation results are provided from the system identification perspective for demonstrating the performance analysis of the proposed algorithms.

For 2-D causal systems, the variables depend both on time, and on spatial coordinates. This article develops two identification algorithms for two-dimensional causal systems. First, a maximum likelihood estimation algorithm is developed for two-dimensional causal systems when there is no missing data. Second, an expectation-maximization based auxiliary model algorithm, and an expectation-maximization based modified Kalman filtering and smoothing algorithm are derived for 2-D causal systems with missing outputs. It is demonstrated that the modified Kalman filtering, and smoothing algorithm is more effective for systems with missing outputs. The effectiveness of these two algorithms is verified by a simulation example.

By combining the coupling identification concept with the gradient search, this study develops a partially coupled generalised extended projection algorithm and a partially coupled generalised extended stochastic gradient algorithm to estimate the parameters of a multivariable output‐error‐like system with autoregressive moving average noise from input–output data. The key is to divide the identification model into several submodels based on the hierarchical identification principle and to establish the parameter estimation algorithm by using the coupled relationship between these submodels. The simulation test results indicate that the proposed algorithms are effective.

Radial basis function-based state-dependent autoregressive (RBF-AR) models are a class of nonlinear combined models. This letter focuses on the parameter estimation for the RBF-AR models. To overcome the estimation difficulty due to the highly nonlinear relations between the parameters and the model output, the separated idea is used to transform the original optimization problem into a quadratic and a nonlinear optimization problems. Applying the hierarchical identification principle and the multi-innovation theory, two interactive algorithms are proposed for the RBF-AR models. In addition, an approach based on data weighting is proposed to overcome the data saturation in the algorithms. The simulation results verify the effectiveness of the proposed algorithms from the aspects of parameter estimation accuracy and prediction performance.

A stochastic gradient (SG)-based particle filter (SG-PF) algorithm is developed for an ARX model with nonlinear communication output in this paper. This ARX model consists of two submodels, one is a linear ARX model and the other is a nonlinear output model. The process outputs (outputs of the linear submodel) transmitted over a communication channel are unmeasurable, while the communication outputs (outputs of the nonlinear submodel) are available, and both of the two-type outputs are contaminated by white noises. Based on the rich input data and the available communication output data, a SG-PF algorithm is proposed to estimate the unknown process outputs and parameters of the ARX model. Furthermore, a direct weight optimization method and the Epanechnikov kernel method are extended to modify the particle filter when the measurement noise is a Gaussian noise with unknown variance and the measurement noise distribution is unknown. The simulation results demonstrate that the SG-PF algorithm is effective.

The identification of the multiple model-based Hammerstein parameter varying systems is studied in this paper. The parameters of the considered systems vary as the systems perform on different operating conditions. For each local model, the input nonlinear output-error structure is introduced to describe the dynamical property. Allocating an exponential weighting function to each local model, the nonlinear dynamics of the global system is approximated by combining all local models. The variational Bayesian (VB) approach is adopted to find the solution to the problem of parameter estimation. For the parameter uncertainties, instead of the point estimation, the posterior distribution of each model parameters is obtained under the framework of the VB approach. Two numerical simulation examples and an experiment carried on a multitank system have been employed to demonstrate that the proposed approach can work effectively.

In the previous work, the parameters of radial basis function network based autoregressive (RBF-AR) models are estimated offline and no longer updated afterward. In this letter, an adaptive learning algorithm is proposed for the RBF-AR models. The proposed strategy is that the nonlinear parameters are previously determined by an off-line variable projection method; and once new samples are available, the linear parameters are updated. The linear adaptive algorithm adopted in this letter is the multi-innovation least squares method, due to its high performance. The simulation results show that with the adaption of the linear parameters, the prediction performance of the RBF-AR models may be significantly improved, which demonstrates the effectiveness of the proposed algorithm.

Separable nonlinear models are very common in various research fields, such as machine learning and system identification. The variable projection (VP) approach is efficient for the optimization of such models. In this paper, we study various VP algorithms based on different matrix decompositions. Compared with the previous method, we use the analytical expression of the Jacobian matrix instead of finite differences. This improves the efficiency of the VP algorithms. In particular, based on the modified Gram-Schmidt (MGS) method, a more robust implementation of the VP algorithm is introduced for separable nonlinear least-squares problems. In numerical experiments, we compare the performance of five different implementations of the VP algorithm. Numerical results show the efficiency and robustness of the proposed MGS method-based VP algorithm.

This study focuses on the parameter estimation of Hammerstein non‐linear systems with unknown time‐delay and coloured noise. By means of the compressed sensing recovery theory and the auxiliary model identification idea, an orthogonal matching pursuit based iterative algorithm is proposed for estimating the system parameters and the time delay. The proposed algorithms can give highly accurate parameter estimates from a small number of observations. Two simulation examples are provided to test the proposed algorithms.

In this study, the authors consider the parameter estimation problem of the response signal from a highly non‐linear dynamical system. The step response experiment is taken for generating the measured data. Considering the stochastic disturbance in the industrial process and using the gradient search, a multi‐innovation stochastic gradient algorithm is proposed through expanding the scalar innovation into an innovation vector in order to obtain more accurate parameter estimates. Furthermore, a hierarchical identification algorithm is derived by means of the decomposition technique and interaction estimation theory. Regarding to the coupled parameter problem between subsystems, the authors put forward the scheme of replacing the unknown parameters with their previous parameter estimates to realise the parameter estimation algorithm. Finally, several examples are provided to access and compare the behaviour of the proposed identification techniques.

This study considers the modelling and identification problems for linear systems based on canonical state space models with d ‐step state‐delay. A recursive least‐squares parameter identification algorithm is presented. The basic idea is to drive a parameter identification model for such d ‐step state‐delay systems, to replace the unknown noise terms and unknown state variables in the formation vector with their estimated residuals and estimated states, and to compute the state estimates of the system in the state estimation algorithm using the estimated parameters. The simulation results indicate that the proposed parameter and state estimation algorithm can capture the dynamics of the system.

A two‐stage recursive least‐squares identification method and a two‐stage multi‐innovation stochastic gradient method are derived for Box–Jenkins (BJ) systems. The key is to decompose a BJ system into two subsystems, one containing the parameters of the system model and the other containing the parameters of the noise model, and then to estimate the parameters of the system model and the noise model, respectively. The simulation examples indicate that the proposed algorithms can generate highly accurate parameter estimates and require small computational burden.

This letter focuses on identification problems of a Hammerstein-Wiener system with an output error linear element embedded between two static nonlinear elements. A hierarchical least squares algorithm is presented for the Hammerstein-Wiener system by using the auxiliary model identification idea and the hierarchical identification principle. The major contributions of the present study are that the identification model is formulated by using the auxiliary model identification idea (the estimate of the unknown internal variable is replaced with the output of an auxiliary model) and that the bilinear parameter vectors in the identification model are estimated by using the hierarchical identification principle. The proposed hierarchical identification approach is computationally more efficient than the existing over-parametrization method.

A multiinnovation least-squares (MILS) identification algorithm is presented for linear regression models with unknown parameter vectors by expanding the innovation length in the traditional recursive least-squares (RLS) algorithm from the viewpoint of innovation modification. Because the proposed MILS algorithm uses p innovations (not only the current innovation but also past innovations) at each iteration (with the integer p > 1 being an innovation length), the accuracy of parameter estimation is improved, compared with that of the RLS algorithm. Performance analysis and simulation results show that the proposed MILS algorithm is consistently convergent. Moreover, a new interval-varying MILS algorithm is proposed, for which the key is to dynamically change the interval in order to deal with cases where some measurement data are missing. Furthermore, an auxiliary-model-based MILS algorithm is derived for pseudolinear models corresponding to output error moving average systems with colored noises. Finally, the proposed algorithms are applied to model an experimental water level control system.

For pseudo-linear regression identification models corresponding output error systems with colored measurement noises, a difficulty of identification is that there exist unknown inner variables and unmeasurable noise terms in the information vector. This paper presents an auxiliary model based multi-innovation stochastic gradient algorithm by using the auxiliary model technique and by expanding the scalar innovation to an innovation vector. Compared with single-innovation stochastic gradient algorithm, the proposed approach can generate highly accurate parameter estimates. The simulation results confirm theoretical findings.

In order to reduce computational burden of identification methods for multivariable systems, a hierarchical least squares (HLS) algorithm is developed. The basic idea is to use the hierarchical identification principle to decompose the identification model of the multivariable system into several submodels with smaller dimensions and fewer variables, and then to identify the parameter vector of each submodel. The analysis indicates that the parameter estimation error given by the proposed algorithm converges to zero under the persistent excitation. Also, the algorithm has much less computational efforts than the recursive least squares algorithm and is easy to implement on computer. Finally, we test the proposed algorithm by an example.

In this paper, we derive a mathematical model for dual-rate systems and present a stochastic gradient identification algorithm to estimate the model parameters and an output estimation algorithm to compute the intersample outputs based on the dual-rate input-output data directly. Moreover, we investigate convergence properties of the parameter and intersample estimation, and we test the proposed algorithms with example systems, including an experimental water-level system.

Crosstalk between multiple services transmitting through the same telephone cable is the primary limitation to digital subscriber line (DSL) services. From a spectrum management point of view, it is important to have an accurate map of all the services that generate crosstalk into a given pair. This paper on crosstalk identification is motivated by an important practical consideration: the signals constituting the crosstalk are transmitted at different rates in xDSL systems. Therefore, we here propose to use the "blocking technique," we derive blocked state-space models for multirate xDSL networks, and we set up the mapping relationship between available input and output data. Further, we use the least-squares principle to identify the crosstalk functions, and study the convergence rate and upper bound of the parameter-estimation error. Finally, we illustrate and verify the theoretical findings with simulation examples