This article proposes a novel macromodeling method for high-frequency nonlinear circuits, utilizing an attention-based deep recurrent neural network (ATDRNN). This method leverages the attention mechanism within the RNN, comparing each time step with other time steps to determine their similarities. It then applies some coefficients as weights to the features of each time step based on these similarities, enhancing the RNN’s ability to focus on more informative features. Consequently, this approach allows for more accurate modeling of nonlinear circuits. Additionally, having comprehensive signal information and similarities between various time steps mitigates the vanishing gradient problem commonly faced by RNNs. The models derived from this method not only exhibit superior accuracy compared to the conventional RNNs, but also run much faster than existing transistor-level models in circuit simulators. The effectiveness of the proposed method is demonstrated by modeling two nonlinear circuits, namely 2-coupled and 3-coupled line high-speed interconnects driven by multi-stage buffers.

In the computer-aided design (CAD) area, the recurrent neural network (RNN) has shown notable functionality in generating fast and high-performance models rather than the models in simulation tools. Predicting time sequences is a pervasive and challenging problem that may require identifying the dependencies between sequences that RNN is capable of performing. Despite all its features, conventional RNN still faces challenges such as limited accuracy and a large number of parameters. Therefore, we propose new macromodeling methods for nonlinear circuits called the Clockwork-RNN (CWRNN) and its hybrid version which is a more powerful but simpler implementation of a conventional RNN architecture with relatively little model complexity. In addition, CWRNN inherently models complex dependencies without the need for a large number of parameters. As a result, the computational cost is less than conventional RNN. Moreover, understanding and implementing the CWRNN is relatively simple and provides great flexibility in architectural configuration by introducing modules with several clock rates of exponents of 2. In addition to the above new modeling technique, we proposed the Hybrid-Module CWRNN as another new modeling method that utilizes modules of various exponents of different numbers resulting in further accuracy improvement of the CWRNN. Furthermore, the models obtained from the proposed techniques required much smaller simulation times compared to the current models used in simulation tools. Three nonlinear high-frequency examples have been utilized to verify the benefits of the proposed modeling methods.

This paper proposes a new hybrid structure and microwave modeling method that combines polynomial regression with batch-normalized deep feedforward neural network (BN-DFN) to be used in high-dimensional microwave circuit modeling. Utilizing the proposed BN-DFN method results in a remarkably faster training procedure compared to the conventional DFN. In addition, the superiority of the BN-DFN method over DFN in terms of accuracy prepares this opportunity to perform high-dimensional microwave modeling using fewer training data in comparison with the modeling with conventional DFN. The results show that a data reduction of about 40-80% can be achieved for microwave applications used in this paper using the proposed method. Also, in this paper, a hybrid polynomial regression BN-DFN (HPBN-DFN) is proposed to further improve the accuracy of the proposed BN-DFN method. The proposed HPBN-DFN method fine-tunes the predicted values of the BN-DFN by passing them through a polynomial regression stage for increasing accuracy. The proposed methods are verified through two high-dimensional parameter-extraction modeling examples of microwave filters.

A new structure and macromodeling approach which is an advance over high-order recurrent neural network named bidirectional high-order deep recurrent neural network (BIHODRNN) is proposed in this paper for the first time for nonlinear circuits. In the proposed structure, besides the fully connected weights to the neurons of multiple previous time steps of the same hidden layer in conventional high-order recurrent neural network (HORNN), there are additional fully connected weights to the neurons of that hidden layer for multiple next times steps. Due to more training parameters compared to conventional RNN and HORNN, the proposed BIHODRNN can train and predict more complex relationships in a faster and more efficient way and can better capture long-term dependencies. Also, because of bidirectional structure with multiple orders to the next time steps, it can predict the output signals beyond the training time intervals with much better accuracy. To improve the accuracy of the proposed BIHODRNN even more, another structure and method called Hybrid BIHODRNN was presented in this paper. By combining layers of different orders and different directionality in Hybrid BIHODRNN, the training parameters are significantly decreased leading to the reduction of overfitting problem and increasing the model accuracy. Furthermore, the proposed BIHODRNN and its hybrid version need smaller number of training data compared to the HODRNN for generating a model with similar accuracy. Moreover, two proposed approaches are notably faster than the transistor-level models in circuit simulators for acquiring similar accuracy. The superiorities of the proposed approaches are investigated by modeling two nonlinear circuit examples, namely, 5-coupled and 3-coupled line high-speed interconnects both driven by a four-stage driver.

In this paper, for the first time, the deep gated recurrent unit (Deep GRU) is used as a new macromodeling approach for nonlinear circuits. Similar to Long Short-Term Memory (LSTM), the GRU has gating units that control the information flow and makes the network less prone to the vanishing gradient problem. Having a smaller number of gates causes GRU to have fewer parameters compared to LSTM leading to better model accuracy. Using the gates leads gradient formulations to have additive nature which helps them to be more resistant to vanishing and consequently learn long sequences of data. The proposed macromodeling method is capable of modeling nonlinear circuits more accurately and using fewer parameters compared to the conventional LSTM macromodeling method. To further improve the GRU performance, a regularization technique called Gaussian dropout is applied in this paper on deep GRU (GDGRU) to reduce the overfitting problem resulting in better test error. Additionally, the models obtained from the proposed techniques are remarkably faster than the original transistor-level models. To verify the superiority of the proposed method, time-domain modeling of three nonlinear circuits is provided. For these circuits, the comparisons of the accuracy and speed between the conventional recurrent neural network (RNN), the LSTM, and the proposed macromodeling methods are provided.

This paper describes a new technique for training feedforward neural networks. We employ the proposed algorithm for robust neural network training purpose. Conventional neural network training algorithms based on the gradient descent often encounter local minima problems. Recently, some evolutionary algorithms are getting a lot more attention about global search ability but are less-accurate for complicated training task of neural networks. The proposed technique hybridizes local training algorithm based on quasi-Newton method with a recent global optimization algorithm called particle swarm optimization (PSO). The proposed technique provides higher global convergence property than the conventional global optimization technique. Neural network training for some benchmark problems is presented to demonstrate the proposed algorithm. The proposed algorithm achieves more accurate and robust training results than the quasi-Newton method and the conventional PSOs.

A new technique based on neural networks is presented for dynamic modeling of nonlinear telecommunication circuits in continuous time domain. The proposed feedforward dynamic neural network (FDNN) model can be developed directly from input-output large-signal measurements or simulations, without having to rely on internal details of the circuit. New formulations are derived in order to handle the important circuit-load effects in system level simulation. The resulting model is fast and can be used with connections to other circuit models, allowing us to perform efficient high-level system simulation and design. It is observed that the proposed FDNN approach provides the best overall performance of being much faster than original detailed system simulation and much more accurate than the conventional behavioral modeling approach. Examples of feedforward dynamic modeling of amplifiers, mixer and their use in telecommunication system simulation are presented, demonstrating the increased efficiency in designing telecommunications systems using the proposed technique.

Neural networks as an emerging modeling technique have gained much attention in the microwave area. Due to the convergence difficulty of simulators or equipment limits where parameters are sampled at extremes, the simulated or measured training data often have both gross errors and small errors. A new training method is presented in this paper which incorporates the Huber concept into a quasi-Newton method. The proposed method can recognize the gross errors and small errors and treat them differently. Therefore this Huber training method is much more robust than traditional least-square l/sub 2/ methods, which is demonstrated through two examples, modeling of a quadratic function and transmission lines.

Accurate simulation of large interconnect networks has become a necessity to address signal-integrity issues in current high-speed very-large-scale-integration designs. To accurately characterize a dispersive system of interconnects at higher frequencies, a full-wave analysis is required. However, conventional circuit simulation of interconnects with full-wave models is extremely CPU expensive. Recently published moment-matching techniques provide a generalized approach to lumped/distributed circuit response approximations. However, these techniques are based on quasi-transverse electromagnetic mode (TEM) assumption and have no mechanism to handle full-wave models. In this paper, we present a new method to extend model-reduction techniques for simulation of full-wave models. The following three new results are presented in this paper: 1) a generalized method to combine modal results from a full-wave analysis into circuit simulators; 2) a new algorithm for moment generation involving full-wave models; 3) deviations associated with quasi-TEM approximations compared to full-wave models at higher frequencies. The proposed algorithm yields a speed up of 1 to 2 orders of magnitude for a comparable accuracy with conventional techniques. In addition, the proposed method can be used for a mixed simulation involving distributed models with frequency dependent/independent RLCG parameters, full-wave interconnect models and measured subnetworks along with nonlinear terminations.

This paper describes a new moment-generation algorithm for efficient simulation of linear subnetworks characterized by measured or tabulated data using moment-matching techniques. The subnetwork moments are computed by performing an integration in time-domain on the measured data. The proposed technique is more accurate as it relies on integration as compared to the previously published approaches which depend on the differentiation of measured data in frequency-domain for computation of moments. Using the new moment-generation technique, the CFH (Complex Frequency Hopping) algorithm has been extended to handle measured subnetworks. Also a generalized stencil for measured data for inclusion in circuit simulators and to facilitate efficient moment-generation has been presented. Examples and comparison with conventional simulations are provided. The method is accurate while it is faster than the conventional approach by 1 to 2 orders of magnitude.

An important yet challenging task for neural network based speech recognizers is the effective processing of temporal information in speech signals. A high-order fully recurrent neural network is developed to effectively handle the sequential nature of speech signals and to accommodate both temporal and spectral variations. The proposed neural network has 4 layers, namely, the input layer, self organizing map, fully recurrent hidden layer and output layer. The important characteristics of the hidden neurons and the output neurons are their high-order processing feature. A 2-stage unsupervised/supervised training method is developed. The solution from unsupervised training provides a good starting point for supervised training. The proposed neural network and the training method are applied to isolated word recognition using the TI20 data.

In order to accurately characterize dispersive system of VLSI interconnects at higher frequencies, full wave analysis which takes into account all possible field components and satisfies all boundary conditions is required. However, conventional circuit simulation of interconnects with full wave models is extremely CPU expensive. This paper presents a new method to extend the moment matching technique, complex frequency hopping, to the case of interconnects modeled with full wave analysis. Formulation of circuit equations is modified to incorporate interconnect stencil from full wave analysis. A new algorithm for the moment generation for interconnect networks with full wave models has been developed. Full wave analysis has been carried out with the efficient 'spectral domain approach'. Results have shown that the proposed method is accurate while it yields a speed up of one to three orders of magnitude over conventional simulation techniques.

Signal integrity issues such as delay and crosstalk are important in designing high-speed printed circuits boards and multichip modules. A complete signal integrity analysis and optimization require repeated simulation of distributed networks which can be very CPU intensive. In this paper an efficient approach is presented using neural network models to describe the signal integrity behaviour of a distributed network. The model is used to formulate a signal integrity optimization problem, replacing exact circuit simulations. This approach has been used in analysis and yield optimization of high-speed VLSI interconnects and is much faster than the standard optimization.>

Interconnects in high-speed VLSI circuits and systems exhibit transmission line effects. Due to the complex geometries of interconnections, coupling between various layers, and inhomogeneous insulating materials, these high-speed interconnects need to be modelled as nonuniform frequency dependent transmission lines. In this paper, we describe a method of simulating the transient response of nonuniform high-speed interconnects in its most general form, i.e. nonuniformly distributed, lossy, coupled, multiple lines with frequency-dependent parameters with linear and nonlinear terminations. Transmission line equations are formulated in the frequency domain as an initial value problem and solved using numerical integration. A new algorithm is proposed for overcoming the inherent initial value instability encountered in the solution of transmission line equations. The time domain response is obtained by Numerical Inversion of Laplace Transform (NILT). Nonlinear networks containing nonuniform high-speed interconnects are analyzed using the Piecewise Decomposition Technique. The accuracy and efficiency of the proposed method is illustrated by appropriate examples and comparisons with published results.>

A method based on numerical inversion of the Laplace transform for the transient analysis of nonuniform high-speed interconnects in LSI/VLSI circuits is described. The interconnects are treated as lossy multiconductor nonuniform frequency dependent transmission lines. An algorithm for overcoming the inherent initial value instability encountered while numerically integrating transmission line equations is described. The method is directly compatible with the piecewise decomposition technique (PDT) and can be extended to interconnect networks with nonlinear terminations. Further speed up can be achieved by using parallel processors. Examples and comparisons with published results are presented.>