Hardware acceleration is very important in the proliferation of Deep Neural Network (DNN) technologies. The Single Input Partial Product 2-D (SIPP2D)1convolution-based architecture introduced in [1], [2] implements the convolution operation of convolution-based DNNs efficiently by reading the input pixels once and maximizing their reuse. In this paper, we describe the methodology to extend the DNN accelerator based on SIPP2D-based architectures to incorporate multiple strides. We describe the algorithm for any allowable stride, given any (square) input and kernel sizes. We present an analysis of the frequency of reuse of each input pixel and its complementary pixels for multiple strides and as well as its theoretical performance.

The Electric Circuits Concept Inventory (ECCI) is a set of multiple-choice questions that measures students’ understanding of DC Circuit analysis. The topics include (i) fundamental conceptual circuit topics (ii) Circuit laws and (iii) Circuit analysis techniques. The ECCI has been quite useful for student learning. In this paper, we explore the usefulness of generative AI (GAI) tools such as ChatGPT for teaching an Electric Circuits course. Does ChatGPT understand basic electrical circuits and know Kirchoff’s laws? We present various kinds of Electric Circuits problems and diagnose how successfully ChatGPT is able or unable to solve them. We attempt to answer the question: "Can ChatGPT be a useful tool to help students learn Electric Circuits more easily?". How good are the results of such Generative AI tools ? We present ideas on how the Electric Circuits Concept inventory can be enhanced using ChatGPT for student learning. Finally, the paper will also discuss the rules and regulations universities have deployed for student academic integrity and learning in the age of AI.

Path loss is a major factor affecting the performance of wireless networks in dense urban areas. This paper investigates the path loss models in Lagos Island, Nigeria, a dense urban area with high-rise buildings and high population density. This paper presents a detailed large-scale 3D ray-tracing investigation of the Lagos Island environment. Path loss analysis was conducted using the Close-In path loss model at 700 MHz for a TR/RX height of 20/2 m. The optimal path loss prediction model for the investigated environment was compared with existing empirical models, and the results show favorable agreement. The Close-in path loss model had a better prediction accuracy with an RMSE of 0.4331 dB, the ECC-33 path loss model achieved an accuracy with the least RMSE of 0.6743 dB. The EGLI path loss prediction model showed a pessimistic performance with the highest RMSE of 2.2496 dB, followed by Hata-Okumura with 1.9606 dB and COST231 extension-to-Hata path loss model with 1.9399 dB. Network service providers can adapt the projected 4G LTE network path loss prediction model to benchmark-related wireless propagation environments. The findings of this study are important for network service providers in Lagos Island and other dense urban areas. The study provides insights into the factors that affect path loss in these areas, and it can help network service providers optimize their transmit power and improve the performance of their wireless networks in areas where 5G is still in development and pilot trials like Nigeria and other parts of developing nations.

Hardware acceleration of Deep Neural Networks (DNNs) is very critical to many edge applications. The acceleration solutions available today are typically for GPU, CPU, FPGA and ASIC platforms. The Single Partial Product 2-D Convolution known as SPP2D is a hardware architecture for fast 2-D convolution which can be used for implementing a convolutional neural network (CNN). The SPP2D based CNNs prevent the re-fetching of input pixels for the calculation of partial products and it computes the output for any input size and kernel with low latency and low power consumption compared to some other popular techniques. SPP2D based VGGNet-16 rivals the performance of other existing implementations as well as a FFO based CNN architecture which takes a novel approach to compute convolution results using row-wise inputs as opposed to traditional tile-based processing. In this paper, we present an SPP2D based hardware accelerator for a channel compressed network. We find that the low power SPP2D implementation gives a better performance in terms of low power and low execution time compared to other compressed contemporary designs. A compressed network requires less on-chip memory thus reducing the most power consuming task of moving data from off-chip to on-chip. This results in a considerable reduction in power consumption due to the reduction in memory traffic. Channel-pruned SPP2D accelerator is a low power design of 298 mW which is about $0.01\times$ to $0.37\times$ of the existing works while also having a low execution time of 0.9 secs.

Autonomous vehicle systems (a.k.a. self-driving cars) are becoming ubiquitous and are going to be a part of our high-technology future. At Santa Clara University, we have introduced a new class for senior undergraduates/graduate students to teach the basic principles involved in autonomous vehicle systems. This paper describes the outline and main components of the course, findings from the recent offerings of the course and some lessons learned so far. We hope this can help other institutions planning to develop similar course.

In recent years, Deep Neural Networks (DNNs) have achieved state-of-the-art results in various fields like Computer Vision, Natural Language Processing and Speech Recognition. Of all the DNN architectures, Convolutional Neural Networks (CNNs) have been most effective in tasks like image classification and object detection. The high performance of the CNNs comes at the cost of computational complexity. Currently Graphics Processing Units (GPUs) are used to accelerate CNN training and inference on workstations and data servers. Though popular, GPUs are not suitable for embedded applications because they are not energy efficient. ASIC and FPGA accelerators have the potential to run CNNs that are optimized for energy and performance.In this paper we present an architecture which takes a novel approach to compute convolution results using row-wise inputs as opposed to traditional tile-based processing. We are able to exceed the results of state of the art architectures when implemented on an inexpensive PYNQ Z1 board running at 100Mhz. The total latency to run the convolution layers in the VGG16 benchmark is nearly 1.5x lower for our architecture than state of the art architectures.

Deep neural networks have revolutionized the technology industry as hardware capable of implementing them has become widely available. They have been utilized in various applications such as image and video processing, self driving cars and speech processing. Two Dimensional (2-D) Convolutions are widely used in Deep Neural Networks. There are several techniques available to perform this operation. They can be implemented using methods such as sliding window, matrix multiplication, vector multiplication etc. In this paper, we introduce a new and improved (2-D) convolution method called Single Partial Product 2-D Convolution (SPP2D Convolution) that will help calculate 2-D convolution in a fast and expedient manner. We demonstrate that the new SPP2D convolution will prevent recalculation of partial weights and we present theoretical analysis of our technique compared to some other popular techniques. According to our analysis, our technique can reduce the clock cycles related to input reuse by at least 3 times in comparison with the technique adopted in the work done in and about 9 times than the standard sliding window approach.

In this paper, we develop a kernel adaptive filter for quaternion data based on minimum error entropy cost function. We apply generalized Hamilton-real (GHR) calculus that is applicable to Hilbert space for evaluating the cost function gradient to develop the quaternion kernel minimum error entropy (MEE) algorithm. The MEE algorithm minimizes Renyis quadratic entropy of the error between the filter output and desired response or indirectly maximizing the error information potential. Here, the approach is applied to quaternions for improving performance for biased or non-Gaussian signals compared with the minimum mean square error criterion of the kernel least mean square algorithm. Simulation results are used to verify the performance of the algorithm. Convergence is very fast and is shown to out-perform existing algorithms.

This paper presents a novel method to reduce the area of the Bose Chaudhuri Hocquenghen (BCH) multimode encoder based on a re-encoding scheme. Previous methods for multimode BCH use several linear-feedback shift registers (LFSR) cascaded in series to achieve area efficient encoder, but for longer BCH codes the critical path becomes an issue for high throughput. A new encoding scheme is proposed to reduce the critical path for long BCH code. Without sacrificing the latency, this method reduces the hardware complexity by reusing the same module for the encoder and the syndrome generator, which is the first stage of the BCH decoder. The experimental results show that, in the case of BCH (8191, 7983, 16), there are logic savings of 25% between the encoder and the syndrome generator, and the method provides a reconfigurable error correction capability (tsel).

Motion estimation consumes the major part of time and power in both video compression standards - HEVC and H.264. This paper presents a Fast Motion Estimation algorithm, which targets Full Search quality even at HD resolution. It is an enhancement of existing Fast Motion Estimation algorithms with the main purpose of reducing cost and power consumption for devices performing Motion Estimation while collecting and transmitting video data (used for deep learning). The proposed algorithm is based on dimensionality reduction and uses Content Addressable Memories (CAMs) and locality sensitive signatures to achieve “Quantitative Expression of Similarity”. The algorithm also presents an enhancement to one of the most efficient existing Fast Motion Estimation algorithms for lower resolutions - HMDS. The quality achieved with the new algorithm is only 3dB below Full Search.

We developed scalable narrowband and wideband speech coding schemes based on the internet low bitrate codec (iLBC). Some of these newer codecs used the Discrete Wavelet Transform (DWT) instead of the Modified Discrete Cosine Transform (MDCT). This paper explores the choice of wavelet packet transform (WPT) for an application for a new scalable speech codec for IP networks using the Discrete Wavelet Transform (DWT) to encode the core-layer coding error in the enhancement layer. The issues regarding the design and in particular the choice of wavelet for the wideband codec are discussed. Experimental simulation results show that the DWT is a promising technique to use for encoding highly non-stationary signals such as the speech coding error. The wideband codec achieved speech quality equivalent to ITU-G.718 and similar codecs and is more robust. We also show that the best choice of wavelet depends on many factors including the order and number o f levels of the wavelet tree, delay and how well it approximates the human auditory system.

The use of quaternion data can provide benefit in applications like robotics and image recognition, and particularly for performing transforms in 3-D space. Here, we describe a kernel adaptive algorithm for quaternions. A least mean square (LMS)-based method was used, resulting in the derivation of the quaternion kernel LMS (Quat-KLMS) algorithm. Deriving this algorithm required describing the idea of a quaternion reproducing kernel Hilbert space (RKHS), as well as kernel functions suitable with quaternions. A modified HR calculus for Hilbert spaces was used to find the gradient of cost functions defined on a quaternion RKHS. In addition, the use of widely linear (or augmented) filtering is proposed to improve performance. The benefit of the Quat-KLMS and widely linear forms in learning nonlinear transformations of quaternion data are illustrated with simulations.

An adaptive line enhancer is a self-tuning filter which attempts to retrieve a sinusoid buried in noise. Generally speaking, there is a trade-off between convergence speed and steady-state error. One method to address this is to use a variable step size algorithm, with a large step size for acquisition, and a smaller step size for improved steady-state performance. An alternate topology is based on the convex combination of two adaptive filters: a fast filter handles acquisition, while a slower filter is used to minimize error. In this paper, we present an adaptive line enhancer based on the convex combination of two IIR filters. It achieves both fast tracking and good steady-state performance, albeit at an increase in computational complexity.

Unquestionably, one of the most important developments in this century has been the avalanche advancement of Integrated Circuits technology incorporating solid state systems into areas never before imagined. Students face special challenges to understand these complex systems using conventional instructional methods. We here present a new pedagogical approach using the Variational Thermodynamic principle to analyze and educate engineering students the quasi-static behavior of such circuit and systems at a level of detail not otherwise available either experimentally or by standard methods. Application of this method is expected to enhance students' grasp of the topic up to the design level.

This paper presents a new scalable speech codec for IP networks using the discrete wavelet transform (DWT). The scalable narrowband speech coding scheme based on the internet low bitrate codec (iLBC) was previously presented and achieved speech quality equivalent to G.718 for narrowband signals. Whereas the performance of the core layer was satisfactory, the higher speech quality by the addition of the enhancement layer which employed the modified discrete cosine transform (MDCT) was desired. We propose the utilization of the DWT instead of the MDCT to encode the core-layer coding error in the enhancement layer. The experimental simulation results show that the DWT is a promising technique to use for encoding highly non-stationary signals such as the coding error.

High quality speech at low bit rates makes code excited linear prediction (CELP) the dominant choice for a narrowband coding technique despite the susceptibility to packet loss. One of the few techniques which received attention after the introduction of CELP coding technique is the internet low bitrate codec (iLBC) because of inherent high robustness to packet loss. Addition of rate flexibility and scalability makes the iLBC an attractive choice for voice communication over IP networks. In this paper, performance improvement schemes of multi-rate iLBC and its scalable structure are proposed, and the proposed codec enhanced from the previous work is re-designed based on the subjective listening quality instead of the objective quality. In particular, perceptual weighting and the modified discrete cosine transform (MDCT) with short overlap in weighted signal domain are employed along with the improved packet loss concealment (PLC) algorithm. The subjective evaluation results show that the speech quality of the proposed codec is equivalent to that of state-of-the-art codec, G.718, under both a clean channel condition and lossy channel conditions. This result is significant considering that development of the proposed codec is still in early stage.

The complex kernel least mean square (CKLMS) algorithm is recently derived and allows for online kernel adaptive learning for complex data. Kernel adaptive methods can be used in finding solutions for neural network and machine learning applications. The derivation of CKLMS involved the development of a modified Wirtinger calculus for Hilbert spaces to obtain the cost function gradient. We analyze the convergence of the CKLMS with different kernel forms for complex data. The expressions obtained enable us to generate theory-predicted mean-square error curves considering the circularity of the complex input signals and their effect on nonlinear learning. Simulations are used for verifying the analysis results.

Industry demand for highly-skilled digital VLSI and embedded systems engineers has made the teaching of design a challenging task for undergraduate educators. Increasingly challenging electronic design automation (EDA) environments call for a variety of skills in engineering graduates which necessitate not only industrial skills but also fundementals in basic science and digital design. These urgent needs are addressed by creating a digital logic design course taught at the freshman level that introduces students to VHDL design of digital VLSI systems while including core concepts. Critical thinking, logic synthesis and circuit innovation take priority over conventional analysis techniques. This case study includes example design projects that have been successfully implemented at The University of Akron.

Among the well known issues with CMA equalizers is that they can converge slowly. Some recent work indicates that pre-whitening with an adaptive LMS filter, configured for linear-prediction (LP-CMA), can cause the CMA equalizer to converge faster, for some poor channel cases. While this works well for a high-dispersion channel, when the technique is applied to less problematic channels, convergence can be slower than non-whitened CMA. In this paper, we apply convex combining to adaptively select between an LP-CMA and CMA with no pre-whitening.

Kernel-based adaptive filters present a new opportunity to re-cast nonlinear optimization problems over a Reproducing Kernel Hilbert Space (RKHS), transforming the nonlinear task to linear, where easier and well-known methods may be used. The approach can be seen to yield solutions suitable for sparse adaptive filtering. The new Complex Kernel Least Mean Square algorithm (CKLMS), derived by Bouboulis and Theodoridis, allows kernel-based online adaptive filtering for complex data. Here we report our results on the convergence of CKLMS with the complexified form of the Gaussian kernel. The analysis performed is based on a recent study of the Kernel LMS from Parreira et al. The analysis is used to generate theory-predicted MSE curves which consider the circularity/non-circularity of complex input which to our knowledge has not been considered previously for online nonlinear learning. Simulations are used to verify the theoretical analysis results.

Rate flexibility and high robustness to packet loss are the essential features of speech codec for voice communications over Internet Protocol (IP) networks. Multi-rate internet Low Bit-rate Codec (iLBC) is one of the speech codecs that possesses both properties and was presented by our previous papers. However the speech quality is limited in clean channel conditions because of the limitations of the current frame-independent coding scheme in time domain. In this paper, the benefits of scalable structure constructed by the addition of enhancement layer to the core layer of multi-rate iLBC are presented. The experimental simulation results show that the proposed framework can improve speech quality especially at high bit rates.

The Signals and Systems Concept Inventory (SSCI) is a set of multiple-choice questions that measures students' understanding of fundamental concepts in continuous-time (CT) and discrete-time (DT) signals and systems. In this paper, we discuss and analyze statistically the results of these assessment tests in our undergraduate courses. We use the results to assess the students' performance from year to year and determine evidence of learning outcomes. We offer useful suggestions for future offerings of the courses based on our findings. Some conclusions are made on whether we meet our assessment goals and on the efficacy of the SSCI CT Tests and the impact it has had on our pedagogy.

A recent paper titled "The Complex Gaussian Kernel LMS Algorithm" published by Bouboulis and Theodoridis introduced a complex version of the Gaussian kernel LMS (KLMS) algorithm. In this paper, we extend the concepts of complex and complexified RKHS spaces to develop suitable complex Kernel based adaptive algorithms using the Affine projection algorithm (KAPA) method. We apply the complex Gaussian kernel here, as well as develop APA-based algorithms using other suitable complex kernels. We evaluate the performance of the new algorithms using practical simulation applications. The complex KAPA algorithms are seen to outperform their LMS-based counterparts, particularly for applications where convergence rate is important.

In this paper, we propose a new hardware implementation of channel estimation for MIMO-OFDM. Our target is to minimize hardware resource utilization. At first, proper algorithm is chosen in consideration of hardware feature as well as communication theory for fast prototyping. Based on the algorithm, our pipelined architecture performs channel estimation by simple calculation logic without redundancy. Our experimental results show that the proposed implementation saves at least 43 percent of the hardware resources, while achieving the same performance as known baseline implementations. We also show that more channels can be supported by using our proposed architecture.

The scaling of CMOS Low Noise Amplifiers (LNA) comes with a reduction in supply voltage and increase in field mobility effect resulting in deterioration in linearity. This trade- off between CMOS transistor scaling and linearity is becoming more and more important as we transition into nano-scale CMOS LNA architectures. As a result, new techniques for the linearization of LNAs while maintaining an above average impedance matching, noise figure, power consumption, and gain are required. In this paper, we propose a novel technique for the adaptive post-distortion of a low power capacitor cross-coupled common gate low noise amplifier which results in compensation of the LNAs third order inter-modulation (IM3) product.The proposed technique is shown to provide excellent improvement in the third order input intercept point (IIP3) with the third order inter modulation tone running parallel with the fundamental tone over a 3 GHz to 10.6 GHz frequency range in a two tone test.

In this paper, we present a new hardware-oriented, modified diamond search (HMDS) algorithm, for fast integer pel, motion estimation in H.264/AVC. We also present our co-designed, low power very large scale integration (VLSI) architecture for HMDS. The goal of HMDS is to enable the support of high quality video on low power mobile devices and low bit rate applications which typically use H.264/AVC baseline profile at levels 1-2. Our experiments use standard test sequences ranging from QCIF to high-definition 1280 × 720p video. The proposed VLSI architecture is prototyped as an field-programable gate array (FPGA)-based field programable system-on-chip. Our results show that HMDS on average has better rate-distortion performance and speedup, compared to previous state-of-the-art fast motion estimation algorithms, while its losses compared to full search motion estimation, are insignificant. Our prototyped architecture is more hardware-efficient than previous FPGA-based architectures in terms of power consumption, area, throughput, and memory utilization. We also show that its performance in terms of maximum frequency, minimum frequency, transistor count, and power consumption are comparable to that of state-of-the-art architectures implemented on application-specific integrated circuits.

We present a new Hardware-oriented, Modified Diamond Search (HMDS) algorithm, for fast integer pel, motion estimation in H.264/AVC. This algorithm is particularly suitable for low bit rate applications and low power mobile devices. Our results show that our algorithm on average outperforms previous state-of-the-art fast motion estimation algorithms, while its losses in rate-distortion performance, when compared with Full Search Motion Estimation, are insignificant.

The first Electric Circuits course is a core component of all engineering undergraduate curricula that include not only Electrical but other engineering disciplines such as Computer, Mechanical and Civil Engineering programs worldwide. The Electric Circuits Concept Inventory (ECCI) is a set of multiple choice questions that measures students' understanding of DC Circuit analysis. The topics include (i) fundamental conceptual circuit topics (ii) Circuit laws and (iii) Circuit analysis techniques. In this paper, we develop a rationale for developing a repertoire of classic and representative set of problems that test the major concepts taught in a typical Electric Circuits course. We also present a set of multiple-choice questions suggested for inclusion in the ECCI focusing on the fundamental topics part of DC circuits analysis only. We examine the various concepts tested in each question and relate its importance to an overall course in Electric Circuits. We welcome more additions to this growing repertoire of questions that we plan to build and use in finalizing the ECCI test, especially in AC Circuit analysis topics.

In this paper, we present a prototype FPGA design for an efficient physical layer implementation of a MIMO-OFDM technique. We implement multi-channel MIMO-OFDM systems using single Fast Fourier Transform block that is shared across modulations for the system. Our experimental results show that the proposed implementation saves at least 60 percent of the hardware resources, while achieving the same data rate as known baseline MIMO-OFDM implementations. We also show that as more channels are used, more resources can be saved by using our proposed architecture.

The first Electric Circuits course is a core component of all engineering undergraduate curricula that include not only Electrical but other engineering disciplines such as Computer, Mechanical and Civil Engineering programs worldwide. The Electric Circuits Concept Inventory (ECCI) is a set of multiple choice questions that measures students' understanding of DC Circuit analysis. The topics include (i) fundamental conceptual circuit topics (ii) Circuit laws and (iii) Circuit analysis techniques. In a companion paper [1], we presented a rationale for ECCI. In this paper, we present a set of multiple-choice questions suggested for inclusion in the ECCI focusing on (ii) DC Circuit laws and (iii) DC Circuit analysis techniques. We examine the various concepts tested in each question and relate its importance to a quality overall course in Electric Circuits. We welcome more additions to this growing repertoire of questions that we plan to build and use in finalizing the ECCI test especially in AC Circuit analysis topics.

H.264/AVC is the most powerful technology in video compression/transmission area because of its high coding efficiency and robustness. In this paper, we propose a new hardware architecture of 8times8 integer transform and quantization for H.264 which promises very low resource utilization. In the architecture, each pixel is processed one by one on a simplified pipeline without multiplication. Thus, redundant modules, which are used for block-based or row-based parallel processing, can be reduced. Experimental results show that it can reduce resource usage 30% compared to previously proposed models. It can be used for mobile applications. It covers a wide range of parameters as well.

The least-mean-square (LMS) algorithm is very popular in adaptive filtering applications due to its robustness and efficiency. The frequency domain implementation of the LMS algorithm offers advantages in both reduced computational complexity for long filter lengths and improved convergence performance. The frequency response of the filter can also be tailored to specific requirements, for example limiting the magnitude response. In this paper, we present an algorithm formulated in the frequency domain based on the principle of minimum disturbance, with a penalty function incorporated to limit the adaptive filter magnitude response at any given frequency. This algorithm performed better than existing ones in terms of convergence and gain limiting, especially in colored noise environments. Simulation results are provided to illustrate the performance of this algorithm in comparison to other algorithms that limit the filter magnitude response.

The constant modulus (CM) error surface is multimodal in nature. This leads to possibility of convergence to undesired local minima. Recently, we characterized the various stationary points associated with the CM error surface. In this paper we make use of the stationary points to develop upper and lower bounds on the weight vector, ordering the minima and obtaining a bound on the excess weight error. The simulation results agree with the bounds derived

It is well known that the least mean square (LMS) algorithm convergence speed degrades considerably when the input signal is correlated. On the other hand, the affine projection algorithm (APA) was recently developed and has faster convergence for correlated inputs compared to LMS. Convergence analysis done on APA to date has been based on either a modification of the independence assumption, a special regression model, or a Gaussian regression data model. In this paper, an analysis of the standard APA algorithm under the assumption of a finite strong memory and finite moments for the regression data is done. We prove that under steady state conditions, the weight error covariance is lower bounded and dependent on the step size and not the correlation of the input regression matrix

The solution obtained by the CMA algorithm depends on both initial conditions and signal realizations. This paper uses the LMS method to seek the global minimum of the CM performance measure only. The resulting increase in computation is not prohibitive.

This paper uses Newton's method to seek the global minimum of the constant modulus performance measure. Unlike the common practice of using the constant modulus adaptive algorithm, the new approach does not suffer from local minima. The paper also discusses some implementation issues of the new algorithm.

This paper presents a detailed performance analysis of third order nonlinear adaptive systems based on the Wiener model. In earlier work, we proposed the discrete Wiener model for adaptive filtering applications for any order. However, we had focused mainly on first and second-order nonlinear systems in our previous analysis. Now, we present new results on the analysis of third order systems. All the results can be extended to higher-order systems. The Wiener model has many advantages over other models such as the Volterra model. These advantages include less number of coefficients and faster convergence. The Wiener model performs a complete orthogonalization procedure to the truncated Volterra series and this allows us to use linear adaptive filtering algorithms like the LMS to calculate all the coefficients efficiently. Unlike the Gram-Schmidt procedure, this orthogonalization method is based on the nonlinear discrete Wiener model. It contains three sections: a single-input multi-output linear with memory section, a multi-input, multi-output nonlinear no-memory section and a multi-input, single-output amplification and summary section. Computer simulation results are also presented to verify the theoretical performance analysis results.

In MPEG-2 HDTV, the bandwidth and size of memory requirements are very large. These requirements increase the system design difficulty and the hardware cost is high. There are a lot of compression algorithms that can reduce the size of the memory, but the algorithms are too complex. We propose a simple algorithm to provide low complexity hardware and high quality video. Our algorithm is based on the wavelet transform (WT) and we use two integer wavelet bases, so there are no multipliers needed. This will reduce the cost a lot in VLSI implementation. Following the WT, we use a variable-length coder to encode the data. For the memory access problem, we use group of macroblock segment coding and a hierarchical access method to reduce the access time.

In this paper, by exploiting the total least-square (TLS) closed-form solution and using state-space structure in Krein space, we show that the solution of the TLS problems can be computed via the recursive Kalman filtering algorithm. This makes it possible to use the TLS for real-time applications.

We present a set of algorithms for feed-forward multilayer neural networks based on the QR and the inverse-QR recursive least-squares algorithms. These algorithms possess excellent numerical stability, fast convergence characteristics compared to the backpropagation algorithm and require much fewer iterations to train the neural networks. We apply these algorithms to practical problems of pattern recognition of different patterns and also for optimization with excellent results. We compare these algorithms with the previously reported ones which are also based on the least squares method and found the one based on the inverse QR method to be superior to the others. The computational complexity comparison of these algorithms is also presented.>

Alternative implementations are presented for the frequency-domain LMS (least-mean-square) adaptive filter first introduced by Narayam and Peterson (1981). The implementation suggested there was a bank of bandpass filters implemented by the discrete Fourier transform (DFT). One of the present implementations is based on the recent paper of B. Widrow et al. (1987) that shows that it is possible to compute the DFT by using the LMS steepest-descent algorithm of Widrow and Hoff. This new implementation is very modular. and requires fewer computations for large filter lengths. This is because the transform part requires O(2N) computations per input sample where N is the filter length while the conventional method of taking FFTs requires O(N log N) computations per input sample. Some results of analysis of convergence are presented. Simulations of practical applications of frequency-domain LMS adaptive filters are carried out to determine its suitability, and comparisons are made with simulations using the FFT. A similar structure cannot be suggested for real data. Instead, a frequency-sampling implementation is proposed, using the recursive computation of the discrete Hartley transform.>