A number of neural networks can be formulated as the linear-in-the-parameters models. Training such networks can be transformed to a model selection problem where a compact model is selected from all the candidates using subset selection algorithms. Forward selection methods are popular fast subset selection approaches. However, they may only produce suboptimal models and can be trapped into a local minimum. More recently, a two-stage fast recursive algorithm (TSFRA) combining forward selection and backward model refinement has been proposed to improve the compactness and generalization performance of the model. This paper proposes unified two-stage orthogonal least squares methods instead of the fast recursive-based methods. In contrast to the TSFRA, this paper derives a new simplified relationship between the forward and the backward stages to avoid repetitive computations using the inherent orthogonal properties of the least squares methods. Furthermore, a new term exchanging scheme for backward model refinement is introduced to reduce computational demand. Finally, given the error reduction ratio criterion, effective and efficient forward and backward subset selection procedures are proposed. Extensive examples are presented to demonstrate the improved model compactness constructed by the proposed technique in comparison with some popular methods.

In this paper, we consider robust system identification under sparse outliers and random noises. In our problem, system parameters are observed through a Toeplitz matrix. All observations are subject to random noises and a few are corrupted with outliers. We reduce this problem of system identification to a sparse error correcting problem using a Toeplitz structured real-numbered codingmatrix. We prove the performance guarantee of Toeplitz structured matrix in sparse error correction. Thresholds on the percentage of correctable errors for Toeplitz structured matrices are also established. When both outliers and observation noise are present, we have shown that the estimation error goes to 0 asymptotically as long as the probability density function for observation noise is not “vanishing” around 0.

We consider the robust localization of radioactive sources by using their gamma-ray count at the smallest number of sensors needed to theoretically localize. We formulate a class of non-convex cost functions and consider their gradient descent optimization. We show that in N-dimensions, if there are exactly N + 1 sensors and the source lies in their open convex hull, then this convex hull is devoid of false stationary points. Thus we augment gradient descent with random projections into the convex hull, when an estimate leaves it. We argue that convergence in probability to the correct source location, will occur. Simulations demonstrate the efficacy of this algorithm.

In this paper, we study the linear zero-forcing equalizability of a communication channel. Necessary and sufficient conditions are given in terms of zeros of the transfer function. Detailed procedures of equalizer design for minimum sensitivity with respect to modeling errors and white noise are also presented. It is found that the worst-case effect of channel mismatch on the performance may be modeled as equivalent losses in SNR.

This paper presents a new method to the blind equalization of a class of linear time-varying mobile channels for differentially encoded channel input signals. Through re-parameterization, the discrete system can be modeled equivalently as a single input multiple output system. Unlike some existing methods, this new algorithm does not require prior knowledge on the cyclic frequency of the channel response. The algorithm is simple and relies only on a decision feedback structure and the differential encoding nature of the channel input signals. The convergence analysis of the decision feedback system is presented along with simulation examples.

We study the linear zero-forcing equalizability of a communication channel. Necessary and sufficient conditions are given in terms of the zeros of the transfer function. Detailed procedures of equalizer design for minimum sensitivity with respect to modeling errors and white noise are also presented. It is found that the worst case effect of channel mismatch on the performance may be modeled as equivalent losses in signal-to-noise ratio.

The so-called analytic center approach to parameter estimation has been proposed as an alternative to the well-known least squares approach. This new approach offers a parameter estimate that is consistent with the past data observations, has a simple geometric interpretation, and is computable sequentially. We study the asymptotic performance of the analytic center approach and show that the resulting estimate converges to the true parameter asymptotically, provided some mild conditions are satisfied. These conditions involve some weak persistent excitation and independence between noise and regressor, similar to the least squares case. This result is used to derive a new parameter estimation approach which offers both good transient and asymptotic performance.