This paper studies the external division operator, an external division (an affine combination with positive and negative weights) of two proximity operators. We show that the external division operator is cocoercive under some condition, and it can be expressed as the proximity operator of a certain weakly convex function. We then consider using the external division operator as an alternative to the proximity operator in the proximal gradient algorithm, which we show converges to a minimizer of the cost function penalized by the weakly convex function under some conditions. Our analysis covers the case when the fidelity function is convex but not strongly convex. In simulations, we employ the octagonal shrinkage and clustering algorithm for regression in the external division operator, and we show that significant improvements are attained in signal recovery with structured sparsity in both overdetermined and underdetermined cases.

This paper presents a graph learning framework to produce sparse and accurate graphs from network data. While our formulation is inspired by the graphical lasso, a key difference is the use of a nonconvex alternative of the ℓ1norm as well as a quadratic term to ensure overall convexity. Specifically, the weakly-convex minimax concave penalty (MCP) is used, which is given by subtracting the Huber function from the ℓ1norm, inducing a less-biased sparse solution than ℓ1. In our framework, the graph Laplacian is represented by a linear transform of the vector corresponding to its upper triangular part. Via a reformulation relying on the Moreau decomposition, the problem can be solved by the primal-dual splitting method. An admissible choice of parameters for provable convergence is presented. Numerical examples show that the proposed method significantly outperforms its ℓ1-based counterpart for sparse grid graphs.

This paper introduces a non-parametric learning framework to combat outliers in online, multi-output, and nonlinear regression tasks. A hierarchical-optimization problem underpins the learning task: Search in a reproducing kernel Hilbert space (RKHS) for a function that minimizes a sample average ℓp-norm (1 ≤ p ≤ 2) error loss defined on data contaminated by noise and outliers, under affine constraints defined as the set of minimizers of a quadratic loss on a finite number of faithful data devoid of noise and outliers (side information). To surmount the computational obstacles inflicted by the choice of loss and the potentially infinite dimensional RKHS, approximations of the ℓp-norm loss, as well as a novel twist of the criterion of approximate linear dependency are devised to keep the computational-complexity footprint of the proposed algorithm bounded over time. Numerical tests on datasets showcase the robust behavior of the advocated framework against different types of outliers, under a low computational load, while satisfying at the same time the affine constraints, in contrast to the state-of-the-art methods which are constraint agnostic.

We propose a novel problem formulation for sparsity-aware adaptive filtering based on the nonconvex minimax concave (MC) penalty, aiming to obtain a sparse solution with small estimation bias. We present two algorithms: the first algorithm uses a single firm-shrinkage operation, while the second one uses double soft-shrinkage operations. The twin soft-shrinkage operations compensate each other, promoting sparsity while avoiding a serious increase of biases. The whole cost function is convex in certain parameter settings, while the instantaneous cost function is always nonconvex. Numerical examples show the superiority compared to the existing sparsity-aware adaptive filtering algorithms in system mismatch and sparseness of the solution.

We previously proposed a joint learning scheme of Gaussian parameters (scales and centers) and coefficients for online nonlinear estimation. The instantaneous squared error cost in terms of the Gaussian scales, however, tends to have shallow slopes when the initial guess is far from optimal, causing extremely slow convergence. In this paper, we propose steepening the cost function by adding a squared distance function from the instantaneously-optimal scale. Numerical examples show that the use of the steepened cost ameliorates the convergence behaviors of the scale parameters in inappropriate initial-scale settings.

This paper presents an automatic kernel weighting technique for multikernel adaptive filtering. The full potential of the multikernel adaptive filtering approach can only be achieved when the kernels are weighted appropriately. The proposed technique balances the dominance of the kernels by making the mean eigenvalues of their associated autocorrelation matrices be equal to each other. The overall complexity of the proposed approach is low because the mean eigenvalues can be computed efficiently. The numerical results verify that the proposed technique balances the coefficient updates and yields reasonable performance.

This work proposes convolutional-sparse-coded dynamic mode decomposition (CSC-DMD) by unifying extended dynamic mode decomposition (EDMD) and convolutional sparse coding. EDMD is a data-driven method of analysis used to describe a nonlinear dynamical system with a linear time-evolution equation. Compared with existing EDMD methods, CSC-DMD has the advantage of reflecting the spatial structure of a target. As an example, the proposed method is applied to river bed shape estimation from the water surface observation. This estimation problem is reduced to sparsityaware signal restoration with a hard constraint given by the CSC-DMD prediction, where the algorithm is derived by the primal-dual splitting method. A time series set of water surface and bed shape measured through an experimental river setup is used to train and test the system. From the result, the efficacy of the proposed method is verified.

We present a convexly-constrained beamformer design for brain activity reconstruction from non-invasive electroencephalography (EEG) signals. An intrinsic gap between the output variance and the mean squared errors is highlighted that occurs due to the presence of interfering activities correlated with the desired activity. The key idea of the proposed beamformer is reducing this gap without amplifying the noise by imposing a quadratic constraint that bounds the total power of interference leakage together with the distortionless constraint. The proposed beamformer can be implemented efficiently by the multi-domain adaptive filtering algorithm. Numerical examples show the clear advantages of the proposed beamformer over the minimum-variance distortionless response (MVDR) and nulling beamformers.

We propose an efficient adaptive update method for the kernel parameters: the kernel coefficients, scales and centers. The mirror descent and the steepest descent method for squared error cost function are employed to update the kernel scales and centers, respectively. Although the problem considered in this paper is nonconvex, we reduce the possibility of falling into local minima by using a novel multiple initialization scheme to grow the dictionary without great increases of the dictionary size. Through computer experiments, we show that the proposed algorithm enjoys a high adaptation-capability while maintaining a small dictionary size, without detailed tuning of the initial kernel parameters.

We propose a novel automatic shrinkage tuning technique for the adaptive proximal forward-backward splitting (APFBS) algorithm. The shrinkage tuning aims to choose an appropriate value of the shrinkage parameter and achieve minimal system mismatch as possible. The system mismatch is approximated based on time-averaged second-order statistics. Numerical examples show that the proposed method achieves performance fairly close to that with a manually chosen shrinkage parameter for colored input signals at some signal to noise ratio (SNR).

This paper proposes an extension of multichannel non-negative matrix factorization (MNMF) that simultaneously solves source separation and dereverberation. While MNMF was originally formulated under an underdetermined problem setting where sources can outnumber microphones, a determined counterpart of MNMF, which we call the determined MNMF (DMNMF), has recently been proposed with notable success. This approach is particularly notable in that the optimization process can be more than 30 times faster than the underdetermined version owing to the fact that it involves no matrix inversion computations. One drawback as regards all methods based on instantaneous mixture models, including MNMF, is that they are weak against long reverberation. To overcome this drawback, this paper proposes an extension of DMNMF using a frequency-domain convolutive mixture model. The optimization process of the proposed method consists of iteratively updating (i) the spectral parameters of each source using the majorization-minimization algorithm, (ii) the separation matrix using iterative projection, and (iii) the dereverberation filters using multichannel linear prediction. Experimental results showed that the proposed method yielded higher separation performance and dereverberation performance than the baseline method under highly reverberant environments.

We propose a hybrid dictionary approach for distributed kernel-based adaptive learning of a nonlinear function by a network of nodes. The hybrid dictionary incorporates a local part to improve learning of high frequency components in the function within the local domain of each node and a global part to provide a consensus estimate of the function over the whole region of interest. We apply our scheme to the reconstruction of a spatial distribution by a network of mobile nodes. Performance evaluations show that high frequency components are reconstructed accurately by our hybrid dictionary approach while common schemes are not able to recover them completely.

We present an exact analytic expression of the contributions of the kernel principal components to the relevant information in a nonlinear regression problem. A related study has been presented by Braun, Buhmann, and Müller in 2008, where an upper bound of the contributions was given for a general supervised learning problem but with “uncentered” kernel PCAs. Our analysis clarifies that the relevant information of a kernel regression under explicit centering operation is contained in a finite number of leading kernel principal components, as in the “uncentered” kernel-Pca case, if the kernel matches the underlying nonlinear function so that the eigenvalues of the centered kernel matrix decay quickly. We compare the regression performances of the least-square-based methods with the centered and uncentered kernel PCAs by simulations.

Non-orthogonal multiple access (NOMA) has emerged as a promising radio access technique for enabling the performance enhancements promised by the fifth-generation (5G) networks in terms of connectivity, latency, and spectrum efficiency. In the NOMA uplink, detection based on successive interference cancellation (SIC) with device clustering has been suggested. If the receivers are equipped with multiple antennas, SIC can be combined with minimum mean-squared error (MMSE) beamforming. However, there exists a tradeoff between the NOMA cluster size and the incurred SIC error. Larger clusters lead to larger errors but they are desirable from the spectrum efficiency and connectivity point of view. To enable the deployment of large clusters, we propose a novel online learning detection method for the NOMA uplink. We design an online adaptive filter in the sum space of linear and Gaussian reproducing kernel Hilbert spaces (RKHSs). Such a sum space design is robust against variations of a dynamic wireless network that can deteriorate the performance of a purely nonlinear adaptive filter. We demonstrate by simulations that the proposed method outperforms (symbol level) MMSE-SIC based detection for large cluster sizes.

Motivated by the success of reinforcement learning (RL) for discrete-time tasks such as AlphaGo and Atari games, there has been a recent surge of interest in using RL for continuous-time control of physical systems (cf. many challenging tasks in OpenAI Gym and DeepMind Control Suite). Since discretization of time is susceptible to error, it is methodologically more desirable to handle the system dynamics directly in continuous time. However, very few techniques exist for continuous-time RL and they lack flexibility in value function approximation. In this paper, we propose a novel framework for model-based continuous-time value function approximation in reproducing kernel Hilbert spaces. The resulting framework is so flexible that it can accommodate any kind of kernel-based approach, such as Gaussian processes and kernel adaptive filters, and it allows us to handle uncertainties and nonstationarity without prior knowledge about the environment or what basis functions to employ. We demonstrate the validity of the presented framework through experiments.

We previously introduced a framework called time-domain spectrogram factorization (TSF), which realizes nonnegative matrix factorization (NMF)-like source separation in the time domain. This framework is particularly noteworthy in that, while maintaining the ability of NMF to obtain a parts-based representation of magnitude spectra, it allows us to (i) circumvent the commonly made assumption with the NMF approach that the magnitude spectra of source components are additive and (ii) take account of the interdependence of the phase/amplitude components at different time-frequency points. In particular, the second factor has been overlooked despite its potential importance. Our previous study revealed that the conventional TSF algorithm was relatively slow due to large matrix inversions, and the early stopping of the algorithm often resulted in poor separation accuracy. To overcome this problem, this paper presents an iterative TSF solver using projected gradient updates. Simulation results show that the proposed TSF approach yields higher source separation performance than NMF and the other variants including the original TSF.

We previously introduced a phase-aware variant of the non-negative matrix factorization (NMF) approach for audio source separation, which we call the “Complex NMF (CNMF).” This approach makes it possible to realize NMF-like signal decompositions in the complex time-frequency domain. One limitation of the CNMF framework is that the divergence measure is limited to only the Euclidean distance. Some previous studies have revealed that for source separation tasks with NMF, the generalized Kullback-Leibler (KL) divergence tends to yield higher accuracy than when using other divergence measures. This motivated us to believe that CNMF could achieve even greater source separation accuracy if we could derive an algorithm for a KL divergence counterpart of CNMF. In this paper, we start by defining the notion of the “dual” form of the CNMF formulation, derived from the original Euclidean CNMF, and show that a KL divergence counterpart of CNMF can be developed based on this dual formulation. We call this “KL-CNMF”. We further derive a convergence-guaranteed iterative algorithm for KL-CNMF based on a majorization-minimization scheme. The source separation experiments revealed that the proposed KL-CNMF yielded higher accuracy than the Euclidean CNMF and NMF with varying divergences.

We present a variant of the regularized dual averaging (RDA) algorithm for stochastic sparse optimization. Our approach differs from the previous studies of RDA in two respects. First, a sparsity-promoting metric is employed, originated from the proportionate-type adaptive filtering algorithms. Second, the squared-distance function to a closed convex set is employed as a part of the objective functions. In the particular application of online regression, the squared-distance function is reduced to a normalized version of the typical squared-error (least square) function. The two differences yield a better sparsity-seeking capability, leading to improved convergence properties. Numerical examples show the advantages of the proposed algorithm over the existing methods including ADAGRAD and adaptive proximal forward-backward splitting (APFBS).

Exploiting the sparsity in learning algorithms is a key to achieve excellent performances of adaptive filters. This can be realized by the adaptive proximal forward-backward splitting with carefully chosen parameters. In this paper, we propose an automatic parameter tuning based on a minimization principle of a stochastic approximation of the system-mismatch. The proposed approximation has a Tikhonov-type regularization term, which aims to minimize the disturbance by the update of the adaptive filter and mitigates overfitting to an instantaneous observation. Thanks to these properties, the proposed method realizes adaptive parameter tuning without any user-defined parameters, unlike our previous method that utilizes the user-defined parameter to avoid over-fitting. A numerical example demonstrates the efficacy of the proposed parameter tuning.

The autocorrelation matrix of the kernelized input vector is well approximated by the squared Gram matrix (scaled down by the dictionary size). This holds true under the condition that the input covariance matrix in the feature space is approximated by its sample estimate based on the dictionary elements, leading to a couple of fundamental insights into online learning with kernels. First, the eigenvalue spread of the autocorrelation matrix relevant to the hyperplane projection along affine subspace algorithm is approximately a square root of that for the kernel normalized least mean square algorithm. This clarifies the mechanism behind fast convergence due to the use of a Hilbertian metric. Second, for efficient function estimation, the dictionary needs to be constructed in general by taking into account the distribution of the input vector, so as to satisfy the condition. The theoretical results are justified by computer experiments.

This paper elucidates the underlying structures of ℓp-regularized least squares problems in the nonconvex case of 0p-constrained optimization (Pcp) and 2) an ℓp-penalized (unconstrained) optimization (Lλp). It is shown that the solution path of (Lλp) is discontinuous and also a part of the solution path of (Pcp). As an alternative to the solution path, a critical path is considered, which is a maximal continuous curve consisting of critical points. Critical paths are piecewise smooth, as can be seen from the viewpoint of the variational method, and generally contain non-optimal points, such as saddle points and local maxima as well as global/local minima. Our study reveals multiplicity (non-monotonicity) in the correspondence between the regularization parameters of (Pcp) and (Lλp). Two particular paths of critical points connecting the origin and an ordinary least squares (OLS) solution are studied further. One is a main path starting at an OLS solution, and the other is a greedy path starting at the origin. Part of the greedy path can be constructed with a generalized Minkowskian gradient. This paper of greedy path leads to a nontrivial close-link between the optimization problem of ℓp-regularized least squares and the greedy method of orthogonal matching pursuit.

This paper presents a stochastic behavior analysis of a kernel-based stochastic restricted-gradient descent method. The restricted gradient gives a steepest ascent direction within the so-called dictionary subspace. The analysis provides the transient and steady state performance in the mean squared error criterion. It also includes stability conditions in the mean and mean-square sense. The present study is based on the analysis of the kernel normalized least mean square (KNLMS) algorithm initially proposed by Chen et al. Simulation results validate the analysis.

We investigate the use of compactly supported kernels (CSKs) for the kernel normalized least mean square (KNLMS) algorithm proposed initially by Richard et al. in 2009. The use of CSKs yields sparse kernelized input vectors, offering an opportunity for complexity reduction. We propose a simple two-step method to compute the kernelized input vectors efficiently. In the first step, it computes an over-estimation of the support of the kernelized input vector based on a certain ℓ1-ball. In the second step, it identifies the exact support by detailed examinations based on an ℓ2-ball. Also, we employ the identified support given by the second step for coherence construction. The proposed method reduces the amount of ℓ2-distance evaluations, leading to the complexity reduction. The numerical examples show that the proposed algorithm achieves significant complexity reduction.

We propose a novel multikernel adaptive filtering algorithm based on the iterative projections in the sum space of reproducing kernel Hilbert spaces. We employ linear and Gaussian kernels, envisioning an application to partially-linear-system identification/estimation. The algorithm is derived by reformulating the hyperplane projection along affine subspace (HYPASS) algorithm in the sum space. The projection is computable by virtue of Minh's theorem proved in 2010 as long as the input space has nonempty interior. Numerical examples show the efficacy of the proposed algorithm.

The existing kernel filtering algorithms are classified into two categories depending on what space the optimization is formulated in. This paper bridges the two different approaches by focusing on the isomorphism between the dictionary subspace and a Euclidean space with the inner product defined by the kernel matrix. Based on the isomorphism, we propose a novel kernel adaptive filtering algorithm which adaptively refines the dictionary and thereby achieves excellent performance with a small dictionary size. Numerical examples show the efficacy of the proposed algorithm.

Effective utilization of sparsity of the system to be estimated is a key to achieve excellent adaptive filtering performances. This can be realized by the adaptive proximal forward-backward splitting (APFBS) with carefully chosen parameters. In this paper, we propose a systematic parameter tuning based on a minimization principle of an unbiased MSE estimate. Thanks to the piecewise quadratic structure of the proposed MSE estimate, we can obtain its minimizer with low computational load. A numerical example demonstrates the efficacy of the proposed parameter tuning by its excellent performance over a broader range of SNR than a heuristic parameter tuning of the APFBS.

The solution path of the least square problem under ℓp-regularization (0 <; p <; 1) is studied, where the Lagrangian multiplier λ due to the constraint is the parameter of the path. It is first proven that the least square solution of an unconstrained overdetermined linear system is connected with the origin, under a mild condition, by a continuous path of critical points of an ℓp-regularized squared error function. Based on this fact, it is proven that every sparsest least square solution of an underdetermined system is connected with the origin by a critical-point path. The existence theorem holds more generally for any least square solution whose support has its associated submatrix of the fat sensing matrix be full column rank. This is a sufficient condition for the existence, and allows to reduce the underdetermined problem to an overdetermined one with the off-support variable(s) nullified. A necessary condition is that the gradient of the ℓpregularizer with respect to the support variables lies in the row space of the submatrix (which is not necessarily full column rank).

In this paper, we propose a novel scheme to supervised nonnegative matrix factorization (NMF). We formulate the supervised NMF as a sparse optimization problem assuming the availability of a set of basis vectors, some of which are irrelevant to a given matrix to be decomposed. The proposed scheme is presented in the context of music transcription and musical instrument recognition. In addition to the nonnegativity constraint, we introduce three regularization terms: (i) a block ℓ1norm to select relevant basis vectors, and (ii) a temporal-continuity term plus the popular ℓ1norm to estimate correct activation vectors. We present a state-of-the-art convex-analytic iterative solver which ensures global convergence. The number of basis vectors to be actively used is obtained as a consequence of optimization. Simulation results show the efficacy of the proposed scheme both in the case of perfect/imperfect basis matrices.

We propose a novel kernel adaptive filtering algorithm, dubbed Parallel HYperslab Projection along Affine Sub-Spaces (Φ-PASS), which reuses observed data efficiently. We first derive its fully-updating version that projects the current filter onto multiple hyperslabs in parallel along the dictionary subspace. Each hyperslab accommodates one of the data observed up to the present time instant. The algorithm is derived with the adaptive projected subgradient method (APSM) based on which a convergence analysis is presented. We then generalize the algorithm so that only a few coefficients, whose associated dictionary-data are coherent to the datum of each hyperslab, can be updated selectively for low complexity. This is accomplished by performing the hyperslab projections along affine subspaces defined with the selected dictionary-data. Numerical examples show the efficacy of the proposed algorithm.

We propose a novel sparsity-aware adaptive filtering algorithm based on iterative use of weighted soft-thresholding. The weights are determined based on a rough local approximation of the ℓpnorm (0 <; p <; 1). The proposed algorithm operates the weighted soft-thresholding for enhancing the sparsity, following estimation error managements with the affine projection. The proposed weighting technique alleviates an extra bias of no benefit caused by shrinking dominant coefficients. The numerical examples demonstrate that the proposed weighting technique outperforms the existing one when the situation changes under the fixed parameter settings.

The ℓp-constrained least squares, which is denoted by (Pc), for 0p-penalized least squares (Qλ) which reformulates (Pc) with the Lagrangian multiplier. The greedy path is a multi-valued function of λ and is formed by a collection of multiple critical paths of (Qλ).

In this paper, we propose an acceleration technique of the adaptive filtering scheme called adaptive proximal forward-backward splitting method. For accelerating the convergence rate, the proposed method includes a step to shift the current estimate in the direction of the difference between the current and previous estimates based on the Fast Iterative Shrinkage/Thresholding Algorithm (FISTA). The computational complexity for this additional step is fairly low compared to the overall complexity of the algorithm. As an example of the proposed method, we derive an acceleration of the composition of the Adoptively Weighted Soft-Thresholding (AWST) operator and the exponentially weighted adaptive parallel projection. AWST shrinks the estimated filter coefficients to zero for exploiting the sparsity of the system to be estimated and the exponentially weighted adaptive parallel projection algorithm realizes high accuracy by utilizing all available information at each iteration. This accelerated method improves the steady-state mismatch drastically with its con vergence speed as fast as the proportionate affine projection algorithm.

In this paper, we propose a novel adaptive filtering algorithm based on an iterative use of (i) the proximity operator and (ii) the parallel variable-metric projection. Our time-varying cost function is a weighted sum of squared distances (in a variable-metric sense) plus a possibly nonsmooth penalty term, and the proposed algorithm is derived along the idea of proximal forward-backward splitting in convex analysis. For application to sparse-system identification problems, we employ the (weighted) ℓ1norm as the penalty term, leading to a time-varying soft-thresholding operator. As the simple example of the proposed algorithm, we present the variable-metric affine projection algorithm composed with the time-varying soft-thresholding operator. Numerical examples demonstrate that the proposed algorithms notably outperform their counterparts without soft-thresholding both in convergence speed and steady-state mismatch, while the extra computational complexity due to the additional soft-thresholding is negligibly low.

We propose multi-domain adaptive filtering based on the idea of feasibility splitting - dealing with feasibility in individual domains. The proposed approach provides a useful and mathematically rigorous framework to incorporate multiple pieces of information (expressed in different domains) efficiently. Indeed, it processes such multiple pieces of information by means of the metric projection in each individual domain; this is a significant advantage over existing single-domain approaches. Also we provide a reasonable strategy to treat the case where the available prior information is inconsistent. A convergence analysis and numerical examples are presented to support the proposed method.

This paper addresses the minimal subset selection of antennas achieving designated channel capacity. This is one of the most natural approaches to alleviating the power consumption in MIMO systems, while it is a mathematically challenging nonlinearly-constrained sparse optimization (ℓ0-norm minimization) problem. We present an efficient algorithmic solution, to this highly combinatorial problem, using convex and differentiable relaxations of the (ℓ0-norm. The proposed algorithm is based on the hybrid steepest descent method for the subgradient projection operator together with the soft-thresholding technique, minimizing the Moreau envelope of the (ℓ1-norm subject to the capacity constraint. The simulation results show that the proposed algorithm realizes a near optimal solution to the original nonlinearly-constrained sparse optimization problem.

We present a rigorous deterministic analysis of the variable-metric adaptive filtering algorithms (including the transform-domain, LMS/Newton, and proportionate adaptive filters) by using the framework of variable-metric adaptive projected subgradient method (Yukawa et al. 2007). Under small metric-fluctuations, we present the useful properties of (i) monotone approximation - with respect to a certain constant metric - indicating the stability of the algorithm and (ii) convergence to an asymptotically optimal point. Numerical examples show the advantage of the variable-metric adaptive filtering algorithms and suggest the validity of the analysis.

In this paper, we propose an efficient design of proportionality factors in the recently established algorithm named Krylov-proportionate normalized least mean-square (KPNLMS), which is an extention of the PNLMS algorithm to nonsparse (or dispersive) unknown systems by means of a Krylov subspace. The designing task takes a form of minimizing the number of iterations that is needed for an upper bound of the system mismatch to reach a specified target value. The minimization is performed under several constraints related to numerical stability, computational requirements, and nonnegativity, and its closed-form solution is derived. Numerical examples demonstrate that the proposed design significantly reduces the number of iterations needed to achieve target values of system mismatch especially when a low level of system mismatch is required.

This paper proposes a fast converging adaptive filtering algorithm named Krylov-proportionate normalized least mean-square (KPNLMS) by extending the proportionate normalized least mean square (PNLMS) algorithm. PNLMS is known to exhibit faster convergence than the standard NLMS algorithm for sparse unknown systems. The proposed algorithm attains similar effects for non-sparse unknown systems by constructing, based on the multistage Wiener filter (MWF) representation, an orthonormal basis with which the unknown system has a sparse structure. The proposed algorithm can be analyzed by the adaptive parallel variable-metric projection framework. Numerical studies for non-sparse unknown systems are presented, comparing KPNLMS and the MWF-based reduced-rank method.

This paper presents a new approach to efficient acoustic echo cancellation (AEC) based on reduced-rank adaptive filtering equipped with selective-decimation and adaptive interpolation. We propose a novel structure of an AEC scheme that jointly optimizes an interpolation filter, a decimation unit, and a reduced-rank filter. With a practical choice of parameters in AEC, the total computational complexity of the proposed reduced-rank scheme with the normalized least mean square (NLMS) algorithm is approximately half of that of the full-rank NLMS algorithm. We discuss the convergence properties of the proposed scheme and present a convergence condition. First, we examine the performance of the proposed scheme in a single-talk situation with an error-minimization criterion adopted in the decimation selection. Second, we investigate the potential of the proposed scheme in a double-talk situation by employing an ideal decimation selection. In addition to mean squared error (MSE) and power spectrum analysis of the echo estimation error, subjective assessments based on absolute category rating are performed, and the results demonstrate that the proposed structure provides significant improvements compared to the full-rank NLMS algorithm.

In this paper, we propose a novel adaptive filtering algorithm named adaptive parallel variable-metric projection (APVP) algorithm, which includes the proportionate normalized least mean square (PNLMS) algorithm as its special example. The proposed algorithm is based on parallel projection (onto multiple closed convex sets) with time-varying metrics. A convergence analysis of the proposed algorithm is presented with the aid of the adaptive projected subgradient method. Numerical examples demonstrate that the proposed algorithm realizes echo cancellation superior to the conventional algorithms.

This paper indicates that an appropriate design of metric leads to significant improvements in the adaptive projected subgradient method (APSM), which unifies a wide range of projection-based algorithms [including normalized least mean square (NLMS) and affine projection algorithm (APA)]. The key is to incorporate a priori (or a posteriori) information on characteristics of an estimandum, a system to be estimated, into the metric design. We propose a family of efficient adaptive filtering algorithms based on a parallel use of quadratic-metric projection, which assigns every point to the nearest point in a closed convex set in a quadratic-metric sense. We present two versions: (1) constant-metric and (2) variable-metric, i.e., the metric function employed is (1) constant and (2) variable among iterations. As a constant-metric version, adaptive parallel quadratic-metric projection (APQP) and adaptive parallel min-max quadratic-metric projection (APMQP) algorithms are naturally derived by APSM, being endowed with desirable properties such as convergence to a point optimal in asymptotic sense. As a variable-metric version, adaptive parallel variable-metric projection (APVP) algorithm is derived by a generalized APSM, enjoying an extended monotone property at each iteration. By employing a simple quadratic-metric, the computational complexity of the proposed algorithms is kept linear with respect to the filter length. Numerical examples demonstrate the remarkable advantages of the proposed algorithms in an application to acoustic echo cancellation.

It is well-known that the Capon beamformer is sensitive to array steering vector errors and may result into a worse performance than classical data-independent beamformers. This paper follows a different path from the well-established diagonal loading techniques and designs a robust Capon beamformer by a recent extension of the adaptive projected subgradient method. The proposed method marks a computational complexity of O(N2), where N is the number of array elements. The simulation results show that the proposed beamformer achieves excellent performance especially in cases where the diagonal loading techniques face difficulties, i.e. in cases where the interference to noise ratio (INR) is moderately larger than SNR

This paper proposes a novel heuristic approach to the infeasible power control problem [i.e., no power allocation can provide users with specified quality of service (QoS)] in wireless communication systems. In such infeasible cases, it has been in great demand to increase the number of accepted users (i.e., users provided with the specified QoS) by certain distributed algorithms, which require no communication among base stations (BSs). The number of accepted users is often suppressed by a small number of users in severe environments, hence removing such users obviously helps many other users be accepted. The proposed algorithm, which is fully distributed, detects and removes such users based on simple criteria including: (a) the QoS is below the necessary level, and (b) the BS is already transmitting with the maximum possible power to the user. Simulation results demonstrate that the proposed algorithm significantly increases the number of accepted users with low computational complexity

The contribution of this paper is twofold. We first clarify geometrically an inherent difference in convergence speed between two adaptive algorithms, projected-NLMS (PNLMS) and constrained-NLMS (CNLMS), both of which are widely used for linearly constrained adaptive filtering problems. A simple geometric interpretation suggests that CNLMS converges faster than PNLMS especially in the challenging situations of the adaptive beamforming where there exist spatially-correlated interferences (i.e., interferences that have small angular separation with the desired signal). To enhance the advantage of CNLMS in convergence speed while keeping linear computational complexity, we then propose an efficient adaptive beamformer that utilizes multiple data at each iteration by extending the constrained parallel projection algorithm to complex cases. The simulation results demonstrate that the proposed beamformer exhibits even faster convergence than the constrained affine projection algorithm (CAPA) as well as CNLMS

The adaptive projected subgradient method offers a unified mathematical perspective for the adaptive (set-membership/set-theoretic) filtering schemes. In this paper, we introduce an overview of its recent theoretical advances and successful applications to robust signal processing problems including the stereo acoustic echo canceling, the MAI suppression in DS/CDMA receivers, and the robust adaptive beamforming with array antenna systems

This paper presents a family of multiple access interference (MAI) suppression receivers based on the adaptive projected subgradient method. The proposed scheme can be applied to many different channel models and modulations in a unified manner. Moreover, it is suitable for both blind and nonblind receivers. The adaptive projected subgradient method realizes excellent convergence to a set including an optimal solution with high probability by asymptotically minimizing a sequence of nonnegative convex functions. Simulation results show much better (variable) tradeoff between speed and performance at steady-state as compared to existing techniques

The paper presents two novel blind set-theoretic adaptive filtering algorithms for multiple access interference (MAI) suppression in DS/CDMA systems. We naturally formulate the problem of MAI suppression as minimizing asymptotically a sequence of cost functions under some linear constraint defined by the desired user's signature. The proposed algorithms embed the constraint in the direction of adaptation, and thus the adaptive filter moves toward the optimal filter without stepping away from the constraint set. In addition, using parallel processors, the proposed algorithms attain good performance behavior with low computational complexity. Geometric interpretation clarifies an advantage of the proposed methods over some conventional methods. Simulation results demonstrate that the proposed algorithms achieve much faster convergence than conventional methods with a moderate number of concurrent processors.