We propose a proximal variable smoothing algorithm for nonsmooth optimization problem with sum of three functions involving weakly convex composite function. The proposed algorithm is designed as a time-varying forward-backward splitting algorithm with two steps: (i) a time-varying forward step with the gradient of a smoothed surrogate function, designed with the Moreau envelope, of the sum of two functions; (ii) the backward step with the proximity operator of the remaining function. For the proposed algorithm, we present a convergence analysis in terms of a stationary point by using a newly smoothed surrogate stationarity measure. As an application of the target problem, we also present a formulation of multiple-input-multiple-output (MIMO) signal detection with phase-shift keying. Numerical experiments demonstrate the efficacy of the proposed formulation and algorithm.

The equilibrium selection problem in the generalized Nash equilibrium problem (GNEP) has recently been studied as an optimization problem, defined over the set of all variational equilibria achievable through a lower-level non-cooperative game among players. However, to make such a selection fair for every player, we have to rely on an unrealistic assumption, that is, the availability of a trusted center that does not induce any bias for every player. In this paper, we study a new equilibrium selection problem, named the hierarchical Nash equilibrium problem (HNEP), and propose an iterative algorithm for solving the HNEP. The HNEP is designed to ensure a fair selection without assuming any trusted center. More precisely, the HNEP is the GNEP for an upper-level non-cooperative game defined over the set of all variational equilibria of the lower-level non-cooperative game. The proposed algorithm for the HNEP is established by applying the hybrid steepest descent method to a variational inequality defined over the fixed point set of a quasi-nonexpansive operator. Numerical experiments show the effectiveness of the proposed equilibrium selection problem and its algorithmic solution.

It is well known that the minimum ℓ2-norm solution of the convex LASSO model, sayx⋆, is a continuous piecewise linear function of the regularization parameter λ, and its signed sparsity pattern is constant within each linear piece (Osborne 2000, Efron et al. 2004). The current study is an extension of this classic result, proving that the aforementioned properties extend to the min-norm solution mapx⋆(y, λ), whereyis the observed signal, for a generalization of LASSO termed the scaled generalized minimax concave (sGMC) model. The sGMC model adopts a nonconvex debiased variant of the ℓ1-norm as sparse regularizer, but its objective function is overall-convex. Based on the geometric properties ofx⋆(y, λ), we propose an extension of the least angle regression (LARS) algorithm, which iteratively computes the closed-form expression ofx⋆(y, λ) in each linear zone. Under suitable conditions, the proposed algorithm provably obtains the whole solution mapx⋆(y, λ) within finite iterations. Numerical experiments demonstrate the efficiency and reduced estimation error of the proposed algorithm compared to the conventional LARS. Notably, our proof techniques for establishing continuity and piecewise linearity ofx⋆(y, λ) are novel, and they lead to two side contributions: (a) our proofs establish continuity of the sGMC solution set as a set-valued mapping of (y, λ); (b) to prove piecewise linearity and piecewise constant sparsity pattern ofx⋆(y, λ), we do not require any assumption that previous work relies on (whereas to prove some additional properties ofx⋆(y, λ), we use a different set of assumptions from previous work).

We propose a variable smoothing algorithm for solving nonconvexly constrained nonsmooth optimization problems. The target problem has two issues that need to be addressed: (i) the nonconvex constraint and (ii) the nonsmooth term. To handle the nonconvex constraint, we translate the target problem into an unconstrained problem by parameterizing the nonconvex constraint in terms of a Euclidean space. We show that under a certain condition, these problems are equivalent in view of finding a stationary point. To find a stationary point of the parameterized problem, the proposed algorithm performs the gradient descent update for the smoothed version of the parameterized problem with replacement of the nonsmooth function by the Moreau envelope, inspired by a variable smoothing algorithm [Böhm-Wright, J. Optim. Theory Appl., 2021] specialized for unconstrained nonsmooth optimization. We also present a convergence analysis of the proposed algorithm as well as its application to a nonconvex reformulation of the sparse spectral clustering.

For the constrained LiGME model, a nonconvexly regularized least squares estimation model, under its overall convexity condition, we newly present an iterative algorithm of guaranteed convergence to its globally optimal solution. The proposed algorithm can deal with two different types of constraints simultaneously. The first type constraint, called the asymptotic constraint, requires the limit of estimation sequence to achieve the corresponding condition. The second type constraint, called the early constraint, requires every vector in estimation sequence to achieve the corresponding condition. We also propose nonconvex and robustness enhancements of fused lasso models for sparse piecewise constant signal estimations, possibly under nonzero baseline assumptions, to which the proposed enhancements with two types of constraints can achieve robustness against possible model mismatch as well as higher estimation accuracy compared with conventional fused lasso models.

The generalized minimax concave (GMC) penalty is a nonconvex sparse regularizer which can preserve the overall-convexity of the sparse least squares problem. In this paper, we study the solution path of a special but important instance of the GMC model termed the scaled GMC (sGMC) model. We show that despite the nonconvexity of the regularizer, there exists a solution path of the sGMC model which is piecewise linear as a function of the regularization parameter, and we propose an efficient algorithm for computing a solution path of this type. Our algorithm is an extension of the well-known least angle regression (LARS) algorithm for LASSO, hence we term the proposed algorithm LARS-sGMC. The proposed algorithm is provably correct and finitely terminating under suitable assumptions. Numerical experiments verify the correctness of LARS-sGMC, and demonstrate the usefulness of LARS-sGMC (with proper model selection criterion) for finding the optimal regularization parameter of the sGMC model.

In this paper, we propose a novel sparse adaptive filtering algorithm termed compensated shrinkage affine projection algorithm (CS-APA). Our cost function is the sum of a time-varying data fidelity term and a difference-of-convex (DC) type nonconvex sparse regularizer. The regularizer includes the well known MC and SCAD penalty as special instances, thus leading to sparse estimation with small bias. Leveraging the DC structure of the regularizer, the nonconvex forward-backward splitting algorithm can be applied to the cost function, whereby the proposed CS-APA is derived. We present several favourable properties of CS-APA, including its mean stability analysis. Numerical examples demonstrate the superiority of CS-APA with comparisons to existing methods.

Optimization problem with orthogonality constraints, whose feasible region is called the Stiefel manifold, has rich applications in data sciences. The severe non-linearity of the Stiefel manifold has hindered the utilization of optimization mechanisms developed specially over a vector space for the problem. In this paper, we present a global parametrization of the Stiefel manifold entirely by a single fixed vector space with the Cayley transform, say Global Cayley Parametrization (G-CP), to solve the problem through optimization over a vector space. The G-CP has key properties for solving the problem with G-CP and for applications to orthogonality constraint stochastic/distributed optimization problems. A numerical experiment shows that G-CP strategy outperforms the standard strategy with a retraction [Absil-Mahony-Sepulchre, 08].

In this paper, we propose a novel strategy which utilizes an inherent algebraic property of simultaneously diagonalizable matrix tuples, i.e., commutativity, for both (i) reducing approximate CP decomposition of a higher-order tensor to Approximate Simultaneous Diagonalization (ASD) and (ii) solving the ASD. By using a commutativity criterion, we design a matrix tuple with matrix slices of a given tensor. Then, for the ASD of the designed tuple, we use the Approximate-Then-Diagonalize-Simultaneously (ATDS) algorithm which utilizes commutativity to solve ASD. Numerical experiments show that the proposed strategy achieves better performance than conventional ones particularly when matrices to be estimated has almost collinear columns.

In this paper, we propose a new linearly involved convexity-preserving model for signal recovery by extending the idea in the generalized minimax concave (GMC) penalty [Se-lesnick' 17]. The proposed model can use nonconvex penalties but maintain the overall convexity and is applicable to much more general scenarios of signal recovery than the original GMC model. We also propose a new iterative algorithm which has theoretical guarantee of convergence to a global minimizer of the proposed model. A numerical experiment for noise suppression shows excellent edge-preserving performance of the proposed smoother in comparison with the standard convex TV smoother.

In this paper, we present a novel formulation of Approximate Simultaneous Diagonalization (ASD) with a nonconvex feasibility problem to find a structured low rank matrix whose building blocks are the Kronecker sums of given multiple matrices. To tackle this feasibility problem, we propose an alternating projection algorithm that can generate a matrix sequence approaching monotonically to a solution. By this algorithm, simultaneously diagonalizable matrices are obtained in the neighborhood of the given matrices which are not necessarily diagonalizable simultaneously. By using further the Diagonalize-One-then-Diagonalize-the-Other (DODO) method, we can obtain finally a common similarity transformation which diagonalizes the simultaneously diagonalizable matrices. Numerical experiments show that, compared with a Jacobi-like method, the proposed algorithm achieves a better approximation to the desired common similarity transformation.

Expressing multidimensional information as a value in hypercomplex number systems (e.g., quaternion, octonion, etc.) has great potential, in data sciences, e.g., signal processing, to enjoy their nontrivial algebraic benefits which are not available in standard real or complex vector systems. Strategic utilizations of such benefits would include, e.g., hypercomplex singular value decomposition (SVD) and low rank approximation of matrices. In real world applications, e.g., representing color images, of hypercomplex number systems, all attributes are often restricted to be non-negative. In this paper, we formulate non-negative matrix completion problem in hypercomplex domain as a convex optimization problem in real domain. These formulation is based on algebraic translations of Cayley-Dickson (C-D) linear systems. We then derive an algorithmic solution to hypercomplex low rank matrix completion with non-negative constraint based on a proximal splitting technique. Numerical experiments are performed in a scenario of high-dimensional hypercomplex matrix completion problem and show that the proposed algorithm recovers much more faithfully the original information, masked randomly by noise, than a part-wise state-of-art algorithms.

We propose a novel automatic shrinkage tuning technique for the adaptive proximal forward-backward splitting (APFBS) algorithm. The shrinkage tuning aims to choose an appropriate value of the shrinkage parameter and achieve minimal system mismatch as possible. The system mismatch is approximated based on time-averaged second-order statistics. Numerical examples show that the proposed method achieves performance fairly close to that with a manually chosen shrinkage parameter for colored input signals at some signal to noise ratio (SNR).

Identification of piecewise continuous Hammerstein systems, which consist of the cascade of memoryless piecewise continuous systems followed by linear systems, is an important problem in engineering. In the identification process, major existing approaches approximate exact minimization of the non-convex cost function for the piecewise continuous system, which degrades the identification accuracy in some occasions. In this paper, we propose an alternating minimization approach for identification of the piecewise continuous Hammerstein system by alleviating the difficulty in minimization of the cost function for the piecewise continuous system. We first decompose this minimization into quadratic subproblems by sorting the magnitudes of input signals. Then, based on this decomposition, the proposed method exactly minimizes the cost function by finite comparison of the solutions of the subproblems. Numerical examples show the effectiveness of the proposed method.

Expressing multidimensional information as a value in hypercomplex number systems (e.g., quaternion, octonion, etc.) has great potential, in signal processing applications, to enjoy their nontrivial algebraic benefits which are not available in standard real or complex vector systems. Strategic utilizations of such benefits would include, e.g., hypercomplex singular value decomposition (SVD) and low rank approximation of matrices. In this paper, as powerful mathematical tools for wider signal processing applications, we first propose novel definitions of SVD and best low rank approximation of matrices based on algebraic translations of Cayley-Dickson (C-D) number systems. We then derive an algorithmic solution to hypercomplex tensor completion problem based on a convex optimization technique. Numerical experiments in a scenario of color tensor completion problem show that the proposed algorithm recovers much more faithfully the original color information, masked randomly by noise, than a part-wise real tensor completion algorithm.

In 2013, Nguyen and Yamada proposed Adaptive normalized quasi-Newton algorithm and its adaptive step size for accurate and stable extraction of the first generalized eigenvector. The adaptive step size is determined by an upper bound of the condition number of a time-varying matrix. However, the employed upper bound is fairly tight only when the size of matrix is small, which degrades the performance of the algorithm for general case. In this paper, we propose new adaptive step sizes with aid of tighter upper bounds of the condition number. The proposed adaptive step sizes can be implemented efficiently, which are the same calculation order with the original adaptive step size. Numerical experiments show that the proposed adaptive step sizes succeed in extending the applicability of the algorithm.

The hybrid steepest descent method (HSDM) [Yamada, '01] was introduced as a low-computational complexity tool for solving convex variational-inequality problems over the fixed-point set of non-expansive mappings in Hilbert spaces. Motivated by results on decentralized optimization, this study introduces an HSDM variant that extends, for the first time, the applicability of HSDM to affinely constrained composite convex minimization tasks over Euclidean spaces; the same class of problems solved by the popular alternating direction method of multipliers and primal-dual methods. The proposed scheme shows desirable attributes for large-scale optimization tasks that have not been met, partly or all-together, in any other member of the HSDM family of algorithms: tunable computational complexity, a step-size parameter which stays constant over recursions, promoting thus acceleration of convergence, no boundedness constraints on iterates and/or gradients, and the ability to deal with convex losses which comprise a smooth and a non-smooth part, where the smooth part is only required to have a Lipschitz-continuous derivative. Convergence guarantees and rates are established. Numerical tests on synthetic data and on colored-image inpainting underline the rich potential of the proposed scheme for large-scale optimization tasks.

Finding a common point of multiple closed sets in a real Hilbert space has been an important task in a wide range of signal processing. In this paper, we study asymptotic properties of the parallel projection method (PPM) for closed sets satisfying a special feasibility condition, which holds in the context of certain sparse signal processing. Our analysis guarantees that the cluster point set of PPM is exactly the intersection of the closed sets, and the distance to each set along a sequence generated by PPM with arbitrary initial point converges to zero. Moreover, under certain additional assumptions, we prove that the sequence converges to a point in the intersection of the closed sets, while existing analyses gave only local behaviors of PPM.

Exploiting the sparsity in learning algorithms is a key to achieve excellent performances of adaptive filters. This can be realized by the adaptive proximal forward-backward splitting with carefully chosen parameters. In this paper, we propose an automatic parameter tuning based on a minimization principle of a stochastic approximation of the system-mismatch. The proposed approximation has a Tikhonov-type regularization term, which aims to minimize the disturbance by the update of the adaptive filter and mitigates overfitting to an instantaneous observation. Thanks to these properties, the proposed method realizes adaptive parameter tuning without any user-defined parameters, unlike our previous method that utilizes the user-defined parameter to avoid over-fitting. A numerical example demonstrates the efficacy of the proposed parameter tuning.

Nguyen and Yamada [NY'13] proposed an adaptive algorithm for fast and stable extraction of the first generalized Hermitian eigenvector and mentioned the extension to the first r generalized eigenvector extraction based on the nested orthogonal complement structure [NTY'12]. However, we recently found that the estimates of the eigenvectors are not expressed ideally in the time-varying coordinate system and can change drastically in a certain situation, which may cause numerical instability. In this paper, we propose a new expression of the estimates along with time-varying coordinate system. This modification can be done efficiently with additional multiplications of orthogonal complement matrices. Numerical experiments show that the modified scheme has better stability compared with the original scheme [NTY'12].

Spline is a piecewise polynomial and has been widely used for interpolation and smoothing of observed data. In this paper, with the use of the sufficient condition, derived by Heß and Schmidt, for the nonnegativity of bivariate splines on square grid, we propose two-dimensional positive spline interpolation/smoothing on square grid for estimation of positive continuous functions. Moreover, we newly derive a sufficient condition for the nonnegativity on triangular grid and propose positive spline interpolation/smoothing on triangular grid. Then we estimate a two-dimensional probability density function (PDF) from its histogram by using the idea of the positive spline smoothing. Numerical experiments show the effectiveness of the newly derived sufficient condition and the proposed PDF estimator.

This paper proposes fast iterative algorithms for solving convexly constrained spline smoothing through a characterization of solutions in a primal-dual space. In view of achievements for fast implementations of spline interpolation, the update of the proposed algorithm is designed as the composition of solving a spline interpolation problem and computing the projection onto the constraint set. In addition, the update of the proposed algorithm is performed in an efficient dimensional space having the same size as given observations. These desired properties significantly reduce the computational cost in the update, which is demonstrated by a numerical example.

In this paper, we propose a new modeling technique of signals having time-varying spectral patterns for supervised source separation. Typical examples of such signals are instrumental sounds having several segments such as “attack” and “sustain”. In the proposed technique, a given signal is modeled as a linear combination of multiple bases which are obtained by using reduced-rank representation of the given signal, where the number of bases is determined automatically. The proposed technique is used to generate the basis matrix in the context of supervised source separation, which improves conventional source separation methods.

Spline is a continuous function piecewise-defined by polynomials and is widely used for interpolation and smoothing of observed data. In 1994, Heβ and Schmidt proposed a positive quartic C2-spline interpolation for estimation of a non-negative and twice continuously differentiable function. In this paper, first we generalize the positive quartic C2-spline interpolation to the positive quartic C2-spline smoothing. Then we propose two estimation methods of a probability density function from its histogram by extending the ideas of the positive quartic C2-spline interpolation and smoothing. Finally numerical experiments show the effectiveness of the proposed methods.

Two-dimensional (2D) phase unwrapping is a reconstruction problem of a continuous phase, defined over 2D-domain, from its wrapped samples. In our previous work, we presented a two-step phase unwrapping algorithm which first constructs, as the real and imaginary parts of a complex function, a pair of piecewise polynomials having no common zero over the domain, then estimates the unwrapped phase by applying the algebraic phase unwrapping. In this paper, we propose a preprocessing of the above algorithm for avoiding the appearance of zeros of the complex function in the first step. The proposed preprocessing is implemented by a convex optimization and resampling, and its effectiveness is shown in a terrain height estimation by the interferometric synthetic aperture radar.

This paper proposes a second-order discrete total generalized variation (TGV) for arbitrary graph signals, which we call the graph TGV (G-TGV). The original TGV was introduced as a natural higher-order extension of the well-known total variation (TV) and is an effective prior for piecewise smooth signals. Similarly, the proposed G-TGV is an extension of the TV for graph signals (G-TV) and inherits the capability of the TGV, such as avoiding staircasing effect. Thus the G-TGV is expected to be a fundamental building block for graph signal processing. We provide its applications to piecewise-smooth graph signal inpainting and 3D mesh smoothing with illustrative experimental results.

This paper proposes a new vectorial total variation prior (VTV) for color images. Different from existing VTVs, our VTV, named the decorrelated vectorial total variation prior (D-VTV), measures the discrete gradients of the luminance component and that of the chrominance one in a separated manner, which significantly reduces undesirable uneven color effects. Moreover, a higher-order generalization of the D-VTV, which we call the decorrelated vectorial total generalized variation prior (D-VTGV), is also developed for avoiding the staircasing effect that accompanies the use of VTVs. A noteworthy property of the D-VT(G)V is that it enables us to efficiently minimize objective functions involving it by a primal-dual splitting method. Experimental results illustrate their utility.

Phase unwrapping is a reconstruction problem of the continuous phase function from its finite wrapped samples. Especially the two-dimensional phase unwrapping has been a common key for estimating many crucial physical information, e.g, the surface topography measured by interferometric synthetic aperture radar. However almost all two-dimensional phase unwrapping algorithms are suffering from either the path dependence or the excess smoothness of the estimated result. In this paper, to guarantee the path independence and the appropriate smoothness of the estimated result, we present a novel algebraic approach by combining the ideas in the algebraic phase unwrapping with techniques for a piecewise polynomial interpolation of two-dimensional finite data sequence.

For the nonlinear acoustic echo cancellation, we present an adaptive learning of the saturation effect of the amplifier and the room propagation in terms of the hard-clipping and the FIR system. The conventional learning algorithms are based on a gradient descent method, i.e., rely on local information, which results in a major drawback that the estimation of the hard-clipping is trapped in local minima. In this paper, we solve this drawback by exploiting global information embodied as a set including the desired hard-clipping with high-probability. The proposed adaptive learning of the hard-clipping is designed to track the sets with a projection-based algorithm. In the adaptive learning of the FIR system, we propose the use of the Huber loss function for the robustness against the error in the estimation of the hard-clipping. Numerical examples show that the proposed algorithm is never trapped in the local minima and has an excellent steady-state behavior.

This paper proposes to use the Total Generalized Variation (TGV) of second order in a constrained form for image processing, which we call the TGV constraint. The main contribution is twofold: i) we present a general form of convex optimization problems with the TGV constraint, which is, to the best of our knowledge, the first attempt to use TGV as a constraint and covers a wide range of problem formulations sufficient for image processing applications; and ii) a computationally-efficient algorithmic solution to the problem is provided, where we mobilize several recently-developed proximal splitting techniques to handle the complicated structured set, i.e., the TGV constraint. Experimental results illustrate the potential applicability and utility of the TGV constraint.

We consider the problem of electroencephalography (EEG) and magnetoencephalography (MEG) source localization using beamforming techniques. Specifically, we propose a reduced-rank extension of the recently derived multi-source activity index (MAI), which itself is an extension of the classical neural activity index to the multi-source case. We show that, for uncorrelated dipole sources and any nonzero rank constraint, the proposed reduced-rank multi-source activity index (RR-MAI) achieves the global maximum when evaluated at the true source positions. Therefore, the RR-MAI can be used to localize multiple sources simultaneously. Furthermore, we propose another version of the RR-MAI which can be seen as a natural generalization of the proposed index to arbitrarily correlated sources. We present a series of numerical simulations showing that the RR-MAI can achieve a more precise source localization than the full-rank MAI in the case when the EEG/MEG forward model becomes ill-conditioned, which in our settings corresponds to the case of closely positioned sources and low signal-to-noise ratio.

We propose to detect edges of reflections, which we call the REF-edges, from a single image via convex optimization. Our method is designed based on two observations on reflections: (i) reflections have almost monotone color and (ii) color around REF-edges varies smoothly. The first one can be translated into the property that gradients around REF-edges distribute linearly in the RGB color space, which we call the REF-linearity. The second one can be interpreted as follows: color differences around REF-edges are small; for an entry of REF-edges, gradients among its surrounding entries have small variance. Using the above properties, we characterize REF-edges as a solution of a constrained convex optimization problem. The optimization problem is solved by the Alternating Direction Method of Multipliers (ADMM). Experiments using real-world images with reflections show the utility of our proposed method.

This paper is concerned with the mean-square performance of the hyperslab-based adaptive projected subgradient method, a set theoretic estimation tool that has been successfully applied in a wide variety of signal processing tasks. Using energy-conservation arguments, general performance results are derived without restricting the regression data to being Gaussian or white. Numerical simulations are provided to illustrate the theoretical developments.

Effective utilization of sparsity of the system to be estimated is a key to achieve excellent adaptive filtering performances. This can be realized by the adaptive proximal forward-backward splitting (APFBS) with carefully chosen parameters. In this paper, we propose a systematic parameter tuning based on a minimization principle of an unbiased MSE estimate. Thanks to the piecewise quadratic structure of the proposed MSE estimate, we can obtain its minimizer with low computational load. A numerical example demonstrates the efficacy of the proposed parameter tuning by its excellent performance over a broader range of SNR than a heuristic parameter tuning of the APFBS.

Using a novel characterization of texture, we propose an image decomposition technique that can effectively decomposes an image into its cartoon and texture components. The characterization rests on our observation that the texture component enjoys a blockwise low-rank nature with possible overlap and shear, because texture, in general, is globally dissimilar but locally well patterned. More specifically, one can observe that any local block of the texture component consists of only a few individual patterns. Based on this premise, we first introduce a new convex prior, named the block nuclear norm (BNN), leading to a suitable characterization of the texture component. We then formulate a cartoon-texture decomposition model as a convex optimization problem, where the simultaneous estimation of the cartoon and texture components from a given image or degraded observation is executed by minimizing the total variation and BNN. In addition, patterns of texture extending in different directions are extracted separately, which is a special feature of the proposed model and of benefit to texture analysis and other applications. Furthermore, the model can handle various types of degradation occurring in image processing, including blur+missing pixels with several types of noise. By rewriting the problem via variable splitting, the so-called alternating direction method of multipliers becomes applicable, resulting in an efficient algorithmic solution to the problem. Numerical examples illustrate that the proposed model is very selective to patterns of texture, which makes it produce better results than state-of-the-art decomposition models.

We propose a new convex regularizer, named the local color nuclear norm (LCNN), for color image recovery. The LCNN is designed to promote a property inherent in natural color images - in which their local color distributions often exhibit strong linearity - and is thus expected to reduce color artifact effectively. In addition, the very nature of LCNN allows us to incorporate it into various types of color image recovery formulations, with the associated convex optimization problems solvable using proximal splitting techniques. Applications of LCNN are demonstrated with illustrative numerical examples.

The phase unwrapping, which is a problem to reconstruct the continuous phase function of an unknown complex function from its finite observed samples, has been a key for estimating useful physical quantity in many signal and image processing applications. In the light of the functional data analysis, it is natural to estimate first the unknown complex function by a certain piecewise complex polynomial and then to compute the exact unwrapped phase of the piecewise complex polynomial with the algebraic phase unwrapping algorithms. In this paper, we propose several useful extensions and numerical stabilization of the algebraic phase unwrapping along the real axis. The proposed extensions include (i) removal of a certain critical assumption premised in the original algebraic phase unwrapping, and (ii) algebraic phase unwrapping for a pair of bivariate polynomials. Moreover, in order to resolve certain numerical instabilities caused by the coefficient growth in an inductive step in the original algorithm, we propose to compute directly a certain subresultant sequence without passing through the inductive step.

The hypercomplex (e.g., complex, quaternion) valued linear model often arises in the signal processing field and attract increasing attention recently. In this paper, we present an algebraic translation of a hypercomplex valued linear systems into a real valued linear model. This translation is designed by taking advantage of isomorphism between hypercomplex numbers and multi-dimensional real vectors and enables us to straightforwardly apply real valued optimization frameworks to various estimation problems for the hypercomplex linear model. We also clarify the useful algebraic properties of the translation. As an application to hypercomplex valued adaptive filtering problems, we derived Am-adaptive projected subgradient method (Am-APSM) for hypercomplex valued system identification problems, and show that many hypercomplex adaptive filtering algorithms can be viewed as a special case of this algorithm. Numerical example shows that a new algorithm derived from proposed algorithm outperforms existing hypercomplex adaptive algorithms.

This paper proposes a likelihood constrained optimization framework for Poisson image restoration. The likelihood constrained problem considered in this paper is the minimization of convex priors over the level set of the negative-log-likelihood function of the Poisson distribution. It has advantages in parameter selection compared with the minimization of the weighted sum of convex priors and the negative-log-likelihood function, which has been used in conventional methods. The level set is characterized as the fixed point set of a certain quasi-nonexpansive operator, which enables us to apply the hybrid steepest descent method to solve the constrained problem. The proposed framework not only can handle the level set of any convex function whose subgradient is available but also does not require any computationally-expensive procedure such as operator inversion and inner loop. Illustrative numerical examples are also presented.

Observing that sparse systems are almost smooth, we propose to utilize the newly-introduced adaptively-weighted total variation (AWTV) for sparse system identification. In our formulation, a sparse system identification problem is posed as a sequential suppression of a time-varying cost function: the sum of AWTV and a data-fidelity term. In order to handle such a non-differentiable cost function efficiently, we propose a time-varying extension of a primal-dual splitting type algorithm, named the adaptive primal-dual splitting method (APDS). APDS is free from operator inversion or other highly complex operations, resulting in computationally efficient implementation in online manner. Moreover, APDS realizes that the sequence defined in a certain product space monotonically approaches the solution set of the current cost function, i.e., the sequence generated by APDS pursues desired replicas of the unknown system in each time-step. Our scheme is applied to a network echo cancellation problem where it shows excellent performance compared with conventional methods.

We consider the problem of dipole source signals estimation in electroencephalography (EEG) using beamforming techniques in ill-conditioned settings. We take advantage of the link between the linearly constrained minimum-variance (LCMV) beamformer in sensor array processing and the best linear unbiased estimator (BLUE) in linear regression modeling. We show that the recently introduced reduced-rank extension of BLUE, named minimum-variance pseudo-unbiased reduced-rank estimator (MV-PURE), achieves much lower estimation error not only than LCMV beamformer, but also than the previously derived reduced-rank principal components (PC) and cross-spectral metrics (CSM) beamformers in ill-conditioned settings. The practical scenarios where the considered estimation model becomes ill-conditioned are discussed, then we show the applicability of MV-PURE dipole source estimator under those conditions through realistic simulations.

We propose a proximal splitting approach to regularized distributed estimation over networks employing diffusion adaptation strategies. Playing a central role in the proposed framework is the so-called proximity operator, which is a generalization of the convex projection mapping, that enables us to handle convex regularization terms efficiently. The diffusion algorithms developed using the proximal formalism endow networks with new learning abilities and open up possibilities for enhancing performance of the networks by utilizing more general convex penalties. We present performance analysis of the proposed method and provide simulations to demonstrate its feasibility in recovering sparse signals.

The Minimum-Variance Pseudo-Unbiased Reduced-rank Estimator (MV-PURE) is designed, as a natural reduced-rank extension of the Gauss-Markov estimator, for the unknown deterministic vector in ill-conditioned linear regression model. In this paper, we propose a novel rank-selection for the MV-PURE to achieve a small Mean Square Error (MSE). The proposed rank-selection is realized by minimizing an unbiased estimate of the predicted-MSE, not of the MSE. Our unbiased estimate can be applicable to any noise distribution with zero mean and a finite covariance matrix, while Stein-type unbiased criteria cannot in general. We apply the proposed selection to an image restoration problem and introduce its efficient O(m log m) implementation by using a special structure found in typical blur matrices, where the blur matrix is of size m×m. A numerical example demonstrates that the MV-PURE with the proposed rank-selection achieves a MSE comparable with the minimal MSE for the unknown vector among all possible ranks.

We propose a novel framework to obtain an artifact-free enlarged image from a given JPEG image. The proposed formulation based on a newly introduced JPEG image acquisition model realizes decompression and super-resolution interpolation simultaneously using multi-order total variation, so that we can drastically reduce artifacts appearing in JPEG images such as block noise and mosquito noise, without generating staircasing effect, which is typical in existing total variation-based JPEG decompression methods. We also present a computationally-efficient optimization scheme, derived as a special case of a primal-dual splitting type algorithm, for solving the convex optimization problem associated with the proposed formulation. Numerical examples show that the proposed method works effectively compared with existing methods.

This letter establishes a novel analysis of the Adaptive Projected Subgradient Method (APSM) in the intersection of the stochastic and robust estimation paradigms. Utilizing classical worst-case bounds on the noise process, drawn from the robust estimation methodology, the present study demonstrates that the hyperslab-inspired version of the APSM generates a sequence of estimates which converges to a point located, with probability one, arbitrarily close to the estimand. Numerical tests and comparisons with classical time-adaptive algorithms corroborate the theoretical findings of the study.

In this paper, we propose a novel missing region recovery method by promoting blockwise low-rankness. It is natural to assume that images often have local repetitive structures. Hence, any small block extracted from an image is expected to be a low-rank matrix. Based on this assumption, we formulate missing region recovery as a convex optimization problem via newly introduced block nuclear norm which promotes blockwise low-rankness of an image with missing regions. An iterative scheme for approximating a global minimizer of the problem is also presented. The scheme is based on the alternating direction method of multipliers (ADMM) and allows us to restore missing regions efficiently. Experimental results reveal that the proposed method can recover missing regions with detailed local structures.

The stochastic MV-PURE estimator is a linear estimator for stochastic linear model that is highly robust to mismatches in model knowledge and which is specially designed for efficient estimation in noisy and ill-conditioned cases. To date, its properties were analyzed in the theoretical settings of perfect model knowledge and thus could not explain clearly the reason behind its superior performance compared to the Wiener filter observed in simulations in practical cases of imperfect model knowledge. In this paper we derive closed form expressions of the mean-square-error (MSE) of both Wiener filter and the stochastic MV-PURE estimator for the case of perturbed singular values of a model matrix in the linear model considered. These expressions provide in particular conditions under which the stochastic MV-PURE estimator achieves smaller MSE not only than Wiener filter, but also than its full-rank version, the minimum-variance distortionless (MVDR) estimator in such settings. We provide numerical simulations confirming the main theoretical results presented.

We propose variants of Alternating Direction Method of Multipliers (ADMM) employing simplified updates under additional assumptions. ADMM iteratively solves the minimization of the sum of two nonsmooth convex functions. Each iterations of ADMM itself consists of solving a certain convex optimization problem which often requires the use of some iterative solver. Such inner iterations cause slow convergence. Our proposed algorithms avoid some of inner iterations by employing simplified updates. An efficacy of the proposed algorithm is shown in an image super-resolution problem. In this application, the resultant algorithm does not require matrix inversion which causes inner iterations of the original ADMM. A numerical example in the image super-resolution setting demonstrates that our proposed algorithms reduce CPU time to about 70-80 percent of the original ADMM.

We propose a novel sparsity-aware adaptive filtering algorithm based on iterative use of weighted soft-thresholding. The weights are determined based on a rough local approximation of the ℓpnorm (0 <; p <; 1). The proposed algorithm operates the weighted soft-thresholding for enhancing the sparsity, following estimation error managements with the affine projection. The proposed weighting technique alleviates an extra bias of no benefit caused by shrinking dominant coefficients. The numerical examples demonstrate that the proposed weighting technique outperforms the existing one when the situation changes under the fixed parameter settings.

In this paper, we propose an acceleration technique of the adaptive filtering scheme called adaptive proximal forward-backward splitting method. For accelerating the convergence rate, the proposed method includes a step to shift the current estimate in the direction of the difference between the current and previous estimates based on the Fast Iterative Shrinkage/Thresholding Algorithm (FISTA). The computational complexity for this additional step is fairly low compared to the overall complexity of the algorithm. As an example of the proposed method, we derive an acceleration of the composition of the Adoptively Weighted Soft-Thresholding (AWST) operator and the exponentially weighted adaptive parallel projection. AWST shrinks the estimated filter coefficients to zero for exploiting the sparsity of the system to be estimated and the exponentially weighted adaptive parallel projection algorithm realizes high accuracy by utilizing all available information at each iteration. This accelerated method improves the steady-state mismatch drastically with its con vergence speed as fast as the proportionate affine projection algorithm.

We address the problem of recovering a low-rank matrix that has a small fraction of its entries arbitrarily corrupted. This problem is recently attracting attention as nontrivial extension of the classical PCA (principal component analysis) problem with applications in image processing and model/system identification. It was shown that the problem can be solved via a convex optimization formulation when certain conditions hold. Several algorithms were proposed in the sequel, including interior-point methods, iterative thresholding and accelerated proximal gradients. Based on algorithms from rank minimization and sparse vector recovery, we propose a computationally efficient greedy algorithm that scales better to large problem sizes than existing algorithms.

In this paper, we propose a novel adaptive filtering algorithm based on an iterative use of (i) the proximity operator and (ii) the parallel variable-metric projection. Our time-varying cost function is a weighted sum of squared distances (in a variable-metric sense) plus a possibly nonsmooth penalty term, and the proposed algorithm is derived along the idea of proximal forward-backward splitting in convex analysis. For application to sparse-system identification problems, we employ the (weighted) ℓ1norm as the penalty term, leading to a time-varying soft-thresholding operator. As the simple example of the proposed algorithm, we present the variable-metric affine projection algorithm composed with the time-varying soft-thresholding operator. Numerical examples demonstrate that the proposed algorithms notably outperform their counterparts without soft-thresholding both in convergence speed and steady-state mismatch, while the extra computational complexity due to the additional soft-thresholding is negligibly low.

This paper presents an efficient link probability control strategy for distributed estimation problems over probabilistic diffusion networks where the mean usage of communication resources is restricted. The proposed algorithm controls link probabilities so that the estimation error is minimized under given resource constraints. Simulation results show that the probabilistic diffusion least-mean squares (LMS) algorithm with the proposed probability control not only outperforms those with static probabilities but also reduces the amount of communications among nodes.

We propose multi-domain adaptive filtering based on the idea of feasibility splitting - dealing with feasibility in individual domains. The proposed approach provides a useful and mathematically rigorous framework to incorporate multiple pieces of information (expressed in different domains) efficiently. Indeed, it processes such multiple pieces of information by means of the metric projection in each individual domain; this is a significant advantage over existing single-domain approaches. Also we provide a reasonable strategy to treat the case where the available prior information is inconsistent. A convergence analysis and numerical examples are presented to support the proposed method.

This paper addresses the minimal subset selection of antennas achieving designated channel capacity. This is one of the most natural approaches to alleviating the power consumption in MIMO systems, while it is a mathematically challenging nonlinearly-constrained sparse optimization (ℓ0-norm minimization) problem. We present an efficient algorithmic solution, to this highly combinatorial problem, using convex and differentiable relaxations of the (ℓ0-norm. The proposed algorithm is based on the hybrid steepest descent method for the subgradient projection operator together with the soft-thresholding technique, minimizing the Moreau envelope of the (ℓ1-norm subject to the capacity constraint. The simulation results show that the proposed algorithm realizes a near optimal solution to the original nonlinearly-constrained sparse optimization problem.

We present a rigorous deterministic analysis of the variable-metric adaptive filtering algorithms (including the transform-domain, LMS/Newton, and proportionate adaptive filters) by using the framework of variable-metric adaptive projected subgradient method (Yukawa et al. 2007). Under small metric-fluctuations, we present the useful properties of (i) monotone approximation - with respect to a certain constant metric - indicating the stability of the algorithm and (ii) convergence to an asymptotically optimal point. Numerical examples show the advantage of the variable-metric adaptive filtering algorithms and suggest the validity of the analysis.

We present an algorithm that minimizes asymptotically a sequence of non-negative convex functions over diffusion networks. To account for possible node failures, position changes, and/or reachability problems (because of moving obstacles, jammers, etc), the algorithm can cope with dynamic networks and cost functions, a desirable feature for online algorithms where information arrives sequentially. Many projection-based algorithms can be straightforwardly extended to diffusion networks with the proposed scheme. We use the acoustic source localization problem in sensor networks as an example of a possible application.

The stochastic MV-PURE estimator has recently emerged as the robust solution for frequently occuring in practice problem of linear estimation in ill-conditioned and imperfectly known linear stochastic model. In this paper we provide theoretical results showing that the stochastic MV-PURE estimator can be used to the greatest effect in highly noisy settings. In such settings, we discuss the relation between the stochastic MV-PURE estimator and the well-known reduced rankWiener filter. We verify the theoretical results presented by a means of numerical simulations.

This paper presents an efficient adaptive combination strategy for diffusion algorithms over adaptive networks in order to improve the robustness against the spatial variation of SNR over the network. The diffusion least-mean square (LMS) algorithm with the proposed combination rule and its mean transient analysis are included. Simulation results show that the diffusion LMS algorithm with our combiners outperforms those with existing static combiners and the incremental LMS algorithm.

The subgradient projection operator has been utilized as a computationally efficient tool not only for suppression but also for minimization of convex functions in many applications. In this paper, we propose a systematic scheme to improve significantly the monotone approximation ability, of the subgradient projection, to the level set of a convex function. The proposed scheme is based on a simple observation: the level set of a convex function does not change by composing any zero-crossing monotonically increasing function. A numerical example demonstrates the effectiveness of the proposed scheme in an application to a simple boosting problem.

One of the main drawbacks of the OFDM modulation is the high peak-to-average power ratio (PAPR) of the transmitted signal. In this study, we devise a low-complexity transmitter that mitigates the PAPR problem and satisfies multiple requirements of a given system, such as error vector magnitude (EVM) constraints, minimum transmitted power of the data symbols, among others. To reduce the PAPR, we use the adaptive projected subgradient method to suppress a sequence of convex cost functions over closed convex sets describing desired properties of the transmitted signal. Numerical examples show that the proposed scheme reduces the PAPR to reasonable levels with few iterations.

This paper presents an analysis of the steady-state mean-square error of an adaptive filtering algorithm using the metric projection onto a closed hyperslab, which we refer to as the hyperslab projection algorithm (HSPA). HSPA is not only a generalization of both the normalized least mean square (NLMS) algorithm and the set-membership NLMS (SM-NLMS) algorithm but also a special case of the adaptive parallel subgradient projection (PSP) method. It is known that HSPA possesses both fast convergence and robustness against noise. The approach of this paper is to employ the energy conservation relation, which enables us to avoid the transient analysis of HSPA. Under different assumptions, we obtain two results, which are generalizations of well-known results of the steady-state performance of NLMS. Extensive simulations show the good match between the theories and experiments.

We introduce adaptive linear filters based on the adaptive projected subgradient method that are suitable for online implementation of multiple access interference (MAI) suppression in OSTBC-MIMO systems. The proposed adaptive filters track the optimal solution of a new cost function that is robust against channel state information (CSI) mismatch. The adaptive update algorithm is based on projections onto closed convex sets that contain the optimal solution with high probability. The main features of the adaptive filters are that no matrix inversion of a sample covariance matrix is required and that a low-complexity recursive implementation is possible. Convergence analysis and simulation results show the effectiveness of the proposed schemes.

The goal of this paper is the development of a novel efficient online kernel-based algorithm for classification. The spirit of the algorithm stems from the recently introduced adaptive projected subgradient method. This is a general convex analytic tool that employs projections onto a sequence of convex sets and it can be considered as a generalization of the celebrated APA algorithm, widely used in classical adaptive filtering.

In this paper, we propose a novel adaptive filtering algorithm named adaptive parallel variable-metric projection (APVP) algorithm, which includes the proportionate normalized least mean square (PNLMS) algorithm as its special example. The proposed algorithm is based on parallel projection (onto multiple closed convex sets) with time-varying metrics. A convergence analysis of the proposed algorithm is presented with the aid of the adaptive projected subgradient method. Numerical examples demonstrate that the proposed algorithm realizes echo cancellation superior to the conventional algorithms.

This paper indicates that an appropriate design of metric leads to significant improvements in the adaptive projected subgradient method (APSM), which unifies a wide range of projection-based algorithms [including normalized least mean square (NLMS) and affine projection algorithm (APA)]. The key is to incorporate a priori (or a posteriori) information on characteristics of an estimandum, a system to be estimated, into the metric design. We propose a family of efficient adaptive filtering algorithms based on a parallel use of quadratic-metric projection, which assigns every point to the nearest point in a closed convex set in a quadratic-metric sense. We present two versions: (1) constant-metric and (2) variable-metric, i.e., the metric function employed is (1) constant and (2) variable among iterations. As a constant-metric version, adaptive parallel quadratic-metric projection (APQP) and adaptive parallel min-max quadratic-metric projection (APMQP) algorithms are naturally derived by APSM, being endowed with desirable properties such as convergence to a point optimal in asymptotic sense. As a variable-metric version, adaptive parallel variable-metric projection (APVP) algorithm is derived by a generalized APSM, enjoying an extended monotone property at each iteration. By employing a simple quadratic-metric, the computational complexity of the proposed algorithms is kept linear with respect to the filter length. Numerical examples demonstrate the remarkable advantages of the proposed algorithms in an application to acoustic echo cancellation.

Constrained normalized adaptive filters are used as a computationally efficient class of receivers to decrease multiple access interference (MAI). Some receivers estimate the amplitude and/or use different normalization parameters in order to improve the convergence speed. In this paper, we derive the steady-state performance of this class of receivers, from which it is revealed that the normalization parameters that aim at increasing the convergence speed deteriorate the steady-state performance if the step-size is not changed. Additionally, we prove that an estimate of the desired user's amplitude can greatly improve the steady-state performance. Computer simulations show remarkably good agreement with our analysis

It is well-known that the Capon beamformer is sensitive to array steering vector errors and may result into a worse performance than classical data-independent beamformers. This paper follows a different path from the well-established diagonal loading techniques and designs a robust Capon beamformer by a recent extension of the adaptive projected subgradient method. The proposed method marks a computational complexity of O(N2), where N is the number of array elements. The simulation results show that the proposed beamformer achieves excellent performance especially in cases where the diagonal loading techniques face difficulties, i.e. in cases where the interference to noise ratio (INR) is moderately larger than SNR

This paper proposes a novel heuristic approach to the infeasible power control problem [i.e., no power allocation can provide users with specified quality of service (QoS)] in wireless communication systems. In such infeasible cases, it has been in great demand to increase the number of accepted users (i.e., users provided with the specified QoS) by certain distributed algorithms, which require no communication among base stations (BSs). The number of accepted users is often suppressed by a small number of users in severe environments, hence removing such users obviously helps many other users be accepted. The proposed algorithm, which is fully distributed, detects and removes such users based on simple criteria including: (a) the QoS is below the necessary level, and (b) the BS is already transmitting with the maximum possible power to the user. Simulation results demonstrate that the proposed algorithm significantly increases the number of accepted users with low computational complexity

This paper presents a mathematically novel low-rank linear statistical estimator named minimum-variance pseudo-unbiased low-rank estimator for applications to ill-conditioned linear inverse problems. Based on a simple fact: `any low-rank estimator can not be a (uniformly) unbiased estimator', we introduce pseudo-unbiased low-rank estimator, as an ideal low-rank extension of unbiased estimators. The minimum-variance pseudo-unbiased low-rank estimator minimizes the variance of estimate among all pseudo-unbiased low-rank estimators, hence it is characterized as a solution to a double layered nonconvex optimization problem. The main theorem presents an algebraic structure of the minimum-variance pseudo-unbiased low-rank estimator in terms of the singular value decomposition of the model matrix in the linear statistical model. The minimum-variance pseudo-unbiased low-rank estimator is not only a best low-rank extension of the minimum-variance unbiased estimator (i.e., Gauss-Markov estimator) but also a nontrivial generalization of the Marquardt's low-rank estimator (Marquardt 1970)

The contribution of this paper is twofold. We first clarify geometrically an inherent difference in convergence speed between two adaptive algorithms, projected-NLMS (PNLMS) and constrained-NLMS (CNLMS), both of which are widely used for linearly constrained adaptive filtering problems. A simple geometric interpretation suggests that CNLMS converges faster than PNLMS especially in the challenging situations of the adaptive beamforming where there exist spatially-correlated interferences (i.e., interferences that have small angular separation with the desired signal). To enhance the advantage of CNLMS in convergence speed while keeping linear computational complexity, we then propose an efficient adaptive beamformer that utilizes multiple data at each iteration by extending the constrained parallel projection algorithm to complex cases. The simulation results demonstrate that the proposed beamformer exhibits even faster convergence than the constrained affine projection algorithm (CAPA) as well as CNLMS

The adaptive projected subgradient method offers a unified mathematical perspective for the adaptive (set-membership/set-theoretic) filtering schemes. In this paper, we introduce an overview of its recent theoretical advances and successful applications to robust signal processing problems including the stereo acoustic echo canceling, the MAI suppression in DS/CDMA receivers, and the robust adaptive beamforming with array antenna systems

In this paper, we propose a method that recovers a smooth high-resolution image from several blurred and roughly quantized low-resolution images. For compensation of the quantization effect we introduce a measurement of smoothness originally used for suppression of block noises in a JPEG compressed image [Schultz & Stevenson '94]. With a simple operator that approximates to the convex projection onto constraint set defined for each quantized image [Hasegawa et al. '05], we propose a method that minimizes these cost functions, which are smooth convex functions, over the intersection of all constraint sets, i.e. the set of all images satisfying all quantization constraints simultaneously, by using hybrid steepest descent method [Yamada & Ogura '04]. Finally in the numerical example we compare images derived by the proposed method, POCS based conventional method, and generalized proposed method minimizing smoothed total variation and energy of output of Laplacian

This paper presents a family of multiple access interference (MAI) suppression receivers based on the adaptive projected subgradient method. The proposed scheme can be applied to many different channel models and modulations in a unified manner. Moreover, it is suitable for both blind and nonblind receivers. The adaptive projected subgradient method realizes excellent convergence to a set including an optimal solution with high probability by asymptotically minimizing a sequence of nonnegative convex functions. Simulation results show much better (variable) tradeoff between speed and performance at steady-state as compared to existing techniques

We present a novel adaptive super-resolution of videos based on an embedded constraint version of adaptive projected subgradient method (Yamada & Ogura, Numerical Functional Analysis and Optimization, vol 25, no.7&8, p.593-617, 2004). The super-resolution image recovery problem is formulated as an estimation of linear time-varying systems, which is a modified version of (Elad & Feuer, IEEE Trans. on Image Proc., vol.8, no.3, p.387-395, 1999). Our method efficiently improves the estimation accuracy by simple iterative operations which can be processed on parallel systems. Robustness to additive noise as well as inaccurate estimation of degradation parameters, is realized by incorporating stochastic information of the noise.

The paper presents two novel blind set-theoretic adaptive filtering algorithms for multiple access interference (MAI) suppression in DS/CDMA systems. We naturally formulate the problem of MAI suppression as minimizing asymptotically a sequence of cost functions under some linear constraint defined by the desired user's signature. The proposed algorithms embed the constraint in the direction of adaptation, and thus the adaptive filter moves toward the optimal filter without stepping away from the constraint set. In addition, using parallel processors, the proposed algorithms attain good performance behavior with low computational complexity. Geometric interpretation clarifies an advantage of the proposed methods over some conventional methods. Simulation results demonstrate that the proposed algorithms achieve much faster convergence than conventional methods with a moderate number of concurrent processors.

An efficient scheme is presented to the color super-resolution problem for recovery of a color high-resolution image with knowledge of multiple Bayer filtered low-resolution images. To recover a visually natural high-resolution image, we restrict fairly smooth initial candidates to all images satisfying all bounds imposed on the several nonsmooth convex color total variations as well as a non-smooth convex inter cross correlation measure among color channels. In the proposed scheme, the data-fidelity is optimized in a systematic way, over all initial candidates, with the hybrid steepest descent method for quasi-nonexpansive mappings [Yamada & Ogura 2004], by minimizing successively an weighted average of pure mean square errors between the low-resolution transforms of the high-resolution estimate and the multiple low-resolution images. Numerical examples show that the proposed scheme recovers visually natural high resolution images by resolving the tradeoff between noise suppression and edge preservation of the recovered image while keeping fair inter channel cross correlation among color channels.

In this paper, we present a novel iterative MPEG super-resolution method based on an embedded constraint version of adaptive projected subgradient method [Yamada & Ogura 2003]. We propose an efficient operator that approximates convex projection onto a set characterizing framewise quantization, whereas a conventional method can only handle a convex projection defined for each DCT coefficient of a frame. By using the operator, the proposed method generates a sequence that efficiently approaches to a solution of super-resolution problem defined in terms of quantization error of MPEG compression.

We introduce a locally reduced-rank optimal filtering that is a generalization of the globally reduced-rank optimal filtering studied extensively as a fundamental tool in signal processing applications. After formulating the problem of locally reduced-rank optimal filtering, we present a closed form solution to the problem in terms of SVD. Moreover, in a way similar to the techniques shown recently by Y. Hua et al. (see IEEE Trans. Sig. Processing, vol.49, p.457-69, 2001), we deduce a numerical algorithm converging globally and exponentially to the solution without passing any computation of the eigenvalue decomposition (or SVD). A numerical example shows that the proposed algorithm converges efficiently to the locally reduced-rank optimal filter that realizes an ideal trade-off between rank-reduction and estimation accuracy.

In this paper, we propose a novel higher order time-frequency distribution (GDH) for discrete time signals. This distribution is defined over the original discrete time-frequency grids through a delicate discretization of an equivalent expression of a higher order distribution, for continuous time signals, in [Fonollosa & Nikias 1993]. We also present a constructive design method, for the kernel of the GDH, by which the distribution satisfies (i) the alias free condition as well as (ii) the marginal conditions. A numerical example shows that the proposed distribution reasonably suppresses the artifacts which are observed severely in a simple higher order generalization of the Wigner distribution.

It is well-known that the unbiased estimate of the covariance matrix of a real vector wide sense stationary process is not necessarily positive semidefinite. By defining the real Hilbert space of all symmetric matrices, the conditions for a symmetric matrix to be positive definite, block Toeplitz, as well as to satisfy other design constraints, are formed as closed convex sets. This paper demonstrates that the problem of approximating the unbiased estimate of the covariance matrix of a real vector wide sense stationary process over the intersection of those closed convex sets in an optimal way can be resolved by the Hybrid Steepest Descent Method. An optimal solution is also provided even when inconsistent constraints are met, i.e., whenever the intersection of the closed convex sets is empty. The numerical results exhibit significant improvement of the proposed method over the standard estimates of the covariance matrix.

This paper presents a novel robust adaptive filtering scheme based on the interactive use of statistical noise information and an extension of the ideas developed originally for efficient algorithmic solutions to the convex feasibility problems. The statistical noise information is quantitatively formulated as stochastic property closed convex sets by the simple design formulae developed. The proposed adaptive algorithm is computationally efficient and robust to noise because it requires only an iterative parallel projection onto a series of closed half spaces highly expected to contain the unknown system to be identified. The numerical examples show that the proposed adaptive filtering scheme achieves low estimation error and realizes dramatically fast and stable convergence even for highly colored excited input signals in severely noisy situations.

We propose a simple set-theoretic blind deconvolution scheme based on a previously developed convex projection technique called hybrid steepest descent methods. The scheme is essentially motivated by Kundur and Hatzinakos's (see IEEE Signal Processing Magazine, vol.13, no.3, p.43-63, 1996 and IEEE Trans. Signal Processing, vol.46, p.375-90, 1998) idea that minimizes a certain cost function uniformly reflecting all a priori information such as the (i) nonnegativity of the true image and the (ii) support size of the original object. The most remarkable feature of the proposed scheme is that one can utilize each a priori information separately from other ones, where some partial information are treated in a set theoretic sense while the others are incorporated in a cost function to be minimized.

Recently, a novel set-theoretic linear blind deconvolution scheme was developed by applying hybrid steepest descent method to Kundur and Hatzinakos' simple a priori information on the original object, where the performance of the scheme seems relatively sensitive to the additive measurement noise. In this paper, we remark some well-known nonlinear filtering techniques which realize immediate effect to suppress the influence of the additive measurement noise in the input to the scheme. Numerical examples show ϵ-separating nonlinear pre-filtering techniques work suitably to this noisy blind deconvolution problem.

The convex set feasibility framework has been widely applied to signal and image processing problems including signal deconvolution, tomographic reconstruction, band limited extrapolation, image restoration and image synthesis. In this paper, we consider convex constrained versions of inconsistent signal feasibility problems. First we derive some new properties of variational nonexpansive operators and convex projections. Based on these properties and fixed point theorems, we propose some types of algorithms called constrained parallel projection methods (CPPM) that solve the convex constrained versions of inconsistent signal feasibility problems.

One dimensional (1-D) perfect reconstruction (PR) QMF banks have been studied extensively. If all the analysis filters are linear phase in a PR QMF bank, we call it a 1-D linear phase PR QMF bank. Nguyen and Vaidyanathan showed a necessary condition for 1-D linear phase PRQMF banks [1989]. Recently, two dimensional (2-D) PR QMF banks have been studied. This paper shows a necessary condition for 2-D linear phase PR and MF banks. Our result is easily generalized to M dimensional linear phase PR QMF banks. In a 2-D system, subsampling is defined by a subsampling matrix D, where D is a 2*2 nonsingular matrix of integers. The sampling retains only samples at points m=(m/sub 1/,m/sub 2/)/sup T/ such that m=Dn, where n=(n/sub 1/,n/sub 2/)/sup T/ is an arbitrary integer vector. One out of every mod det(D) mod samples of the sequence is retained. A 2-D N channel analysis/synthesis filter bank is shown. We assume that all channels share the same subsampling matrix D such that mod det(D) mod =N (maximally decimated). For a matrix B, (B)/sub (i,j/) denotes the (i,j) element of B, diag(a/sub 0/,a/sub 1/,...,a/sub N-1/) denotes an N*/spl times/N diagonal matrix whose (i,i) element is a/sub i-1/.>

This paper presents a design method of linear phase PR QMF banks of the second approach. H/sub 1/(z) can be optimized not only in L/sub 2/-norm sense, but also in L/sub /spl infin//-norm sense (essentially, in any sense). Even in L/sub 2/-norm sense, the proposed method is more efficient than the Lagrange multiplier method. The proposed design method can be extended to M(>2)-channel systems immediately. Simulation results are shown for 2- and 3-channel systems, respectively.>

This paper describes a 200 Mbit/s multiservice optical local area network (LAN) using a synchronous TDM loop structure. The LAN consists of a central supervisory node and multiple service nodes connected by an optical fiber loop. Each service node supports communication channels which have access to allocated time slots in TDM frames continuously circulating on the loop. Multiple independent communication paths of various speeds up to 140 Mbits/s and various modes including point-to-point, ring, and multicast, can be provided between the channels on the loop. The ring will be useful to support ring networks, such as a token ring. The structure of this LAN is quite suitable for integration of multiple services, including video, image, data, and voice, since each service can independently choose its own speed, access method, and mode. In this development, various LSI-based high-speed hardware technologies including compact E/O and O/E modules, GaAs 4 × 4 matrix switch LSI's, and high-speed TDM-processor LSI's, which are versatilely applicable to high-speed LAN's ranging from 100 Mbits/s up to 560 Mbits/s, have been successfully introduced and compact LAN equipment has been obtained. This paper deals mainly with the system and hardware structure of this LAN, together with high-speed hardware technologies. An outline of firmware and network operation, and an application example are also described.