This paper proposes an image restoration method using a convolutional sparse coding (CSC) unrolling network with a box constraint and total variation. Unlike conventional deep unrolling methods, the proposed method constructs an interpretable lightweight network with restoration stability. Specifically, we design a new constrained convex optimization problem that incorporates CSC, a box constraint, and total variation (TV). The box constraint ensures that the image values fall within a certain range, making the restoration process stable. In addition, combining total variation and CSC leads to high interpretability and representation with a small number of parameters. We develop an optimization algorithm based on the primal-dual splitting (PDS) method. Then, by unrolling the algorithm, we construct the proposed lightweight network. Experimental results demonstrate the superiority of the proposed method in image restoration accuracy and lightweightness of the proposed network in the number of parameters.

This paper proposes a method for sampling graph signals by designing a flexible sampling operator via a difference-of-convex (DC) based algorithm. Departing from conventional methods limited to bandlimited signals, our method extend the generalized sampling theory to handle graph signals beyond bandlimitedness. Our method aims to design a flexible sampling operator that mixes vertex values, while controlling the number of sample-contributive vertices. The operator design is formulated as a feasibility problem with an invertibility constraint for the best possible recovery and a constraint controlling the number of sample-contributive vertices. We reformulate the problem as a DC-like optimization problem by using the nuclear norm to obtain a tight relaxation of the invertibility constraint. To solve this problem, we present a DC-based algorithm. The effectiveness of our approach is demonstrated through sampling and recovery experiments on various graph signal models.

This letter introduces a graph learning approach leveraging prior knowledge of graph topology. For this, we integrate the concept of polytopic uncertainty into existing approaches that learn graph Laplacians and adjacency matrices, constraining the solution space to a polytopic set. Our approach offers improved accuracy with reduced computational cost by focusing on a smaller solution space, effectively excluding implausible topologies. Numerical experiments demonstrate superior learned graph quality compared to existing approaches across various signal models and noise levels.

This paper proposes a new regularization function specific to Spatiotemporal (ST) fusion, named temporally-guided total variation (TGTV). ST fusion is a promising approach to address a trade-off between the temporal and spatial resolution of satellite images. In general, satellite images are severely degraded by noise due to the observation instrument and environment. However, existing ST fusion methods designed to be robust to noise have some limitations, such as only being robust to local noise and oversmoothing without capturing detailed spatial structure. To address these challenges, TGTV is designed according to a reference image that is temporally different from a target image, but is expected to have a similar spatial structure. Then, we provide a new robust ST fusion framework based on TGTV. Experimental results show that our method performs as well as or better than several state-of-the-art ST fusion methods in noiseless cases and outperforms them in noisy cases.

We propose a hyperspectral (HS) anomaly detection method using a novel characterization of anomalies. Among HS anomaly detection approaches, decomposition-based methods, which simultaneously estimate a background part and an anomaly part from an HS image, have attracted much attention. In these methods, various approaches have been proposed for mathematical modeling of the background part, but the anomaly part is mostly modeled by an ℓ1-norm or an ℓ2,1-norm. However, these norms have limited ability to promote the exact sparsity of the anomaly part, leading to detection failure. In this paper, we introduce a difference-of-convex (DC) approach to HS anomaly detection. First, we design a DC function that properly models the sparsity of the anomaly part. Next, we formulate a constrained DC optimization problem that decomposes a given HS image into the two parts and noise. Then, we develop an efficient solver for the problem based on the proximal linearized DC algorithm (PLDC) and the preconditioned primal-dual splitting method (P-PDS). Finally, we demonstrate the effectiveness of our method compared to state-of-the-art methods through experiments on several HS anomaly detection datasets.

We propose a new deep plug-and-play (PnP) algorithm for constrained image restoration with guaranteed theoretical convergence. The PnP strategy, which incorporates off-the-shelf Gaussian denoisers into proximal splitting algorithms, has demonstrated outstanding performance in a variety of image restoration tasks. However, theoretically guaranteeing its convergence under reasonable assumptions is extremely challenging. In particular, an existing PnP method based on a primal-dual splitting algorithm (PnP-PDS) faces instability due to the absence of theoretical convergence guarantees, despite its flexibility in solving constrained image restoration without matrix inversion. In this paper, we address this instability issue by establishing theoretical conditions for the convergence of PnP-PDS with deep neural network (DNN)-based denoisers, and then provide its application to constrained image restoration. We also demonstrate experimentally that our convergent PnP-PDS achieves state-of-the-art performance in two image restoration tasks. These findings support the practicality of our theoretical results in terms of both performance and stability.

In this paper, we propose a risk-managed sparse index tracking framework. In this approach, we impose market-graph neutrality and turnover sparsity on the index tracking problem. Historically, sector neutrality has been researched to diversify investment across various sectors, preventing the portfolio from being biased toward specific industries. This strategy can act as a failsafe in scenarios where assets linked to a particular industry may fall simultaneously due to industry-wide events. However, pre-defined sectors may not always be suitable for all situations. In response, we propose replacing these sectors by grouping assets through graph clustering on a market graph. We refer to using these newly defined market-graph clusters to ensure diversified investments as market-graph neutrality. Additionally, to offset the probable rise in transaction costs caused by the frequent redefinition of these clusters, we introduce turnover sparsity to our formulation. We confirm our hypothesis that clusters generated from actual data could perform better than pre-defined heuristically and exhibit the advantageous results of our method through experiments on a real-world finance dataset, specifically, the S&P500 dataset.

This paper proposes a novel spatiotemporal (ST) fusion framework for satellite images, named Robust Optimization-based Spatiotemporal Fusion (ROSTF). ST fusion is a promising approach to resolve a trade-off between the temporal and spatial resolution of satellite images. Although many ST fusion methods have been proposed, most of them are not designed to explicitly account for noise in observed images, despite the inevitable influence of noise caused by the measurement equipment and environment. Our ROSTF addresses this challenge by formulating noise removal and ST fusion as a unified optimization problem. First, we define observation models for satellite images that may be contaminated with random noise, outliers, and/or missing values. Next, we introduce certain assumptions that naturally hold between the observed images and the target high-resolution image. Then, based on these models and assumptions, we formulate the fusion problem as a constrained optimization problem and develop an efficient algorithm based on a preconditioned primal-dual splitting method for solving the problem. The performance of ROSTF was verified using simulated and real data. The results show that ROSTF performs comparably to several state-of-the-art ST fusion methods in noiseless cases and outperforms them in noisy cases.

Road administrators require fine-scaled information regarding road surface conditions to ensure efficient operation during winter periods. However, conventional models offer low-resolution information at a scale comparable to meteorological meshes or the spatial configuration of road weather information systems. Additionally, few methods have been proposed for predicting road surface conditions specifically in urban areas, where roads frequently experience shading from surrounding buildings. This study proposes a statistical approach for predicting road surface temperature and conditions in urban road networks. The complicated accumulated distribution of solar radiation along each road is calculated and used as an effective explanatory variable that considers the complex shading effects of nearby structures. The proposed model adopts a Bayesian spatiotemporal hierarchical framework for predicting road surface temperature using a solar radiation variable. Furthermore, a spatial machine learning model is implemented to estimate road surface conditions. The model classifies icy road conditions into six distinct types, achieving a sensitivity of 0.7712 and a balanced accuracy of 0.8637. Ultimately, the model provides significant information required for decision-making processes aimed at ensuring efficient winter road management. These results indicate that the applicability of the proposed approach can extend beyond the studied area, demonstrating its potential for broader implementation.

Spatiotemporal fusion (ST fusion) is a feasible solution to resolve a tradeoff between the temporal and spatial resolutions of satellite images. Although many ST fusion methods have been proposed, most methods have not been developed that explicitly take noise in observed images into account, despite the inevitable influence of noise caused by the observation equipment and environment. In this paper, we propose an optimization-based ST fusion method that is robust to noise. First, we introduce observation models for noisy satellite images and make certain assumptions on the relationship between the observed images and the target high-resolution image. Next, based on these models and assumptions, we formulate the fusion problem as a constrained optimization problem and develop an efficient algorithm based on a primal-dual splitting method for solving the problem. The performance of the proposed method was verified using simulated and real data, and the results illustrate that our method outperforms state-of-the-art ST fusion methods for both noiseless and noisy satellite images.

This work proposes a novel formulation of convolutional-sparse-coded dynamic mode decomposition (CSC-DMD) incorporating a deep learning framework. CSC-DMD is a high-dimensional data analysis method with a convolutional synthesis dictionary and applicable to analyze dynamics such as seismic motions and river flows. An authors’ previous work has shown the effectiveness of CSC-DMD for riverbed state estimation. However, there still remains a room to improve the performance in expressing evolution of temporal and spatial changes in riverbed shape. Hence, this work proposes to adopt multi-resolution convolutional dictionary by introducing a deep learning framework so that the capability of simultaneously capturing local and global features is added to CSC-DMD. The significance of the proposed method is verified by evaluation of riverbed state estimation for time-series data of water surface and riverbed shape obtained through an experimental setup of river model.

We propose an efficient foreground-background separation (FBS) method for (possibly noisy) video data. Most existing FBS methods model the background as a low-rank component. However, this approach is computationally expensive because it requires matrix/tensor decomposition of high-dimensional videos. In this paper, we first introduce a new background model, named static scene constraint (SSC), to FBS. SSC plays a role in accurately capturing the static background by keeping the temporal gradient of the background component to zero. In addition, SSC is formulated as a convex constraint using differences in the temporal direction, which eliminates the need for matrix/tensor decomposition in optimization and significantly reduces the computational cost compared to existing low-rank-based background models. Second, we formulate the FBS problem as a convex optimization problem involving SSC and develop an efficient solver based on a preconditioned primal-dual splitting algorithm, which can automatically determine the appropriate stepsizes based on problem structure. Finally, we demonstrate the efficiency and effectiveness of our method compared with state-of-the-art FBS methods through experiments using infrared and electron microscope videos.

Hyperspectral (HS) anomaly detection is the task of identifying pixels with spectral signatures that differ significantly from surrounding pixels. Most existing anomaly detection methods do not take into account the effect of noise in HS images, or if they do, it is only Gaussian noise. In practice, however, it is inevitable that non-Gaussian noise is superimposed on HS images due to sensor failure and/or calibration errors, resulting in considerable degradation of the detection performance of these methods. In this paper, we propose a method to achieve robust anomaly detection even when HS images contain various types of noise. Specifically, we newly formulate a constrained convex optimization problem that decomposes a given HS image into a background part, an anomaly part, and three types of noise. We also develop an efficient solver for the problem based on a preconditioned version of a primal-dual splitting algorithm, which can automatically determine the appropriate stepsizes. Experiments on HS datasets show that the proposed method 1) achieves state-of-the-art detection performance both in noise-free and noisy cases and 2) is much more robust against noise than existing methods.

We propose a new regularization function, named Spatio-Spectral Structure Tensor Total Variation (S3TTV), for hyperspectral image (HSI) denoising. Spatio-Spectral Total Variation (SSTV), defined using spatio-spectral second-order differences, is widely known as a regularization function for HSI that can effectively remove noise while avoiding spatial over-smoothing. However, since SSTV only refers to the information of neighboring pixels or bands, it can corrupt semi-local spatial structure in the process of noise removal. To resolve this problem, we formulate S3TTV, which is defined by the sum of the nuclear norms of matrices consisting of spatio-spectral second-order differences in small spatial blocks (we call these matrices as spatio-spectral structure tensors). With this formulation, S3TTV can capture not only the similarity of semi-local spatial structure between adjacent bands but also the spectral correlation across all bands. We also formulate the HSI denoising problem as a convex optimization problem involving S3TTV and develop an efficient algorithm based on a diagonally preconditioned primal-dual splitting method to efficiently solve this problem. Finally, we demonstrate the effectiveness of S3TTV by comparing it with state-of-the-art HSI regularization models through mixed noise removal experiments.

This paper proposes an epigraphically-relaxed linearly-involved generalized Moreau-enhanced (ER-LiGME) model for layered mixed norm regularization. Group sparse and low-rank (GSpLr)-aware modeling using ℓ1/nuclear-norm-based layered mixed norms has succeeded in precise high dimensional signal recovery, e.g., images and videos. Our previous work significantly expands the potential of the GSpLr-aware modeling by epigraphical relaxation (ER). It enables us to handle a (even non-proximable) deeply-layered mixed norm minimization by decoupling it into a norm and multiple epigraphical constraints (if each proximity operator is available). One problem with typical SpLr modeling is that it suffers from the underestimation effect due to the ℓ1and nuclear norm regularization. To circumvent this problem, LiGME penalty functions, which modify conventional sparsity and low-rankness promoting convex functions to nonconvex ones while keeping overall convexity, have been proposed conventionally. In this work, we integrate the ER technique with the LiGME model to realize deeply-layered (possibly non-proximable) mixed norm regularization and show its effectiveness in denoising and compressed sensing reconstruction.

The spatio-spectral total variation (SSTV) model has been widely used as an effective regularization of hyperspectral images (HSI) for various applications such as mixed noise removal. However, since SSTV computes local spatial differences uniformly, it is difficult to remove noise while preserving complex spatial structures with fine edges and textures, especially in situations of high noise intensity. To solve this problem, we propose a new TV-type regularization called Graph-SSTV (GSSTV), which generates a graph explicitly reflecting the spatial structure of the target HSI from noisy HSIs and incorporates a weighted spatial difference operator designed based on this graph. Furthermore, we formulate the mixed noise removal problem as a convex optimization problem involving GSSTV and develop an efficient algorithm based on the primal-dual splitting method to solve this problem. Finally, we demonstrate the effectiveness of GSSTV compared with existing HSI regularization models through experiments on mixed noise removal. The source code will be available at https://www.mdi.c.titech.ac.jp/publications/gsstv.

Removing stripe noise, i.e., destriping, from remote sensing images is an essential task in terms of visual quality and subsequent processing. Most existing destriping methods are designed by combining a particular image regularization with a stripe noise characterization that cooperates with the regularization, which precludes us to examine and activate different regularizations to adapt to various target images. To resolve this, two requirements need to be considered: a general framework that can handle a variety of image regularizations in destriping, and a strong stripe noise characterization that can consistently capture the nature of stripe noise, regardless of the choice of image regularization. To this end, this article proposes a general destriping framework using a newly introduced stripe noise characterization, namedflatness constraint (FC), where we can handle various regularization functions in a unified manner. Specifically, we formulate the destriping problem as a nonsmooth convex optimization problem involving a general form of image regularization and the FC. The constraint mathematically models that the intensity of each stripe is constant along one direction, resulting in a strong characterization of stripe noise. For solving the optimization problem, we also develop an efficient algorithm based on a diagonally preconditioned primal-dual splitting algorithm (DP-PDS), which can automatically adjust the step sizes. The effectiveness of our framework is demonstrated through destriping experiments, where we comprehensively compare combinations of a variety of image regularizations and stripe noise characterizations using hyperspectral images (HSIs) and infrared (IR) videos.

The paper proposes a new high spatial resolution hyperspectral (HR-HS) image estimation method based on convex optimization. The method assumes a low spatial resolution HS (LR-HS) image and a guide image as observations, where both observations are contaminated by noise. Our method simultaneously estimates an HR-HS image and a noiseless guide image, so the method can utilize spatial information in a guide image even if it is contaminated by heavy noise. The proposed estimation problem adopts hybrid spatio-spectral total variation as regularization and evaluates the edge similarity between HR-HS and guide images to effectively use apriori knowledge on an HR-HS image and spatial detail information in a guide image. To efficiently solve the problem, we apply a primal-dual splitting method. Experiments demonstrate the performance of our method and the advantage over several existing methods.

We propose a design method of sampling matrices for graph signals that guarantees perfect recovery for arbitrary graph signal subspaces. When the signal subspace is known, perfect reconstruction is always possible from the samples with an appropriately designed sampling matrix. However, most graph signal sampling methods so far design sampling matrices based on the bandlimited assumption and sometimes violates the perfect reconstruction condition for the other signal models. In this paper, we formulate an optimization problem for the design of the sampling matrix that guarantees perfect recovery, thanks to a generalized sampling framework for standard signals. In experiments with various signal models, our sampling matrix presents better reconstruction accuracy both for noiseless and noisy situations.

This paper proposes an effective and efficient destriping method for remote-sensing data. Destriping of remote-sensing data is an essential task because stripe noise not only degrades the visual quality but also seriously affects subsequent processing. We formulate the destriping problem as a convex optimization problem involving zero-gradient constraints, where the constraints are designed to exploit the fact that the spatial and temporal gradients of stripe noise equal to zero. Our method imposes such strong constraints on stripe noise, and thus can fully capture the nature of stripe noise, leading to very effective destriping. Also, operations required for handling the zero-gradient constraints in optimization are simple, which enables us to develop an efficient algorithm for solving the problem by a primal-dual splitting method. We demonstrate the advantages of our method over existing methods on destriping experiments using remote-sensing data.

The randomized subspace Newton convex methods for the sensor selection problem are proposed. The randomized subspace Newton algorithm is straightforwardly applied to the convex formulation, and the customized method in which the part of the update variables are selected to be the present best sensor candidates is also considered. In the converged solution, almost the same results are obtained by original and randomized-subspace-Newton convex methods. As expected, the randomized-subspace-Newton methods require more computational steps while they reduce the total amount of the computational time because the computational time for one step is significantly reduced by the cubic of the ratio of numbers of randomly updating variables to all the variables. The customized method shows superior performance to the straightforward implementation in terms of the quality of sensors and the computational time.

This paper proposes an epigraphical reformulation (ER) technique for non-proximable mixed norm regularization. Various regularization methods using mixed norms have been proposed, where their optimization relies on efficient computation of the proximity operator of the mixed norms. Although the sophisticated design of mixed norms significantly improves the performance of regularization, the proximity operator of such a mixed norm is often unavailable. Our ER decouples a non-proximable mixed norm function into a proximable norm and epigraphical constraints. Thus, it can handle a wide range of non-proximable mixed norms as long as the proximal operator of the outermost norm, and the projection onto the epigraphical constraints can be efficiently computed. Moreover, we prove that our ER does not change the minimizer of the original problem despite using a certain inequality approximation. We also provide a new structure-tensor-based regularization as an application of our framework, which illustrates the utility of ER.

Hyperspectral (HS) imaging based on compressed sensing (CS) is actively studied to capture an HS image in one shot. Although CS can reconstruct an HS image from a much less number of random observations, capturing an HS image of high spatial and spectral resolution (HR-HS image) is still difficult because of current imaging systems. In this paper, we propose a new methodology of HS imaging, named compressed HS pansharpening. Specifically, the concept enables to generate an HR-HS image from a compressively-sensed observation with the help of a panchromatic (PAN) image. For a realistic setting, the concept assumes that both a CS observation and a PAN image are contaminated by noise. Then, an HR-HS and a clean PAN image are simultaneously estimated from the noisy pair by solving a newly-formulated optimization problem. In the experiments, we demonstrate the utility of our proposed methodology.

Compressed sensing (CS) reconstruction is essential in capturing hyperspectral (HS) images by one-shot. However, existing approaches for compressed HS imaging assume that compressed observation is contaminated by Gaussian noise, and so they are sensitive to the other type of noise and outliers. To resolve the above problem, we propose a new methodology compressed HS imaging that can handle mixed Gaussian-sparse noise. For robust estimation, our proposed method simultaneously estimates a clean HS image and sparse noise by solving a convex optimization problem. Experimental results illustrate the utility of our proposed framework.

This work proposes convolutional-sparse-coded dynamic mode decomposition (CSC-DMD) by unifying extended dynamic mode decomposition (EDMD) and convolutional sparse coding. EDMD is a data-driven method of analysis used to describe a nonlinear dynamical system with a linear time-evolution equation. Compared with existing EDMD methods, CSC-DMD has the advantage of reflecting the spatial structure of a target. As an example, the proposed method is applied to river bed shape estimation from the water surface observation. This estimation problem is reduced to sparsityaware signal restoration with a hard constraint given by the CSC-DMD prediction, where the algorithm is derived by the primal-dual splitting method. A time series set of water surface and bed shape measured through an experimental river setup is used to train and test the system. From the result, the efficacy of the proposed method is verified.

This paper proposes a randomized optimization framework for constrained signal reconstruction, where the word "constrained" implies that data-fidelity is imposed as a hard constraint instead of adding a data-fidelity term to an objective function to be minimized. Such formulation facilitates the selection of regularization terms and hyperparameters, but due to the non-separability of the data-fidelity constraint, it does not suit block-coordinate-wise randomization as is. To resolve this, we give another expression of the data-fidelity constraint via epigraphs, which enables to design a randomized solver based on a stochastic proximal algorithm with randomized epigraphical projection. Our method is very efficient especially when the problem involves non-structured large matrices. We apply our method to CT image reconstruction, where the advantage of our method over the deterministic counterpart is demonstrated.

A hyperspectral (HS) image has high spectral resolution information but low spatial resolution information. To get an HS image of high spatial and spectral resolution (high-spatial HS image), fusion techniques are actively studied, which synthesize an HS image of low spatial and high spectral resolution and a multispectral (MS) image. The techniques can generate a high-spatial HS image by exploiting the high spectral and spatial resolution information of HS and MS images, respectively. However, the methods do not evaluate the edge similarity between generated HS and observed MS images, and do not denoise the MS image. As a result, when an observed MS image is noisy, these methods produce artifacts and spectral distortion. To tackle this problem, we propose a new HS and MS data fusion method using a hybrid spatio-spectral total variation (HSSTV), which is a regularization for HS image restoration. The method not only generates a high-spatial HS image but also denoises a given MS image, so that we obtain a high-spatial HS image even if the observed images are contaminated by noise. In the experiments, we demonstrate the advantages of our method over existing fusion methods and the effectiveness of HSSTV for MS image restoration.

This paper proposes multiscale structure-tensor total variation (MSTV) for image recovery. Gradient vectors in local patches usually have similar directions, and thus each local gradient matrix (the set of gradient vectors) tends to be low rank. STV introduces this property by calculating the sum of nuclear norms from all the 10-cal gradient matrices over the input image. By STV regularization, fine textures are recovered efficiently. However, since STV only considers differences of vertically and horizontally adjacent pixels, if neighboring samples are not reliable due to severe degradation, a latent image cannot be recovered efficiently. In this work, we assume that, for any two target pixels in a local patch, two vectors consisting of multiple differences not only between each target and adjacent pixels but also each target and further distant pixels exhibit a similar direction. According to this assumption, our MSTV firstly applies wavelet-based multiscale decomposition to vertical/horizontal gradient vectors and then evaluates the sum of nuclear norms of all the local wavelet coefficients. Experimental results show that the MSTV improves both numerical reconstruction error and subjective visual quality, compared with the conventional STV.

This work proposes a novel restoration model for optical coherence tomography (OCT) data. The authors have been developing a multi-frequency swept (MS) en-face OCT device that can help understand the mechanism of the sensory epithelium in the cochlear. Although the device has merit in acquiring moving tissues, the broadened light gives a weak response; thus, some signal restorations are demanded. This work proposes the introduction of a formulation for OCT data restoration as a convex optimization problem by assuming a latent refractive index distribution. An algorithm to solve the problem with the primal-dual splitting (PDS) framework is then derived. The PDS has an advantage of requiring no inverse matrix operation and being able to handle high-dimensional data. The significance of the proposed model is verified by simulations on artificial data, followed by an experiment with the actual observation of 256 256 2000 voxels.

This paper proposes a new mixed noise removal method using a hybrid spatio-spectral total variation (HSSTV) for hy-perspectral (HS) images. HSSTV effectively evaluates spatial and spectral piecewise smoothness and would be a powerful regularization for HS image restoration. Existing mixed noise removal methods evaluate a-priori knowledge of an HS image via multiple regularizations. However, they do not appropriately evaluate spatial piecewise smoothness, resulting in oversmoothing or artifacts. Moreover, parameter settings in existing methods are troublesome tasks because multiple balancing parameters are interdependent. In contrast, thanks to HSSTV, the proposed method can restore a clean HS image while keeping sharp edges and details. In addition, parameter settings in our method are much easier than existing ones because data fidelity is imposed as hard constraints. In the experiments, we demonstrate the advantages of our proposed method over existing regularizations.

This work proposes a volumetric data restoration method, especially for data acquired through an optical coherence tomography (OCT) device. OCT is a technique for acquiring a tomographic image of a specimen object in a few μm scale by using a near infrared laser. The authors have been trying dynamic observation of epithelium in cochlear of the inner ear. Currently, there is a problem to remove the influence of the measurement process as well as noise due to image sensor sensitivity. Therefore, in this work, on the assumption that specimen objects follow some sort of signal generation model, an OCT volumetric data restoration method is proposed. The proposed technique adopts the primal-dual plug-and-play (PDPnP) method, where the generation model is represented by a sparsity-aware regularization term explicitly or implicitly. The significance of the proposed method is verified by simulation on artificial data, followed by an experiment with actual observation data.

Tensor factorization with hard and/or soft constraints has played an important role in signal processing and data analysis. However, existing algorithms for constrained tensor factorization have two drawbacks: (i) they require matrix-inversion; and (ii) they cannot (or at least is very difficult to) handle structured regularizations. We propose a new tensor factorization algorithm that circumvents these drawbacks. The proposed method is built upon alternating optimization, and each subproblem is solved by a primal-dual splitting algorithm, yielding an efficient and flexible algorithmic framework to constrained tensor factorization. The advantages of the proposed method over a state-of-the-art constrained tensor factorization algorithm, called AO-ADMM, are demonstrated on regularized nonnegative tensor factorization.

Acquiring high-resolution hyperspectral (HS) images is a very challenging task. To this end, hyperspectral pansharpening techniques have been widely studied, which estimate an HS image of high spatial and spectral resolution (high HS image) from a pair of an HS image of high spectral resolution but low spatial resolution (low HS image) and a high spatial resolution panchromatic (PAN) image. However, since these methods do not fully utilize the piecewise-smoothness of spectral information on HS images in estimation, they tend to produce spectral distortion when the low HS image contains noise. To tackle this issue, we propose a new hyperspectral pansharpening method using a spatio-spectral regularization. Our method not only effectively exploits observed information but also properly promotes the spatio-spectral piecewise-smoothness of the resulting high HS image, leading to high quality and robust estimation. The proposed method is reduced to a nonsmooth convex optimization problem, which is efficiently solved by a primal-dual splitting method. Our experiments demonstrate the advantages of our method over existing hyperspectral pansharpening methods.

This paper proposes color affine subspace pursuit (CASSP) for color artifact removal. Local patches in natural color images tend to exhibit a line distribution, so-called a color line. According to this characteristic, a convex-optimization-based image recovery with a local color nuclear norm (LCNN) has conventionally been introduced to promote the color line property of local patches and succeeded in removing color artifacts. It is, however, often the case that a local patch does not form a line distribution, but a union of affine subspaces (UoAS), e.g., a patch consisting of two different colors. In such regions, the LCNN often results in color fading or color smearing. This paper promotes the UoAS property, i.e., the color line or plane distribution for each affine subspace in local patches by using CASSP. Our cost function for the CASSP consists of the LCNN for each centered color distribution cluster. Experimental results show that the CASSP improves both numerical reconstruction error and subjective visual quality, compared with the LCNN.

We propose a novel sampling method of graph signals that simulta- neously inherits the characteristics of the sampled signals in the vertex and graph frequency domains. In the conventional sampling of graph signals, the sampled signal can retain the characteristics only in one (vertex or graph frequency) domain, but it is not suitable for another domain. The proposed method forms an optimization prob- lem to design appropriate sampling matrices in the graph frequency domain for inheriting the characteristics of the sampled signals in the vertex domain. We perform downsampling of graph signals based on the proposed approach and apply it to denoising with graph Laplacian pyramid along with the comparison to the existing methods.

We propose an edge-preserving filtering method with a novel use of the L0gradient. Our method, termed as the L0gradient projection, is formulated as the minimization of a quadratic data-fidelity to an input image subject to the constraint that the L0gradient, the number of non-zero gradients, of the output image is less than a user-given parameter α. This strategy is much more intuitive than the conventional approach, the L0gradient minimization, that minimizes the sum of the L0gradient plus the quadratic data-fidelity, because one can directly impose a desired degree of flatness by α, which is impossible in the L0gradient minimization. We also provide an efficient algorithm based on the so-called alternating direction method of multipliers for solving the nonconvex optimization problem associated with the L0gradient projection. The utility of the L0gradient projection is illustrated by experiments.

The hybrid steepest descent method (HSDM) [Yamada, '01] was introduced as a low-computational complexity tool for solving convex variational-inequality problems over the fixed-point set of non-expansive mappings in Hilbert spaces. Motivated by results on decentralized optimization, this study introduces an HSDM variant that extends, for the first time, the applicability of HSDM to affinely constrained composite convex minimization tasks over Euclidean spaces; the same class of problems solved by the popular alternating direction method of multipliers and primal-dual methods. The proposed scheme shows desirable attributes for large-scale optimization tasks that have not been met, partly or all-together, in any other member of the HSDM family of algorithms: tunable computational complexity, a step-size parameter which stays constant over recursions, promoting thus acceleration of convergence, no boundedness constraints on iterates and/or gradients, and the ability to deal with convex losses which comprise a smooth and a non-smooth part, where the smooth part is only required to have a Lipschitz-continuous derivative. Convergence guarantees and rates are established. Numerical tests on synthetic data and on colored-image inpainting underline the rich potential of the proposed scheme for large-scale optimization tasks.

We propose a new regularization technique, named Hybrid Spatio-Spectral Total Variation (HSSTV), for hyperspectral image (HSI) restoration. Popular regularization techniques for HSIs are total variation functions (TV), and there have been proposed a variety of TVs for HSI restoration. However, they do not fully exploit both spatial and spectral smoothness, which are the underlying properties of HSIs, and/or they result in computationally expensive optimization. Our proposed HSSTV is designed to evaluate the two properties via two types of discrete differences of an HSI, leading to much more effective regularization than existing TVs for HSI restoration. HSSTV is defined with local discrete difference operators and the ℓ1/mixed ℓ1,2norm, so that optimization problems involving it can be efficiently solved by proximal splitting methods, such as the so-called alternating direction method of multipliers. Experimental results illustrate the advantages of HSSTV over state-of-the-art methods.

The frame rate of existing stereo cameras is not enough to track quick hand or finger actions. It also requires lots of computational cost to find correspondence between stereo images to compute distance. The recently commercialized 3D position sensors such as TOF cameras or Leap Motion needs strong illumination to ensure sufficient optical energy for the high frame rate sensing. To overcome these problems, this paper proposes to use a pair of optical-mouse-sensors as a stereo image sensor to measure 3D-velocity and use it to extrapolate 3D position measured by a low-frame-rate stereo camera. It is shown that quick hand actions are tracked under ordinary in-door lighting condition. As 2D velocities are computed inside the optical-mouse-sensors, computation and communication costs are drastically reduced.

This paper presents a simple but effective way of enhancing tactile stimulus by a mechanism with springs to preserve elastic energies charged in a prior energy-charging phase and discharge them to enhance the force to hit a finger in the stimulating phase. With this mechanism, a small and light stimulator attached to the fingertip is developed and demonstrated to generate the tactile feedback strong enough to make people feel as if their fingers collide with a virtual object. It is also shown that the durations of the two phases can be as short as a few milliseconds so that the latency in tactile feedback can be negligible. The performance of the mechanism and the effectiveness of its tactile feedback are evaluated for in-air keypress and swipe operations.

Simulator sickness is an issue in virtual reality environments. In a virtual world, sensory conflict between visual sensation and self-motion perception occurs readily. Contradiction between visual and vestibular sensation is a dominant cause of motion sickness. Vection is a visually evoked illusion of self-motion. Vection occurs when a stationary human experiences locomotor stimulation over a wider area of the field of view, and senses motion when in fact there is none. Strong vection has been associated with simulator sickness. In this poster, the authors present results of a pilot study based on a hypothesis that simulator sickness can be mitigated by passively responding to the body sway. Commercially available air cushions were applied for VR environments. Measurable mitigation of simulator sickness was achieved by physically responding to vection. Allowing body sway encourages moderating the sensory conflict between visual sensation and self-motion perception. Also, the shapes of air cushions on seat backs were found to be an important variable.

We propose a new plug-and-play image restoration method based on primal-dual splitting. Existing plug-and-play image restoration methods interpret any off-the-shelf Gaussian denoiser as one step of the so-called alternating direction method of multipliers (ADMM). This makes it possible to exploit the power of such a highly-customized Gaussian denoising method for general image restoration tasks in a plug-and-play fashion. However, the ADMM-based plug-and-play approach (ADMMPnP) has several limitations: 1) it often requires a problem-specific iterative method in solving a subproblem, which results in a computationally expensive inner loop; and 2) it is specialized to handle the formulation of a regularization (plug-and-play) term plus a data-fidelity term, so that it does not allow to impose hard constraints useful for image restoration. Our approach resolves these issues by leveraging the nature of primal-dual splitting, yielding a very flexible plug-and-play image restoration method. Experimental results demonstrate that the proposed method is much more efficient than ADMMPnP with an inner loop, whereas it keeps the same efficiency as ADMMPnP in the case where the subproblem of ADMMPnP can be solved efficiently.

Minimizing L0gradient, the number of the non-zero gradients of an image, together with a quadratic data-fidelity to an input image has been recognized as a powerful edge-preserving filtering method. However, the L0gradient minimization has an inherent difficulty: a user-given parameter controlling the degree of flatness does not have a physical meaning since the parameter just balances the relative importance of the L0gradient term to the quadratic data-fidelity term. As a result, the setting of the parameter is a troublesome work in the L0gradient minimization. To circumvent the difficulty, we propose a new edge-preserving filtering method with a novel use of the L0gradient. Our method is formulated as the minimization of the quadratic data-fidelity subject to the hard constraint that the L0gradient is less than a user-given parameter α. This strategy is much more intuitive than the L0gradient minimization because the parameter α has a clear meaning: the L0gradient value of the output image itself, so that one can directly impose a desired degree of flatness by α. We also provide an efficient algorithm based on the so-called alternating direction method of multipliers for computing an approximate solution of the nonconvex problem, where we decompose it into two subproblems and derive closed-form solutions to them. The advantages of our method are demonstrated through extensive experiments.

We propose a new regularization function, named as Arranged Structure tensor Total Variation (ASTV), for multichannel image restoration. Since the standard structure tensor is a matrix whose eigenvalues well encodes local neighborhood information of an image, there has been proposed vectorial total variation based on the structure tensor for image regularization. However, the correlation among the channels cannot be measured by the structure tensor because the discrete differences of all the channels are just summed up in the entries of the structure tensor. On the other hand, ASTV is based on a newly-defined arranged structure tensor that becomes an approximately low-rank matrix when multichannel images have strong correlation among their channels. This suggests that penalizing the nuclear norm of the arranged structure tensor is a reasonable regularization for multichannel images, leading to the definition of ASTV. Experimental results illustrate the advantage of ASTV over a state-of-the-art vectorial total variation based on the structure tensor.

We propose an efficient image restoration framework based on stochastic optimization. Image restoration usually requires some iterative methods for solving optimization problems that characterize restored images, where the multiplication of the observation matrix Φ ϵ Rm × n and variables has to be computed at each iteration. If an efficient implementation of the multiplication (e.g., using FFT) is unavailable, its computational cost becomes O(MN), which is quite expensive since both N and M are usually large in image restoration. Our method needs to load and apply only a part of the observation matrix of size M/b × N (b: the number of parts), so that the computational cost is only O(MN/b). Moreover, the proposed method accepts various nonsmooth objectives effective for image restoration. Experiments on compressed sensing reconstruction and non-uniform deblurring show the advantage of the proposed method over state-of-the-art proximal optimization methods.

We propose an image colorization method using fast soft-thresholding of singular values (singular value thresholding). An image colorization method with nuclear norm minimization (NNM) has been proposed and brings good results. NNM usually requires iterative application of singular value decomposition (SVD) for singular value thresholding. However, the computational cost of SVD in the colorization method becomes too expensive to handle high-resolution images. In this paper, we reduce its computational cost by using Chebyshev polynomial approximation (CPA). Singular value thresholding is expressed by a multiplication of certain matrices derived from the characteristic of CPA. As a result, our CPA-based technique makes the image colorization method much more efficient. In addition, we replace the optimization method used in the image colorization method by alternating direction method of multipliers, which further accelerates the computation. Experimental results verify the effectiveness of our method with respect to the computation time and the approximation precision.

Although non-local image denoising has attracted much research effort due to its superior performance, little attention has focused on its color extension. Most existing non-local color image denoising methods process the color channels of an input image separately. However, in order to improve the performance of color image denoising, all color channels should be processed jointly for fully utilizing the interchannel dependency. This paper proposes a new non-local and inter-channel dependency aware prior, named weighted tensor nuclear norm (WTNN), and it is defined on a 3rd-order tensor from a patch cluster of an input image. We also present an effective algorithm for color image denoising using the WTNN. Experimental results clearly show that the proposed algorithm outperforms a state-of-the-art color image denoising method, known as CBM3D.

When we use a tablet style handheld device such as a smart phone as a part of a virtual really system, its most outstanding future: the touch screen dominating the most area of the front face should be incorporated into the system effectively and beneficially. For example, the visual information displayed on the screen can be merged with the surrounding or background scenes and the intuitive touch operation can be performed in a suitable scenario. However, if the means of the interaction is limited to the touch operation, finger operation on the front screen must be performed even for unsuitable scenarios, and the fingers or hands occluding the visual information disturb our immersive experience. To deal with this situation, we propose a multi-modal interactive tablet that uses its cameras, accelerometer, track ball and pressure sensors implemented on its rear and side for operations ensuring visibility. The pressing and ball-rotating operation on the rear and the side and the tactile feedback generated by voice-coil-based actuators assist and guide the multi-modal interaction. The effectiveness of the multi-modality with the rear and the side operation and the tactile feedback is evaluated by an experiment.

This paper proposes a second-order discrete total generalized variation (TGV) for arbitrary graph signals, which we call the graph TGV (G-TGV). The original TGV was introduced as a natural higher-order extension of the well-known total variation (TV) and is an effective prior for piecewise smooth signals. Similarly, the proposed G-TGV is an extension of the TV for graph signals (G-TV) and inherits the capability of the TGV, such as avoiding staircasing effect. Thus the G-TGV is expected to be a fundamental building block for graph signal processing. We provide its applications to piecewise-smooth graph signal inpainting and 3D mesh smoothing with illustrative experimental results.

In this paper, we propose a fast and an approximate solution of non-local/local filters using Chebyshev polynomial approximation (CPA). A non-local/local filter is generally expressible in a matrix form. From the matrix notation, image denoising performance is improved by filtering the eigenvalues of the filter matrix. However, it requires much execution time due to computational complexity of eigendecomposition. To reduce the computational cost, we apply the CPA to eigenvalue filtering, leading to an eigendecomposition-free procedure. Moreover, a fast SURE-based parameter optimization is possible by using the CPA. It enables us to determine a suitable filtering parameter efficiently. Numerical examples illustrate that the proposed method is significantly faster than conventional methods while it maintains high approximate precision.

This paper aims to introduce a convex prior based on gradient-domain image decomposition (GID) for image recovery. As a useful class of convex priors, total variation (TV) and its variants have been widely investigated. Among them, total generalized variation (TGV) is paid much attention recently, because it provides rich visual quality around edges and gradation regions in recovered images. This paper gives a general perspective: the TGV can be regarded as a GID and thus can be extended to more general formulation. Specifically, by introducing some priors promoting desired properties on gradient components which cannot be treated efficiently by the TGV, the restored image would attain better visual quality. As a practical instance, we incorporate block nuclear norm (BNN), which can characterize local low rankness, into the GID framework to keep the visual quality of well-patterned textures. Consequently, the GID provides better subjective visual quality and less reconstruction error in missing pixel recovery, than the TGV-based and the TV/BNN-based regularization methods.

The front face of the tablet style smartphone or computer is dominated by a touch screen. As a finger operation on the touch screen disturbs its visibility, it is assumed a finger touches the screen instantly. Under such restriction, use of the rear surface of the tablet for tactile display is promising as the fingers constantly touch the back face and feel the tactile information. In our presentation, various tactile feedback mechanisms implemented on the back face are demonstrated and the latency of the feedback and its effect on the usability are evaluated for different communication means to control actuators such as wireless LAN, Bluetooth and audio signals. It is shown that the audio signal is promising to generate quick tactile feedback.

The front surface of the tablet style smartphone or computer is dominated by a touch screen. As a finger operation on the touch screen disturbs its visibility, it is assumed a finger touches the screen instantly. Under such restriction, use of the rear surface of the tablet for tactile display is promising as the fingers holding the tablet constantly touch it and feel the feedback steadily. In this paper, a slim design of tactile feedback mechanism that can be easily installed on the back of existing tablets is given and its mechanical performance regarding electricity consumption, latency and force is evaluated. Human capability in perceiving the tactile information on the display is also evaluated.

This paper proposes a new vectorial total variation prior (VTV) for color images. Different from existing VTVs, our VTV, named the decorrelated vectorial total variation prior (D-VTV), measures the discrete gradients of the luminance component and that of the chrominance one in a separated manner, which significantly reduces undesirable uneven color effects. Moreover, a higher-order generalization of the D-VTV, which we call the decorrelated vectorial total generalized variation prior (D-VTGV), is also developed for avoiding the staircasing effect that accompanies the use of VTVs. A noteworthy property of the D-VT(G)V is that it enables us to efficiently minimize objective functions involving it by a primal-dual splitting method. Experimental results illustrate their utility.

When high ISO sensitivity is used to acquire images of dark scenes, their detail textures are often deteriorated by sensor noise. On the other hand, using flash photography with artificial light, one can shorten the exposure time, and obtain a sharp image under the low ISO sensitivity. However, the use of flash light changes the color tone and often generates unnatural images due to a specific color temperature of the additional light. This paper presents a new efficient method for flash/no-flash image integration. In contrast to conventional integration methods assuming that the flash image has a sharp texture without any noise, our method can successfully remove noise. Specifically, our method separately handles regions within the reach of the flash light and other regions out of range of the flash light, because the two regions have much different characteristics. As for the former well-exposed regions, we transferred the detail of the flash image to no-flash image by optimization and component separation. As for the latter under-exposed regions, an optimization based joint bilateral filtering that uses information of a flash image is performed to remove noise. Experimental results show the effectiveness of our method compared to the conventional methods.

This paper proposes to use the Total Generalized Variation (TGV) of second order in a constrained form for image processing, which we call the TGV constraint. The main contribution is twofold: i) we present a general form of convex optimization problems with the TGV constraint, which is, to the best of our knowledge, the first attempt to use TGV as a constraint and covers a wide range of problem formulations sufficient for image processing applications; and ii) a computationally-efficient algorithmic solution to the problem is provided, where we mobilize several recently-developed proximal splitting techniques to handle the complicated structured set, i.e., the TGV constraint. Experimental results illustrate the potential applicability and utility of the TGV constraint.

We propose to detect edges of reflections, which we call the REF-edges, from a single image via convex optimization. Our method is designed based on two observations on reflections: (i) reflections have almost monotone color and (ii) color around REF-edges varies smoothly. The first one can be translated into the property that gradients around REF-edges distribute linearly in the RGB color space, which we call the REF-linearity. The second one can be interpreted as follows: color differences around REF-edges are small; for an entry of REF-edges, gradients among its surrounding entries have small variance. Using the above properties, we characterize REF-edges as a solution of a constrained convex optimization problem. The optimization problem is solved by the Alternating Direction Method of Multipliers (ADMM). Experiments using real-world images with reflections show the utility of our proposed method.

Using a novel characterization of texture, we propose an image decomposition technique that can effectively decomposes an image into its cartoon and texture components. The characterization rests on our observation that the texture component enjoys a blockwise low-rank nature with possible overlap and shear, because texture, in general, is globally dissimilar but locally well patterned. More specifically, one can observe that any local block of the texture component consists of only a few individual patterns. Based on this premise, we first introduce a new convex prior, named the block nuclear norm (BNN), leading to a suitable characterization of the texture component. We then formulate a cartoon-texture decomposition model as a convex optimization problem, where the simultaneous estimation of the cartoon and texture components from a given image or degraded observation is executed by minimizing the total variation and BNN. In addition, patterns of texture extending in different directions are extracted separately, which is a special feature of the proposed model and of benefit to texture analysis and other applications. Furthermore, the model can handle various types of degradation occurring in image processing, including blur+missing pixels with several types of noise. By rewriting the problem via variable splitting, the so-called alternating direction method of multipliers becomes applicable, resulting in an efficient algorithmic solution to the problem. Numerical examples illustrate that the proposed model is very selective to patterns of texture, which makes it produce better results than state-of-the-art decomposition models.

We propose a new convex regularizer, named the local color nuclear norm (LCNN), for color image recovery. The LCNN is designed to promote a property inherent in natural color images - in which their local color distributions often exhibit strong linearity - and is thus expected to reduce color artifact effectively. In addition, the very nature of LCNN allows us to incorporate it into various types of color image recovery formulations, with the associated convex optimization problems solvable using proximal splitting techniques. Applications of LCNN are demonstrated with illustrative numerical examples.

This paper proposes a likelihood constrained optimization framework for Poisson image restoration. The likelihood constrained problem considered in this paper is the minimization of convex priors over the level set of the negative-log-likelihood function of the Poisson distribution. It has advantages in parameter selection compared with the minimization of the weighted sum of convex priors and the negative-log-likelihood function, which has been used in conventional methods. The level set is characterized as the fixed point set of a certain quasi-nonexpansive operator, which enables us to apply the hybrid steepest descent method to solve the constrained problem. The proposed framework not only can handle the level set of any convex function whose subgradient is available but also does not require any computationally-expensive procedure such as operator inversion and inner loop. Illustrative numerical examples are also presented.

Observing that sparse systems are almost smooth, we propose to utilize the newly-introduced adaptively-weighted total variation (AWTV) for sparse system identification. In our formulation, a sparse system identification problem is posed as a sequential suppression of a time-varying cost function: the sum of AWTV and a data-fidelity term. In order to handle such a non-differentiable cost function efficiently, we propose a time-varying extension of a primal-dual splitting type algorithm, named the adaptive primal-dual splitting method (APDS). APDS is free from operator inversion or other highly complex operations, resulting in computationally efficient implementation in online manner. Moreover, APDS realizes that the sequence defined in a certain product space monotonically approaches the solution set of the current cost function, i.e., the sequence generated by APDS pursues desired replicas of the unknown system in each time-step. Our scheme is applied to a network echo cancellation problem where it shows excellent performance compared with conventional methods.

We propose a novel framework to obtain an artifact-free enlarged image from a given JPEG image. The proposed formulation based on a newly introduced JPEG image acquisition model realizes decompression and super-resolution interpolation simultaneously using multi-order total variation, so that we can drastically reduce artifacts appearing in JPEG images such as block noise and mosquito noise, without generating staircasing effect, which is typical in existing total variation-based JPEG decompression methods. We also present a computationally-efficient optimization scheme, derived as a special case of a primal-dual splitting type algorithm, for solving the convex optimization problem associated with the proposed formulation. Numerical examples show that the proposed method works effectively compared with existing methods.

In this paper, we propose a novel missing region recovery method by promoting blockwise low-rankness. It is natural to assume that images often have local repetitive structures. Hence, any small block extracted from an image is expected to be a low-rank matrix. Based on this assumption, we formulate missing region recovery as a convex optimization problem via newly introduced block nuclear norm which promotes blockwise low-rankness of an image with missing regions. An iterative scheme for approximating a global minimizer of the problem is also presented. The scheme is based on the alternating direction method of multipliers (ADMM) and allows us to restore missing regions efficiently. Experimental results reveal that the proposed method can recover missing regions with detailed local structures.

We propose variants of Alternating Direction Method of Multipliers (ADMM) employing simplified updates under additional assumptions. ADMM iteratively solves the minimization of the sum of two nonsmooth convex functions. Each iterations of ADMM itself consists of solving a certain convex optimization problem which often requires the use of some iterative solver. Such inner iterations cause slow convergence. Our proposed algorithms avoid some of inner iterations by employing simplified updates. An efficacy of the proposed algorithm is shown in an image super-resolution problem. In this application, the resultant algorithm does not require matrix inversion which causes inner iterations of the original ADMM. A numerical example in the image super-resolution setting demonstrates that our proposed algorithms reduce CPU time to about 70-80 percent of the original ADMM.

Solving image restoration problems requires the use of efficient regularization terms that represent certain features of the original image. Natural images generally have three features: smooth regions, textures, and edges. However, conventional optimization techniques typically adopt only one or two reg-ularization terms, and there is no regularized optimization problem that represents such features exactly and completely. By applying three regularization terms corresponding to these three features, we can restore images more efficiently in ill-posed conditions. We propose here total variation (TV), wavelet, and curvelet regularized optimization for image restoration. These regularization terms correspond exactly to the smooth region, textures, and edges. We also present an algorithm to solve the proposed optimization problem, and ensure its convergence. Experimental results revealed that our optimization technique was more effective for image restoration than conventional methods.

Colorization is a method that adds color components to a grayscale image using only a few representative pixels provided by the user. A novel approach to image compression called colorization-based coding has recently been proposed. It automatically extracts representative pixels from an original color image at an encoder and restores a full color image by using colorization at a decoder. However, previous studies on colorization-based coding extract redundant representative pixels and do not extract the pixels required for suppressing coding error. This paper focuses on the colorization basis that restricts the decoded color components. From this viewpoint, we propose a new colorization-based coding method. Experimental results revealed that our method can drastically suppress the information amount (number of representative pixels) compared conventional colorization based-coding while objective quality is maintained.