Missing values in high-dimensional, mixed-type datasets pose significant challenges for data imputation, particularly under Missing Not At Random (MNAR) mechanisms. Existing methods struggle to integrate local and global data characteristics, limiting performance in MNAR and high-dimensional settings. We propose an innovative framework, RefiDiff, combining local machine learning predictions with a novel Mamba-based denoising network efficiently capturing long-range dependencies among features and samples with low computational complexity. RefiDiff bridges the predictive and generative paradigms of imputation, leveraging pre-refinement for initial warm-up imputations and post-refinement to polish results, enhancing stability and accuracy. By encoding mixed-type data into unified tokens, RefiDiff enables robust imputation without architectural or hyperparameter tuning. RefiDiff outperforms state-of-the-art (SOTA) methods across missing-value settings, demonstrating strong performance in MNAR settings and superior out-of-sample generalization. Extensive evaluations on nine real-world datasets demonstrate its robustness, scalability, and effectiveness in handling complex missingness patterns.

In recent years, deep learning techniques have made significant strides in molecular generation for specific targets, driving advancements in drug discovery. However, existing molecular generation methods present significant limitations: those operating at the atomic level often lack synthetic feasibility, drug-likeness, and interpretability, while fragment-based approaches frequently overlook comprehensive factors that influence protein–molecule interactions. To address these challenges, we propose a novel fragment-based molecular generation framework tailored for specific proteins. Our method begins by constructing a protein subpocket and molecular arm concept-based neural network, which systematically integrates interaction force information and geometric complementarity to sample molecular arms for specific protein subpockets. Subsequently, we introduce a diffusion model to generate molecular backbones that connect these arms, ensuring structural integrity and chemical diversity. Our approach improves synthetic feasibility and binding affinity, with a 4% increase in drug-likeness and a 6% improvement in synthetic feasibility. Furthermore, by integrating explicit interaction data through a concept-based model, our framework enhances interpretability, offering valuable insights into the molecular design process.

The integration of single-cell RNA sequencing (scRNA-seq) and spatial transcriptomics (ST) data is crucial for understanding gene expression in spatial context. Existing methods for such integration have limited performance, with structural similarity often below 60\%, We attribute this limitation to the failure to consider causal relationships between genes. We present CausalGeD, which combines diffusion and autoregressive processes to leverage these relationships. By generalizing the Causal Attention Transformer from image generation to gene expression data, our model captures regulatory mechanisms without predefined relationships. Across 10 tissue datasets, CausalGeD outperformed state-of-the-art baselines by 5- 32\% in key metrics, including Pearson's correlation and structural similarity, advancing both technical and biological insights.

The development of single-cell and spatial transcriptomics has revolutionized our capacity to investigate cellular properties, functions, and interactions in both cellular and spatial contexts. Despite this progress, the analysis of single-cell and spatial omics data remains challenging. First, single-cell sequencing data are high-dimensional and sparse, and are often contaminated by noise and uncertainty, obscuring the underlying biological signal. Second, these data often encompass multiple modalities, including gene expression, epigenetic modifications, metabolite levels, and spatial locations. Integrating these diverse data modalities is crucial for enhancing prediction accuracy and biological interpretability. Third, while the scale of single-cell sequencing has expanded to millions of cells, high-quality annotated datasets are still limited. Fourth, the complex correlations of biological tissues make it difficult to accurately reconstruct cellular states and spatial contexts. Traditional feature engineering approaches struggle with the complexity of biological networks, while deep learning, with its ability to handle high-dimensional data and automatically identify meaningful patterns, has shown great promise in overcoming these challenges. Besides systematically reviewing the strengths and weaknesses of advanced deep learning methods, we have curated 21 datasets from nine benchmarks to evaluate the performance of 58 computational methods. Our analysis reveals that model performance can vary significantly across different benchmark datasets and evaluation metrics, providing a useful perspective for selecting the most appropriate approach based on a specific application scenario. We highlight three key areas for future development, offering valuable insights into how deep learning can be effectively applied to transcriptomic data analysis in biological, medical, and clinical settings.

Understanding drug responses at the single-cell level is crucial for identifying biomarkers and uncovering resistance mechanisms. However, existing models predominantly rely on genomic profiles, while overlooking drug structure-function relationships and showing limited generalization to novel drugs with distinct structures. To address this limitation, we propose a novel framework that integrates drug structural information with genomic data. Specifically, we develop a graph-aware Transformer to capture interatomic relations and generate joint representations linking atomic features to genomic profiles. To overcome the scarcity of single-cell drug response data, we propose a novel predictive framework that leverages prior knowledge from bulk RNA datasets through meta-pretraining and few-shot transfer learning. Furthermore, we introduce a position-based feature extraction network and a gene gradient attribution algorithm to identify key resistance genes and drug action pathways. Pre-trained on 223 drugs across 14 tissues and tested on seven single-cell datasets, our model achieves an approximate 5% improvement in accuracy for known drugs and about 20% increase in generalization to unseen drugs. This approach provides an effective method for studying drug resistance mechanisms at single-cell level, particularly for novel compounds.

In this paper, we propose a novel concept factorization method that seeks factor matrices using a cross-order positive semi-definite neighbor graph, which provides comprehensive and complementary neighbor information of the data. The factor matrices are learned with bipartite graph partitioning, which exploits explicit cluster structure of the data and is more geared towards clustering application. We develop an effective and efficient optimization algorithm for our method, and provide elegant theoretical results about the convergence. Extensive experimental results confirm the effectiveness of the proposed method.

Long-term time-series forecasting remains challenging due to the difficulty in capturing long-term dependencies, achieving linear scalability, and maintaining computational efficiency. We introduce TimeMachine, an innovative model that leverages Mamba, a state-space model, to capture long-term dependencies in multivariate time series data while maintaining linear scalability and small memory footprints. TimeMachine exploits the unique properties of time series data to produce salient contextual cues at multi-scales and leverage an innovative integrated quadruple-Mamba architecture to unify the handling of channel-mixing and channel-independence situations, thus enabling effective selection of contents for prediction against global and local contexts at different scales. Experimentally, TimeMachine achieves superior performance in prediction accuracy, scalability, and memory efficiency, as extensively validated using benchmark datasets.

Multi-view subspace clustering (MVSC) has drawn significant attention in recent study. In this paper, we propose a novel approach to MVSC. First, the new method is capable of preserving high-order neighbor information of the data, which provides essential and complicated underlying relationships of the data that is not straightforwardly preserved by the first-order neighbors. Second, we design log-based nonconvex approximations to both tensor rank and tensor sparsity, which are effective and more accurate than the convex approximations. For the associated shrinkage problems, we provide elegant theoretical results for the closed-form solutions, for which the convergence is guaranteed by theoretical analysis. Moreover, the new approximations have some interesting properties of shrinkage effects, which are guaranteed by elegant theoretical results. Extensive experimental results confirm the effectiveness of the proposed method.

Video-based point cloud compression (V-PCC) is a state-of-the-art moving picture experts group (MPEG) standard for point cloud compression. V-PCC can be used to compress both static and dynamic point clouds in a lossless, near lossless, or lossy way. Many objective quality metrics have been proposed for distorted point clouds. Most of these metrics are full-reference metrics that require both the original point cloud and the distorted one. However, in some real-time applications, the original point cloud is not available, and no-reference or reduced-reference quality metrics are needed. Three main challenges in the design of a reduced-reference quality metric are how to build a set of features that characterize the visual quality of the distorted point cloud, how to select the most effective features from this set, and how to map the selected features to a perceptual quality score. We address the first challenge by proposing a comprehensive set of features consisting of compression, geometry, normal, curvature, and luminance features. To deal with the second challenge, we use the least absolute shrinkage and selection operator (LASSO) method, which is a variable selection method for regression problems. Finally, we map the selected features to the mean opinion score in a nonlinear space. Although we have used only 19 features in our current implementation, our metric is flexible enough to allow any number of features, including future more effective ones. Experimental results on the Waterloo point cloud dataset version 2 (WPC2.0) and the MPEG point cloud compression dataset (M-PCCD) show that our method, namely PCQAML, outperforms state-of-the-art full-reference and reduced-reference quality metrics in terms of Pearson linear correlation coefficient, Spearman rank order correlation coefficient, Kendall's rank-order correlation coefficient, and root mean squared error.

Graph neural networks have been widely used for a variety of learning tasks. Link prediction is a relatively under-studied graph learning task, with current state-of-the-art models based on one- or two-layer shallow graph auto-encoder (GAE) architectures. In this paper, we overcome the limitation of current methods for link prediction of non-Euclidean network data, which can only use shallow GAEs and variational GAEs. Our proposed methods innovatively incorporate standard auto-encoders (AEs) into the architectures of GAEs to capitalize on the intimate coupling of node and edge information in complex network data. Empirically, extensive experiments on various datasets demonstrate the competitive performance of our proposed approach. Theoretically, we prove that our deep extensions can inclusively express multiple polynomial filters with different orders. The codes of this paper are available at https://github.com/xinxingwu-uk/DGAE.

In this paper, we propose a novel nonconvex approach to robust principal component analysis for HSI denoising, which focuses on simultaneously developing more accurate approximations to both rank and column-wise sparsity for the low-rank and sparse components, respectively. In particular, the new method adopts the log-determinant rank approximation and a novell2,lognorm, to restrict the local low-rank or column-wisely sparse properties for the component matrices, respectively. For thel2,log-regularized shrinkage problem, we develop an efficient, closed-form solution, which is namedl2,log-shrinkage operator. The new regularization and the corresponding operator can be generally used in other problems that require column-wise sparsity. Moreover, we impose the spatial-spectral total variation regularization in the log-based nonconvex RPCA model, which enhances the global piece-wise smoothness and spectral consistency from the spatial and spectral views in the recovered HSI. Extensive experiments on both simulated and real HSIs demonstrate the effectiveness of the proposed method in denoising HSIs.

Feature selection reduces the dimensionality of data by identifying a subset of the most informative features. In this paper, we propose an innovative framework for unsupervised feature selection, called fractal autoencoders (FAE). It trains a neural network to pinpoint informative features for global exploring of representability and for local excavating of diversity. Architecturally, FAE extends autoencoders by adding a one-to-one scoring layer and a small sub-neural network for feature selection in an unsupervised fashion. With such a concise architecture, FAE achieves state-of-the-art performances; extensive experimental results on fourteen datasets, including very high-dimensional data, have demonstrated the superiority of FAE over existing contemporary methods for unsupervised feature selection. In particular, FAE exhibits substantial advantages on gene expression data exploration, reducing measurement cost by about 15% over the widely used L1000 landmark genes. Further, we show that the FAE framework is easily extensible with an application.

Generative adversarial network (GAN) has become one of the most important neural network models for classical unsupervised machine learning. A variety of discriminator loss functions have been developed to train GAN's discriminators and they all have a common structure: a sum of real and fake losses that only depends on the actual and generated data respectively. One challenge associated with an equally weighted sum of two losses is that the training may benefit one loss but harm the other, which we show causes instability and mode collapse. In this paper, we introduce a new family of discriminator loss functions that adopts a weighted sum of real and fake parts, which we call adaptive weighted loss functions or aw-loss functions. Using the gradients of the real and fake parts of the loss, we can adaptively choose weights to train a discriminator in the direction that benefits the GAN's stability. Our method can be potentially applied to any discriminator model with a loss that is a sum of the real and fake parts. For our experiments, SN-GAN, AutoGAN, and BigGAN are used. Experiments validated the effectiveness of our loss functions on unconditional and conditional image generation tasks, improving the baseline results by a significant margin on CIFAR-10, STL-10, and CIFAR-100 datasets in Inception Scores (IS) and Fréchet Inception Distance (FID) metrics.

It is a challenging task to remove heavy and mixed types of noise from Hyperspectral images (HSIs). In this paper, we propose a novel nonconvex approach to RPCA for HSI denoising, which adopts the log-determinant rank approximation and a novel $\ell_{2,\text{l}\text{o}\text{g}}$ norm, to restrict the low-rank or column-wise sparse properties for the component matrices, respectively. For the $\ell_{2,\text{l}\text{o}\text{g}}$-regularized shrinkage problem, we develop an efficient, closed-form solution, which is named $\ell_{2,\text{l}\text{o}\text{g}}$-shrinkage operator, which can be generally used in other problems. Extensive experiments on both simulated and real HSIs demonstrate the effectiveness of the proposed method in denoising HSIs.

Feature selection, as a vital dimension reduction technique, reduces data dimension by identifying an essential subset of input features, which can facilitate interpretable insights into learning and inference processes. Algorithmic stability is a key characteristic of an algorithm regarding its sensitivity to perturbations of input samples. In this paper, we propose an innovative unsupervised feature selection algorithm attaining this stability with provable guarantees. The architecture of our algorithm consists of a feature scorer and a feature selector. The scorer trains a neural network (NN) to globally score all the features, and the selector adopts a dependent sub-NN to locally evaluate the representation abilities for selecting features. Further, we present algorithmic stability analysis and show that our algorithm has a performance guarantee via a generalization error bound. Extensive experimental results on real-world datasets demonstrate superior generalization performance of our proposed algorithm to strong baseline methods. Also, the properties revealed by our theoretical analysis and the stability of our algorithm-selected features are empirically confirmed.

In this article, we introduce a discriminative ridge regression approach to supervised classification. It estimates a representation model while accounting for discriminativeness between classes, thereby enabling accurate derivation of categorical information. This new type of regression model extends the existing models, such as ridge, lasso, and group lasso, by explicitly incorporating discriminative information. As a special case, we focus on a quadratic model that admits a closed-form analytical solution. The corresponding classifier is called the discriminative ridge machine (DRM). Three iterative algorithms are further established for the DRM to enhance the efficiency and scalability for real applications. Our approach and the algorithms are applicable to general types of data including images, high-dimensional data, and imbalanced data. We compare the DRM with current state-of-the-art classifiers. Our extensive experimental results show the superior performance of the DRM and confirm the effectiveness of the proposed approach.

Edge features contain important information about graphs. However, current state-of-the-art neural network models designed for graph learning, \eg, graph convolutional networks (GCN) and graph attention networks (GAT), inadequately utilize edge features, especially multi-dimensional edge features. In this paper, we build a new framework for a family of new graph neural network models that can more sufficiently exploit edge features, including those of undirected or multi-dimensional edges. The proposed framework can consolidate current graph neural network models, e.g., GCN and GAT. The proposed framework and new models have the following novelties: First, we propose to use doubly stochastic normalization of graph edge features instead of the commonly used row or symmetric normalization approaches used in current graph neural networks. Second, we construct new formulas for the operations in each individual layer so that they can handle multi-dimensional edge features. Third, for the proposed new framework, edge features are adaptive across network layers. As a result, our proposed new framework and new models are able to exploit a rich source of graph edge information. We apply our new models to graph node classification on several citation networks, whole graph classification, and regression on several molecular datasets. Compared with the current state-of-the-art methods, i.e., GCNs and GAT, our models obtain better performance, which testify to the importance of exploiting edge features in graph neural networks.

Robust principal component analysis (RPCA) has drawn significant attentions due to its powerful capability in recovering low-rank matrices as well as successful appplications in various real world problems. The current state-of-the-art algorithms usually need to solve singular value decomposition of large matrices, which generally has at least a quadratic or even cubic complexity. This drawback has limited the application of RPCA in solving real world problems. To combat this drawback, in this paper we propose a new type of RPCA method, RES-PCA, which is linearly efficient and scalable in both data size and dimension. For comparison purpose, AltProj, an existing scalable approach to RPCA requires the precise knowlwdge of the true rank; otherwise, it may fail to recover low-rank matrices. By contrast, our method works with or without knowing the true rank; even when both methods work, our method is faster. Extensive experiments have been performed and testified to the effectiveness of proposed method quantitatively and in visual quality, which suggests that our method is suitable to be employed as a light-weight, scalable component for RPCA in any application pipelines.

Spectral clustering has found extensive use in many areas. Most traditional spectral clustering algorithms work in three separate steps: similarity graph construction; continuous labels learning; discretizing the learned labels by k-means clustering. Such common practice has two potential flaws, which may lead to severe information loss and performance degradation. First, predefined similarity graph might not be optimal for subsequent clustering. It is well-accepted that similarity graph highly affects the clustering results. To this end, we propose to automatically learn similarity information from data and simultaneously consider the constraint that the similarity matrix has exact c connected components if there are c clusters. Second, the discrete solution may deviate from the spectral solution since k-means method is well-known as sensitive to the initialization of cluster centers. In this work, we transform the candidate solution into a new one that better approximates the discrete one. Finally, those three subtasks are integrated into a unified framework, with each subtask iteratively boosted by using the results of the others towards an overall optimal solution. It is known that the performance of a kernel method is largely determined by the choice of kernels. To tackle this practical problem of how to select the most suitable kernel for a particular data set, we further extend our model to incorporate multiple kernel learning ability. Extensive experiments demonstrate the superiority of our proposed method as compared to existing clustering approaches.

In this article, we introduce a novel, general methodology, called integrate and conquer, for simultaneously accomplishing the tasks of feature extraction, manifold construction, and clustering, which is taken to be superior to building a clustering method as a single task. When the proposed novel methodology is used on two-dimensional (2D) data, it naturally induces a new clustering method highly effective on 2D data. Existing clustering algorithms usually need to convert 2D data to vectors in a preprocessing step, which, unfortunately, severely damages 2D spatial information and omits inherent structures and correlations in the original data. The induced new clustering method can overcome the matrix-vectorization-related issues to enhance the clustering performance on 2D matrices. More specifically, the proposed methodology mutually enhances three tasks of finding subspaces, learning manifolds, and constructing data representation in a seamlessly integrated fashion. When used on 2D data, we seek two projection matrices with optimal numbers of directions to project the data into low-rank, noise-mitigated, and the most expressive subspaces, in which manifolds are adaptively updated according to the projections, and new data representation is built with respect to the projected data by accounting for nonlinearity via adaptive manifolds. Consequently, the learned subspaces and manifolds are clean and intrinsic, and the new data representation is discriminative and robust. Extensive experiments have been conducted and the results confirm the effectiveness of the proposed methodology and algorithm.

Many similarity-based clustering methods work in two separate steps including similarity matrix computation and subsequent spectral clustering. However similarity measurement is challenging because it is usually impacted by many factors, e.g., the choice of similarity metric, neighborhood size, scale of data, noise and outliers. Thus the learned similarity matrix is often not suitable, let alone optimal, for the subsequent clustering. In addition, nonlinear similarity often exists in many real world data which, however, has not been effectively considered by most existing methods. To tackle these two challenges, we propose a model to simultaneously learn cluster indicator matrix and similarity information in kernel spaces in a principled way. We show theoretical relationships to kernel k-means, k-means, and spectral clustering methods. Then, to address the practical issue of how to select the most suitable kernel for a particular clustering task, we further extend our model with a multiple kernel learning ability. With this joint model, we can automatically accomplish three subtasks of finding the best cluster indicator matrix, the most accurate similarity relations and the optimal combination of multiple kernels. By leveraging the interactions between these three subtasks in a joint framework, each subtask can be iteratively boosted by using the results of the others towards an overall optimal solution. Extensive experiments are performed to demonstrate the effectiveness of our method.

Spectral clustering based subspace clustering methods have emerged recently. When the inputs are 2-dimensional (2D) data, most existing clustering methods convert such data to vectors as preprocessing, which severely damages spatial information of the data. In this paper, we propose a novel subspace clustering method for 2D data with enhanced capability of retaining spatial information for clustering. It seeks two projection matrices and simultaneously constructs a linear representation of the projected data, such that the sought projections help construct the most expressive representation with the most variational information. We regularize our method based on covariance matrices directly obtained from 2D data, which have much smaller size and are more computationally amiable. Moreover, to exploit nonlinear structures of the data, a nonlinear version is proposed, which constructs an adaptive manifold according to updated projections. The learning processes of projections, representation, and manifold thus mutually enhance each other, leading to a powerful data representation. Efficient optimization procedures are proposed, which generate non-increasing objective value sequence with theoretical convergence guarantee. Extensive experimental results confirm the effectiveness of proposed method.

Construction of a reliable similarity matrix is fundamental for graph-based clustering methods. However, most of the current work is built upon some simple manifold structure, whereas limited work has been conducted on nonlinear data sets where data reside in a union of manifolds rather than a union of subspaces. Therefore, we construct a similarity graph to capture both global and local manifold structures of the input data set. The global structure is exploited based on the self-expressive property of data in an implicit feature space using kernel methods. Since the similarity graph computation is independent of the subsequent clustering, the final results may be far from optimal. To overcome this limitation, we simultaneously learn similarity graph and clustering structure in a principled way. Experimental studies demonstrate that our proposed algorithms deliver consistently superior results to other state-of-the-art algorithms.

Subspace clustering methods have been widely studied recently. When the inputs are two-dimensional (2-D) data, existing subspace clustering methods usually convert them into vectors, which severely damages inherent structures and relationships from original data. In this letter, we propose a novel subspace clustering method for 2-D data. It directly uses 2-D data as inputs such that the learning of representations benefits from inherent structures and relationships of the data. It simultaneously seeks image projection and representation coefficients such that they mutually enhance each other and lead to powerful data representations. An efficient algorithm is developed to solve the proposed objective function with provable decreasing and convergence property. Extensive experimental results verify the effectiveness of the new method.

We introduce a new supervised learning model using a discriminative regression approach. This new model estimates a regression vector to represent the similarity between a test example and training examples while seamlessly integrating the class information in the similarity estimation. This distinguishes our model from usual regression models and locally linear embedding approaches, rendering our method suitable for supervised learning problems in high-dimensional settings. Our model is easily extensible to account for nonlinear relationship and applicable to general data, including both high- and low-dimensional data. The objective function of the model is convex, for which two optimization algorithms are provided. These two optimization approaches induce two scalable solvers that are of mathematically provable, linear time complexity. Experimental results verify the effectiveness of the proposed method on various kinds of data. For example, our method shows comparable performance on low-dimensional data and superior performance on high-dimensional data to several widely used classifiers; also, the linear solvers obtain promising performance on large-scale classification.

Matrix factorization is a useful technique for data representation in many data mining and machine learning tasks. Particularly, for data sets with all nonnegative entries, matrix factorization often requires that factor matrices be nonnegative, leading to nonnegative matrix factorization (NMF). One important application of NMF is for clustering with reduced dimensions of the data represented in the new feature space. In this paper, we propose a new graph regularized NMF method capable of feature learning and apply it to clustering. Unlike existing NMF methods that treat all features in the original feature space equally, our method distinguishes features by incorporating a feature-wise sparse approximation error matrix in the formulation. It enables important features to be more closely approximated by the factor matrices. Meanwhile, the graph of the data is constructed using cleaner features in the feature learning process, which integrates feature learning and manifold learning procedures into a unified NMF model. This distinctly differs from applying the existing graph-based NMF models after feature selection in that, when these two procedures are independently used, they often fail to align themselves toward obtaining a compact and most expressive data representation. Comprehensive experimental results demonstrate the effectiveness of the proposed method, which outperforms state-of-the-art algorithms when applied to clustering.

Matrix factorization is often used for data representation in many data mining and machine-learning problems. In particular, for a dataset without any negative entries, nonnegative matrix factorization (NMF) is often used to find a low-rank approximation by the product of two nonnegative matrices. With reduced dimensions, these matrices can be effectively used for many applications such as clustering. The existing methods of NMF are often afflicted with their sensitivity to outliers and noise in the data. To mitigate this drawback, in this paper, we consider integrating NMF into a robust principal component model, and design a robust formulation that effectively captures noise and outliers in the approximation while incorporating essential nonlinear structures. A set of comprehensive empirical evaluations in clustering applications demonstrates that the proposed method has strong robustness to gross errors and superior performance to current state-of-the-art methods.

Top-N recommender systems have been investigated widely both in industry and academia. However, the recommendation quality is far from satisfactory. In this paper, we propose a simple yet promising algorithm. We fill the user-item matrix based on a low-rank assumption and simultaneously keep the original information. To do that, a nonconvex rank relaxation rather than the nuclear norm is adopted to provide a better rank approximation and an efficient optimization strategy is designed. A comprehensive set of experiments on real datasets demonstrates that our method pushes the accuracy of Top-N recommendation to a new level.

Recommender systems play an increasingly important role in online applications to help users find what they need or prefer. Collaborative filtering algorithms that generate predictions by analyzing the user-item rating matrix perform poorly when the matrix is sparse. To alleviate this problem, this paper proposes a simple recommendation algorithm that fully exploits the similarity information among users and items and intrinsic structural information of the user-item matrix. The proposed method constructs a new representation which preserves affinity and structure information in the user-item rating matrix and then performs recommendation task. To capture proximity information about users and items, two graphs are constructed. Manifold learning idea is used to constrain the new representation to be smooth on these graphs, so as to enforce users and item proximities. Our model is formulated as a convex optimization problem, for which we need to solve the well known Sylvester equation only. We carry out extensive empirical evaluations on six benchmark datasets to show the effectiveness of this approach.

Recovering low-rank matrices is a problem common in many applications of data mining and machine learning, such as matrix completion and image denoising. Robust Principal Component Analysis (RPCA) has emerged for handling such kinds of problems; however, the existing RPCA approaches are usually computationally expensive, due to the fact that they need to obtain the singular value decomposition (SVD) of large matrices. In this paper, we propose a novel RPCA approach that eliminates the need for SVD of large matrices. Scalable algorithms are designed for several variants of our approach, which are crucial for real world applications on large scale data. Extensive experimental results confirm the effectiveness of our approach both quantitatively and visually.

Robust principal component analysis (RPCA) has been widely used for recovering low-rank matrices in many data mining and machine learning problems. It separates a data matrix into a low-rank part and a sparse part. The convex approach has been well studied in the literature. However, state-of-the-art algorithms for the convex approach usually have relatively high complexity due to the need of solving (partial) singular value decompositions of large matrices. A non-convex approach, AltProj, has also been proposed with lighter complexity and better scalability. Given the true rank r of the underlying low rank matrix, AltProj has a complexity of O(r2dn), where d × n is the size of data matrix. In this paper, we propose a novel factorization-based model of RPCA, which has a complexity of O(kdn), where k is an upper bound of the true rank. Our method does not need the precise value of the true rank. From extensive experiments, we observe that AltProj can work only when r is precisely known in advance, however, when the needed rank parameter r is specified to a value different from the true rank, AltProj cannot fully separate the two parts while our method succeeds. Even when both work, our method is about 4 times faster than AltProj. Our method can be used as a light-weight, scalable tool for RPCA in the absence of the precise value of the true rank.

The importance of accurate recommender systems has been widely recognized by academia and industry. However, the recommendation quality is still rather low. Recently, a linear sparse and low-rank representation of the user-item matrix has been applied to produce Top-N recommendations. This approach uses the nuclear norm as a convex relaxation for the rank function and has achieved better recommendation accuracy than the state-of-the-art methods. In the past several years, solving rank minimization problems by leveraging nonconvex relaxations has received increasing attention. Some empirical results demonstrate that it can provide a better approximation to original problems than convex relaxation. In this paper, we propose a novel rank approximation to enhance the performance of Top-N recommendation systems, where the approximation error is controllable. Experimental results on real data show that the proposed rank approximation improves the Top-N recommendation accuracy substantially.

We propose a new subspace clustering method that integrates feature selection into subspace clustering. Rather than using all features to construct a low-rank representation of the data, we find such a representation using only relevant features, which helps in revealing more accurate data relationships. Two variants are proposed by using both convex and nonconvex rank approximations. Extensive experimental results confirm the effectiveness of the proposed method and models.

Matrix rank minimization problem is in general NP-hard. The nuclear norm is used to substitute the rank function in many recent studies. Nevertheless, the nuclear norm approximation adds all singular values together and the approximation error may depend heavily on the magnitudes of singular values. This might restrict its capability in dealing with many practical problems. In this paper, an arctangent function is used as a tighter approximation to the rank function. We use it on the challenging subspace clustering problem. For this nonconvex minimization problem, we develop an effective optimization procedure based on a type of augmented Lagrange multipliers (ALM) method. Extensive experiments on face clustering and motion segmentation show that the proposed method is effective for rank approximation.

Numerous applications in data mining and machine learning require recovering a matrix of minimal rank. Robust principal component analysis (RPCA) is a general framework for handling this kind of problems. Nuclear norm based convex surrogate of the rank function in RPCA is widely investigated. Under certain assumptions, it can recover the underlying true low rank matrix with high probability. However, those assumptions may not hold in real-world applications. Since the nuclear norm approximates the rank by adding all singular values together, which is essentially a l1-norm of the singular values, the resulting approximation erroris not trivial and thus the resulting matrix estimator can be significantly biased. To seek a closer approximation and to alleviate the above-mentioned limitations of the nuclear norm, we propose a nonconvex rank approximation. This approximation to the matrix rank is tighter than the nuclear norm. To solve the associated nonconvex minimization problem, we develop an efficient augmented Lagrange multiplier based optimization algorithm. Experimental results demonstrate that our method outperforms current state-of-the-art algorithms in both accuracy and efficiency.

A number of machine learning and computer vision problems, such as matrix completion and subspace clustering, require a matrix to be of low-rank. To meet this requirement, most existing methods use the nuclear norm as a convex proxy of the rank function and minimize it. However, the nuclear norm simply adds all nonzero singular values together instead of treating them equally as the rank function does, which may not be a good rank approximation when some singular values are very large. To reduce this undesirable weighting effect, we use a log-determinant function as a non-convex rank approximation which reduces the contributions of large singular values while keeping those of small singular values close to zero. We apply the method of augmented Lagrangian multipliers to optimize this non-convex rank approximation-based objective function and obtain closed-form solutions for all subproblems of minimizing different variables alternatively. The log-determinant low-rank optimization method is used to solve subspace clustering problem, for which we construct an affinity matrix based on the angular information of the low-rank representation to enhance its separability property. Extensive experimental results on face clustering and motion segmentation data demonstrate the effectiveness of the proposed method.

Matrix rank minimizing subject to affine constraints arises in many application areas, ranging from signal processing to machine learning. Nuclear norm is a convex relaxation for this problem which can recover the rank exactly under some restricted and theoretically interesting conditions. However, for many real-world applications, nuclear norm approximation to the rank function can only produce a result far from the optimum. To seek a solution of higher accuracy than the nuclear norm, in this letter, we propose a rank approximation based on Logarithm-Determinant. We consider using this rank approximation for subspace clustering application. Our framework can model different kinds of errors and noise. Effective optimization strategy is developed with theoretical guarantee to converge to a stationary point. The proposed method gives promising results on face clustering and motion segmentation tasks compared to the state-of-the-art subspace clustering algorithms.

Quantitative susceptibility mapping (QSM) is a magnetic resonance imaging technique that reveals tissue magnetic susceptibility. It relies on having a high quality field map, typically acquired with a relatively long echo spacing and long final TE. Applications of QSM outside the brain require the removal of fat contributions to the total signal phase. However, current water/fat separation methods applied on typical data acquired for QSM suffer from three issues: inadequacy when using large echo spacing, over-smoothing of the field maps and high computational cost. In this paper, the general phase wrap and chemical shift problem is formulated using a single species fitting and is solved using graph cuts with conditional jump moves. This method is referred as simultaneous phase unwrapping and removal of chemical shift (SPURS). The result from SPURS is then used as the initial guess for a voxel-wise iterative decomposition of water and fat with echo asymmetric and least-squares estimation (IDEAL). The estimated 3-D field maps are used to compute QSM in body regions outside of the brain, such as the liver. Experimental results show substantial improvements in field map estimation, water/fat separation and reconstructed QSM compared to two existing water/fat separation methods on 1.5T and 3T magnetic resonance human data with long echo spacing and rapid field map variation.

This paper presents an accurate, efficient, and scalable algorithm for minimizing a special family of convex functions, which have a lp loss function as an additive component. For this problem, well-known learning algorithms often have well-established results on accuracy and efficiency, but there exists rarely any report on explicit linear scalability with respect to the problem size. The proposed approach starts with developing a second-order learning procedure with iterative descent for general convex penalization functions, and then builds efficient algorithms for a restricted family of functions, which satisfy the Karmarkar's projective scaling condition. Under this condition, a light weight, scalable message passing algorithm (MPA) is further developed by constructing a series of simpler equivalent problems. The proposed MPA is intrinsically scalable because it only involves matrix-vector multiplication and avoids matrix inversion operations. The MPA is proven to be globally convergent for convex formulations; for nonconvex situations, it converges to a stationary point. The accuracy, efficiency, scalability, and applicability of the proposed method are verified through extensive experiments on sparse signal recovery, face image classification, and over-complete dictionary learning problems.

This paper introduces a minimax framework for multiclass classification, which is applicable to general data including, in particular, imagery and other types of high-dimensional data. The framework consists of estimating a representation model that minimizes the fitting errors under a class of distortions of interest to an application, and deriving subsequently categorical information based on the estimated model. A variety of commonly used regression models, including lasso, elastic net and ridge regression, can be regarded as special cases that correspond to specific classes of distortions. Optimal decision rules are derived for this classification framework. By using kernel techniques the framework can account for nonlinearity in the input space. To demonstrate the power of the framework we consider a class of signal-dependent distortions and build a new family of classifiers as new special cases. This family of new methods-minimax classification with generalized multiplicative distortions-often outperforms the state-of-the-art classification methods such as the support vector machine in accuracy. Extensive experimental results on images, gene expressions and other types of data verify the effectiveness of the proposed framework.

Markov Decision Processes (MDPs) are extremely useful for modeling and solving sequential decision making problems. Graph-based MDPs provide a compact representation for MDPs with large numbers of random variables. However, the complexity of exactly solving a graph-based MDP usually grows exponentially in the number of variables, which limits their application. We present a new variational framework to describe and solve the planning problem of MDPs, and derive both exact and approximate planning algorithms. In particular, by exploiting the graph structure of graph-based MDPs, we propose a factored variational value iteration algorithm in which the value function is first approximated by the multiplication of local-scope value functions, then solved by minimizing a Kullback-Leibler (KL) divergence. The KL divergence is optimized using the belief propagation algorithm, with complexity exponential in only the cluster size of the graph. Experimental comparison on different models shows that our algorithm outperforms existing approximation algorithms at finding good policies.

This paper develops a novel supervised method for predicting earthquake ground motions in the wavelet domain. The training input is a set of seismological predictors related to seismic source, path and local site conditions, and the training output consists of the weights from a multiway analysis of ground motions. We treat wavelet transforms of acceleration records as images and extract essential patterns from them using tensor decomposition. The decomposition weights of these patterns are then linked to seismological variables using general regression neural network (GRNN). The resulting nonparametric model is then used to predict the wavelet image of an accelerogram for a given set of seismological variables. The predicted image can be transformed back to the time domain using inverse wavelet transform for subsequent processing to match a given design spectrum. Unlike conventional ground motion models, the proposed approach retains the time domain characteristics of ground motions. Pearson's correlation coefficient between the vectorized forms of actual and predicted wavelet images has been used as the similarity metric in assessing the prediction capability of the resulting model. Experimental results demonstrate the ability of the proposed model to predict significant patterns in the seismic energy distribution.

We study the marginal-MAP problem on graphical models, and present a novel approximation method based on direct approximation of the sum operation. A primary difficulty of marginal-MAP problems lies in the non-commutativity of the sum and max operations, so that even in highly structured models, marginalization may produce a densely connected graph over the variables to be maximized, resulting in an intractable potential function with exponential size. We propose a chain decomposition approach for summing over the marginalized variables, in which we produce a structured approximation to the MAP component of the problem consisting of only pairwise potentials. We show that this approach is equivalent to the maximization of a specific variational free energy, and it provides an upper bound of the optimal probability. Finally, experimental results demonstrate that our method performs favorably compared to previous methods.

Subspace embedding is a powerful tool for extracting salient information from matrix, and it has numerous applications in image processing. However, its applicability has been severely limited by the computational complexity of O(N3) (N is the number of the points) which usually arises in explicitly evaluating the eigenvalues and eigenvectors. In this paper, we propose an implicit subspace embedding method which avoids explicitly evaluating the eigenvectors. Also, we show that this method can be seamlessly incorporated into the unsupervised multi-scale image segmentation framework and the resulted algorithm has a running time of genuine O(N). Moreover, we can explicitly determine the number of iterations for the algorithm by estimating the desired size of the subspace, which also controls the amount of information we want to extract for this unsupervised learning. We performed extensive experiments to verify the validity and effectiveness of our method, and we conclude that it only requires less than 120 seconds (CPU 3.2G and memory 16G) to cut a 1000∗1000 color image and orders of magnitude faster than original multi-scale image segmentation with explicit spectral decomposition while maintaining the same or a better segmentation quality.

Selecting features for multiclass classification is a critically important task for pattern recognition and machine learning applications. Especially challenging is selecting an optimal subset of features from high-dimensional data, which typically have many more variables than observations and contain significant noise, missing components, or outliers. Existing methods either cannot handle high-dimensional data efficiently or scalably, or can only obtain local optimum instead of global optimum. Toward the selection of the globally optimal subset of features efficiently, we introduce a new selector--which we call the Fisher-Markov selector--to identify those features that are the most useful in describing essential differences among the possible groups. In particular, in this paper we present a way to represent essential discriminating characteristics together with the sparsity as an optimization objective. With properly identified measures for the sparseness and discriminativeness in possibly high-dimensional settings, we take a systematic approach for optimizing the measures to choose the best feature subset. We use Markov random field optimization techniques to solve the formulated objective functions for simultaneous feature selection. Our results are noncombinatorial, and they can achieve the exact global optimum of the objective function for some special kernels. The method is fast; in particular, it can be linear in the number of features and quadratic in the number of observations. We apply our procedure to a variety of real-world data, including mid--dimensional optical handwritten digit data set and high-dimensional microarray gene expression data sets. The effectiveness of our method is confirmed by experimental results. In pattern recognition and from a model selection viewpoint, our procedure says that it is possible to select the most discriminating subset of variables by solving a very simple unconstrained objective function which in fact can be obtained with an explicit expression.

This brief presents a dynamical system approach to vector quantization or clustering based on ordinary differential equations with the potential for real-time implementation. Two examples of different pattern clusters demonstrate that the model can successfully quantize different types of input patterns. Furthermore, we analyze and study the stability of our dynamical system. By discovering the equilibrium points for certain input patterns and analyzing their stability, we have shown the quantizing behavior of the system with respect to its vigilance parameter. The proposed system is applied to two real-world problems, providing comparable results to the best reported findings. This validates the effectiveness of our proposed approach.

Regularization technique has become a principle tool for statistics and machine learning research and practice. However, in most situations, these regularization terms are not well interpreted, especially on how they are related to the loss function and data. In this paper, we propose a robust minimax framework to interpret the relationship between data and regularization terms for a large class of loss functions. We show that various regularization terms are essentially corresponding to different distortions to the original data matrix. This minimax framework includes ridge regression, lasso, elastic net, fused lasso, group lasso, local coordinate coding, multiple kernel learning, etc., as special cases. Within this minimax framework, we further gave mathematically exact definition for a novel representation called sparse grouping representation (SGR), and proved sufficient conditions for generating such group level sparsity. Under these sufficient conditions, a large set of consistent regularization terms can be designed. This SGR is essentially different from group lasso in the way of using class or group information, and it outperforms group lasso when there appears group label noise. We also gave out some generalization bounds in a classification setting.

Prepulse inhibition (PPI) refers to the reduction in startle reaction towards a startle-eliciting “pulse” stimulus when it is shortly preceded by a sub-threshold “prepulse” stimulus. PPI deficits have been seen in patients with schizophrenia and animal models of this mental disorder. The goal of this study was to provide an alternative method for the analysis of PPI data. The new method is expected to be more reliable and sensitive than the existing conventional method. We applied the Kernel density estimation (KDE) in the analysis of PPI data. KDE is a non-parametric method of estimating the probability density function of a random variable and is widely used in inferring population statistics based on limited, noisy samples of continuous random variables. Our results showed that the KDE method performed better than the conventional method and offered some advantages which are of significant in the post-session analysis of PPI data and in performing animal experiments.

Extracting features from high-dimensional data is a critically important task for pattern recognition and machine learning applications. High-dimensional data typically have much more variables than observations, and contain significant noise, missing components, or outliers. Features extracted from high-dimensional data need to be discriminative, sparse, and can capture essential characteristics of the data. In this paper, we present a way to constructing multivariate features and then classify the data into proper classes. The resulting small subset of features is nearly the best in the sense of Greenshtein's persistence; however, the estimated feature weights may be biased. We take a systematic approach for correcting the biases. We use conjugate gradient-based primal-dual interior-point techniques for large-scale problems. We apply our procedure to microarray gene analysis. The effectiveness of our method is confirmed by experimental results.

This paper constructs a class of generalized embeddings of multiplicative watermarks. Ordinary multiplicative and additive methods are included as special cases. The new watermarks automatically adapt to the local contents of host signals, benefiting the perceptual quality. The decoding makes use of the optimal generalized correlation detector. The host interference is precanceled at the embedder side and very high gains are obtained in terms of decoding capability. We develop performance analysis for this new class of embeddings. It turns out that the plain multiplicative watermark is far outperformed by the new embedding. Further, the multiplicative watermark with host interference rejection is still suboptimal. The best embeddings and configurations are specified for typical scenarios. Our construction and performance analyses of the generalized embedding offer a class of new methods. The construction and analyses are confirmed by empirical experiments.

As one of the fundamental issues in wireless sensor networks (WSNs), the sensor localization problem has recently received extensive attention. In this work, we investigate this problem from a novel perspective by treating it as a functional dual of target tracking. In traditional tracking problems, static location-aware sensors track and predict the position and/or velocity of a moving target. As a dual, we utilize a moving location assistant (LA) (with a global positioning system (GPS) or a predefined moving path) to help location-unaware sensors to accurately discover their positions. We call our proposed system Landscape. In Landscape, an LA (an aircraft, for example) periodically broadcasts its current location (we call it a beacon) while it moves around or through a sensor field. Each sensor collects the location beacons, measures the distance between itself and the LA based on the received signal strength (RSS), and individually calculates their locations via an Unscented Kalman Filter (UKF)-based algorithm. Landscape has several features that are favorable to WSNs, such as high scalability, no intersensor communication overhead, moderate computation cost, robustness to range errors and network connectivity, etc. Extensive simulations demonstrate that Landscape is an efficient sensor positioning scheme for outdoor sensor networks.

Multiplanar reformatting (MPR) of 3D biomedical images is an important technique in visualizing, editing, and interacting with volumetric data. To obtain near real-time interactions with the data in many applications such as telemedicine and teleconsultation, the MPR slices are produced and transmitted dynamically. The MPR compression is an important technique to improve the transmission and display efficiency. We develop a dedicated MPR compression scheme by exploiting the characteristics of MPR slices, especially for thin MPR. Robust regression techniques are applied to predict the current slice from the previous ones. In the presence of scales, rotations, and translations, we make use of the known knowledge of the operations, or the transform domain representations of the image in the case of unknown parameters. The effectiveness of the scheme is confirmed by experimental results.

Despite the fact that sensor networks could often be deployed over three-dimensional (3D) terrains, most approaches on sensor localizations are designed and evaluated considering only two-dimensional (2D) applications. On the other hand, being the foundation of the most previous localization solutions, reliable and sufficient neighborhood-measurements are often hard to achieve for sensor nodes deployed in complex 3D terrains, which makes it difficult to extend those solutions into 3D applications. In the paper, we introduce a robust 3D localization solution called Landscape-3D, in which we treat the localization problem from a novel perspective by taking it as a functional dual of target tracking. Besides several nice features, such as high scalability, high accuracy, zero sensor-to-sensor communication overhead, low computation overhead, etc., one of the most important advantages of Landscape-3D is that it works totally independent of node densities and network topologies, which makes it robust to complex 3D environments. Our simulation model involves various 3D scenarios. Experimental results demonstrate that Landscape-3D is a robust localization approach for sensor networks deployed in complex 3D terrains

We develop principal components analysis for the optimal SNR phased-array magnetic resonance (MR) image recombination. As shown in our analysis, we can achieve the best possible SNR in both weak-noise and noisy cases, without needing to estimate the coil sensitivities or to remove noise effects using polynomial fitting or filtering. We provide both analysis and reconstruction techniques. Our results shed light on the performance of the phased-array image combination and give new insight into good image formation schemes.

In this work, we consider the sensor localization problem from a novel perspective by treating it as a functional dual of target tracking. In traditional tracking problems, static location-aware sensors track and predict the position/speed of a moving target. As a dual, we utilize a moving location-assistant (LA) (with global positioning system (GPS) or pre-defined moving path) to help location-unaware sensors to accurately discover their positions. We call our proposed system Landscape. In Landscape, an LA (an aircraft, for example) periodically broadcasts its current location while it moves around or through a sensor field. Each sensor collects the location beacons, measures the distance between itself and the LA based on received signal strength (RSS), and individually calculates their locations via an unscented Kalman filter (UKF) based algorithm. Our contributions are at least twofold. (1) Landscape is a distributed scheme, it does not rely on measured distances among neighbors (as used by most current proposals), which makes it robust to topology and density; Landscape involves zero sensor-to-sensor communication overhead, and is highly scalable to network size. (2) By introducing UKF in sensor localization problem, we reap multiple benefits: our UKF-based algorithm nicely exploits the constraints increasingly added by the beacons; it elegantly solves the nonlinear problem with low computation cost and complexity; and most importantly, it efficiently reduces the effects of measurement errors, making Landscape robust to ranging errors. Extensive simulations and evaluations against the state-of-the-art systems show that Landscape is a high-performance sensor positioning scheme for outdoor sensor networks.

The increasing nursing shortage makes people turn to cutting-edge technology. Nursing systems utilizing the Bluetooth technique have appeared as an alternative to the shortage. They give patients the freedom of moving around. However, the mobility is confined inside the ward, hospital or house. This paper proposed a nursing system prototype by making use of the existing cell phone network. It has flexibility and allows the patients to go anywhere they like while still providing the same good medical care service. As an additional benefit, the proposed prototype can free people from the trivial typing burden in computer data management.

Digital watermarking is an emerging technique to protect intellectual property right and to transmit secondary data. Communication of secret messages or verification of watermarking patterns can be achieved by detecting watermarks in received signals. This paper investigates efficient designs of watermarking patterns by minimizing the probability of detection errors. For an encoder with the knowledge of a decoder and with side information from host signals, a small signal approximation is used in designing efficient watermark patterns. For a decoder without the knowledge of the host signal, accurate statistical modeling can help achieving optimal decoding performance. The design method opens the door for seeking efficient watermarks for many watermarking systems.

Digital watermarking is an emerging technique to protect intellectual property right and to transmit secondary data. This paper investigates efficient designs of watermarking system that maximizes the error exponents of probability of detection errors. It turns out that the efficacy of the systems can be maximized in doing so. We take a detector-centric approach to achieving our designing goals. First an optimum detector is constricted by taking into account the real statistical characteristics of the multimedia host data. Then, for an encoder with side information of the decoder as well as the host signal, a particular watermark pattern can be constructed to satisfy some desired properties, such as automatic adaptivity to the local content of the host signal, and/or host-interference rejection. A family of automatically host-content adaptive watermarks (AHCAW) is also proposed. The method paves the way towards creating efficient watermarks and building new watermarking systems.

Digital watermarking is a powerful technique to help protect the intellectual property right as well as the security of multimedia content. Multiplicative watermarks have strong robustness, and are well-suited for the copyright protection. We investigate the optimal detection and decoding of multiplicative watermarks in DFT domain. For a class of non-Gaussian distributions, we derive locally optimum detector structure and extend it to decoding multiple-bit message. In watermark detection, optimal thresholding for given false-alarm rates is derived. The optimal decoding of multiple-bit messages in data hiding is constructed using generalized maximum likelihood estimation. Theoretical results are verified using experiments.

The technique of embedding a digital signal into an audio recording or image using techniques that render the signal imperceptible has received significant attention. Embedding an imperceptible, cryptographically secure signal, or watermark, is seen as a potential mechanism that may be used to prove ownership or detect tampering. While there has been a considerable amount of attention devoted to the techniques of spread-spectrum signaling for use in image and audio watermarking applications, there has only been a limited study for embedding data signals in speech. Speech is an uncharacteristically narrow band signal given the perceptual capabilities of the human hearing system. However, using speech analysis techniques, one may design an effective data signal that can be used to hide an arbitrary message in a speech signal. Also included are experiments demonstrating the subliminal channel capacity of the speech data embedding technique developed here.

Image-adaptive watermarking based on sophisticated human perceptual models is capable of embedding watermarks with maximum strengths while incurring no perceptual loss. Very strong robustness as well as high information capacity can be achieved using these schemes. In this paper, the optimum detector for the perceptual-model-based robust watermarking is constructed, and the performance analysis is investigated. The new detector asymptotically is most efficient for weak signals, and particularly it is the most powerful for the perceptual-model-constrained watermarks. The experiments results validate our theoretical analysis.

Multiplicative watermarks have very strong robustness, and they are well-suited for the copyright protection. In this paper, new detector structures for the optimum detection of multiplicative watermarks are derived. It is shown that the observations should be raised to the power of the shape parameter of the distribution before they are correlated with the watermark. For commonly used Gaussian distribution a quadratic correlator is obtained. A generalized linear correlator is also derived based on the Laplacian distribution. The performance analysis of the proposed detector is examined. The theoretical results are verified by the experiments.

Digital watermarking is a promising technique to help protect the data security and intellectual property right. In this paper a combined audio and video watermarking technique is proposed. Content-dependent information is extracted from the audio signal. The extraction uses a well-known feature set for audio, the melfrequency cepstra. The information is encoded and embedded into the video. The algorithm can be applied to the content authentication of audiovisual data in journalism, commerce and law.

An image watermarking technique based on pyramid transforms is proposed. An arbitrary binary pattern is formed into an effective hypothesized pattern and transmitted as a watermark. Multiresolution pyramid transforms are applied to host images, whose characteristics are exploited to embed the watermark. The detector is designed to be effective to a wide range of original signal sources and noise sources. The scheme is designed to achieve efficient trade-offs between perceptual invisibility, robustness and trustworthy detection. The experiments demonstrate that the proposed technique has high imperceptibility, good robustness, and accurate detection. It can be applied to copyright notification, enforcement, and fingerprinting.

This paper presents an additive approach to transform-domain information hiding and the performance analysis for images and video. The watermark embedding method is designed to satisfy the perceptual constraints and improve the detectability as well as the information embedding rate. The statistical behaviors of subband coefficients are modeled by the generalized Gaussian distribution. The structure of the optimum detection is built and the performance of the exact asymptotic detection is evaluated using large deviation theory. Our approach can not only achieve good transparency but also precisely control the detection errors. It can be applied to watermarking, authentication, fingerprinting, and steganography.

This paper presents a digital watermarking system in the DCT-domain. The watermarking system is designed to satisfy strictly the perceptual constraints. Under the constraints, robust and fragile watermarking techniques are designed by varying the watermark strength. The optimum detection structure is constructed based on the Laplacian model, which has the properties that high performance can be achieved for a large variety of statistical models, and the computation is efficient. The detection does not need the original images. It can also detect the tampered regions very accurately. The simulations validate the watermarking system. The system can be used in copyright notification and protection, broadcast monitoring and tracking, and authentication and tamper-proofing.