The Laplace transform is traditionally viewed as a non-orthogonal integral transform, lacking an explicit basis interpretation comparable to the Fourier transform. In this work, we present a novel operator-theoretic framework showing that the Laplace transform can be interpreted as a biorthogonal transform, with its forward and inverse kernels forming a pair of continuous dual frames in an appropriate Hilbert space. We further extend this perspective to bilateral Laplace, Mellin, and fractional Laplace transforms, proving that each can be expressed as a similarity to a unitary operator combined with a bounded, invertible weighting operator. This leads to explicit biorthogonality relations and stable reconstruction formulas, connecting classical integral transforms to modern frame and Riesz basis theory. Discretizations of these transforms naturally yield biorthogonal Riesz bases on finite intervals, enabling stable inversion and sparse representation. The framework unifies classical integral transform theory with contemporary biorthogonal analysis and opens avenues for novel transform designs in signal processing.

In this work, we address the challenge of estimating principal components in the presence of outliers, a problem commonly referred to as robust principal component analysis (PCA). Traditional PCA minimizes the Euclidean norm, making it vulnerable to outliers. To enhance robustness, earlier approaches replaced the Euclidean norm with the l1-norm, which improved resilience to outliers but did not fully eliminate their influence. Subsequent studies proposed the use of the lp-norm (0p-norm with the l0-norm, effectively eliminating the influence of outliers in the estimation process. To maintain rotational invariance, we introduce the l2,0-norm, applying the l2-norm and the l0-norm selectively across features and samples. The proposed formulations are solved using augmented Lagrangians followed by alternating minimization. Extensive experiments on datasets from the UCI Machine Learning Repository demonstrate that our method outperforms previous robust PCA formulations, including R1-PCA, principal component pursuit, and the recently introduced median-of-means-based robust PCA approach.

In the past there have been studies in kernelized versions of sparse recovery algorithms; most notably for l1-norm minimization and orthogonal matching pursuit. However, there has been no work on a kernel version of a related problem – low-rank matrix recovery via nuclear norm minimization. In this paper, we address this hitherto unsolved problem. The ensuing formulation is solved using the Majorisation Minimization (MM) approach. The reason for using MM is to ensure there are no additional hyper-parameters to tune. We apply this technique for Collaborative Representation based Classification (CRC). Experiments have been carried forth on video based face recognition on the VidTIMIT and iSAFE databases. Comparison has been done with other CRC approaches along with two dictionary learning based and two deep learning based techniques. Results show that our method improves over all of the aforesaid.

Antimicrobial resistance (AMR) is a significant global health challenge caused by the misuse and overuse of antibiotics in various sectors, leading to the development of resistant bacteria. In such infections, the first-line antibiotics intended for specific diseases become ineffective, necessitating the repurposing of other antibiotics for treatment. To address this, we have developed a new algorithm for general-purpose drug repositioning based on a matrix completion framework on graphs. Our probabilistic approach combines deep matrix factorization with graph learning to achieve precise drug repurposing. In this study, we curated a new dataset on antibiotic-bacteria associations. Applying our proposed method to this dataset demonstrates that our approach outperforms benchmarks in both general-purpose drug repositioning and three specific AMR case studies.

In this work, we propose a semi-supervised deep feature generation network that accounts for local similarities. It is based on the deep dictionary learning (DDL) framework. The formulation accounts for two unique aspects of hyperspectral classification. First, the fact that the total number of pixels / samples to be labeled is constant; this allows for a semi-supervised formulation allowing only a few pixels / samples to be labeled as training data. Second, the samples / pixels are spatially correlated; this leads to a graph regularization formulation. Our formulation has been benchmarked with state-of-the-art techniques on two popular datasets; the results show that our work improves upon the ones compared against.

MOTIVATION: Dysregulation of a gene's function, either due to mutations or impairments in regulatory networks, often triggers pathological states in the affected tissue. Comprehensive mapping of these apparent gene-pathology relationships is an ever-daunting task, primarily due to genetic pleiotropy and lack of suitable computational approaches. With the advent of high throughput genomics platforms and community scale initiatives such as the Human Cell Landscape project, researchers have been able to create gene expression portraits of healthy tissues resolved at the level of single cells. However, a similar wealth of knowledge is currently not at our finger-tip when it comes to diseases. This is because the genetic manifestation of a disease is often quite diverse and is confounded by several clinical and demographic covariates. RESULTS: To circumvent this, we mined ∼18 million PubMed abstracts published till May 2019 and automatically selected ∼4.5 million of them that describe roles of particular genes in disease pathogenesis. Further, we fine-tuned the pretrained bidirectional encoder representations from transformers (BERT) for language modeling from the domain of natural language processing to learn vector representation of entities such as genes, diseases, tissues, cell-types, etc., in a way such that their relationship is preserved in a vector space. The repurposed BERT predicted disease-gene associations that are not cited in the training data, thereby highlighting the feasibility of in silico synthesis of hypotheses linking different biological entities such as genes and conditions. AVAILABILITY AND IMPLEMENTATION: PathoBERT pretrained model: https://github.com/Priyadarshini-Rai/Pathomap-Model. BioSentVec-based abstract classification model: https://github.com/Priyadarshini-Rai/Pathomap-Model. Pathomap R package: https://github.com/Priyadarshini-Rai/Pathomap.

With the advent of industry 4.0, data-driven techniques have gained a lot of popularity for machine condition monitoring, ensuring reliable and safe operation of the machines. In most practical application scenarios, domain discrepancy may arise between the training (source domain) and test (target domain) data due to various factors like changes in the operating conditions, different sensor locations, etc. Traditional data-driven techniques fail to address this domain shift, and hence domain adaptation techniques are required to ensure reliable performance. This work presents an unsupervised domain adaptation method where labeled data is available only in the source domain via subspace interpolation using deep dictionary learning. Deep dictionaries learn rich representations from the data and hence are used for subspace interpolation to capture the domain shift and form a shared feature space for cross-domain analysis. The proposed method is evaluated for the challenging scenario of adaptation between different but related machines. Experimental results obtained with two publicly available bearing fault datasets are promising; the proposed method significantly outperforms all the state-of-the-art methods.

Co-administration of two or more drugs simultaneously can result in adverse drug reactions. Identifying drug-drug interactions (DDIs) is necessary, especially for drug development and for repurposing old drugs. DDI prediction can be viewed as a matrix completion task, for which matrix factorization (MF) appears as a suitable solution. This paper presents a novel Graph Regularized Probabilistic Matrix Factorization (GRPMF) method, which incorporates expert knowledge through a novel graph-based regularization strategy within an MF framework. An efficient and sounded optimization algorithm is proposed to solve the resulting non-convex problem in an alternating fashion. The performance of the proposed method is evaluated through the DrugBank dataset, and comparisons are provided against state-of-the-art techniques. The results demonstrate the superior performance of GRPMF when compared to its counterparts.

Algorithms for non-negative sparse recovery are either based on modifications of orthogonal matching pursuit or are based on thresholding of non-negative least squares. Both are variants of techniques proposed for sparse recovery. This work is based on the iterative re-weighted least squares (IRLS) approach for sparse recovery. IRLS has been found to be a simple yet versatile approach that can handle both l1-norm and lp-quasi norm (0<p<1). We extend this approach to handle not only sparse recovery but also group-sparse recovery.

This work addresses the problem of hyperspectral image classification when the number of labeled samples is very small (few shot learning). Our work is based on the recently proposed framework of convolutional transform learning. In this work, we propose a semi-supervised version of deep convolutional transform learning. We compare with four recent studies which are tailored for solving the few-shot learning problem in hyperspectral classification. Results show that our method can improve over the state-of-the-art.

Conventional Convolutional Neural Network (CNN) based clustering formulations are based on the encoder-decoder based framework, where the clustering loss is incorporated after the encoder network. The problem with this approach is that it requires training an additional decoder network; this, in turn, means learning additional weights which can lead to over-fitting in data constrained scenarios. This work introduces a Deep Convolutional Transform Learning (DCTL) based clustering framework. The advantage of our proposed formulation is that we do not require learning the additional decoder network. Therefore our formulation is less prone to over-fitting. Comparison with state-of-the-art deep learning based clustering solutions on benchmark image datasets shows that our proposed method improves over the rest in challenging scenarios where there are many clusters with limited samples.

Fault diagnosis plays a vital role in ensuring the normal operation of the machine and safe production. In recent years, data-driven techniques have gained a lot of popularity for machine fault diagnosis. But most of these techniques assume the training and test data have the same distribution. However, in most practical application scenarios, domain discrepancy can be observed between the training (source) and test (target) data due to different factors like changes in the operating conditions, different sensor locations, etc. Classical approaches fail to address such domain discrepancy, which leads to poor performance. The problem becomes more challenging when the target is completely unlabeled. To address this scenario, domain adaptation techniques are used to transfer the knowledge learned from the labeled source domain to the unlabeled target domain. Recently, adversarial network based domain adaptation has been extensively explored for fault diagnosis. But the adversarial loss alone does not guarantee the translation of the source to the desired target domain (class consistent). Here, we propose to use cycle-consistency loss employing 1D-CycleGAN for learning the source to target mapping for unsupervised adaptation for bearing fault diagnosis. The proposed method is evaluated for two different scenarios, with the source and target from (i) same machine but different working conditions and (ii) different but related machines. Experimental results show that while the proposed method performs comparable to the best-performing benchmark for the first case, it significantly outperforms all the state-of-the-art methods for the challenging second case.

Subspace clustering techniques have shown promise in hyperspectral image segmentation. The fundamental assumption in subspace clustering is that the samples belonging to different clusters/segments lie in separable subspaces. What if this condition does not hold? We surmise that even if the condition does not hold in the original space, the data may be nonlinearly transformed to a space where it will be separable into subspaces. In this work, we propose a transformation based on the tenets of deep dictionary learning (DDL). In particular, we incorporate the sparse subspace clustering (SSC) loss in the DDL formulation. Here DDL nonlinearly transforms the data such that the transformed representation (of the data) is separable into subspaces. We show that the proposed formulation improves over the state-of-the-art deep learning techniques in hyperspectral image clustering.

In clustering-based hyperspectral band selection techniques, 2-D images of each band are usually taken as input samples. Some form of feature extraction on these images is performed before they are input to the clustering algorithm. The clustering algorithm returns the cluster centroids; the bands closest to the centroids are selected as representative bands for each cluster. In this work, we propose a joint representation learning and clustering framework. We embed the popular$K$-means clustering loss into the newly developing framework of deep transform learning and solve the ensuing formulation via alternating direction method of multipliers (ADMM). We combine clustering with feature extraction. Application of our proposed solution to the hyperspectral band selection problem shows that we improve over the state of the art by a reasonable margin.

Accelerated population ageing experienced in the last few decades is an unprecedented phenomenon. Currently, this is more in the developing countries. Soon three-fourths of the elderly will be in the developing world. From 1990 to 2025, the elderly population in Asia will rise from 50 per cent of the world's elderly to 58 per cent, in Africa and Latin America from 5 to 7 per cent, but in Europe the figure will drop from 19 to 12 per cent of the world's elderly. The life span has increased in India from 32 yr in 1947 to more than 62 yr now. From the morbidity point of view, almost 50 per cent of the Indian elderly have chronic diseases and 5 per cent suffer from immobility. There are several vulnerable groups and a big disadvantaged lot are elderly females who are one of the fastest growing segments, which will increase to become 4 times the current figure, by 2025. In spite of professional disinterest in the speciality, recent trends indicate the beginning of sensitization of medical teachers, advancing speciality of psychosocial gerontology and availability of some research funds. Importance of training of health professionals and priorities in gerontological research are also under consideration. Infections still take a heavy toll of our elderly population apart from well known degenerative disorders. Limitations of a developing country further influence the morbidity pattern in various ways. Nutritional deficiencies are common and often subclinical thus escaping the desired interventions. Coronary heart disease, hypertension, mental and many other disorders in the elderly have been reported as isolated observations highlighting differences from those made in the Western countries. Socio-economically, the traditional support of extended families is rapidly undergoing erosion making the elderly further vulnerable. This causes more emotional and psychological problems while the State finds itself helpless in providing a comprehensive care to its large chunk of elderly population. It will be important to surmise and predetermine the future factors that are going to modify the diverse patterns of morbidity, disability and mortality in regional context.

Single-cell RNA sequencing has been proved to be advantageous in discerning molecular heterogeneity in seemingly similar cells in a tissue. Due to the paucity of starting RNA, a large fraction of transcripts fail to amplify during the polymerase chain reaction cycle. This gets compounded by trivial biological noise such as variability in the cell cycle specific genes. As a result expression matrix obtained from a single-cell study is highly sparse with a large number of missing values. This hinders downstream analysis of single-cell expression data. It has been observed that feature engineering significantly improves the analysis outcomes. Feature extraction methods such as principal component analysis and zero-inflated factor analysis have been shown to be useful for subsequent steps of data analysis including clustering. However, too little or no visible efforts have been observed for developing feature selection techniques, which offer transparency for the analyst's consumption. We propose SelfE, a novel$l_{2,0}$-minimization algorithm that determines an optimal subset of feature vectors that preserves sub-space structures as observed in the data. We compared SelfE with the commonly used feature selection methods for single-cell expression data analysis.

The objective of this letter is to propose a novel computational method to learn the state of an appliance (ON / OFF) given the aggregate power consumption recorded by the smart-meter. We formulate a multi-label classification problem where the classes correspond to the appliances. The proposed approach is based on our recently introduced framework of convolutional transform learning. We propose a deep supervised version of it relying on an original multi-label cost. Comparisons with state-of-the-art techniques show that our proposed method improves over the benchmarks on popular non-intrusive load monitoring datasets.

Insights from multiple imaging modalities have recently been applied in solving many computer vision related applications. In this paper, we model the cross-modal dependencies between different modalities for Multimodal Image Super-Resolution (MISR), i.e., enhance the Low Resolution (LR) image of target modality with the guidance of a High Resolution (HR) image from another modality. We introduce a joint optimization based transform learning frame-work referred to as Joint Coupled Transform Learning (JCTL) to combine the information from multiple modalities to generate the HR image of the target modality. All the necessary intermediate steps and the corresponding closed form solution updates are pro-vided. The performance of the proposed JCTL is benchmarked against the state-of-the-art MISR approaches on different multi-modal datasets with different upscaling factors. The results show better performance with the proposed JCTL approach compared to other state-of-the-art techniques both in terms of PSNR and SSIM.

The performance of existing methods for multisensor fusion are severely affected by the lack of significant amount of labeled data. In most practical scenarios, the amount of unlabeled data is huge in comparison to labeled data. To address this problem, a novel autoencoder based multi-sensor fusion framework for semi-supervised learning is proposed in this work. Here, both labeled and unlabeled data are used for learning the latent representation from each sensor. Subsequently, the latent representation of all the sensors are combined to perform classification. A joint optimization formulation is presented for learning the sensor-specific latent representation, their encoder and decoder weights and the classification weights together. This ensures discriminative features to be learnt from individual sensors that aids in classification. The requisite solution steps and the closed form updates for the joint learning of all the parameters are given. Experiment results presented on two datasets from different domains demonstrate the generalizability and superior performance of the proposed AutoFuse compared to state-of-the-art methods with relatively less complexity and the ability to work with partially annotated data.

This work proposes a new approach to deep neural network (DNN) compression. We employ black-box function approximation techniques from signal processing to compress. DNN, in general, can approximate non-smooth and piecewise smooth functions. With only this assumption, we model the function that the DNN has learnt as a piecewise linear function. This is a standard function approximation approach. We compared our approach with two state-of-the-art techniques - spatial singular value decomposition and channel pruning with weight reconstruction; and one of state-of-practice tool - OpenVINO. Two well known 1D DNN models for time series classification - ResNet and InceptionTime were compressed. Results show that our model yields better compression at comparable losses in accuracy on majority of the datasets.

Curbing hate speech is undoubtedly a major challenge for online microblogging platforms like Twitter. While there have been studies around hate speech detection, it is not clear how hate speech finds its way into an online discussion. It is important for a content moderator to not only identify which tweet is hateful but also to predict which tweet will be responsible for accumulating hate speech. This would help in prioritizing tweets that need constant monitoring. Our analysis reveals that for hate speech to manifest in an ongoing discussion, the source tweet may not necessarily be hateful; rather, there are plenty of such non-hateful tweets which gradually invoke hateful replies, resulting in the entire reply threads becoming provocative.

This work proposes a new approach for dynamical modeling; we call it sequential transform learning. This is loosely based on the transform (analysis dictionary) learning formulation. This is the first work on this topic. Transform learning, was originally developed for static problems; we modify it to model dynamical systems by introducing a feedback loop. The learnt transform coefficients for the t th instant are fed back along with the t + 1st sample, thereby establishing a Markovian relationship. Furthermore, the formulation is made supervised by the label consistency cost. Our approach keeps the best of two worlds, marrying the interpretability and uncertainty measure of signal processing with the function approximation ability of neural networks. We have carried out experiments on one of the most challenging problems in dynamical modeling - stock forecasting. Benchmarking with the state-of-the-art has shown that our method excels over the rest.

Subspace clustering assumes that the data is separable into separate subspaces. Such a simple assumption, does not always hold. We assume that, even if the raw data is not separable into subspaces, one can learn a representation (transform coefficients) such that the learnt representation is separable into subspaces. To achieve the intended goal, we embed subspace clustering techniques (locally linear manifold clustering, sparse subspace clustering and low rank representation) into transform learning. The entire formulation is jointly learnt; giving rise to a new class of methods called transformed subspace clustering (TSC). In order to account for non-linearity, kernelized extensions of TSC are also proposed. To test the performance of the proposed techniques, benchmarking is performed on image clustering and document clustering datasets. Comparison with state-of-the-art clustering techniques shows that our formulation improves upon them.

Convolutional transform learning is an unsupervised framework we introduced recently, for feature generation based on learnt convolutions. In this work, we propose a supervised formulation for convolutional transform so as to address the multi-label classification problem. Unlike the simple multiclass classification, in multi-label problems, each sample can correspond to multiple classes simultaneously, making the problem quite challenging. We propose to make use of a label consistency penalty and develop a suitable minimization algorithm for the training step. We illustrate the performance of the developed formulation on the practical problem of nonintrusive load monitoring. Comparisons with popular techniques show that our proposed approach yields better results on benchmark datasets.

This work proposes matrix completion via deep matrix factorization on graphs. The work is motivated by the success of two very recent studies on (shallow) matrix completion on graphs and deep matrix factorization (without graphs). We show that the proposed deep matrix factorization on graphs improves over both - shallow techniques on graphs and deep matrix factorization. Experiments are carried out on the challenging real-life problem of modeling drug-target interactions.

Construction of learning model under computational and energy constraints, particularly in highly limited training time requirement is a critical as well as unique necessity of many practical IoT applications that use time series sensor signal analytics for edge devices. Yet, majority of the state-of-the-art algorithms and solutions attempt to achieve high performance objective (like test accuracy) irrespective of the computational constraints of real-life applications. In this paper, we propose Instant Adaptive Learning that characterizes the intrinsic signal processing properties of time series sensor signals using linear adaptive filtering and derivative spectrum to efficiently construct a low-cost learning model followed by standard classification algorithms. Our empirical studies on a number of time series sensor signals from publicly available time series database (UCR) demonstrate that with slight trade-off in performance, the proposed method achieves very fast learning capability.

Load Disaggregation has gained much popularity in the recent times, owing to the advantages it brings to energy utility companies. Many modeling techniques ranging from Dictionary Learning to HMM-based techniques to Neural Network based modeling have been proposed in the literature to solve this problem. However, scalability and computational lightness, have been two main areas of concern associated with the problem modeling. In this work, the authors propose to use Multi-Label Auto-Encoder architecture to solve this problem. The proposed architecture incurs minimum instrumentation cost, which makes it truly non-intrusive. The use of superposed appliance class labels of interest in the discriminative penalizing term of the architecture, ensures that disaggregation is achieved without the need to train a separate model for each appliance class of interest.

In drug target interaction (DTI) the interactions of some (a subset) drugs on some (a subset) targets are known. The goal is to predict the interactions of all drugs on all targets. One approach is to formulate this as a matrix completion problem, where the matrix of interactions having drugs along the rows and targets along the columns is partially filled. So far standard matrix completion approaches such as nuclear norm minimization and matrix factorization have been used to address the problem. In this work, we propose a deep matrix factorization approach to improve the prediction results. Experiments have been performed on benchmark databases and comparison carried out with some state-of-the-art algorithms. Empirically our proposed deep method, outperforms all the techniques compared against.

Increasing population indicates that energy demands need to be managed in the residential sector. Prior studies have reflected that the customers tend to reduce a significant amount of energy consumption if they are provided with appliance-level feedback. This observation has increased the relevance of load monitoring in today's tech-savvy world. Most of the previously proposed solutions claim to perform load monitoring without intrusion, but they are not completely non-intrusive. These methods require historical appliance-level data for training the model for each of the devices. This data is gathered by putting a sensor on each of the appliances present in the home which causes intrusion in the building. Some recent studies have proposed that if we frame Non-Intrusive Load Monitoring (NILM) as a multi-label classification problem, the need for appliance-level data can be avoided. In this paper, we propose Multi-label Restricted Boltzmann Machine(ML-RBM) for NILM and report an experimental evaluation of proposed and state-of-the-art techniques.

This work introduces certain supervised formulations for transform learning. Transform learning is the analysis equivalent of dictionary learning. Four different types of supervision penalties are proposed. The first one is class-sparsity, which imposes common sparse support within representations of each class. The second one imposes similarity among intra-class features in terms of a low-rank constraint (high cosine similarity). The third penalty enforces features of the same class to be nearby each other and features of different classes to be far apart. The final formulation is the well known label-consistency formulation which learns a linear map from the feature space to the class targets. For the first time, we show how transform learning (and its supervised versions can be kernelized). Finally this work also introduces stochastic regularization techniques like DropOut and DropConnect into the transform learning formulation. Experiments have been carried out on two different problems - computer vision and biomedial signal analysis. In both the problems, our method excels over all existing ones.

The paper addresses the problem of estimating age and gender from frontal photos. Most prior studies on deep learning based estimation formulate it as a convolutional neural network based classification problem. For gender it is a two class problem; for age, several age brackets are created to form classes. In this work we formulate it as a regression problem. This is a natural way to handle both gender and age. Gender can be represented as a single variable possible of taking binary values (male or female) whereas age can be represented by a single variable taking non-negative real values. We formulate regression on the newly proposed deep dictionary learning framework. Prior work on this topic, is on unsupervised representation learning; in this work we in-built regression into the deep dictionary learning framework making the formulation supervised. Testing has been done on several state-of-the-art datasets - Adience, MORPH, ChaLearn LAP, LFWA and CelebA. Our method yields age and gender estimation results better than the state-of-the-art.

Researchers have explored the importance of Siamese networks in deep learning. With recent developments in deep learning and the effectiveness of deep dictionary learning, this research proposes the architecture of Siamese Deep Dictionary Learning. We first propose the architecture followed by solving the optimization problem. The experimental effectiveness is demonstrated on five different image databases pertaining to two classification problems: face verification and kinship verification. The experiments show that the proposed Siamese Deep Dictionary Learning yields comparable results compared to state-of-the-art algorithms on all five databases.

This letter proposes a new image analysis tool called label-consistent transform learning. Transform learning is a recent unsupervised representation learning approach; we add supervision by incorporating a label consistency constraint. The proposed technique is especially suited for hyperspectral image classification problems owing to its ability to learn from fewer samples. We have compared our proposed method with the state-of-the-art techniques such as label-consistent K-singular value decomposition, stacked autoencoder, deep belief network, convolutional neural network, and generative adversarial network. Our method yields considerably better results than all the aforesaid techniques.

The term ``blind denoising'' refers to the fact that the basis used for denoising is learned from the noisy sample itself during denoising. Dictionary learning- and transform learning-based formulations for blind denoising are well known. But there has been no autoencoder-based solution for the said blind denoising approach. So far, autoencoder-based denoising formulations have learned the model on a separate training data and have used the learned model to denoise test samples. Such a methodology fails when the test image (to denoise) is not of the same kind as the models learned with. This will be the first work, where we learn the autoencoder from the noisy sample while denoising. Experimental results show that our proposed method performs better than dictionary learning (K-singular value decomposition), transform learning, sparse stacked denoising autoencoder, and the gold standard BM3D algorithm.

The problem of load estimation from sensor signals holds significance in the field of intelligent manufacturing. The goal of this work is to estimate the axial and spindle load values in a Computer Numerical Control machine from input sensor readings like spindle speed, feed rate, tool positions' etc. This can be viewed as a standard regression problem. Here, we propose a novel deep learning based regression technique that incorporates regression within the stacked autoencoder framework. Unlike the popular heuristic pretraining, fine-tuning approach, we solve all the parameters of the problem jointly. A variable splitting Augmented Lagrangian approach is employed to solve the ensuing optimization problem. Comparisons on standard regression models like linear, Least Absolute Shrinkage and Selection Operator (LASSO), Support Vector Regression and the traditional stacked autoencoder have shown that our technique considerably outperforms them.

In this paper, we present a kernelized dictionary learning framework for carrying out regression to model signals having a complex nonlinear nature. A joint optimization is carried out where the regression weights are learnt together with the dictionary and coefficients. Relevant formulation and dictionary building steps are provided. To demonstrate the effectiveness of the proposed technique, elaborate experimental results using different real-life datasets are presented. The results show that non-linear dictionary is more accurate for data modeling and provides significant improvement in estimation accuracy over the other popular traditional techniques especially when the data is highly non-linear.

We introduce deep transform learning - a new tool for deep learning. Deeper representation is learnt by stacking one transform after another. The model is akin to a feedforward neural network. The first layer learns the transform and features from the input training samples. Subsequent layers use the features (after activation) from the previous layers as training input. However, this explanation is only given for intuitive understanding; the ensuing problem is not solved in a greedy fashion. All the layers are solved jointly. Experiments have been carried out with other state-of-the-art. Results on classification and clustering show that our proposed technique is better than all the said techniques, at least on the benchmark datasets compared on.

Classification using cross-datasets (where a classifier trained using annotated image set A is used to test similar images of set B due to lack of training images in B) is important for many classification problems especially in biomedical imaging. We propose a discriminative autoencoder, useful for addressing the challenge of classification using cross-datasets. Our autoencoder learns an encoder and decoder such that the distances between the representations of the same class is minimized whereas the distances between the representations of different classes are maximized. We derive a fast algorithm to solve the aforementioned problem using the Augmented Lagrangian Alternating Directions Method of Multipliers (ADMM) approach. ADMM is a faster alternative to back-propagation which is used in standard autoencoders. The proposed method outperforms state-of-the-art representation learning tools in terms of classification results in breast cancer related histopathological image set MITOS and AMIDA and some of the benchmark image datasets.

This work addresses the problem of reconstructing biomedical signals from their lower dimensional projections. Traditionally Compressed Sensing (CS) based techniques have been employed for this task. These are transductive inversion processes; the problem with these approaches is that the inversion is time-consuming and hence not suitable for real-time applications. With the recent advent of deep learning, Stacked Sparse Denoising Autoencoder (SSDAE) has been used for learning inversion in an inductive setup. The training period for inductive learning is large but is very fast during application - capable of real-time speed. This work proposes a new approach for inductive learning of the inversion process. It is based on Coupled Analysis Dictionary Learning. Results on Biomedical signal reconstruction show that our proposed approach is very fast and yields result far better than CS and SSDAE.

Energy disaggregation is a single channel blind source separation problem where the task is to estimate the consumption of each electrical appliance given the total meter reading. A recent approach to solve this problem is to model the appliances by individual dictionaries. However prior studies based on off-the-shelf dictionary learning techniques do not account for non-linear perturbations in electrical systems. This work models such perturbations as sparse error and applies robust versions of dictionary learning for disaggregation. On top of the basic (unsupervised) robust dictionary learning formulation, we propose two supervised variants. Comparison with state-of-the-art techniques show marked improvement with our proposed methods on benchmark REDD dataset.

Recent studies have experimentally shown that autoencoder based formulations for collaborative filtering can outperform other approaches. However, prior studies in this area, were either based on only ratings information; and at most accounted for either user or item metadata but not both. The metadata was appended to the ratings and just passed on as inputs. This is the most that can be done in the standard neural network based autoencoder formulation. However, collaborative filtering is a highly under-determined problem, therefore being able to utilize maximum information will naturally boost results. Previous autoencoder based formulations either had to leave out user metadata or item metadata. This is the first work, that proposes to modify the autoencoder to account for both user and item metadata. Results using explicit metadata (e.g. age, gender, occupation for users and genre for movies) and implicit metadata (neighborhood information) on our proposed formulation improves upon the state-of -the-art techniques considerably.

In recent years autoencoder based collaborative filtering for recommender systems have shown promise. In the past, several variants of the basic autoencoder based approach has been proposed - marginalized denoising autoencoder and stacked denoising autoencoder. However, these are not new developments; just applications of existing machine learning techniques on collaborative filtering. In this work we propose a fundamentally new architecture of hierarchical autoencoder. In a normal stacked denoising autoencoder, reconstruction only happens at the final layer, the intermediate layers are not directly responsible. In our proposed hierarchical model every layer reconstructs; each layer provides complimentary information. The output from all the layers are fused to yield the final result. Experiments of benchmark collaborative filtering datasets show the superiority of our technique over the state-of-art.

Dictionary learning (DL) is one of the popular sparse coding machine learning techniques. In image processing literature, every input image is represented as the sparse linear combination of basis vectors. DL has been shown to have wide applications for image restoration as well as pattern recognition problems. In DL, the input image is factorized into dictionary and sparse codes. This factorization always leaves a residual or approximation error. Very few works in the literature had focused on to leverage the information present in this residual. In this paper, we use residuals within our framework and show that the restoration performance or accurate prediction of missing label in multi-spectral images can be significantly improved over conventional DL based techniques. We initially show that the higher order frequencies are propagated through residuals. Then we show that incorporating this residual in the image restoration methodology can significantly improve the outcomes. Finally, we propose a technique to solve the problem of missing label prediction by using a restoration based deep residual dictionary learning framework.

This is the first work that introduces supervision into the deep dictionary learning framework and solves it in an optimal fashion. The derivation for solving the ensuing formulation is based on the state-of-the-art optimization paradigm that includes proximal variable splitting, augmented Lagrangians and alternating direction method of multipliers. Our proposed formulation can handle both single label and multi-label classification problems. Experiments have been carried out on benchmark datasets. Comparison has been carried out with both well known and modern techniques. In every case, our proposed solution surpasses others.

This work addresses the problem of extracting deeply learned features directly from compressive measurements. There has been no work in this area, existing deep learning tools only give good results when applied on the full signal (that too usually after pre-processing). These techniques require the signal to be reconstructed first. In this work we show that by learning directly from the compressed domain, considerably better results can be obtained. This work extends the recently proposed framework of deep matrix factorization in combination with blind compressed sensing, hence the term 'deep blind compressed sensing'. Simulation experiments have been carried out on imaging via single pixel camera, under-sampled biomedical signals (arising in wireless body area network) and compressive hyperspectral imaging. In all cases, the superiority of our proposed deep blind compressed sensing can be envisaged.

Currently there are two predominant ways to train deep neural networks. The first one uses restricted Boltzmann machine (RBM) and the second one auto encoders. RBMs are stacked in layers to form deep belief network (DBN), the final representation layer is attached to the target to complete the deep neural network. Auto encoders are nested one inside the other to form stacked auto encoders, once the stcaked auto encoder is learnt the decoder portion is detached and the target attached to the deepest layer of the encoder to form the deep neural network. This work proposes a new approach to train deep neural networks using dictionary learning as the basic building block, the idea is to use the features from the shallower layer as inputs for training the next deeper layer. One can use any type of dictionary learning (unsupervised, supervised, discriminative etc.) as basic units till the pre-final layer. In the final layer one needs to use the label consistent dictionary learning formulation for classification. We compare our proposed framework with existing state-of-the art deep learning techniques on benchmark problems, we are always within the top 10 results. In actual problems of age and gender classification, we are better than the best known techniques.

Dictionary learning follows a synthesis framework; the dictionary is learnt such that the data can be synthesized / re-generated from the coefficients. Transform learning on the other hand is based on analysis formulation; it learns a transform so as to generate the coefficients. The basic formulations of dictionary learning and transform learning employ a Euclidean cost function for the data fidelity term. Such cost functions are optimal when the noise / error in the system is Normally distributed, but not in the presence of sparse but large outliers. For such heavy tailed noise distributions, minimizing the absolute distance is more robust. There are several papers on robust dictionary learning. This work introduces robust transform learning. Experiments carried out on image analysis and impulse denoising elucidate the superiority of our method.

Soft biometric modalities have shown their utility in different applications including reducing the search space significantly. This leads to improved recognition performance, reduced computation time, and faster processing of test samples. Some common soft biometric modalities are ethnicity, gender, age, hair color, iris color, presence of facial hair or moles, and markers. This research focuses on performing ethnicity and gender classification on iris images. We present a novel supervised auto-encoder based approach, Deep Class-Encoder, which uses class labels to learn discriminative representation for the given sample by mapping the learned feature vector to its label. The proposed model is evaluated on two datasets each for ethnicity and gender classification. The results obtained using the proposed Deep Class-Encoder demonstrate its effectiveness in comparison to existing approaches and state-of-the-art methods.

Face sketch to digital image matching is an important challenge of face recognition that involves matching across different domains. Current research efforts have primarily focused on extracting domain invariant representations or learning a mapping from one domain to the other. In this research, we propose a novel transform learning based approach termed as DeepTransformer, which learns a transformation and mapping function between the features of two domains. The proposed formulation is independent of the input information and can be applied with any existing learned or hand-crafted feature. Since the mapping function is directional in nature, we propose two variants of DeepTransformer: (i) semi-coupled and (ii) symmetrically-coupled deep transform learning. This research also uses a novel IIIT-D Composite Sketch with Age (CSA) variations database which contains sketch images of 150 subjects along with age-separated digital photos. The performance of the proposed models is evaluated on a novel application of sketch-to-sketch matching, along with sketch-to-digital photo matching. Experimental results demonstrate the robustness of the proposed models in comparison to existing state-of-the-art sketch matching algorithms and a commercial face recognition system.

In this paper we propose a joint optimization technique for coupled autoencoder which learns the autoencoder weights and coupling map (between source and target) simultaneously. The technique is applicable to any transfer learning problem. In this work, we propose a new formulation that recasts deblurring as a transfer learning problem; it is solved using the proposed coupled autoencoder. The proposed technique can operate on-the-fly; since it does not require solving any costly inverse problem. Experiments have been carried out on state-of-the-art techniques; our method yields better quality images in shorter operating times.

In this work we propose to regularize the encoding and decoding weights of an autoencoder using low-rank penalty in the form of nuclear norm. Such a formulation models redundancy in the network. We show that our proposed method yields better classification accuracy (on an average) and denoising results than other stochastic and deterministic regularization techniques used in deep autoencoders. Our method is also considerably faster compared to these techniques. The experiments have been carried out on benchmark deep learning datasets.

We introduce deep transform learning - a new tool for deep learning. Deeper representation is learnt by stacking one transform after another. The learning proceeds in a greedy way. The first layer learns the transform and features from the input training samples. Subsequent layers use the features (after activation) from the previous layers as training input. Experiments have been carried out with other deep representation learning tools - deep dictionary learning, stacked denoising autoencoder, deep belief network and PCANet (a version of convolutional neural network). Results show that our proposed technique is better than all the said techniques, at least on the benchmark datasets (MNIST, CIFAR-10 and SVHN) compared on.

Classification is an important pattern recognition paradigm with a multitude of applications in popular research problems. Utilizing multiple data representations to improve the accuracy of classification has been explored in literature. However, approaches such as combining classifiers using majority voting and score level fusion do not utilize the underlying structure of the data which is available at the representation stage itself. In this paper, we propose a kernelization based extension to the group sparse representation classifier which can utilize multiple representations of input data to improve classification performance. By using a kernel, these representations are processed in a higher dimensional space where they are more separable, without substantially increasing computational costs. The proposed algorithm selects the ideal kernel to use along with its parameters automatically as part of the training process. We evaluate the proposed algorithm on three challenging biometric problems namely, cross distance face recognition, RGB-D face recognition, and multimodal biometrics to showcase its efficacy. Experimentally, we observe that the proposed algorithm can efficiently combine multiple data representations to further improve classification performance.

This work addresses the problem of cold and warm start arising in recommender systems. Usually a latent factor model based on matrix factorization is used for collaborative filtering (warm start recommender system). Only in recent times, a handful of papers have been published that uses autoencoders for the same task; these studies have shown to yield better results than matrix factorization. This is the first work that proposes a comprehensive autoencoder based formulation to address both the cold and warm start problem. It makes use of both available rating's of users on items as well as associated user and item metadata. The proposed method has been compared with state-of-the-art methods and have shown to supersede them.

A recent work introduced the concept of deep dictionary learning. In deep dictionary learning, the first level proceeds like standard dictionary learning; in sub-sequent layers the (scaled) output coefficients from the previous layer are used as inputs for dictionary learning. This is an unsupervised deep learning approach. The features from the final / deepest layer are employed for subsequent analysis and classification. The seminal paper of stacked denoising autoencoders have shown that robust deep models can be learnt when noisy data is used for training stacked autoencoders instead of clean data. We adopt this idea into the deep dictionary learning framework; instead of using only clean data we augment the training dataset by adding noise; this improves robustness. Experimental evaluation on benchmark deep learning datasets and real world problem of AD classification show that our proposal yields considerable improvement.

Traditional stacked autoencoders have an equal number of encoders and decoders. However, while fine-tuned as a deep neural network the decoder portion is detached and never used. This begs the question: ‘do we need equal number of decoders and encoders’? In this study we explore asymmetric autoencoders — unequal number of encoders and decoders. We specifically address two tasks — 1. Classification capacity as deep neural network and 2. Compressibility of stacked autoencoder. For both the problems, our asymmetric autoencoders have several encoders but a single decoders. We find that such autoencoders are more accurate compared to traditional symmetrically stacked autoencoders for classification accuracy and also yield slightly better results on compression problems.

This work proposes a new deep learning method which we call robust deep dictionary learning RDDL. RDDL is suitable for learning representations from signals corrupted with sparse but large outliers such as artifacts and noise that are more heavy tailed than Gaussian distributions. Such outliers are common in biomedical signals e.g. EEG and ECG. RDDL learns multiple levels of non-linear dictionaries for representing the data. Instead of the standard Euclidean cost function that is usually employed in dictionary learning, we propose a robust l1-norm cost function. In order to achieve sparse representation, an l1-norm is imposed on the learned representation. The `depth' arises from the fact that multiple levels of dictionaries are learnt. The full formulation is solved in a greedy fashion, one layer at a time. To study the extent of usefulness of RDDL, we first benchmark it with two wellknown deep learning tools - the stacked denoising autoencoder and the deep belief network methods; experiments are carried out on benchmark deep learning datasets - MNIST, CIFAR-10 and SVHN. In all cases, our method yields the best results. Then the proposed method is used for learning representations of ECG data (containing arficacts) and for their classification using the MIT-BIH arrhythmia classification database. We compare it with traditional techniques as well as on deep learning tools. Our method yields the best results.

In this work we propose a new framework for combined feature extraction and classification. The base idea stems from the sparse representation based classification; where in the training samples from each class are assumed to form a basis for representing the same. Later studies learned a basis for each class using dictionary learning; these were shallow techniques where only one level of dictionary was learnt. In this work we propose to learn multiple levels of dictionaries for each class. We test our technique on benchmark deep learning datasets. We compare our proposed method with deep (stacked autoencoder, deep belief network) techniques and shallow (support vector machine and label consistent dictionary learning) techniques; ours yield the best results overall. We also carry out an empirical analysis with perturbations. We find that our method is more robust compared to other deep learning techniques in the presence of different kinds of noise, missing features and varying amounts of training data.

Autoencoders are deep learning architectures that learn feature representation by minimizing the reconstruction error. Using an autoencoder as baseline, this paper presents a novel formulation for a class sparsity based supervised encoder, termed as CSSE. We postulate that features from the same class will have a common sparsity pattern/support in the latent space. Therefore, in the formulation of the autoencoder, a supervision penalty is introduced as a jointsparsity promoting l2;1-norm. The formulation of CSSE is derived for a single hidden layer and it is applied for multiple hidden layers using a greedy layer-bylayer learning approach. The proposed CSSE approach is applied for learning face representation and verification experiments are performed on the LFW and PaSC face databases. The experiments show that the proposed approach yields improved results compared to autoencoders and comparable results with state-ofthe-art face recognition algorithms.

This paper proposes a new framework for deep learning that has been particularly tailored for hyperspectral image classification. We learn multiple levels of dictionaries in a robust fashion. The last layer is discriminative that learns a linear classifier. The training proceeds greedily; at a time, a single level of dictionary is learned and the coefficients used to train the next level. The coefficients from the final level are used for classification. Robustness is incorporated by minimizing the absolute deviations instead of the more popular Euclidean norm. The inbuilt robustness helps combat mixed noise (Gaussian and sparse) present in hyperspectral images. Results show that our proposed techniques outperform all other deep learning methods-deep belief network, stacked autoencoder, and convolutional neural network. The experiments have been carried out on both benchmark deep learning data sets (MNIST, CIFAR-10, and Street View House Numbers) as well as on real hyperspectral imaging data sets.

In movies, film stars portray another identity or obfuscate their identity with the help of silicone/latex masks. Such realistic masks are now easily available and are used for entertainment purposes. However, their usage in criminal activities to deceive law enforcement and automatic face recognition systems is also plausible. Therefore, it is important to guard biometrics systems against such realistic presentation attacks. This paper introduces the first-of-its-kind silicone mask attack database which contains 130 real and attacked videos to facilitate research in developing presentation attack detection algorithms for this challenging scenario. Along with silicone mask, there are several other presentation attack instruments that are explored in literature. The next contribution of this research is a novel multilevel deep dictionary learning-based presentation attack detection algorithm that can discern different kinds of attacks. An efficient greedy layer by layer training approach is formulated to learn the deep dictionaries followed by SVM to classify an input sample as genuine or attacked. Experimental are performed on the proposed SMAD database, some samples with real world silicone mask attacks, and four existing presentation attack databases, namely, replay-attack, CASIA-FASD, 3DMAD, and UVAD. The results show that the proposed algorithm yields better performance compared with state-ofthe-art algorithms, in both intra-database and cross-database experiments.

Kinship verification has a number of applications such as organizing large collections of images and recognizing resemblances among humans. In this paper, first, a human study is conducted to understand the capabilities of human mind and to identify the discriminatory areas of a face that facilitate kinship-cues. The visual stimuli presented to the participants determine their ability to recognize kin relationship using the whole face as well as specific facial regions. The effect of participant gender and age and kin-relation pair of the stimulus is analyzed using quantitative measures such as accuracy, discriminability index d' , and perceptual information entropy. Utilizing the information obtained from the human study, a hierarchical kinship verification via representation learning (KVRL) framework is utilized to learn the representation of different face regions in an unsupervised manner. We propose a novel approach for feature representation termed as filtered contractive deep belief networks (fcDBN). The proposed feature representation encodes relational information present in images using filters and contractive regularization penalty. A compact representation of facial images of kin is extracted as an output from the learned model and a multi-layer neural network is utilized to verify the kin accurately. A new WVU kinship database is created, which consists of multiple images per subject to facilitate kinship verification. The results show that the proposed deep learning framework (KVRL-fcDBN) yields the state-of-the-art kinship verification accuracy on the WVU kinship database and on four existing benchmark data sets. Furthermore, kinship information is used as a soft biometric modality to boost the performance of face verification via product of likelihood ratio and support vector machine based approaches. Using the proposed KVRL-fcDBN framework, an improvement of over 20% is observed in the performance of face verification.

This work addresses the problem of estimating proton density and T1 maps from two partially sampled K-space scans such that the total acquisition time remains approximately the same as a single scan. Existing multi-parametric non-linear curve fitting techniques require a large number (8 or more) of echoes to estimate the maps - resulting in prolonged (clinically infeasible) acquisition times. Our simulation results show that our method yields very accurate and robust results from only two partially sampled scans (total scan time being the same as a single echo MRI). We model PD and T1 maps to be sparse in some transform domain. The PD map is recovered via standard Compressed Sensing based recovery technique. Estimating the T1 map requires solving an analysis prior sparse recovery problem from non-linear measurements, since the relationship between T1 values and intensity values / K-space samples is not linear. For the first time in this work, we propose an algorithm for analysis prior sparse recovery for non-linear measurements. We have compared our approach with the only existing technique based on matrix factorization from non-linear measurements, our method yields considerably superior results.

In this work we address the problem of recovering sparse solutions to non-linear inverse problems. We look at two variants of the basic problem - the synthesis prior problem when the solution is sparse and the analysis prior problem where the solution is co-sparse in some linear basis. For the first problem, we propose non-linear variants of the Orthogonal Matching Pursuit (OMP) and CoSamp algorithms, for the second problem we propose a non-linear variant of the Greedy Analysis Pursuit (GAP) algorithm. We empirically test the success rates of our algorithms on exponential and logarithmic functions.

This paper proposes a sparse regression based approach for accurate continuous Blood Pressure (BP) monitoring. ECG and Finger PPG signals serve as the input; from which 32 parameters are extracted. Not all parameters are indicative of BP; to automatically trim the redundant parameters a sparse regression based approach is proposed. To build the BP predicting model the necessary parameters and their corresponding weights are learned using data from 99 subjects. The learned model is applied on 10 test subjects. The ground truth BP is measured using a clinically proven, professional automatic digital BP monitor OMRON HBP1300., The BP prediction results show that the SBP/DBP mean absolute error and error standard deviation, with OMRON monitor as a reference, is 4.43/2.46 and 4.90/3.31 mmHg respectively, which falls under the standard allowable error mentioned by Association for the Advancement of Medical Instrumentation for estimation of BP. We have compared our work with other BP prediction techniques (Linear Regression and Feed Forward Neural Network) and have seen that our proposed method yields considerably better results, especially for diastolic BP.

It is well known that the Euclidean norm is sensitive to outliers; yet it is widely used for minimizing it is easy. Dictionary learning is no exception - the l2-norm allows for easy update of the basis/dictionary atoms. In this work, we propose a robust dictionary learning method that is based on minimizing the robust l1-norm. The ensuing optimization is solved using the Split Bregman approach. We apply the proposed technique to signal (energy and water) disaggregation and show that it excels over existing dictionary learning techniques (based on l2-norm).

Face recognition under uncontrolled environment persists to be an unresolved problem having challenges such as varying pose, illumination, occlusion etc. In this research, we propose an algorithm for identification of faces with pose and illumination variations. An adaptive dictionary learning framework built upon group sparse representation classifier is presented in order to learn dictionary parameters and pose invariant sparse codes for given images. Low rank regularization is utilized for dictionary learning, to address the noise present in training samples that can hinder the discriminative power of the learnt dictionary. Experimental results illustrate state-of-the-art performance on the CMU Multi-PIE dataset.

Compressed sensing has found several applications in hyperspectral imaging because it helps in reducing the size of data to be captured or to be transmitted to ground stations. This work is based on the reconstruction of a hyperspectral image from compressive measurements. There are various hardware models proposed in the literature for compressed sensing of hyperspectral images. This work considers the reconstruction of full hyperspectral images from its compressive measurements when they are contaminated by mixed noise. A generic mixed noise model has been considered that explicitly accounts for the presence of both Gaussian and impulse noise. The quantitative and qualitative experimental results with synthetic and real image demonstrate the capabilities of proposed method.

The nearest subspace classifier (NSC) assumes that the samples of every class lie on a separate subspace and it is possible to classify a test sample by computing the distance between the test sample and the subspaces. The sparse representation based classification (SRC) generalizes the NSC - it assumes that the samples of any class can lie on a union of subspaces. By calculating the distance between the test sample and these subspaces, one can classify the test sample. Both NSC and SRC hinge on the assumption that the distance between the test sample and correct subspace will be small and approximately Normally distributed. Based on this assumption, these studies proposed using an l2-norm measure. It is well known that l2-norm is sensitive to outliers (large deviations at few locations). In order to make the NSC and SRC robust and improve their performance we propose to employ the l1-norm based distance measure. Experiments on benchmark classification problems, face recognition and character recognition show that the proposed method indeed improves upon the basic versions of NSC and SRC; in fact our proposed robust NSC and robust SRC yield even better results than support vector machine and neural network.

This work proposes to learn autoencoders with sparse connections. Prior studies on autoencoders enforced sparsity on the neuronal activity; these are different from our proposed approach - we learn sparse connections. Sparsity in connections helps in learning (and keeping) the important relations while trimming the irrelevant ones. We have tested the performance of our proposed method on two tasks - classification and denoising. For classification we have compared against stacked autneencoders, contractive autoencoders, deep belief network, sparse deep neural network and optimal brain damage neural network; the denoising performance was compared against denoising autoencoder and sparse (activity) autoencoder. In both the tasks our proposed method yields superior results.

In this work we propose a classification framework called class-wise deep dictionary learning (CWDDL). For each class, multiple levels of dictionaries are learnt using features from the previous level as inputs (for first level the input is the raw training sample). It is assumed that the cascaded dictionaries form a basis for expressing test samples for that class. Based on this assumption sparse representation based classification is employed. Benchmarking experiments have been carried out on some deep learning datasets (MNIST and its variations, CIFAR and SVHN); our proposed method has been compared with Deep Belief Network (DBN), Stacked Autoencoder, Convolutional Neural Net (CNN) and Label Consistent KSVD (dictionary learning). We find that our proposed method yields better results than these techniques and requires much smaller run-times. The technique is applied for Brain Computer Interface (BCI) classification problems using EEG signals. For this problem our method performs significantly better than Convolutional Deep Belief Network(CDBN).

To elongate the battery life of sensors worn in wireless body area networks, recent studies have advocated compressing the acquired biological signals before transmitting them. The signals are compressed using compressive sensing (CS), by projecting them onto a lower dimension. The original signals are then recovered using CS recovery techniques at the base station, where the computational power is assumed to be abundant. This assumption however is not entirely true when a mobile phone acts as the base station. The computational capacity of a mobile phone is limited; therefore solving the CS recovery problem in the phone would be time consuming. In many cases (e,g. heart stroke detection or monitoring applications) this latency cannot be tolerated. In this work we propose a new technique to solve the inverse problem using stacked autoencoders. We show that the reconstruction of the proposed method can be done in real-time, and there is only a slight degradation in accuracy compared to CS based inversion methods.

This letter introduces a hyperspectral denoising algorithm based on spatio-spectral total variation. The denoising problem has been formulated as a mixed noise reduction problem. A general noise model has been considered which accounts for not only Gaussian noise but also sparse noise. The inherent structure of hyperspectral images has been exploited by utilizing 2-D total variation along the spatial dimension and 1-D total variation along the spectral dimension. The denoising problem has been formulated as an optimization problem whose solution has been derived using the split-Bregman approach. Experimental results demonstrate that the proposed algorithm is able to reduce a significant amount of noise from real noisy hyperspectral images. The proposed algorithm has been compared with existing state-of-the-art approaches. The quantitative and qualitative results demonstrate the superiority of the proposed algorithm in terms of peak signal-to-noise ratio, structural similarity, and the visual quality.

This paper addresses the problem of impulse denoising from hyper-spectral images. Impulse noise is sparse; removing impulse noise requires minimizing an l1-norm data fidelity term. Prior studies have exploited the intra-band spatial correlation (leading to sparsity in transform domain) and inter-band spectral-correlation (joint-sparsity) of hyper-spectral images for Gaussian denoising. In this work, we propose to learn the joint-sparsity promoting dictionary adaptively from the data for impulse denoising problems. Unlike dictionary learning techniques, the sparsifying dictionary is not learnt in an offline training phase. We follow the Blind Compressed Sensing (BCS) framework - dictionary learning and denoising proceeds simultaneously. The optimization problem that arises out of our formulation is solved using the Split Bregman approach. The proposed algorithm, when compared against prior techniques (on real hyper-spectral datasets) shows more than 5dB improvement in PSNR on average.

In Wireless Body Area Networks (WBAN) the energy consumption is dominated by sensing and communication. Previous techniques exploited the sparsity of the signal (in transform domains) to reduce communication costs for EEG transmission. For the first time, in this work, we propose to jointly exploit sparsity and rank-deficiency of the multi-channel signal ensemble in order to reduce both sensing and communication power consumptions. We test our method with state-of-the-art recovery techniques and find that the reconstruction accuracy from our method is considerably better and that too at lower energy consumption.

Modeling a temporal image sequence as a super-position of sparse and low-rank component stems from studies in principal component pursuit (PCP). Recently this technique was applied for dynamic MRI reconstruction with two modifications. First, unlike the original PCP, the problem was to recover the image sequence from under-sampled measurements. Second, the sparse component of the signal was not sparse in itself but in a transform domain. Recent studies in dynamic MRI reconstruction showed that, instead of using a fixed sparsity basis, better recovery results can be achieved when the sparsifying dictionary is adaptively learned from the data using Blind Compressed Sensing (BCS) framework. In this work, we demonstrate that learning the sparsity basis using BCS like techniques improve the recovery accuracy from PCP when applied to dynamic MRI reconstruction problems.

The goal of this work is to improve dictionary learning techniques for classification. We primarily focus on the label consistent K-SVD technique. We improve the consistency between the linear classification and class labels by introducing a sigmoid function. The second improvement is in replacing the Euclidean norm for the consistency constraint by a robust lp-norm (0<;p<;1); this makes the inconsistency robust to changes in magnitude. We compare our proposed modifications with existing work on label consistent KSVD and Sparse Classifier on the Extended YaleB and the AR face databases. Our proposed formulations show considerable improvement in accuracy compared to the baselines.

This work addresses the problem of recovering dynamic MR sequences from their undersampled projections using the Blind Compressed Sensing (BCS) framework. In BCS, reconstructing the sparse signal and estimating the sparsifying basis proceeds simultaneously. Best results in CS based dynamic MRI reconstruction have been achieved when both spatial correlation and temporal redundancies were exploited. Prior studies in BCS dynamic MRI reconstruction either accounted for spatial redundancy or temporal correlation, but not both. In this work, we propose to jointly exploit spatio-temporal correlation within the BCS framework. The improvement in reconstruction results is significant compared to prior BCS techniques in dynamic MRI.

Hyperspectral image denoising is an important preprocessing step in the analysis of hyperspectral images in several applicaitons domains. These images often gets corrupted by various kinds of noise during acquisition process. There are several studies on reducing Gaussian noise from hyperspectral images. This work addresses the problem of reducing mixed noise from hyperspectral images; in particular a mixture of Gaussian and impulse noise has been considered. The proposed image acquisition model explicitly accounts for both Gaussian and impulse noise as additive noise. This mixed noise reduction problem has been formulated as synthesis prior optimization problem which exploits inherent spatio-spectral correlation present in hyperspectral images. Split-Bregman based approach has been utilized to solve resulting optimization problem. Experiements were conducted using both synthetic noise as well as real noisy hyperspectral images. Experimental results have been quantified using peak signal to noise ratio (PSNR) and structural similarity index (SSIM). A comparative study with an existing low-rank based image denoising approaches has also been carried out. Both quantitative and qualitative results suggest the superiority of proposed approach.

Compressive hyperspectral imaging is an inverse problem of reconstructing a hyperspectral image from its low-dimensional measurements. Single-pixel architecture has been extended in various studies to acquire compressive hyperspectral images. Compressed sensing requires prior knowledge about sparsifying transform domain in which signal has sparse representation. This work utilizes Blind Compressed Sensing (BCS) framework which does not require any fixed sparsifying transform to sparsify hyperspectral image. The proposed method outperforms the existing Kronecker compressed sensing based method.

This work proposes the use of an elastic-net to reconstruct Magnetic Resonance Images from their partially sampled K-space. The resulting elastic-net formulation of this problem is composed of two terms - the first term promotes sparsity and the other one promotes a grouping effect. The advantage of using an elastic-net for MRI reconstruction is that it can recover the hierarchically correlated sparse wavelet coefficients of the image. We develop two reconstruction methods via two elastic-net formulations - the synthesis prior and the analysis prior. We also impose non-convex sparsity penalties. There are no existing algorithms that solve such problems; hence we derive efficient algorithms for solving them. The experimental results show that our proposed analysis prior method outperforms state-of-the-art in MRI reconstruction.

In this work we derive algorithms for solving two problems - the first one is the combined l1-norm (sparsity) and nuclear norm (low rank) regularized least squares problem and the second one is the l2, 1-norm (joint sparsity) and nuclear norm regularized least squares problem. There are no efficient general purpose solvers for these problems; our work plugs this gap by deriving Split Bregman based algorithms for solving the said problems. Both algorithms are applicable for recovering hyperspectral images from their compressive measurements obtained via the single pixel camera. We show that our proposed techniques significantly outperform previous methods in terms of recovery accuracy.

The Kaczmarz algorithm is popular for iteratively solving an over determined system of linear equations. Randomized version of the Kaczmarz algorithm can converge exponentially and independent of number of equations. Recently an algorithm for finding sparse solution to a linear system of equations has been proposed based on weighted randomized Kaczmarz algorithm. These algorithms solves single measurement vector problem, however there are applications where multiple-measurements are available. In this work, the objective is to solve a multiple measurement vector problem with common sparse support by modifying the sparse randomized Kaczmarz algorithm. We have also modeled the problem of face recognition from video as the multiple measurement vector problem and solved using our proposed technique. We have compared the proposed algorithm with state-of-art spectral projected gradient algorithm for multiple measurement vectors on both real and synthetic datasets. The Monte Carlo simulations confirms that our proposed algorithm has better recovery and convergence rate than the MMV version of spectral projected gradient algorithm under fairness constraints.

In this paper we address the problem of recovering a matrix, with inherent low rank structure, from its lower dimensional projections. This problem is frequently encountered in wide range of areas including pattern recognition, wireless sensor networks, control systems, recommender systems, image/video reconstruction etc. Both in theory and practice, the most optimal way to solve the low rank matrix recovery problem is via nuclear norm minimization. In this paper, we propose a Split Bregman algorithm for nuclear norm minimization. The use of Bregman technique improves the convergence speed of our algorithm and gives a higher success rate. Also, the accuracy of reconstruction is much better even for cases where small number of linear measurements are available. Our claim is supported by empirical results obtained using our algorithm and its comparison to other existing methods for matrix recovery. The algorithms are compared on the basis of NMSE, execution time and success rate for varying ranks and sampling ratios.

Multi-spectral images capture more information about a scene as compared to RGB images and have various scientific applications. But the high resolution multi-spectral cameras are very expensive which limits their wide applicability as compared to normal digital RGB cameras. In this paper a multi-spectral filter array design is proposed to capture multiple bands using the single-sensor architecture. The use of single-sensor can help in reducing the cost and size of multi-spectral cameras while simultaneously eliminating the image registration problem. Fast linear demosaicing technique is also proposed to interpolate missing values from under-sampled raw image. Experimental results show the superiority of proposed technique over state of art multi-spectral demosaicing technique.

This work addresses the problem of dynamic MRI reconstruction from partially sampled K-space. When the frames of the dynamic MRI sequences are stacked as columns of a matrix, the resultant matrix is both sparse (in a transform domain) and rank-deficient. The dynamic MRI sequence is reconstructed by solving an optimization problem that minimizes a sum of sparsity and rank-deficiency penalties subject to data constraints (K-space data acquisition model). In this work, we propose a non-convex optimization problem for dynamic MRI reconstruction where the sparsity penalty is an lp-norm and the rank-deficiency penalty is the Schatten-q norm (0p-norm and Schatten-q norm minimization problem; hence we derive a new algorithm based on the Majorization Minimization method. Our proposed method shows considerable improvement in reconstruction results over state-of-the-art techniques in dynamic MRI reconstruction.

In this paper we propose a new approach to video based face recognition. Our work is based on the Sparse Classification approach which assumes that each test sample can be formed by a linear combination of the training samples of the correct class. Based on this assumption, we formulate the classification problem as one of joint sparse recovery of Multiple Measurement Vectors (MMV). This requires solving an NP hard problem. This problem has not been solved earlier; thus we derive an algorithm for solving it. The experimental evaluation is carried on the VidTIMIT database. The proposed method is compared against an HMM based method for video based face recognition and the modified Sparse Classification method. The results show that the proposed method outperforms both these methods.

This paper proposes a Majorization-Minimization approach for solving the synthesis and analysis prior joint-sparse multiple measurement vector reconstruction problem. The proposed synthesis prior algorithm yielded the same results as the Spectral Projected Gradient (SPG) method. The analysis prior algorithm is the first to be proposed for this problem. It yielded considerably better results than the proposed synthesis prior algorithm. For problems of a given size, the run times for our proposed algorithms are fixed; unlike SPG where the reconstruction time also depends on the support size of the vectors.

In this work, we address the problem of low-rank matrix recovery from its under-sampled projections. The recovery is formulated as a Schatten-p norm minimization problem. We proposed a novel algorithm to solve the Schatten-p norm minimization problem based on the FOCUSS (FOCally Under-determined System Solver) approach. We compared our proposed method with state-of-the-art solvers. Experimental evaluation was carried out on two problems - matrix completion and image inpainting. For matrix completion, our proposed method showed better recovery rate than other methods. In the image inpainting problem, our method yields 1.5 dB improvement over the nearest competing algorithm.

This work addresses the problem of real-time reconstruction of Dynamic Contrast Enhanced Magnetic Resonance Images (DCE MRI) from partially sampled K-space. The difference image between the current and previous frame is modeled as a rank deficient matrix. It is reconstructed from via nonconvex Schatten-p norm minimization. We develop a FOCUSS (FOCally Under-determined System Solver) based algorithm for solving the said minimization problem. The proposed technique has been compared with state-of-the-art online and offline reconstruction techniques. It yields slightly worse reconstruction results than the offline technique but considerably superior results, both in terms of accuracy and speed compared to the online technique.

This work addresses the problem of real-time online reconstruction of dynamic magnetic resonance imaging sequences. The proposed method reconstructs the difference between the previous and the current image frames. This difference image is sparse. We recover the sparse difference image from its partial k-space scans by using a nonconvex compressed sensing algorithm. As there was no previous fast enough algorithm for real-time reconstruction, we derive a novel algorithm for this purpose. Our proposed method has been compared against state-of-the-art offline and online reconstruction methods. The accuracy of the proposed method is less than offline methods but noticeably higher than the online techniques. For real-time reconstruction we are also concerned about the reconstruction speed. Our method is capable of reconstructing 128 × 128 images at the rate of 6 frames/s, 180 × 180 images at the rate of 5 frames/s and 256 × 256 images at the rate of 2.5 frames/s.

The main challenge faced by wireless sensor networks today is the problem of power consumption at the sensor nodes. Over time, researchers have developed different strategies to address this issue. Such strategies are strongly model dependent and/or application specific. In this work, we take a fresh look at the problem of power consumption in wireless sensor networks from a signal processing perspective. The main idea is simple. Sample only a subset of all the sensor nodes at a given instant and transmit them (this reduces both sampling and communication cost for all the nodes combined). At the central unit (sink) use smart mathematical tools (matrix completion algorithms) to estimate the data for the entire network. We have showed that, if about 1% reconstruction error is allowed, only 20% of the sensors need to sample and transmit at a given instant. This means on an average the life of the network is increased 5-fold. If more error reconstruction error is allowed, even lesser number of sensors need to be active at a given instant leading to more prolonged life of the network.

This work investigates a group-sparse solution to the under-determined system of linear equations b=Ax where the unknown x is formed of a group of vectors xi's. A group-sparse solution has only a few xivectors as non-zeroes while the rest are zeroes. To seek a group-sparse solution generally a convex optimization problem is solved. Such an optimization criterion is unsuitable when the system is highly under-determined or when some of the vector xi's are themselves sparse. For such cases, we propose an alternate non-convex optimization problem. Simulation results show that the proposed method yields significantly improved results (2 orders of magnitude) over the standard method. We also apply the proposed group-sparse optimization in a novel fashion to the problem of color imaging. The new method shows an improvement of more than 1dB over the standard method.

Compressed sensing (CS) of color images can be formulated as a group-sparsity promoting inverse problem. In the past, group-sparsity constraint was imposed on the CS synthesis prior formulation with an orthogonal transform to solve the inverse problem. The objective of this work is to empirically show that better results can be obtained if a group-sparsity constraint is imposed on the CS analysis prior formulation with a redundant transform. This problem requires solving a group-sparsity promoting inverse problem which has not been addressed earlier. Therefore we derive a new algorithm for solving it based on the Majorization-Minimization approach. Experimental results corroborate that analysis prior with a redundant transform gives far superior (about 1.5dB) improvement compared to synthesis prior with orthogonal transform.

The computational cost for most classification algorithms is dependent on the dimensionality of the input samples. As the dimensionality could be high in many cases, particularly those associated with image classification, reducing the dimensionality of the data becomes a necessity. The traditional dimensionality reduction methods are data dependent, which poses certain practical problems. Random projection (RP) is an alternative dimensionality reduction method that is data independent and bypasses these problems. The nearest neighbor classifier has been used with the RP method in classification problems. To obtain higher recognition accuracy, this study looks at the robustness of RP dimensionality reduction for several recently proposed classifiers--sparse classifier (SC), group SC (along with their fast versions), and the nearest subspace classifier. Theoretical proofs are offered regarding the robustness of these classifiers to RP. The theoretical results are confirmed by experimental evaluations.

Recently a new classification assumption was proposed in [1]. It assumed that the training samples of a particular class approximately form a linear basis for any test sample belonging to that class. The classification algorithm in [1] was based on the idea that all the correlated training samples belonging to the correct class are used to represent the test sample. The Lasso regularization was proposed to select the representative training samples from the entire training set (consisting of all the training samples). Lasso however tends to select a single sample from a group of correlated training samples and thus does not promote the representation of the test sample in terms of all the training samples from the correct group. To overcome this problem, we propose two alternate regularization methods, elastic net and sum-over-l2-norm. Both these regularization methods favor the selection of multiple correlated training samples to represent the test sample. Experimental results on benchmark datasets show that our regularization methods give better recognition results compared to [1].

The problem of recognizing a face from a single sample available in a stored dataset is addressed. A new method of tackling this problem by using the Fisherface method on a generic dataset is explored. The recognition scheme is also extended to multiscale transform domains like wavelet, curvelet and contourlet. The proposed method in the transform domain shows better recognition errors than the SPCA algorithm and Eigenface selection method, both of which are specially tailored for recognizing faces from single samples.

This paper addresses the problem of identifying faces when the training face database consists of one face image of each person. It proposes a new approach that synthesizes new face samples of varying degrees of edge information; the synthesized images are generated from the original image and form non-linear approximations of the latter. The approximation is framed as anl1minimization problem in a transform domain. The paper also shows that a voting based approach to recognize faces from single available samples yields better results than previous works that only augmented the available database. The proposed approach yields considerably better results (about 6% increase in recognition accuracy) than the SPCA method, which was tailored for addressing this problem.

This paper deals with automatic recognition of offline handwritten Bangla characters. Bangla is the second most popular script among SAARC countries. A new class of features based on Curvelet transform has been used in our classification scheme. The classifier used was SVM with one-against-rest class model. The training and test set were morphologically deformed to get five versions of the same character and each version has been subject to individual SVM classifier. Five classifier outputs obtained in this way have been combined by simple majority voting scheme. The overall recognition accuracy of 95.5% has been obtained on the data set. It is hoped that the Curvelet transform along with such multi-classifier scheme will be useful in other handwritten character data as well.