Subspace clustering assumes that the data is separable into separate subspaces. Such a simple assumption, does not always hold. We assume that, even if the raw data is not separable into subspaces, one can learn a representation (transform coefficients) such that the learnt representation is separable into subspaces. To achieve the intended goal, we embed subspace clustering techniques (locally linear manifold clustering, sparse subspace clustering and low rank representation) into transform learning. The entire formulation is jointly learnt; giving rise to a new class of methods called transformed subspace clustering (TSC). In order to account for non-linearity, kernelized extensions of TSC are also proposed. To test the performance of the proposed techniques, benchmarking is performed on image clustering and document clustering datasets. Comparison with state-of-the-art clustering techniques shows that our formulation improves upon them.

Convolutional transform learning is an unsupervised framework we introduced recently, for feature generation based on learnt convolutions. In this work, we propose a supervised formulation for convolutional transform so as to address the multi-label classification problem. Unlike the simple multiclass classification, in multi-label problems, each sample can correspond to multiple classes simultaneously, making the problem quite challenging. We propose to make use of a label consistency penalty and develop a suitable minimization algorithm for the training step. We illustrate the performance of the developed formulation on the practical problem of nonintrusive load monitoring. Comparisons with popular techniques show that our proposed approach yields better results on benchmark datasets.

This work introduces certain supervised formulations for transform learning. Transform learning is the analysis equivalent of dictionary learning. Four different types of supervision penalties are proposed. The first one is class-sparsity, which imposes common sparse support within representations of each class. The second one imposes similarity among intra-class features in terms of a low-rank constraint (high cosine similarity). The third penalty enforces features of the same class to be nearby each other and features of different classes to be far apart. The final formulation is the well known label-consistency formulation which learns a linear map from the feature space to the class targets. For the first time, we show how transform learning (and its supervised versions can be kernelized). Finally this work also introduces stochastic regularization techniques like DropOut and DropConnect into the transform learning formulation. Experiments have been carried out on two different problems - computer vision and biomedial signal analysis. In both the problems, our method excels over all existing ones.

This letter proposes a new image analysis tool called label-consistent transform learning. Transform learning is a recent unsupervised representation learning approach; we add supervision by incorporating a label consistency constraint. The proposed technique is especially suited for hyperspectral image classification problems owing to its ability to learn from fewer samples. We have compared our proposed method with the state-of-the-art techniques such as label-consistent K-singular value decomposition, stacked autoencoder, deep belief network, convolutional neural network, and generative adversarial network. Our method yields considerably better results than all the aforesaid techniques.

We introduce deep transform learning - a new tool for deep learning. Deeper representation is learnt by stacking one transform after another. The model is akin to a feedforward neural network. The first layer learns the transform and features from the input training samples. Subsequent layers use the features (after activation) from the previous layers as training input. However, this explanation is only given for intuitive understanding; the ensuing problem is not solved in a greedy fashion. All the layers are solved jointly. Experiments have been carried out with other state-of-the-art. Results on classification and clustering show that our proposed technique is better than all the said techniques, at least on the benchmark datasets compared on.

Dictionary learning follows a synthesis framework; the dictionary is learnt such that the data can be synthesized / re-generated from the coefficients. Transform learning on the other hand is based on analysis formulation; it learns a transform so as to generate the coefficients. The basic formulations of dictionary learning and transform learning employ a Euclidean cost function for the data fidelity term. Such cost functions are optimal when the noise / error in the system is Normally distributed, but not in the presence of sparse but large outliers. For such heavy tailed noise distributions, minimizing the absolute distance is more robust. There are several papers on robust dictionary learning. This work introduces robust transform learning. Experiments carried out on image analysis and impulse denoising elucidate the superiority of our method.

We introduce deep transform learning - a new tool for deep learning. Deeper representation is learnt by stacking one transform after another. The learning proceeds in a greedy way. The first layer learns the transform and features from the input training samples. Subsequent layers use the features (after activation) from the previous layers as training input. Experiments have been carried out with other deep representation learning tools - deep dictionary learning, stacked denoising autoencoder, deep belief network and PCANet (a version of convolutional neural network). Results show that our proposed technique is better than all the said techniques, at least on the benchmark datasets (MNIST, CIFAR-10 and SVHN) compared on.