We present a simple approach to make pre-trained Vision Transformers (ViTs) interpretable for fine-grained analysis, aiming to identify and localize the traits that distinguish visually similar categories, such as bird species. Pretrained ViTs, such as DINO, have demonstrated remarkable capabilities in extracting localized, discriminative features. However, saliency maps like Grad-CAM often fail to identify these traits, producing blurred, coarse heatmaps that highlight entire objects instead. We propose a novel approach, Prompt Class Attention Map (Prompt-CAM), to address this limitation. Prompt-CAM learns class-specific prompts for a pre-trained ViT and uses the corresponding outputs for classification. To correctly classify an image, the true-class prompt must attend to unique image patches not present in other classes’ images (i.e., traits). As a result, the true class’s multi-head attention maps reveal traits and their locations. Implementation-wise, Prompt-CAM is almost a "free lunch," requiring only a modification to the prediction head of Visual Prompt Tuning (VPT). This makes Prompt-CAM easy to train and apply, in stark contrast to other interpretable methods that require designing specific models and training processes. Extensive empirical studies on a dozen datasets from various domains (e.g., birds, fishes, insects, fungi, flowers, food, and cars) validate the superior interpretation capability of Prompt-CAM. The source code and demo are available at https://github.com/Imageomics/Prompt_CAM.

We present a novel usage of Transformers to make image classification interpretable. Unlike mainstream classifiers that wait until the last fully connected layer to incorporate class information to make predictions, we investigate a proactive approach, asking each class to search for itself in an image. We realize this idea via a Transformer encoder-decoder inspired by DEtection TRansformer (DETR). We learn ''class-specific'' queries (one for each class) as input to the decoder, enabling each class to localize its patterns in an image via cross-attention. We name our approach INterpretable TRansformer (INTR), which is fairly easy to implement and exhibits several compelling properties. We show that INTR intrinsically encourages each class to attend distinctively; the cross-attention weights thus provide a faithful interpretation of the prediction. Interestingly, via ''multi-head'' cross-attention, INTR could identify different ''attributes'' of a class, making it particularly suitable for fine-grained classification and analysis, which we demonstrate on eight datasets. Our code and pre-trained models are publicly accessible at the Imageomics Institute GitHub site: https://github.com/Imageomics/INTR.

Recent years have witnessed tremendous growth in the publication of research articles as well as in the rise of new research topics. Articles get published in a streaming manner and therefore retrieving and recommending trending topics continuously by updating the trend of topics with time will be beneficial for young researchers. The proposed topic recommendation system is a clustering-based approach that utilizes an autoencoder framework for the generation of clusters. The autoencoder framework considers articles as input in a multiview framework and produces latent data in lower-dimensional space using graph attention-based encoder and decoder networks. The latent data is then partitioned with respect to its different views and finally, a single consensus overlapping partitioning is produced by satisfying all the views. Since the publication of articles is a continuous process, to update the trending topics continuously, the clustering-based approach is kept on and applied iteratively in a sliding window manner. The generated clusters are analyzed to extract the trending topics and recommendations for future scope. An article can belong to multiple topics and the proposed method is developed by considering such criteria and therefore tested with the modified version of the multilabel scientific article data ArXiv, named CSML.ArXiv. The superiority of the proposed method can be observed from its comparisons with existing methods, and a few baseline methods with respect to cluster formation, topic extraction, and trending topic evaluation.

During any natural disaster or unfortunate accident, both civilians and responders need information on an urgent basis. In such events, microblogging sites particularly Twitter plays an important role in providing real-time information. The raw form of microblog tweets is prodigiously informative but massive in size. The end-users and data analysts have to go through millions of tweets before extraction of any information. To ease the process and extract only relevant information, artificial intelligence (AI)-based techniques can be incorporated to generate summaries from the incoming information. Moreover, tweets keep on arriving continuously in a streaming manner, and therefore in ideal cases, the summaries also need to be updated continuously. In this work, we have proposed a clustering-based summary generation approach that takes multiviewed representations of data and utilizes a new variant of generative adversarial network (GAN) named triple-GAN to perform clustering. Triple-GAN consists of three networks, a generator, a discriminator, and a separator. Maintaining equilibrium among these networks requires proper parameter tuning which makes training of GAN difficult. In the literature, GAN-based techniques have been extensively applied to image datasets. In the proposed method, we have explored the usage of GAN for text data in an unsupervised manner and the analysis of the training of GAN has also been reported. The developed method opens up a new direction in utilizing GAN for solving clustering problem of text data. The proposed method is applied to two versions of four disaster-based microblog datasets and obtained results are compared with many existing and a few baseline methods. The comparative study illustrates the superiority and efficacy of the developed method.

In today’s scenario, a large number of scientific articles on various domains are being published everyday, resulting in a rapid change in the trends of research topics. Retrieving the trending topics, evaluating the trends and extracting the scope of topics could be beneficial to young researchers, which can be recommended for future scope. Publication of articles is a continuous process, and so is the evolution of topics as well as the scope. The dynamic behavior of topics can be handled by continuously updating the partitioning of incoming articles arriving in a streaming manner. This article proposes an online clustering approach utilizing the autoencoder, which is capable of handling the online stream of articles. The training is performed by considering the multiple views of articles and posing the topic modeling (TM) problem as a multiview (MV) clustering problem. In addition, we employ an evolutionary-based approach to the latent representation of data to automatically determine the number of clusters. To learn the nonlinear mapping appropriately and to generate clusters effectively, the model simultaneously optimizes the reconstruction loss and clustering loss in an MV framework. The developed method has experimented on ArXiv dataset to determine the trending topics, scope, and recommendation of topics. The superiority of the proposed method in generating clusters as well as in determining scope is shown by comparing the results with many existing and baseline methods.

The feature selection method is the process of selecting only relevant features by removing irrelevant or redundant features amongst the large number of features that are used to represent data. Nowadays, many application domains especially social media networks, generate new features continuously at different time stamps. In such a scenario, when the features are arriving in an online fashion, to cope up with the continuous arrival of features, the selection task must also have to be a continuous process. Therefore, the streaming feature selection based approach has to be incorporated, i.e., every time a new feature or a group of features arrives, the feature selection process has to be invoked. Again, in recent years, there are many application domains that generate data where samples may belong to more than one classes called multi-label dataset. The multiple labels that the instances are being associated with, may have some dependencies amongst themselves. Finding the co-relation amongst the class labels helps to select the discriminative features across multiple labels. In this article, we develop streaming feature selection methods for multi-label data where the multiple labels are reduced to a lower-dimensional space. The similar labels are grouped together before performing the selection method to improve the selection quality and to make the model time efficient. The multi-objective version of the cuckoo search-based approach is used to select the optimal feature set. The proposed method develops two versions of the streaming feature selection method: ) when the features arrive individually and ) when the features arrive in the form of a batch. Various multi-label datasets from various domains such as text, biology, and audio have been used to test the developed streaming feature selection methods. The proposed methods are compared with many previous feature selection methods and from the comparison, the superiority of using multiple objectives and label co-relation in the feature selection process can be established.

The current paper addresses the online feature selection problem in multi-label classification framework where multi-labelled data with features arriving in an online fashion is considered as input. The proposed approach works in two phases, in the first phase, the best subset of features is selected from the initial available set of features using a multi-objective optimization (MOO) based feature selection technique. In the second phase of the proposed feature selection technique, a newly arrived feature is accepted or rejected based on redundancy with respect to the already selected set of features and relevancy of the arrived feature with respect to the class labels. In order to show the efficacy of the proposed algorithm, it is tested on 7 various types of multi-label datasets of different domains such as text, biology, and audio. The obtained results outperform the results obtained by state-of-the-art approaches in majority of the cases.