Vision Transformers have proven their mettle across a variety of computer vision problems, however, their reliance on pretraining with very large-scale datasets such as JFT-300M is also no secret, as large amounts of data is very conductive to effective feature learning. In this paper, however, we propose a novel Vision Transformer architecture PAEViT that aims to effectively learn generalized features from small-sized datasets, usually having only a few hundred images per class at most. By refining the attention scores based on patch-level interactions and modulating it to enhance the trained model’s ability to focus more on task relevant patches, PAEViT is able to significantly improve upon the performance of the regular ViT model, as well as other variants of the same, when the model is being trained solely on small-sized datasets. In data-constrained situations and visual recognition tasks that do not conform well with the existing large-scale datasets, PAEViT can be used to create effective and scalable solutions with all the features and attention scores being based only on relevant data. Our code is publicly available at https://github.com/AkashVermaIN/PAEViT.

Urban roads face major challenges during rainfall, impacting visibility and road texture. Rain degrades the performance of vision-based systems in traffic safety and monitoring. Real-world rainy road datasets are scarce and often lack diversity. This limits the training of models for image translation under rainy conditions. Effective rainy image synthesis is essential for advancing robust autonomous driving systems. This paper introduces a novel methodology to bridge this gap by generating synthetic datasets simulating adverse weather conditions, particularly rainfall, using Generative Adversarial Networks (GANs). We introduce a newly curated dataset of Kathmandu road scenes to provide diverse, real-world clear-weather imagery. Leveraging these datasets, we employ advanced deep learning techniques to synthesize high-quality rainy road scenes. In experiments, RFDETR achieved a mean Average Precision of 0.405 (averaged over IoU$0.50-0.95$) on the RainGAN-augmented dataset 0.605 at IoU 0.50 and 0.416 at IoU 0.75 demonstrating that our augmentation preserves detection performance under simulated rainfall. Proposed GAN also outperforms existing GAN variants, achieving an FID of 8.34 and LPIPS of 0.12 (versus DCGAN 18.45/0.29, Pix2Pix 15.32/0.24, and StyleGAN 10.57/0.18), indicating realism in the generated imagery Generated outputs are evaluated using Fréchet Inception Distance (FID) and Learned Perceptual Image Patch Similarity (LPIPS) metrics to ensure realism and perceptual fidelity. The paper outlines the data creation pipeline, model implementation, evaluation strategy, and broader implications, offering a robust framework for weatheraffected scene synthesis and road safety research.

Unsupervised image retrieval aims to learn the important visual characteristics without any given level to retrieve the similar images for a given query image. The Convolutional Neural Network (CNN)-based approaches have been extensively exploited with self-supervised contrastive learning for image hashing. However, the existing approaches suffer due to lack of effective utilization of global features by CNNs and biased-ness created by false negative pairs in the contrastive learning. In this paper, we propose a TransClippedCLR model by encoding the global context of an image using Transformer having local context through patch based processing, by generating the hash codes through product quantization and by avoiding the potential false negative pairs through clipped contrastive learning. The proposed model is tested with superior performance for unsupervised image retrieval on benchmark datasets, including CIFAR10, NUS-Wide and Flickr25K, as compared to the recent state-of-the-art deep models. The results using the proposed clipped contrastive learning are greatly improved on all datasets as compared to same backbone network with vanilla contrastive learning.

Breast cancer is a disease where breast cells grow out of control and leads to cancer. Various methodologies have been developed to identify breast cancer. In this paper, we have developed an approach to classify breast cancer from histopathology images. The approach makes use of deep learning based architectures by setting same parameters for all while training and testing on them. Thereafter, all the architectures are compared to see which one is most suited for the classification of breast cancer. Previous works on AlexNet, VGG, ResNet have already been published, and here we have tried to see the performance of those models which have less number of trainable parameters, namely DenseNet121, DenseNet169, DenseNet201, EfficientNetB0, EfficientNetB5, EfficientNetV2B0 and EfficientNetV2S. Here, all the experiments are conducted on BreakHis histopathology dataset by utilizing all the images of resolutions 40X, 100X, 200X and 400X of benign and malignant cancer.

The stochastic gradient descent (SGD) optimizers are generally used to train the convolutional neural networks (CNNs). In recent years, several adaptive momentum based SGD optimizers have been introduced, such as Adam, diffGrad, Radam and AdaBelief. However, the existing SGD optimizers do not exploit the gradient norm of past iterations and lead to poor convergence and performance. In this paper, we propose a novel AdaNorm based SGD optimizers by correcting the norm of gradient in each iteration based on the adaptive training history of gradient norm. By doing so, the proposed optimizers are able to maintain high and representive gradient throughout the training and solves the low and atypical gradient problems. The proposed concept is generic and can be used with any existing SGD optimizer. We show the efficacy of the proposed AdaNorm with four state-of-the-art optimizers, including Adam, diffGrad, Radam and AdaBelief. We depict the performance improvement due to the proposed optimizers using three CNN models, including VGG16, ResNet18 and ResNet50, on three benchmark object recognition datasets, including CIFAR10, CIFAR100 and TinyImageNet.

Recently, Transformer has emerged as a new architecture in deep learning by utilizing self-attention without convolution. Transformer is also extended to Vision Transformer (ViT) for the visual recognition with a promising performance on ImageNet. In this paper, we propose a Vision Transformer Hashing (VTS) for image retrieval. We utilize the pre-trained ViT on ImageNet as the backbone network and add the hashing head. The proposed VTS model is fine tuned for hashing under six different image retrieval frameworks with their objective functions. We perform the extensive experiments on CIFAR10, ImageNet, NUS-Wide, and COCO datasets. The proposed VTS based image retrieval outperforms the recent state-of-the-art hashing techniques with a significant margin. We also find the proposed VTS model as the backbone network is better than the existing networks, such as AlexNet and ResNet. The code is released at https://github.com/shivram1987/VisionTransformerHashing.

In this work, we propose a Wavelet-based Deep Auto Encoder-Decoder Network (WDAED) based image compression which takes care of the various frequency components present in an image. Specifically, we demonstrate improvements over prior approaches utilizing this framework by introducing: (a) wavelet transform pre-processing for decomposing image into different frequencies for their separate processing (b) a very deep super-resolution network as a decoder of the convolutional autoencoder in order to achieve a good quality decompressed image. The end-to-end learning is performed for four wavelet sub-bands in parallel, minimizing the computational time. The encoder compresses the image by generating the latent space representations, whereas the decoder transforms the latent space to image space. The algorithm has been tested on various standard datasets i.e., ImageNet, Set 5, Set 14, Live 1, Kodak, Classic 5, General 100 and CLIC 2019 dataset. The proposed algorithm clearly exhibited the compression performance improvement of approximately 5%, 5.5%, and 13% in terms of PSNR, PSNRB and SSIM respectively.

In this letter, we are proposing BoCNN architecture framework for occluded thermal face recognition. We analyzed the performance of Pretrained models using transfer learning and they exhibit promising results for thermal faces without occlusion. The performance degrades with occlusion. We have used different decision level fusion strategies post transfer learning for the performance enhancement of the pretrained models. All the fusion strategies used in the proposed work give better results compared to any of the single CNN architecture.

Stochastic gradient descent (SGD) is one of the core techniques behind the success of deep neural networks. The gradient provides information on the direction in which a function has the steepest rate of change. The main problem with basic SGD is to change by equal-sized steps for all parameters, irrespective of the gradient behavior. Hence, an efficient way of deep network optimization is to have adaptive step sizes for each parameter. Recently, several attempts have been made to improve gradient descent methods such as AdaGrad, AdaDelta, RMSProp, and adaptive moment estimation (Adam). These methods rely on the square roots of exponential moving averages of squared past gradients. Thus, these methods do not take advantage of local change in gradients. In this article, a novel optimizer is proposed based on the difference between the present and the immediate past gradient (i.e., diffGrad). In the proposed diffGrad optimization technique, the step size is adjusted for each parameter in such a way that it should have a larger step size for faster gradient changing parameters and a lower step size for lower gradient changing parameters. The convergence analysis is done using the regret bound approach of the online learning framework. In this article, thorough analysis is made over three synthetic complex nonconvex functions. The image categorization experiments are also conducted over the CIFAR10 and CIFAR100 data sets to observe the performance of diffGrad with respect to the state-of-the-art optimizers such as SGDM, AdaGrad, AdaDelta, RMSProp, AMSGrad, and Adam. The residual unit (ResNet)-based convolutional neural network (CNN) architecture is used in the experiments. The experiments show that diffGrad outperforms other optimizers. Also, we show that diffGrad performs uniformly well for training CNN using different activation functions. The source code is made publicly available at https://github.com/shivram1987/diffGrad.

Computer-aided histopathological image analysis for cancer detection is a major research challenge in the medical domain. Automatic detection and classification of nuclei for cancer diagnosis impose a lot of challenges in developing state-of-the-art algorithms due to the heterogeneity of cell nuclei and dataset variability. Recently, a multitude of classification algorithms have used complex deep learning models for their dataset. However, most of these methods are rigid, and their architectural arrangement suffers from inflexibility and non-interpretability. In this research article, we have proposed a hybrid and flexible deep learning architecture O L ConvNet that integrates the interpretability of traditional object-level features and generalization of deep learning features by using a shallower Convolutional Neural Network (CNN) named as CNN 3L . CNN 3L reduces the training time by training fewer parameters and hence eliminating space constraints imposed by deeper algorithms. We used F1-score and multiclass Area Under the Curve (AUC) performance parameters to compare the results. To further strengthen the viability of our architectural approach, we tested our proposed methodology with state-of-the-art deep learning architectures AlexNet, VGG16, VGG19, ResNet50, InceptionV3, and DenseNet121 as backbone networks. After a comprehensive analysis of classification results from all four architectures, we observed that our proposed model works well and performs better than contemporary complex algorithms.

Local descriptors used in face recognition are robust in a sense that these descriptors perform well in varying pose, illumination, and lighting conditions. The accuracy of these descriptors depends on the precision of mapping the relationship that exists in the local neighborhood of a facial image into microstructures. In this paper, a local gradient hexa pattern is proposed that identifies the relationship among the reference pixel and its neighboring pixels at different distances across different derivative directions. Discriminative information exists in the local neighborhood as well as in different derivative directions. The proposed descriptor effectively transforms these relationships into binary micropatterns discriminating inter-class facial images with optimal precision. The recognition and retrieval performance of the proposed descriptor has been compared with state-of-the-art descriptors, namely, local derivative pattern, local tetra pattern, multiblock local binary pattern, and local vector pattern over the most challenging and benchmark facial image databases, i.e., Cropped Extended Yale B, CMU-PIE, color-FERET, LFW, and Ghallager database. The proposed descriptor has better recognition as well as retrieval rates compared with state-of-the-art descriptors.

Local binary pattern (LBP) is widely adopted for efficient image feature description and simplicity. To describe the color images, it is required to combine the LBPs from each channel of the image. The traditional way of binary combination is to simply concatenate the LBPs from each channel, but it increases the dimensionality of the pattern. In order to cope with this problem, this paper proposes a novel method for image description with multichannel decoded LBPs. We introduce adder- and decoder-based two schemas for the combination of the LBPs from more than one channel. Image retrieval experiments are performed to observe the effectiveness of the proposed approaches and compared with the existing ways of multichannel techniques. The experiments are performed over 12 benchmark natural scene and color texture image databases, such as Corel-1k, MIT-VisTex, USPTex, Colored Brodatz, and so on. It is observed that the introduced multichannel adder- and decoder-based LBPs significantly improve the retrieval performance over each database and outperform the other multichannel-based approaches in terms of the average retrieval precision and average retrieval rate.

A novel image feature descriptor based on the local bit-plane decoded pattern (LBDP) is introduced for indexing and retrieval of biomedical images in this paper. A local bit-plane transformation scheme is proposed to compute the local bit-plane transformed values for each image pixel from the bit-plane binary contents of its each neighboring pixels. The introduced LBDP is generated by finding a binary pattern using the difference of center pixel's intensity value with the local bit-plane transformed values. The efficacy of the LBDP is tested under biomedical image retrieval using average retrieval precision and average retrieval rate. Three benchmark databases Emphysema-CT, NEMA-CT, and Open Access Series of Imaging Studies magnetic resonance imaging are used for the evaluation and comparison of the proposed approach with recent state-of-art methods. The experimental results confirm the discriminative ability and the efficiency of the proposed LBDP for biomedical image indexing and retrieval and prove the outperformance of existing biomedical image retrieval approaches.

Content‐based image retrieval (CBIR) is demanding accurate with efficient retrieval approaches to index and retrieve the most similar images from the huge image databases. This study introduces a novel local neighbourhood‐based robust colour occurrence descriptor (LCOD) to encode the colour information present in the local structure of the image. The colour information is processed in two steps: first, the number of colours is reduced into a less number of shades by quantising the red–green–blue colour space; second, the reduced colour shade information of the local neighbourhood is used to compute the descriptor. A local colour occurrence binary pattern is generated for each pixel of the image by representing each reduced colour shade occurrence in its local neighbourhood using a binary pattern. The descriptor is constructed by summing the local colour occurrence binary patterns of all the pixels in the image. LCOD is tested over the natural and colour texture databases for CBIR and experimental results suggest that LCOD outperforms other state‐of‐the‐art descriptors. The performance of the proposed descriptor is promising in the case of illumination difference, rotation and scaling also and it can be effectively used for accurate image retrieval under various image transformations.

The medical image retrieval plays an important role in medical diagnosis where a physician can retrieve most similar images from template images against a query image of a particular patient. In this letter, a new and efficient image features descriptor based on the local diagonal extrema pattern (LDEP) is proposed for CT image retrieval. The proposed approach finds the values and indexes of the local diagonal extremas to exploit the relationship among the diagonal neighbors of any center pixel of the image using first-order local diagonal derivatives. The intensity values of the local diagonal extremas are compared with the intensity value of the center pixel to utilize the relationship of central pixel with its neighbors. Finally, the descriptor is formed on the basis of the indexes and comparison of center pixel and local diagonal extremas. The consideration of only diagonal neighbors greatly reduces the dimension of the feature vector which speeds up the image retrieval task and solves the “Curse of dimensionality” problem also. The LDEP is tested for CT image retrieval over Emphysema-CT and NEMA-CT databases and compared with the existing approaches. The superiority in terms of performance and efficiency in terms of speedup of the proposed method are confirmed by the experiments.

A new image feature description based on the local wavelet pattern (LWP) is proposed in this paper to characterize the medical computer tomography (CT) images for content-based CT image retrieval. In the proposed work, the LWP is derived for each pixel of the CT image by utilizing the relationship of center pixel with the local neighboring information. In contrast to the local binary pattern that only considers the relationship between a center pixel and its neighboring pixels, the presented approach first utilizes the relationship among the neighboring pixels using local wavelet decomposition, and finally considers its relationship with the center pixel. A center pixel transformation scheme is introduced to match the range of center value with the range of local wavelet decomposed values. Moreover, the introduced local wavelet decomposition scheme is centrally symmetric and suitable for CT images. The novelty of this paper lies in the following two ways: 1) encoding local neighboring information with local wavelet decomposition and 2) computing LWP using local wavelet decomposed values and transformed center pixel values. We tested the performance of our method over three CT image databases in terms of the precision and recall. We also compared the proposed LWP descriptor with the other state-of-the-art local image descriptors, and the experimental results suggest that the proposed method outperforms other methods for CT image retrieval.

The region descriptors using local intensity ordering patterns have become more popular recent years for image matching due to its enhanced discriminative ability. However, the dimension of these descriptors increases rapidly with the slight increase in the number of local neighbors under consideration and becomes unreasonable for image matching due to time constraint. In this paper, we reduce the dimension of the descriptor and matching time significantly while keeping up the comparable performance by considering the number of neighboring sample points in an interleaved manner. The proposed interleaved order based local descriptor (IOLD) considers the local neighbors of a pixel as a set of interleaved neighbors and constructs the descriptor over each set separately and finally combines them to produce a single pattern. We extract the local ordering pattern to cope up with the illumination effect in an inherent rotation invariant manner. The novelty lies with using multiple neighboring sets in an interleaved fashion. We also explored the local intensity order pattern in a multisupport-region scenario. Results are compared over three challenging and widely adopted image matching data sets with other prominent descriptors under various image transformations. Results based on experiments suggest that the proposed IOLD descriptor outperforms in terms of both improved matching performance and reduced matching time. We also found that the amount of improvement is significant under complex illumination difference while showing more robustness toward noise.

The various roles that aggregation prone regions (APRs) are capable of playing in proteins are investigated here via comprehensive analyses of multiple non-redundant datasets containing randomly generated amino acid sequences, monomeric proteins, intrinsically disordered proteins (IDPs) and catalytic residues. Results from this study indicate that the aggregation propensities of monomeric protein sequences have been minimized compared to random sequences with uniform and natural amino acid compositions, as observed by a lower average aggregation propensity and fewer APRs that are shorter in length and more often punctuated by gate-keeper residues. However, evidence for evolutionary selective pressure to disrupt these sequence regions among homologous proteins is inconsistent. APRs are less conserved than average sequence identity among closely related homologues (≥80% sequence identity with a parent) but APRs are more conserved than average sequence identity among homologues that have at least 50% sequence identity with a parent. Structural analyses of APRs indicate that APRs are three times more likely to contain ordered versus disordered residues and that APRs frequently contribute more towards stabilizing proteins than equal length segments from the same protein. Catalytic residues and APRs were also found to be in structural contact significantly more often than expected by random chance. Our findings suggest that proteins have evolved by optimizing their risk of aggregation for cellular environments by both minimizing aggregation prone regions and by conserving those that are important for folding and function. In many cases, these sequence optimizations are insufficient to develop recombinant proteins into commercial products. Rational design strategies aimed at improving protein solubility for biotechnological purposes should carefully evaluate the contributions made by candidate APRs, targeted for disruption, towards protein structure and activity.